I have file 1 which have those lines: ATM 1434.972183 BMPR2 10762.78192 BMPR2 10762.78192 BMPR2 1469.14535 BMPR2 1469.14535 BMPR2 1738.479639 BMS1 4907.841667 BMS1 4907.841667 BMS1 880.4532628 BMS1 880.4532628 BMS1P17 1249.75 BMS1P17 1249.75 BMS1P17 1606.821429 BMS1P17 1606.821429 BMS1P17 1666.333333 BMS1P17 1666.333333 BMS1P17 2108.460317 BMS1P17 2108 And file 2 have a list of words: ATM BMS1 So, the output will be like this: ATM 1434.972183 BMS1 4907.841667 BMS1 4907.841667 BMS1 880.4532628 BMS1 880.4532628 I know it's really a duplicate question, but I tried all types of grep and sed and awk, maybe it will works with you guys with this tiny example but I have a very huge file > 1M lines and all previous way doesn't help it return part of the lines that containing those words although there are other words in file 2 that matches the lines from file 1

I understand that tar --list --file=x will list all files and folders. I am looking to just list the-top level data. Does anyone know how to do that? Alternatively, does anyone know how to list only the top-level files, but all folders including subfolders? Maybe with grep somehow? I'm after something that works on most nix flavors including MacOS.

I have the following file example:

some text is here  
   sometext(1,21);
   sometext(2,9);
   sometext(3,231);
   sometext(10,1112);
   sometext(11,17)
Some text is here

I'm trying to extract the second number in the parentheses of the lines containing sometext, so in the above example, the numbers 21,9,231,1112,17. I didn't find a suitable grep command for the above pattern.

Input file1 is: dog 123 4335 cat 13123 23424 deer 2131 213132 bear 2313 21313 I give the match the pattern from in other file ( like dog 123 4335 from file2). I match the pattern of the line is dog 123 4335 and after printing all lines without match line my output is: cat 13123 23424 deer 2131 213132 bear 2313 21313 If only use without address of line only use the pattern, for example 1s how to match and print the lines?

**I'm not talking about -o option.** [Posix](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03) says: > The search for a matching sequence starts at the beginning of a string and stops when the **first sequence matching the expression is found**, where "first" is defined to mean "begins earliest in the string". If the pattern permits a variable number of matching characters and thus there is more than one such sequence starting at that point, the **longest such sequence is matched**. For example, the BRE "bb*" matches the second to fourth characters of the string "abbbc", and the ERE "(wee|week)(knights|night)" matches all ten characters of the string "weeknights". And I want to **verify** what is being said in posix and this tutorial [regTutorialSite](https://www.regular-expressions.info/posix.html) : > A POSIX-compliant engine will still find the **leftmost match**. If you **apply** Set|SetValue to Set or SetValue **once**, it **will match Set**. How to "apply once"? When i run **grep -o** the result is two strings: Set and SetValue, but not just " one leftmost " . That is, I read about one thing, but in practice I get something else. So, how to see what string was matched by regex? (Perhaps the question was formulated incorrectly or could have been better)

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a pattern, but only to return the portion of the line after the match. The portion before and after the match will consistently vary. I have played with using sed or awk, but have not been able to figure out how to filter the line to either delete the part before the match, or just return the part after the match, either will work. This is an example of a line that I need to filter: 2011-11-07T05:37:43-08:00 isi-udb5-ash4-1(id1) /boot/kernel.amd64/kernel: [gmp_info.c:1758](pid 40370="kt: gmp-drive-updat")(tid=100872) new group: : { 1:0-25,27-34,37-38, 2:0-33,35-36, 3:0-35, 4:0-9,11-14,16-32,34-38, 5:0-35, 6:0-15,17-36, 7:0-16,18-36, 8:0-14,16-32,34-36, 9:0-10,12-36, 10-11:0-35, 12:0-5,7-30,32-35, 13-19:0-35, 20:0,2-35, down: 8:15, soft_failed: 1:27, 8:15, stalled: 12:6,31, 20:1 } The portion I need is everything after "stalled". The background behind this is that I can find out how often something stalls: cat messages | grep stalled | wc -l What I need to do is find out how many times a certain node has stalled (indicated by the portion before each colon after "stalled". If I just grep for that (ie 20:) it may return lines that have soft fails, but no stalls, which doesn't help me. I need to filter only the stalled portion so I can then grep for a specific node out of those that have stalled. For all intents and purposes, this is a freebsd system with standard GNU core utils, but I cannot install anything extra to assist.

I noticed a difference in behavior between an older pcre2grep version (10.22) and a more recent one (10.42), and I am wondering how I can get the old behavior back. Take the following file:

aaa
bbb
XXX
ccc
ddd
eee
XXX
fff
ggg

Back with v10.22 (Debian 9), I could achieve non-greedy multi-line captures:

$ pcre2grep --version
pcre2grep version 10.22 2016-07-29

$ pcre2grep -nM '(.|\n)*?XXX' file
1:aaa
bbb
XXX
4:ccc
ddd
eee
XXX

Notice how it captured two multi-line groups, one starting at line 1 (1:aaa), and a second starting at line 4 (4:ccc). Now, with a more recent version (10.42, Debian 12), its behaviour changed:

$ pcre2grep --version
pcre2grep version 10.42 2022-12-11

$ pcre2grep -nM '(.|\n)*?XXX' file
1:aaa
bbb
XXX
ccc
ddd
eee
XXX

Now I only have one group, starting with 1:aaa. Basically, it seems to ignore the non-greedy operator (?). The result is the same if I omit it:

$ pcre2grep -nM '(.|\n)*XXX' file
1:aaa
bbb
XXX
ccc
ddd
eee
XXX

How can I get the behavior of v10.22 back ? In other words, how can I do non-greedy multiline captures in recent versions of pcre2grep?

Is there a command-line tool to text-search a docx file? I tried grep, but it doesn't work with docx even though it works fine with txt and xml files. I could convert the docx to txt first, but I'd prefer a tool that operates directly on docx files. I need the tool to work under Cygwin. OP edit: Later I found out that the easiest way to achieve the grep is actually to convert those docx to txt then grep over them.

I have a dataset of images labeled/classified by characteristics, where an image can have more than one label. I want to count how many of each identifier I have. A toy dataset is created below, with different colors being the labels.

bballdave025@MY-MACHINE /home/bballdave025/toy
$ touch elephant_grey.jpg && touch zebra_white_black.jpg && touch rubik-s_cube_-_1977-first-prod_by_ErnoRubik-_red_orange_yellow_white_blue_green.jpg && touch Radio_Hotel.Washington_Heights.NYC-USA_green_yellow_blue_red_orange_reddish-purple_teal_grey-brown.jpg && touch Big_Bird__yellow_orange_red.jpg

Let's make it more easily visible. The files in the initially labeled dataset are shown below. (The | awk -F'/' '{print $NF}' is just meant to take off the ./ or path/to/where/the/jpegs/are/ that would otherwise be before the filename.)

$ find . -type f | awk -F'/' '{print $NF}'
Big_Bird__yellow_orange_red.jpg
elephant_grey.jpg
Radio_Hotel.Washington_Heights.NYC-USA_green_yellow_blue_red_orange_reddish-purple_teal_grey-brown.jpg
rubik-s_cube_-_1977-first-prod_by_ErnoRubik-_red_orange_yellow_white_blue_green.jpg
zebra_white_black.jpg

Those are the filenames for labeled versions of the images. The corresponding originals are below:

$ find ../toy_orig_bak/ -type f | awk -F'/' '{print $NF}'
Big_Bird_.jpg
elephant.jpg
Radio_Hotel.Washington_Heights.NYC-USA.jpg
rubik-s_cube_-_1977-first-prod_by_ErnoRubik-.jpg
zebra.jpg

This is to show that the color labels are inserted between the filename and the dot extension. They are separated from each other and from the original filename by a (delimiting) _ character. (There are rules for the label names and for the filenames1.) The only allowed color strings at this initial point are any of {black, white, grey, red, orange, yellow, green, blue, reddish-purple, teal, grey-brown}. I further want to show that other labels may be added, as long as they're part of my controlled vocabulary, something which can be changed only by me. Imagine a file named rainbox.jpg gets put in with the original filenames ( touch ../toy_orig_bak/rainbow.jpg, for those of you following along for reproducibility ). I decide that I want to add indigo and violet to my controlled vocabulary list, so I can create the labeled filename,

$ touch rainbow_red_orange_yellow_green_blue_indigo_violet.jpg

Desired Output Again, I want a count of each of the labels. For the dataset I've set up (including that last labeled picture of a rainbow), the correct output would be

1 black
      3 blue
      3 green
      1 grey
      1 grey-brown
      1 indigo
      4 orange
      4 red
      1 reddish-purple
      1 teal
      1 violet
      2 white
      4 yellow

(The counts were performed somewhat manually, due to my grep confusion.) Attempts and a note on the details of the solution I want Research below My first thought (although I did worry about delimiter consumption) was to look at the surrounding delimiters: '_' before and '_' or '.' after. Here's first my grep attempt

find . -type f -iname "*.jpg" | \
 \
    grep -o "[_]\(black\|white\|grey\|red\|orange\|yellow\|green\|blue\|"\
"reddish-purple\|teal\|grey-brown\|indigo\|violet\)[_.]" | \
 \
        tr -d [_.] | sort | uniq -c

and its output

3 blue
      1 green
      1 grey
      1 orange
      3 red
      1 teal
      1 violet
      1 white
      3 yellow

Which is not the same as before. Here's the comparison.

Before              |   Now
-----------------------|---------------------
      1 black          |
      3 blue           |      3 blue
      3 green          |      1 green
      1 grey           |      1 grey
      1 grey-brown     |
      1 indigo         |
      4 orange         |      1 orange
      4 red            |      3 red
      1 reddish-purple |
      1 teal           |      1 teal
      1 violet         |      1 violet
      2 white          |      1 white
      4 yellow         |      3 yellow
                       |

I know this is happening because the regex engine consumes the second delimiter2. Here is the crux of my main question: (I do want to solve my count problem, and I'll talk about some solutions I've researched and considered myself, but) the detail I want to know is about truly regular expressions and consuming the delimiter. I want to get a count of each identifier string, and I'm wondering if I can do it with approach and (POSIX) Basic Regular Expressions – BRENote 2 and [reddit thread](https://www.reddit.com/r/askscience/comments/5rttyo/do_extended_regular_expressions_still_denote_the/) ([archived as a gist](https://gist.github.com/bballdave025/b2f7a190907146151696eed394079a64)) , specifically with grep . Any of sed, awk, IFS with read, etc. are welcome, too. I'm sure someone has a way solve this problem with Perl (dermis and feline can be divorced by manifold methods), and I'd be glad to get that one, too. Basically, I am absolutely okay with other solutions to the task of getting a count of each identifier string. However, if it's true that there's no way of stepping back the engine with a Basic Regular Expression engine (that's truly regular), I want to know. I've thought of zero-width matches, lookaheads, and look-behinds, but I don't know how these play out in POSIX Basic Regular Expressions or in mathematically/grammatically regular language parsers. One thing I realize I wasn't taking into account The point of the rules (see note \[2\]) was to allow the regex to take advantage of the fact that we should be able to assure ourselves that we're only getting the part of the classified filename with labels, as we only allow one of a finite set of strings preceded by an underscore and followed by either an underscore or a dot, with the dot only happening before the file extension. (I guess we can't be absolutely certain, as the original, pre-labeled filename could have one of the labels immediately preceding the dot - something like a_sunburn_that_is_bright_red.jpg, but that's something for which I check and correct by adding a specific non-label string before the dot and extension.) My regex, imagining that it could get past the delimiter being consumed, would still allow the following example problems

the_new_red_car_-_1989_red_black_silver.jpg

- would return {red, red , silver } as is, - {red, red, black , silver } if working without consuming the 2nd '_', - whereas {red, black , silver } is desired

parrot_at_blue_gold_banquet_-_a_black_tie_affair_yellow_red_green.jpg

- would return {blue, black, yellow, green} as is, - {blue, gold, black, yellow, green} if not consuming the 2nd '_', - whereas {yellow, red, green} is desired Extra points for answers and discussions that take that into account. ; ) Research and ideas There are a few discussions on different StackExchange sites, like [this one](https://web.archive.org/web/20230925145242/https://stackoverflow.com/questions/63821591/how-to-split-a-string-by-underscore-and-extract-an-element-as-a-variable-in-bash) , [that one](https://web.archive.org/web/20250602171231/https://stackoverflow.com/questions/49784912/regex-of-underscore-delimited-string) , [another one](https://web.archive.org/web/20250602171046/https://unix.stackexchange.com/questions/267677/add-quotes-and-new-delimiter-around-space-delimited-words) , but I think the [Unix & Linux discussion here](https://unix.stackexchange.com/a/334551/291375) ([archived](https://web.archive.org/web/20230324152449/https://unix.stackexchange.com/questions/334549/how-do-i-extract-multiple-strings-which-are-comma-delimited-from-a-log-file)) is the best one. I think that one of the approaches in this answer from @terdon ♦ or in the answer with hashes – from @Sobrique – might be useful. I keep thinking that some version of ^.*\([_][]\)\+[.]jpg$ might be key to the situation, but I haven't been able to put together that solution today. If you know how it can help, you're welcome to give an answer using it; I'm going to wait for a fresh brain tomorrow morning. Edit: @ilkkachu successfully used this idea. Why am I doing this? I'm training a CNN to recognize different occurrences (not colors) in pictures of old and often handwritten books. I want to make sure the classes are balanced as I want. Also, I'll compare this with another method that doesn't look at the delimiter to make sure I don't have any problems like a '_yllow' (instead of '_yellow'), or a '_whiteorange' _instead of '_white_orange'). Most of the labels are put on through a Java program I've put together, but I've given a little leeway for people to change the filenames themselves in case of multiple labels for one file. Having given that permission, I have the responsibility of verifying legal labeled filenames. Notes \[1\] The rules for the identifying/classifying labels are: The identifiers can be any of a finite set of strings which can contain only characters in [A-Za-z0-9-] but not underscores.
The bare filenames (without dot and extension) can consist of any ASCII characters except: 1) non-printable/control characters; 2) spaces or tabs ; OR 3) any of [!"#$%&/)(\]\[}{*?] See the next paragraph for the real 3). (Note that this means the bare filenames CAN have an underscore, '_', or even several of them.)
Edit: I had my no-no list of characters as is now crossed (struck) out above when @ilkkachu gave the accepted answer. One option of that answer makes excellent use of the '@' which was then not in the excluded character group, but which I actually don't allow in my filenames. There are other omissions in the original character group. As I actually want it, the above paragraph should be amended with the following. 3) any of '[] ~@#$%^&|/)(}{[*?>\ Edit: Now this compiles as a BRE. (This was the simplest and most-readable BRE I could come up with.) that beautifully crazy character group means that any of { [, ], , ~, @, #, $, %, ^, &, |, \, /, ), (, }, {, [, *, ?, >, `} is not allowed – and neither is any tab (\t, ...), nor any non-printing/control characters. Some of these are already standard on the no-no list for filenames on different OSs, but I give _my_ complete set (when I'm in charge of creating the filenames). \[2\] Here is what I mean by the delimiter being consumed. I'll do my best to illustrate an example with our (Basic) Reg(ular)Ex(pression),

"[_]\(black\|white\|grey\|red\|orange\|yellow\|green\|blue\|"\
"reddish-purple\|teal\|grey-brown\|indigo\|violet\)[_.]"

Here goes. This missing of some of the color strings is happening because the regex engine consumes the second delimiter. For example, using O to denote part of a miss (non-match) and X to denote part of a hit (match), with YYYYY denoting a complete match for the whole regex pattern, we get the following behavior.

Engine goes along looking for '_'

engine is here
       |
       v
rainbow_red_orange_yellow_green_blue_indigo_violet.jpg
OOOOOOO

Matches 
      [_]
          with '_'
engine is at
        |
        v
rainbow_red_orange_yellow_green_blue_indigo_violet.jpg
OOOOOOOX

Matches
 \(...\|red\|...\)
                  with 'red'    
  engine is at
           |
           v
rainbow_red_orange_yellow_green_blue_indigo_violet.jpg
OOOOOOOXXXX

Matches
          [_.]
               with '_'
   engine is at
            |
            v
rainbow_red_orange_yellow_green_blue_indigo_violet.jpg
OOOOOOOXXXXX

We have a whole match! 

rainbow_red_orange_yellow_green_blue_indigo_violet.jpg
OOOOOOOYYYYY

Given the  -o  flag, the engine outputs

'_red_'

The  'tr -d [_.]' takes off the surrounding underscores, 
and our output line becomes 

'red'

The problem now is that the engine cannot go back to find the '_' before 'orange', or at least it can't do so using any process I know about from my admittedly imperfect knowledge of Basic Regular Expressions. As far as a REGULAR expression engine, using a REGULAR grammar and a REGULAR language parser knows, the whole universe in which it's searching now consists of

orange_yellow_green_blue_indigo_violet.jpg

(I don't know if this statement is correct from a mathematical/formal-language point of view, and I'd be interested to know.) And the process continues as from the first, beginning with Engine goes along looking for '_'

orange_yellow_green_blue_indigo_violet.jpg
OOOOOOXXXXXXXX

Match!

orange_yellow_green_blue_indigo_violet.jpg
OOOOOOYYYYYYYY

Engine spits out  '_yellow_'  which is 'tr -d [_.]'-ed

Engine cannot go back, so its search universe is now

green_blue_indigo_violet.jpg

and we continue with

green_blue_indigo_violet.jpg
OOOOOXXXXXX

Match!

green_blue_indigo_violet.jpg
OOOOOYYYYYYOOOOOOYYYYYYYY

That last match being on the '.' from [_.]

More formally, I want to know if it can be done with a real regular expression, i.e. one which can define a regular language and whose language is a context-free language, cf. Wikipedia's Regex article (archived). I think this is the same as a POSIX regular expression, but I'm not sure.
Refs. [A] (archived), [B] (archived), [C] (archived), Dang it, I know there's a missing ending parenthesis up there in the text, somewhere, because I noticed it and went up to fix it. When I got up into the text, I couldn't remember the context of the parenthesis, so it's still there, just mocking me. I found it, and I bolded it! I'll probably take the bold formatting and this note down, soon, but I'm sharing my happiness right now.