Unix & Linux Stack Exchange
Q&A for users of Linux, FreeBSD and other Unix-like operating systems
Latest Questions
1
votes
2
answers
2613
views
Regex101 to Grep/egrep returns nothing
I've been playing with the following regex to cut some content from a markdown file for me which works great in regex101 but can't seem to get grep to work with it. Either it errors or doesn't return anything The Regex is here: https://regex101.com/r/XDImM9/1 or `(?s)##\s\[v0.0.1].+?(?=---)` I've tr...
I've been playing with the following regex to cut some content from a markdown file for me which works great in regex101 but can't seem to get grep to work with it.
Either it errors or doesn't return anything
The Regex is here: https://regex101.com/r/XDImM9/1
or
(?s)##\s\[v0.0.1].+?(?=---)
I've tried using grep with the -P flag which should support PCRE style regex but doesn't seem to do much
grep -P 'm/(?s)##\s\[v0.0.1].+?(?=---)/' CHANAGELOG.md
The sample data I have been working on is
# Changelog
All notable changes to this project will be documented in this file.
Please note that all entries must end with ---
to allow for the auto release body to use the Changelog
## [v0.0.1]
### Added
- Initial Commit/Release
---
Matthew Gialelis
(113 rep)
Dec 19, 2019, 12:00 AM
• Last activity: Jul 31, 2023, 04:35 PM
4
votes
0
answers
5327
views
In journalctl internal grep, is it possible to specify option to only output matched pattern?
I have the following journal messages: ```bash $ journalctl _COMM=kwin_wayland -o cat --since "-10s" ... kwin_screencast: Dropping a screencast frame because the compositor is slow kwin_screencast: Dropping a screencast frame because the compositor is slow js: hello goodbue kwin_screencast: Dropping...
I have the following journal messages:
>Filter output to entries where the MESSAGE= field matches the specified regular expression. PERL-compatible regular expressions are used, see pcre2pattern(3) for a detailed description of the syntax. The normal grep can work in PCRE mode with -P option. And as described here , I can use
$ journalctl _COMM=kwin_wayland -o cat --since "-10s"
...
kwin_screencast: Dropping a screencast frame because the compositor is slow
kwin_screencast: Dropping a screencast frame because the compositor is slow
js: hello
goodbue
kwin_screencast: Dropping a screencast frame because the compositor is slow
kwin_screencast: Dropping a screencast frame because the compositor is slow
...
I want to extract only messages starting with "js: ".
From journalctl manual:
> -g, --grep=>Filter output to entries where the MESSAGE= field matches the specified regular expression. PERL-compatible regular expressions are used, see pcre2pattern(3) for a detailed description of the syntax. The normal grep can work in PCRE mode with -P option. And as described here , I can use
-o
to only match pattern, and \K
in pattern itself to drop the beginning. So this command:
$ journalctl _COMM=kwin_wayland -o cat --since "-10s" | grep -Po "js: \K.*"
hello
gives me pretty much what I want (unfortunately, it also cuts the next line of message).
I would like to use a single command, i.e. not piping to normal grep, but use the -g
option of journalctl.
I can specify such pattern for internal grep:
$ journalctl _COMM=kwin_wayland -o cat --since "-10s" -g "js: \K.*"
js: hello
goodbue
And it prints my message (fortunately, with next line of message). But it is with "js:" part. I want to drop it.
Is it possible to specify journalctl's internal grep option "-o" somehow?
Alternatively, I could use grouping in the pattern, but again, how do I specify that I want to output only that group?
Ashark
(1069 rep)
Jul 16, 2023, 11:20 AM
7
votes
5
answers
17612
views
Local installation of pcre2 not detected while installing R 4.0.4 from source
I am trying to do a build and local install of R 4.0.4 on Red Hat Linux 6.8. There were several unmet dependencies which I resolved by doing local installations (following the procedure in [this][1]). However, I couldn't resolve the issue of `pcre2` with that procedure. This is the configure command...
I am trying to do a build and local install of R 4.0.4 on Red Hat Linux 6.8. There were several unmet dependencies which I resolved by doing local installations (following the procedure in this ). However, I couldn't resolve the issue of
pcre2
with that procedure. This is the configure command I run:
./configure --with-pcre2 --prefix=$HOME/bin/R-4.0.4 --enable-R-shlib LDFLAGS="-L/$HOME/local/zlib-1.2.11/lib -L/$HOME/local/bzip2-1.0.8/lib -L/$HOME/local/xz-5.2.5/lib -L/$HOME/local/pcre2-10.00/lib" CPPFLAGS="-I/$HOME/local/zlib-1.2.11/include"
This is the error I get:
checking whether PCRE support suffices... no
configure: error: PCRE2 library and headers are required, or use --with-pcre1 and PCRE >= 8.32 with UTF-8 support
I also tried configure with a local installation of pcre-8.44
and --with-pcre1
flag but I get the same error.
What should I do so that the configure script detects the pcre2
local installation?
Dronacharya
(171 rep)
Feb 22, 2021, 06:54 AM
• Last activity: Jul 5, 2023, 09:43 AM
1
votes
2
answers
110
views
XMLERR: Element '=").*?(?=")</regex' not closed while using lookaround
I am trying to use lookaround element in PCRE2 regex for Wazuh tool, i need to match strings which are in double quotes and made the below regex however it looks its picking up " element. (? *ERROR: (1226): Error reading XML file 'etc/decoders/local_decoder.xml': XMLERR: Element '=").*?(?=") element...
I am trying to use lookaround element in PCRE2 regex for Wazuh tool, i need to match strings which are in double quotes and made the below regex however it looks its picking up " element.
(?
*ERROR: (1226): Error reading XML file 'etc/decoders/local_decoder.xml': XMLERR: Element '=").*?(?=") element properly
Atul
(1911 rep)
Dec 31, 2022, 07:20 AM
• Last activity: Dec 31, 2022, 11:12 AM
0
votes
1
answers
276
views
Is possible to discard outer brackets in regex and consider inner bracket inside?
I have many patterns with bracket enclosure, I made a regular expression where is not considering brackets and just only what is inside/between them, but exists a problem when the text within brackets contain [] brackets too. Thanks! Regex: `(? (?<=\[).*?\[?(?=\]))` For example, ``` A) [ClusterRecei...
I have many patterns with bracket enclosure,
I made a regular expression where is not considering brackets and just only
what is inside/between them, but exists a problem when the text within brackets
contain [] brackets too.
Thanks!
Regex:
(?(?<=\[).*?\[?(?=\]))
For example,
A) [ClusterReceiver ]
B) [first-second-third-8050-exec-a ]
From above, B) is working perfectly, but A) not
What is being returned for every case (without quotes):
---
B) "first-second-third-8050-exec-a "
A) "ClusterReceiver[99"
What is desired?
------------------
B) "first-second-third-8050-exec-a "
A) "ClusterReceiver"
The problem is when exist [ ]
bracket enclosure within outer [ ]
enclosure.
The worst case is when exists that problem like A),
can you help me by giving a suggestion how to accept at least 1 bracket, in order to have A) as desired "ClusterReceiver" ?
dcubaz
(23 rep)
Nov 16, 2022, 08:14 PM
• Last activity: Nov 16, 2022, 08:27 PM
1
votes
4
answers
736
views
Find lines with first word containing a certain set of letters
The input file exists out of lines containing multiple 10-letter combinations: NGNAEAREAX EAHVSELYCI FNWGNLACQM AWKLRMDHIT PRYMFNYMVM NCNREDEEEQ EAHVSELYCI FNWGNLACQM AWKLRMDHIT PRYMFNYMVM I need a regex (PCRE or BRE/ERE) that finds all lines of which the first word contains the letters to form the...
The input file exists out of lines containing multiple 10-letter combinations:
NGNAEAREAX EAHVSELYCI FNWGNLACQM AWKLRMDHIT PRYMFNYMVM
NCNREDEEEQ EAHVSELYCI FNWGNLACQM AWKLRMDHIT PRYMFNYMVM
I need a regex (PCRE or BRE/ERE) that finds all lines of which the first word contains the letters to form the word "REGEX". So the first word of the line has to contain at least 1 R, 2 E's, 1 G and 1 X.
So the output of the above would be:
NGNAEAREAX EAHVSELYCI FNWGNLACQM AWKLRMDHIT PRYMFNYMVM
Tomyy
(155 rep)
Oct 21, 2022, 10:47 PM
• Last activity: Nov 4, 2022, 07:37 AM
3
votes
2
answers
1029
views
Rename file with the rename tool - moving around square brackets
Rookie question. Following this answer https://unix.stackexchange.com/questions/444825/move-last-part-of-filename-to-front/444826#444826, I'm trying to do the same, except all files in my case contains square brackets. What I want is to move the title to the other side of the brackets (keeping the f...
Rookie question. Following this answer https://unix.stackexchange.com/questions/444825/move-last-part-of-filename-to-front/444826#444826 , I'm trying to do the same, except all files in my case contains square brackets.
What I want is to move the title to the other side of the brackets (keeping the file extension), so this:
title ![s2_e2].mp4
renames to this: [s2_e2]title !.mp4
The first part may contain exclamation marks and spaces, but no other characters which need to be escaped.
I have come up with this, but it only removes the filename until the first square bracket: rename -n 's/^.*\[//' *
Am I on the right path here? And how can I accomplish it with the perl rename tool on Linux?
Thanks!
Nicolas Ø
(45 rep)
Feb 5, 2022, 02:00 PM
• Last activity: Feb 6, 2022, 04:25 PM
3
votes
5
answers
15658
views
Matching Repeating Pattern Using Regex
Let's say I have a file like following 1,2,3-5,6 1,2,3-5,6, 1 1-3 1,2,3-,4,5-7 1,2,3-,4,5-7, 1,2,-3,4,5 1,2,-,3,4 1,2,,,3,4 ,1,2,3 Only combination of following rules should be considered as valid: 1. Ranges `[0-9]+-[0-9]+` 2. Groups `[0-9]+,[0-9]+` 3. Single Numbers `[0-9]+` The lines could ending...
Let's say I have a file like following
1,2,3-5,6
1,2,3-5,6,
1
1-3
1,2,3-,4,5-7
1,2,3-,4,5-7,
1,2,-3,4,5
1,2,-,3,4
1,2,,,3,4
,1,2,3
Only combination of following rules should be considered as valid:
1. Ranges
[0-9]+-[0-9]+
2. Groups [0-9]+,[0-9]+
3. Single Numbers [0-9]+
The lines could ending with comma should also be considered valid
I want to extract only
1,2,3-5,6
1,2,3-5,6,
1
1-3
As the other lines shown below do not match the rules
1,2,3-,4,5-7
1,2,3-,4,5-7,
1,2,-3,4,5
1,2,-,3,4
1,2,,,3,4
,1,2,3
Because some lines have incomplete ranges, some have missing numbers in groups
----
P.S: A PCRE
compatible grep
only solution would be awesome, but other solutions are also welcome
GypsyCosmonaut
(4289 rep)
Jul 25, 2021, 10:50 AM
• Last activity: Sep 11, 2021, 10:13 PM
3
votes
3
answers
1046
views
Extract text starting at specific category header to next category header from a text file
I have a TOML file in the following format (categories may have any name, the sequential numbering is just an example and not guaranteed): ```toml [CATEGORY_1] A=1 B=2 [CATEGORY_2] C=3 D=4 E=5 ... [CATEGORY_N] Z=26 ``` What I want to achieve is to retrieve the text inside a given category. So, if I...
I have a TOML file in the following format (categories may have any name, the sequential numbering is just an example and not guaranteed):
[CATEGORY_1]
A=1
B=2
[CATEGORY_2]
C=3
D=4
E=5
...
[CATEGORY_N]
Z=26
What I want to achieve is to retrieve the text inside a given category.
So, if I specify, let's say, [CATEGORY_1]
I want it to give me the output:
A=1
B=2
I tried using grep
to achieve this task, with the z
flag, so it could interpret newlines as null-byte characters and using this regular expression:
(^\[.*]) # Match the category
((.*\n*)+? # Match the category content in a non-greedy way
(?=\[|$)) # Lookahead to the start of other category or end of line
It wasn't working unless I removed the ^
at beginning of the expression. However, if I do this, it will misinterpret loose pairs of brackets as a category.
Is there a way to do it correctly? If not with grep
, with other tool, such as sed
or awk
.
Educorreia
(225 rep)
Jul 29, 2021, 10:51 AM
• Last activity: Aug 25, 2021, 06:24 PM
9
votes
1
answers
6720
views
How to refer to matched groups in jq gsub?
Using `jq`, I want to search for a pattern via regex, and wrap the **matched** string with something like ` ` tags ```bash $ echo "\"This is a valid json file"\" | jq '. | gsub("valid";"how_to_refer_to_matches";"i") - ``` How to refer to the matched results in the second argument of `gsub`? What if...
Using
jq
, I want to search for a pattern via regex, and wrap the **matched** string with something like tags
$ echo "\"This is a valid json file"\" | jq '. | gsub("valid";"how_to_refer_to_matches";"i") -
How to refer to the matched results in the second argument of gsub
?
What if there are more than 1 matches?
Zeta.Investigator
(1190 rep)
Jul 9, 2021, 06:46 PM
• Last activity: Jul 9, 2021, 07:50 PM
0
votes
1
answers
44
views
grep nonascii without pcre
I want to grep nonascii characters from a lot of .gz files. However the below does not work on CentOS 7.6 (GNU grep v2.20) $ zcat yyyy/yyyymmdd/filname.yyyymmdd.gz | grep --color='auto' -P -n "[\x80-\xFF]" $ zgrep --color='auto' -P -n "[\x80-\xFF]" yyyy/yyyymmdd/filname.yyyymmdd.gz This perl search...
I want to grep nonascii characters from a lot of .gz files. However the below does not work on CentOS 7.6 (GNU grep v2.20)
$ zcat yyyy/yyyymmdd/filname.yyyymmdd.gz | grep --color='auto' -P -n "[\x80-\xFF]"
$ zgrep --color='auto' -P -n "[\x80-\xFF]" yyyy/yyyymmdd/filname.yyyymmdd.gz
This perl search work
$ zcat yyyy/yyyymmdd/filname.yyyymmdd.gz | perl -ne 'print "$. $_" if m/[\x80-\xFF]/'
But how do I do it on mulitple files like the following?
$ zgrep "[\x80-\xFF]" 2020/2020*/filename.2020*.gz
fivelements
(101 rep)
Sep 13, 2020, 02:28 PM
• Last activity: Sep 13, 2020, 02:45 PM
2
votes
1
answers
234
views
Can I change the regex engine used to search in `less`?
I would like to use a perl compatible regex engine in the `less` command line utility. Is that possible?
I would like to use a perl compatible regex engine in the
less
command line utility. Is that possible?
drevicko
(353 rep)
Aug 31, 2020, 12:18 AM
• Last activity: Aug 31, 2020, 01:44 AM
1
votes
2
answers
661
views
Advanced regex: Can't figure a case where (?R) recursive regex can be used
Is anyone here can explain a case where the recursive Perl/PCRE regex `(?R)` can be helpful ? I read - https://regular-expressions.mobi/recursebacktrack.html?wlr=1 - https://perldoc.perl.org/perlre.html#(%3fPARNO)-(%3f-PARNO)-(%3f%2bPARNO)-(%3fR)-(%3f0) but still can't find a use-case.
Is anyone here can explain a case where the recursive Perl/PCRE regex
(?R)
can be helpful ?
I read
- https://regular-expressions.mobi/recursebacktrack.html?wlr=1
- https://perldoc.perl.org/perlre.html#(%3fPARNO)-(%3f-PARNO)-(%3f%2bPARNO)-(%3fR)-(%3f0)
but still can't find a use-case.
Mévatlavé Kraspek
(541 rep)
Jun 29, 2020, 08:33 PM
• Last activity: Jun 30, 2020, 05:07 AM
3
votes
4
answers
8443
views
Help with this non-capturing group with grep?
I am new to GNU/Linux and regex. Recently I've been playing around trying to get to grips with regex. So far I feel I've got a pretty solid foundational understanding. I'm digging PCRE at the moment. This is the practice text file I'm playing about with: > 01234 567890 > > 01111-222111 > > 09876.543...
I am new to GNU/Linux and regex. Recently I've been playing around trying to get to grips with regex. So far I feel I've got a pretty solid foundational understanding. I'm digging PCRE at the moment.
This is the practice text file I'm playing about with:
> 01234 567890
>
> 01111-222111
>
> 09876.543210
I can successfully match the numbers by doing something like this:
(\d{5})[-.]?\s*?(\d{6})
Now I wanted to create a non-capturing group in order to miss out the first 5 digits and only match the last 6. So I guess I throw in (?:)
to represent the non-capturing group followed by whatever I want to not be captured, right? So that would be
(?:\d{5})[-.]?\s*?(\d{6})
I do that and in my terminal using grep -Po
for PCRE and show output I'm still getting a full match as if the non-capturing group did not apply.
Any guidance?
customcup
(131 rep)
May 12, 2020, 12:37 AM
• Last activity: May 19, 2020, 05:21 AM
0
votes
1
answers
325
views
build apache on linux server
I have downloaded pcre from https://ftp.pcre.org/pub/pcre/pcre2-10.34.tar.gz and extracted it to /usr/local/lib when I run ./configure --with-included-apr --with-pcre=/usr/local/lib/pcre2-10.34 I am getting following error > configure: error: Did not find pcre-config script at > /usr/local/lib/pcre2...
I have downloaded pcre from https://ftp.pcre.org/pub/pcre/pcre2-10.34.tar.gz and extracted it to
/usr/local/lib
when I run
./configure --with-included-apr --with-pcre=/usr/local/lib/pcre2-10.34
I am getting following error
> configure: error: Did not find pcre-config script at
> /usr/local/lib/pcre2-10.34
Daniel
(111 rep)
Mar 18, 2020, 06:30 PM
• Last activity: Mar 21, 2020, 03:32 AM
3
votes
1
answers
1618
views
Postfix PCRE maps broken in RHEL8: "error: unsupported dictionary type: pcre"
My Postfix config worked flawlessly in ***RHEL 7*** and now all maps which rely on PCRE in RHEL are busted after migrating the config to ***RHEL 8***. postfix/cleanup[xxxx]: error: unsupported dictionary type: pcre Is **PCRE** support deprecated in Postfix in RHEL 8?!?!?
My Postfix config worked flawlessly in ***RHEL 7*** and now all maps which rely on PCRE in RHEL are busted after migrating the config to ***RHEL 8***.
postfix/cleanup[xxxx]: error: unsupported dictionary type: pcre
Is **PCRE** support deprecated in Postfix in RHEL 8?!?!?
F1Linux
(2744 rep)
Mar 10, 2020, 11:14 PM
• Last activity: Mar 11, 2020, 07:24 AM
1
votes
3
answers
917
views
How to replace a line in text using RegEX?
The problem I have is that is trying to match both sets of delimiter (above and below) **I'm trying to match only the second part of the delimiter below (bolded).** [![enter image description here][1]][1] **This is so I can add a new version made on the same day to multiple files. Using perl, so I c...
The problem I have is that is trying to match both sets of delimiter (above and below)
**I'm trying to match only the second part of the delimiter below (bolded).**
**This is so I can add a new version made on the same day to multiple files. Using perl, so I can get a result like this when I make the replacement**
**How ever according to https://regex101.com/ (and my experience when I ran the command) it selects both sets of delimiters,**
**making a replacement above and below.**
This is the RegEx I'm using
(?!V[0-9]{2}.[0-9]{2}.[0-9]{4}.1)(.*=.$)
And the comand in UNIX:
perl -pe 's#(?!V[0-9]{2}.[0-9]{2}.[0-9]{4}.1)(.*=.$)#-* V02.11.2020.1 11/Feb/2020 Author2 Minor Changed Include lms \n -* ================ ============= ==================== =========== ========================================================/#g' path/to/file
Is there a way to select the one below? Or the problem originates from the Negative Lookahead?
-**********************************************************************
EDIT
I used the command selected by bey0nd
3,$s/ -\* =[=[:space:]]*\// -* V02.11.2020.1\t 11\/Feb\/2020\t Author2\t\t Minor\tChange include 1ms\n\0/1
It helped a lot with readability
But I'm still getting both delimeters (= signs) repalced. I thought that the lookaround function of regex would've helped
**I'm using perl 5 and sed 4.2**
At least I got it to work in regex101.com, but in my version didn't work
Hope someone finds it useful
(-\* =[=[:space:]]*\/)(?!\n.-\*[[:space:]].V[0-9]{2}.[0-9]{2}.[0-9]{4}.1*)





Javier Vazquez
(11 rep)
Feb 12, 2020, 12:55 AM
• Last activity: Feb 13, 2020, 11:46 AM
1
votes
0
answers
1100
views
Postfix not Modifying header Message-Id
I am trying to modify Postfix Message-Ids and I have tried several things but Postfix just ignores the line in header_checks /Message-Id:\s+ / REPLACE Message-Id: The original header reads: `message-id= ` but I want it to read `message-id= ` My `master.cf` file: cleanup unix n - n - 0 cleanup -o hea...
I am trying to modify Postfix Message-Ids and I have tried several things but Postfix just ignores the line in header_checks
/Message-Id:\s+/ REPLACE Message-Id:
The original header reads:
message-id=
but I want it to read
message-id=
My master.cf
file:
cleanup unix n - n - 0 cleanup
-o header_checks=pcre:/etc/postfix/header_checks
The header_checks
file:
/Message-Id:\s+/ REPLACE Message-Id:
My main.cf
file has
always_add_missing_headers = yes
The logs show emails consistently have postfix generated headers without the rewrite happening.
I am using postfix 3
bacardi_rum2
(111 rep)
Feb 7, 2020, 09:10 PM
• Last activity: Feb 7, 2020, 09:42 PM
2
votes
5
answers
165
views
How can I extract strings from one file to insert (modified) in a different file?
problem is the following: I have an xml file with data and I am looking for a small part of the data to write it into a new file: content has been shortened by request: snippet if type=dhcp-client: ```xml yes Firewall ``` snippet if type=static ```xml Firewall 192.168.0.2 255.255.255.0 192.168.0.1 1...
problem is the following:
I have an xml file with data and I am looking for a small part of the data to write it into a new file:
content has been shortened by request:
snippet if type=dhcp-client:
yes
Firewall
snippet if type=static
Firewall
192.168.0.2
255.255.255.0
192.168.0.1
192.168.0.1
ethernet1/4
0.0.0.0/0
the four relevant values are unique (or nonexistent) within the "system" tag
``
things like ip-address might appear again elsewhere outside of
``
but i am only checking for the ones inside system, if the type is not static dont appear, i set it to dhcp-client
this is what I need as a result in a file if the type is dhcp:
type=dhcp-client
this is what I need as a result in a file if the type was static:
type=static
ip-address=192.168.0.2
default-gateway=192.168.0.1
netmask=255.255.255.0
I am not sure how to accomplish this efficiently and integrated inside an existing **PHP** file (so either work with exec or better yet use php only).
I am also limited to tools that are installed by default on an ubuntu server system and would be unable to use other packages.
PS: this is actually the whole/complete use-case, I will not need to produce other output other than these two examples.
Thanks for any help or pointers :)
Questi
(55 rep)
Jan 14, 2020, 09:16 PM
• Last activity: Jan 21, 2020, 01:47 AM
3
votes
1
answers
5873
views
swig error in ubuntu 16.04
For some reason **swig** on my computer is not configured correctly. It gives this output on checking version: $ swig --version swig: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory whereas on reinstalling it the following output is gener...
For some reason **swig** on my computer is not configured correctly.
It gives this output on checking version:
$ swig --version
swig: error while loading shared libraries: libpcre.so.1: cannot open
shared object file: No such file or directory
whereas on reinstalling it the following output is generated:
piyush@piyush-HP:~/test$ sudo apt-get install swig
[sudo] password for piyush:
Reading package lists... Done
Building dependency tree
Reading state information... Done
swig is already the newest version (3.0.8-0ubuntu3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
any suggestions for how to fix it?
thanks.
Piyush Verma
(31 rep)
Feb 1, 2018, 05:55 PM
• Last activity: Dec 4, 2019, 07:01 PM
Showing page 1 of 20 total questions