Perl regular expressions and text processing
Capture named groups, collect all matches, transliterate characters, and compute replacements with /e.
On this page
Perl scalars and strings.
- Capture named groups, collect all matches, transliterate characters, and compute replacements with /e.
- Predict the sample output and explain the state changes.
- Complete the practice task and check the boundary cases.
The concept
Regular expressions are built into Perl. The match operator =~ /pattern/ tests a string, and parentheses capture groups. Named groups, written (?<name>...), are stored in the %+ hash, which makes patterns self-documenting. In list context with the /g flag a match returns every captured piece at once.
The substitution s/pattern/replacement/ changes a string. With /g it replaces all matches, and with /e the replacement is Perl code, so it can compute new text. \U ... \E upper-cases part of the replacement. tr/// transliterates characters one for one, and in scalar context it counts how many characters matched. Copying with (my $copy = $text) =~ s/// changes the copy and leaves the original alone.
Code example
use strict;
use warnings;
my $line = "2024-03-15 ERROR disk full on /dev/sda1";
if ($line =~ /^(?<date>\d{4}-\d{2}-\d{2})\s+(?<level>[A-Z]+)\s+(?<message>.*)$/) {
print "$+{date} | $+{level} | $+{message}\n";
}
my $text = "cat bat rat hat";
my @letters = $text =~ /\b(\w)at\b/g;
print scalar(@letters), " matches: @letters\n";
(my $copy = $text) =~ s/(\w)at/\U$1\Eat/g;
print "$copy\n";
(my $dna = "AACGT") =~ tr/ACGT/TGCA/;
my $count = ($text =~ tr/a//);
print "$dna $count\n";
my $prices = "apple=3, pear=12, fig=7";
(my $doubled = $prices) =~ s/(\d+)/$1 * 2/ge;
print "$doubled\n";
print join("|", split /\s*,\s*/, "a , b,c ,d"), "\n";2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat TTGCA 4 apple=6, pear=24, fig=14 a|b|c|d
Follow the code
Open the walkthrough to inspect the example one step at a time.
Show guided walkthrough
use strict;use warnings; my $line = "2024-03-15 ERROR disk full on /dev/sda1";if ($line =~ /^(?<date>\d{4}-\d{2}-\d{2})\s+(?<level>[A-Z]+)\s+(?<message>.*)$/) { print "$+{date} | $+{level} | $+{message}\n";} my $text = "cat bat rat hat";my @letters = $text =~ /\b(\w)at\b/g;print scalar(@letters), " matches: @letters\n"; (my $copy = $text) =~ s/(\w)at/\U$1\Eat/g;print "$copy\n"; (my $dna = "AACGT") =~ tr/ACGT/TGCA/;my $count = ($text =~ tr/a//);print "$dna $count\n"; my $prices = "apple=3, pear=12, fig=7";(my $doubled = $prices) =~ s/(\d+)/$1 * 2/ge;print "$doubled\n"; print join("|", split /\s*,\s*/, "a , b,c ,d"), "\n";Follow the example
Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.
View the complete dry-run table
| Step / line | What happens | State after step | Output so far |
|---|---|---|---|
| Named captures are read from the %+ hash. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 | |
| In list context with /g, every capture of every match is returned: the first letter of each three-letter word ending in at. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 | |
| Four words matched. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h | |
| Substitute on a copy: \U upper-cases the captured letter. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h | |
| Each word now starts with a capital letter. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat | |
| Transliterate each base to its complement. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat | |
| The complement strand, and the number of letter a characters in $text. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat TTGCA 4 | |
| With /e the replacement is evaluated as Perl code, so every number is doubled. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat TTGCA 4 | |
| The doubled prices. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat TTGCA 4 apple=6, pear=24, fig=14 | |
| split with a regex removes the spaces around the commas. | — | 2024-03-15 | ERROR | disk full on /dev/sda1 4 matches: c b r h Cat Bat Rat Hat TTGCA 4 apple=6, pear=24, fig=14 a|b|c|d |
Behind the scenes
How Perl processes this code
Perl compiles the script internally before executing it. strict and warnings help identify mistakes.
Cost and common mistakes
Matching is linear in the length of the string for these patterns.
- A match in list context returns the captured groups, but in scalar context it returns true or false. The context decides, so check what you assign to.
- Without /g a regex matches only once. Without /e the replacement is plain text, so $1 * 2 would be inserted literally.
- Greedy patterns such as .* take as much as they can. Use .*? when you want the shortest match.
Try it yourself
What does "hello" =~ tr/a-z// return?
Reveal answer and reasoning
5, because tr with an empty replacement counts the characters that match the range and leaves the string unchanged.