Readable Regular Expressions

twitter.com/jeanneboyarsky mastodon.social/@jeanneboyarsky 1
Readable Regular Expressions
Jeanne Boyarsky

Friday, June 23th 2023

KCDC

speakerdeck.com/boyarsky

View Slide

twitter.com/jeanneboyarsky mastodon.social/@jeanneboyarsky
Pause for a Commercial
2
Java certs: 8/11/17

View Slide

Readability
note!

View Slide

Intro

View Slide

Why learning regex matters
5
Anyone see what is wrong?
Match all version 1 TPR reports

View Slide

6
Uh oh :(

View Slide

7
Whew

View Slide

History
8
1943 Patterns in neuroscience
1951 Stephen Keene describing
neural networks
1960’s Pattern matching in text
editors, lexical parsing in
compilers
1980’s PERL
2002 Java 1.4 - regex in core Java

View Slide

Greedy Quantifiers
9
Symbol # j’s?
j 1
j? 0-1
j* 0 or more
j+ 1 or more
j{5} 5
j{5,6} 5-6
j{5,} 5 or more

View Slide

Puzzle time
10
Can you come up with 10 ways of

matching your assigned regex?

• One or more x’s

Note: for this game, you can only have

two x’s in each regex

View Slide

Sample solutions
11
One or more

• x+

• x{1,}

• xx*

• x{1}x*

• x{1,1}x*

• x{1}x{0,}

• x{1,1}x{0,}

• xx{0,}

• x{0,0},x{1,}

• (x|x)+

• etc

View Slide

Common Character Classes
12
Regex Matches
[123] Any of 1, 2 or 3
[1-3] Any of 1, 2 or 3
[^5] Any character but “5”
[a-zA-Z] Letter
\d Digit
\s Whitespace
\w Word character (letter
or digit)

View Slide

Less Common Character Classes
13
Regex Matches Longer form
\D Not digit [^0-9]
\S Not whitespace [^\s]
\W Not word char [^a-zA-Z0-9]
[1-3[x-z]] Union [1-3x-z]
[[m-p]&&[l-n]] Intersection [mn]
[m-p&&[^o]] Subtraction [mnp]
Clarity

Understanding

View Slide

Know your use case
14
Runtime
generated
classes?
[[m-p]&&[l-n]] Rarely
clearer

View Slide

Know your team
15
Team
knows what \D
means?
\D vs [^0-9] Misread
\d vs \\D?

View Slide

Conclusion:
Don’t be clever!

View Slide

Use case
17
Doing
by hand likely
faster….
Do 10 replaces one time

View Slide

Time for Java

View Slide

111-111-1111
222-222-2222
""";
var threeDigits = "[0-9]{3}";
var fourDigits = "[0-9]{4}";
var dash = "-";
var regex = threeDigits + dash
+ threeDigits + dash + fourDigits;
var pattern = Pattern.compile(regex);
Refactored
20

View Slide

111-111-1111
222-222-2222
""";
var threeDigits = “\\d{3}”;
var fourDigits = “\\d{4}”;
var dash = "-";
var regex = threeDigits + dash
Escaping
21

View Slide

111-111-1111
222-222-2222
""";
var areaCodeGroup = "(\\d{3})";
var threeDigits = "\\d{3}";
var fourDigits = "\\d{4}";
var dash = "-";
var regex = areaCodeGroup + dash
System.out.println(matcher.group(1));
Groups
22

View Slide

var numbers = "012";
var regex = "((\\d)(\\d))\\d";
var matcher = pattern.matcher(numbers);
while (matcher.find()) {
System.out.format("%s %s ",
matcher.group(), matcher.group(0));
System.out.format("%s %s ",
matcher.group(1), matcher.group(2));
System.out.format("%s %s",
matcher.group(3), matcher.group(4));
}
What is the output?
23
012 012 01 0

Index out of bounds: no group 4

View Slide

Named capturing groups
24
Group
2 was what
now?
(?)

View Slide

111-111-1111
""";
var areaCodeGroup = "(?\\d{3})";
var threeDigits = "\\d{3}";
var fourDigits = "\\d{4}";
var dash = "-";
var regex = areaCodeGroup + dash
System.out.println(
matcher.group("areaCode"));
Named Capturing Groups
25

View Slide

var string = "Elevation high";
var regex = "[a-zA-Z ]+";
System.out.println(
string.matches(regex));
Exact match
26
That’s
a lot of
ceremony!

View Slide

var before = "123 Sesame Street";
var after = before.replaceAll("\\d", "");
System.out.println(after);
Replace
27
Now
THAT is easy
to read

View Slide

var string = "Mile High City";
string = string.replaceAll("^\\w+", "");
string = string.replaceAll("\\w+$", "");
string = string.strip();
System.out.println(string);
What does this print?
28
High

View Slide

29
With
readability this
time?
var firstWord = "^\\w+";
var lastWord = "\\w+$";
string = string.replaceAll(firstWord, "");
string = string.replaceAll(lastWord, "");

View Slide

var boundaryAndWord = "\\b\\w+";
string = string.replaceAll(
boundaryAndWord, "");
What about now?
30
Blank. Both start of string and spaces are boundaries

View Slide

What does this print?
31
var text = "\\___/";
var regex = "\\_.*/";
System.out.println(text.matches(regex));
false

Need four backslashes in the regex to print true.

View Slide

Beyond the Basics

View Slide

Flags
33
Flag Name Purpose
(?i) CASE_INSENTIVE Case insensitive
ASCII
(?m) MULTILINE ^ and $ match line
breaks
(?s) DOTALL . matches line
break
(?d) UNIX_LINES Only matches \n
(?x) COMMENTS Ignores
whitespace and
# to end of line
+ Unicode ones

View Slide

var html = """

…

Ready!

""";
var body = html.replaceFirst("(?s)^.*", "")
.replaceFirst("(?s).*$", “")
.strip();
System.out.println(body);
Embedding Flag
35
Ready!

View Slide

36
So I
have to say what
I don’t want?
var regex = "(?s).*(.*).*";
var body = html
.replaceFirst(regex, "$1")
.strip();
Ready!

View Slide

37
Huh?
var dotAllMode = "(?s)";
var anyChars = ".*";
var captureAnyChars = "(.*)";
var startBody = "";
var endBody = "";
var bodyPart = startBody
+ captureAnyChars + endBody;
var regex = dotAllMode + anyChars
+ bodyPart + anyChars;
var body = html.replaceFirst(regex, “$1")
.strip();
Ready!

View Slide

Where did the close * go?
38
var text = "* -aa- -b- *";
var pattern =
Pattern.compile("-([a-z]+)-");
var matcher = pattern.matcher(text);
var builder = new StringBuilder();
while(matcher.find())
matcher.appendReplacement(
builder, "x");
System.out.println(builder);
* x x

View Slide

Where did the close * go?
39
var text = "* -aa- -b- *";
var pattern =
builder, "x");
matcher.appendTail(builder);
* x x *

View Slide

What does this do?
40
var text = "* -aa- -b- *";
var pattern =
builder, "$");
IllegalArgumentException: Illegal group reference:
group index is missing

View Slide

Fix
41
var text = "* -aa- -b- *";
var pattern =
var replace = Matcher.quoteReplacement("$");
matcher.appendReplacement(builder, replace);
* $ $

View Slide

Quantifier Types
42
Sample Type Description
z? Greedy Read whole string and backtrack
z?? Reluctant Look at one character at a time
z?+ Possessive Read whole string/never backtrack

View Slide

Comparing
43
var text = "Poem: row row row your boat";
System.out.println(
text.matches(".*(row )+your boat"));
System.out.println(
text.matches(".*?(row )+your boat"));
System.out.println(
text.matches(".*+(row )+your boat"));
true (extra backtracking)

true (faster)

false

View Slide

Lookahead/behind
44
Sample Type
(?=r) Positive lookahead
(?!r) Negative lookahead
(?<=r) Positive lookbehind
(?

View Slide

Looking
45
var text =
“1 fish 2 fish red fish blue fish";
var regex = "\\w+ fish(?! blue)";
1 fish

2 fish

blue fish

View Slide

Indexes
46
var text = "i am sam. I am SAM. Sam i am";
var pattern = Pattern.compile("(?i)sam");
System.out.println(matcher.start()
+ "-" + matcher.end());
5-8

15-18

20-23

View Slide

Debugging Tip
47
Small
regex and build
up
Online
regex checker

View Slide

Sonar 9 Highlights

View Slide

What’s wrong?
49
"[ab]|a"
Redundant. a is a subset of [ab]

View Slide

What’s wrong?
50
changed = changed.
replaceAll("\\.\\.\\.", ";")
Performance. No need for a regex

changed = changed.replace("...", “;");

View Slide

What’s wrong?
51
Pattern regex =
Pattern.compile("myRegex");
Matcher matcher =
regex.matcher("s");
Performance since not static pattern
Readability
tradeoff

View Slide

What’s wrong?
52
<.+?>
Needless backtracking. Instead can write:

<[^>]+>

View Slide

What’s wrong?
53
if (dateString.matches("^(?:(?:31(\\/|-|\\.)(?:0?[13578]|
1[02]))\\1|(?:(?:29|30)(\\/|-|\\.)(?:0?[13-9]|1[0-2])\\2))
(?:(?:1[6-9]|[2-9]\\d)?\\d{2})$|^(?:29(\\/|-|\\.)0?2\\3(?:
(?:(?:1[6-9]|[2-9]\\d)?(?:0[48]|[2468][048]|[13579][26])|
(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\\d|
2[0-8])(\\/|-|\\.)(?:(?:0?[1-9])|(?:1[0-2]))\\4(?:
(?:1[6-9]|[2-9]\\d)?\\d{2})$")) {
handleDate(dateString);
}
Too complicated
I
draw the line way
before this :)

View Slide

What’s wrong?
54
Pattern.compile("$[a-z]+^");
Functionally wrong. Should be:

Pattern.compile("^[a-z]+$");

View Slide

What’s wrong?
55
Pattern.compile("(?=a)b");
If lookahead matches next character = a, it isn’t b

View Slide

What’s wrong?
56
Pattern.compile("[a-zA-Z]");
Assumes only English characters

Pattern.compile("\\p{IsAlphabetic}");
a-z
is clearer if that’s all
you want

View Slide

What’s wrong?
57
String regex = request
.getParameter("regex");
String input = request
.getParameter("input");
return input.matches(regex);
Denial of service opportunity. Need to validate

View Slide

What’s wrong?
58
Pattern.compile("(a|b)*");
Backtracking can overflow stack on large strings. Vs

Pattern.compile("[ab]*");

View Slide

What’s wrong?
59
Pattern.compile("(.|\n)*");
Have dot itself match the line breaks. Better to use:

Pattern.compile("(?s).*");

View Slide

What’s wrong?
60
Pattern.compile("(ab?)*");
Possessive quantifiers disable backtracking

Pattern.compile("(ab?)*+");

View Slide

What’s wrong?
61
str.matches("\\d*?")
? is redundant here and causes backtracking

str.matches("\\d*")

View Slide

What’s wrong?
62
Pattern.compile("a++abc");
Can’t match because ++ is greedy so no “a” left after

Pattern.compile("aa++bc");

View Slide

Beyond English
63
Problem Reason/Fix
"cc̈d̈d".replaceAll("[c̈d̈]
", "X");
Incorrectly assumes Unicode
Graphene Cluster is one code
point. Fix:

"cc̈d̈d".replaceAll("c̈|d̈", "X");
Pattern.compile("söme
pättern",
Pattern.CASE_INSENSITIV
E);
By default, case insensitive is
ASCII only. Fix:

Pattern.compile(“söme pättern",
Pattern.CASE_INSENSITIVE |
Pattern.UNICODE_CASE);
Pattern p =
Pattern.compile("é|ë|
è");
Could be code point or cluster.
Fix:

Pattern p = Pattern.compile("é|
ë|è", Pattern.CANON_EQ);

View Slide

Readable Regular Expressions

Readable Regular Expressions

Jeanne Boyarsky

More Decks by Jeanne Boyarsky

Other Decks in Technology

Featured

Transcript