Regexp-based Refactoring

Transcript

1. Regexp-based Refactoring VilniusRB meetup 2015-10-08
2. Remigijus Jodelis @remigijusj
3. Part 0. Intro to regexps
4. regular expressions special language for search patterns
5. regexp examples /.*\.txt$/ => "hello.txt", ".txt" /#[0-9A-F]{6}/ => "color: #AA88E7;" /\d{4}-\d{2}-\d{2}/ => "today is 2015-10-08"
6. regexp operations string[/answer: (yes|no|(maybe))/, 2] # nil, "maybe" string.gsub(/<.*?>/, "\n") # "This:<br/><p></p>" => "This:\n\n\n"
7. regexp operations url = "http://[a-zA-Z0-9/.?=+]*" string.scan(/<a href="(#{url})"/) # [["http://..."], ...]
8. QUESTION what does it match? /^1?$|^(11+?)\1+$/
9. /^1?$|^(11+?)\1+$/ "", "1", "1111", "111111", "11111111", "111111111", … all the strings of 1 except those whose length is PRIME NUMBER ANSWER
10. Part 1. Challenge
11. let me tell you a story
15. stats / sloc ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- PHP (not really..) 14 7130 573 21648 CSS 1 625 40 4835 Javascript 1 1 0 12 ------------------------------------------------------------------------------- Total size: 1.3 MB
17. worst kind of duplication: lots of text with small anomalies everywhere
18. problems difficult to find & fix layout errors hard to make further modifications
19. challenge accepted
20. Part 2. Progress
22. well... GIT, obviously ! > cd project > git init > git commit -am "status quo"
23. next: clean-up white-space and style
24. keeping strict style is important makes patterns simpler
25. need an idea how to start for real
26. remember this?
28. identifiers come in different contexts <div id="Ellipse2copy6"> <img src="images/kubas_2X2/images/Ellipse2copy7.png" $("#Layer10_ellipse2_mini").hide(); #Layer50_ellipse2_mini_m6 { position: absolute;
29. useful tool: extract.rb rx = Regexp.new(pattern) freq = Hash.new(0) File.read(path).scan(rx) do |it| freq[it] += 1 end freq.keys.sort.each do |word| puts "%s -> %d" % [word, freq[word]] end
30. regexps: here they come /\bid="(.*?)"/ /#(\w+)\s*\{/ /\/(\w+)\.png/
31. basic operations • edit: manually • grep: search project files (list of lines) • count: search, but only check the number • replace: pattern to replacement • interactive replace • test: manually and/or automatically
32. regexps: real life ^ {6}<label id="yes(\d+)" class="slot circle-(\d+)">\1\r\n \s {8}<input type="radio" name="siena\1([abcd][123456])" id=" siena\1\3" value="s\1" />\r\n {6}</label>\r\n <label id="yes3" class="slot circle-2">3 <input type="radio" name="siena3b4" id="siena3b4" value="s3" /> </label>
33. how do i match this <label id="yes3"> <input type="radio" name="siena3b4" id="siena3b4" value="s3" /> <div class="circle-off" id="Ellipse2_back"><img src=" images/circle_grey.png" width="131" height="57" /></div> <div class="circle-sel" id="Ellipse2_visited_back"><img src=" images/circle_green.png" width="131" height="57" /></div> <div class="circle-num" id="_3">3</div> </label>
34. most complicated regexp ^ {6}<label id="yes(\d+)">\r\n {8}<input type="radio" name=" siena\1([abcd][123456])" id="siena\1\2" value="s\1" />\r\n {8} <div class="circle-off" id="(Ellipse\d+)(_back)?"><img src=" images/circle_grey.png" width="(\d+)" height="(\d+)" /></div>\r\n {8}<div class="circle-sel" id="\3_visited\4?"><img src="images/circle_green.png" width="\5" height="\6" /></div>\r\n {8}<div class="circle-num" id="_\1">\1</div>\r\n {6}</label>\r\n
36. points to consider match and count patterns -- before actually updating decide for each step -- edit manually or replace pattern?
37. points to consider systematically increase order, remove anomalies maintain a todo list
38. other usual refactoring steps • splitting modules • renaming functions • moving code around • etc
39. how does code size decrease?
40. 2 weeks and 80 commits later … 1.3 MB got reduced to 270 kB
41. SUCCESS!
42. Part 3. Summing-up
43. there are ways to deal with copy-pasted code but it’s not trivial
44. is it worth it?
45. lesson learned bulk-updating code is not (that) scary
46. lesson learned regexps are extremely useful, sometimes
47. so, next time?
49. just kidding ☺
50. Thank You!