How to Use Real Computer Science in Your Day Job

How to Use Real Computer Science in Your Day Job, as presented to the Columbus Functional Programming User Group

Transcript

1. How to Use Real Computer Science in Your Day Job Craig Stuntz Improving craig.stuntz@improving.com Abstract— When you leave this user group and return to work, do you expect to apply what you’ve learned here to hard problems, or is there just never time or permission to venture outside of fixing “undefined is not a function" in JavaScript? Many of us do use functional languages, machine learning, proof assistants, parsing, and formal methods in our day jobs, and employment by a CS research department is not a prerequisite. As a consultant who wants to choose the most effective tool for the job and keep my customers happy in the process, I’ve developed a structured approach to finding ways to use the tools of the future (plus a few from the 70s!) in the enterprises of today. I’ll share that with you and examine research into the use of formal This talk combines general techniques for finding freedom to use the best tool for the job with my real-world experience using F#, machine learning, parsing, and constructive logic in an organization with conservative architectural guidelines which forbid most of this explicitly. I’ll expand on this in the presentation, but the elevator pitch is: “Look for unpopular but potentially costly problems; find the looming technical debt that other developers are afraid to touch and use formal methods to break it wide open.” The running example is a smoldering tire fire of 3.5 GB of XML, custom-not-quite-XPath, multiple ad- hoc DSLs, and VB6 services used to configure a single web site in an otherwise successful business. I like to hack on compilers. People often say, “I studied
2. https://speakerdeck.com/craigstuntz http://xkcd.com/1301/ Slides
3. Overview 1. Key ideas 1. Seek out “impossible” / unpopular work 2. Distinguish production code from metacode 3. Don’t wait to be asked to do the work you want to do 2. Legacy code migration example 3. Related work
5. https://www.reddit.com/r/ProgrammerHumor/comments/26rzo3/the_functional_way/ Not a Good Sales Pitch 㾑
6. –Steve Yegge “You're actually surrounded by compilation problems. You run into them almost every day. “ http://steve-yegge.blogspot.ca/2007/06/rich-programmer-food.html
7. He doesn’t need my help. 㾑
9. “What are the toxic, impossible problems which hold you back and everyone else is afraid to touch?” https://www.flickr.com/photos/motleye/306334293/
10. –Frank Wilczek “If you don't make mistakes, you're not working on hard enough problems. And that is a big mistake.”
11. –Abelson / Sussman / Sussman, SICP “...we believe that the essential material to be addressed by a subject at this level is not the syntax of particular programming-language constructs, nor clever algorithms for computing particular functions efficiently, nor even the mathematical analysis of algorithms and the foundations of computing, but rather the techniques used to control the intellectual complexity of large software systems.” https://mitpress.mit.edu/sicp/full-text/book/book-Z-H-7.html#%25_chap_Temp_4
12. Running Example: Legacy Code Migration
13. Underwriting pass through? Underwriter Review Underwriter approved? Write contract Original contract Modification from agent Compare contracts No Yes Yes Modify or reject
14. Ratebook 1 Ratebook 2 Ratebook 3 Underwriting rules 1 Underwriting rules 1
15. Contract difference document 5 GB XML configuration Agent
17. Project scale • ~70000 lines net XML deleted from 300+ files • ~10000 lines C# production code generated in ~100 files • ~15000 lines C# test code generated • “Some” hand-written C# • Affects ~1500 product offerings • ~300 manual test cases generated
18. http://homepages.cwi.nl/~landman/docs/Klint2013-dmr-icsm2013-preprint.pdf
21. “It's easier to ask forgiveness than it is to get permission.” Grace Hopper Chips Ahoy, July, 1986
25. Hand Calculate Generalize Automate
26. http://shop.evilmadscientist.com/productsmenu/605
27. http://shop.evilmadscientist.com/productsmenu/605
29. State Plan ID Michigan HMO 2 Michigan HMO 20 Michigan PPO 35 Michigan HDHP 50 Ohio HMO 1 Ohio HMO 24 Ohio PPO 36 Ohio PPO 90 Ohio HDHP 203 Ohio HDHP 305 Texas HMO 56 Texas PPO 67 Texas HDHP 304 Utah HMO 57 Utah PPO 89 Utah PPO 92 (but hundreds of other products it doesn’t apply to) ⇒ state ∈{ MI, OH, TX, UT } ∧ Plan ∈ { HMO, PPO, HDHP }
30. http://commons.wikimedia.org/wiki/File:CART_tree_titanic_survivors.png
31. For Rule #UD07 OR MI OH TX UT HMO HMO HMO HMO AND AND AND AND
32. (State = Michigan ∧ Plan = HMO) ∨ (State = Ohio ∧ Plan = HMO) ∨ (State = Texas ∧ Plan = HMO) ————————————————————— (Plan = HMO) ∧ (State ∈ { Michigan, Ohio, Texas } )
33. “Sometimes we don’t program to ship; we program to understand programming.” Nada Amin Programming Should Eat Itself
37. Production bugs 0 Code generated bugs 0 Bugs in hand-written C# a few Incorrect TFS work items a few, until I automated that, too QA
39. Code I Had to Write • Domain model recovery • Strong static typing as domain modeling • Wrote two parsers and a lexer • AST optimizer • Machine learning/decision tree • Code generator
40. Business Value • Becomes possible to retire legacy service • Had only model of underwriting rules and could provide data to BAs on what current system actually does • Other developers could analyze XML • Foundation for removing XML altogether => more agility
41. Related Work
42. Maintain Lists of • Stuff I already know • Stuff I want to learn • Hard problems at my employer
43. Code Which Learns About Code • Distinguish production code from… • Code which exists to understand other code • Domain Model Recovery • Compilers • Tests • Proofs • Code Generators
44. Start a Clique • Lunch and learn • MOOC study groups • Find math friends in other departments (actuaries, etc.) • Papers We Love!
47. Case Studies • Use of Formal Methods at Amazon Web Services • Is Proof More Cost Effective Than Testing?
48. Conclusion
49. Craig Stuntz @craigstuntz Craig.Stuntz@improving.com http://www.craigstuntz.com http://www.meetup.com/Papers-We-Love-Columbus/