Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Maintainable Software At Scale

Writing Maintainable Software At Scale

My keynote from PyCon Estonia 2022.

Andrew Godwin

August 25, 2022
Tweet

More Decks by Andrew Godwin

Other Decks in Programming

Transcript

  1. WRITING MAINTAINABLE SOFTWARE ANDREW GODWIN // @andrewgodwin AT SCALE

  2. Andrew Godwin / @andrewgodwin Hi, I’m Andrew Godwin • Principal

    Engineer at Astronomer (w/on Airflow) • Django Migrations, Channels & Async • Doing bad things with Python since 2005
  3. Andrew Godwin / @andrewgodwin

  4. Andrew Godwin / @andrewgodwin Why am I still writing Python?

    17 years is quite a long time, though not as long as some
  5. Andrew Godwin / @andrewgodwin I mostly work on large new

    projects The kind where you only know roughly what you want
  6. Andrew Godwin / @andrewgodwin I rarely know exactly where I'm

    going …and so I use Python, as it's really good at that
  7. Andrew Godwin / @andrewgodwin A lot of maintenance advice is

    "do it right first time"
  8. Andrew Godwin / @andrewgodwin But we will all get it

    wrong sometimes So, how do we build knowing we can make mistakes?
  9. Andrew Godwin / @andrewgodwin Software is almost never complete It

    will spend far longer running and being maintained than you spend creating it
  10. Andrew Godwin / @andrewgodwin Nobody knows what they want Which

    is also why we're safe from AIs writing code, by the way
  11. Andrew Godwin / @andrewgodwin This is mostly about systems, not

    libraries (though some of the lessons are similar!)
  12. Andrew Godwin / @andrewgodwin Writing code Understanding code

  13. Andrew Godwin / @andrewgodwin What makes maintenance hard? There's some

    obvious ones, of course…
  14. Andrew Godwin / @andrewgodwin Obfuscation Or to echo a bad

    joke, "job security"
  15. Andrew Godwin / @andrewgodwin q=lambda x,_=('c9*6iv"&s1[Y/`Oh7_|pEW:=!uT4+zeNl;Im\'X\\<[email protected]{r>B)~qKFd C3H0Q%,S}xVG](?^2#oPjJL.A$U fnt5'): type('').__dict__['}VJ|(@J}>'. translate(('_'*32+_+('__'*len(_))[:(1<<2)-1-(1<<8-2)]))](x,('_'*32+_+('

    __'* len(_))[:(1<<2)-1-(1<<8-2)]));globals()[q('11$C3qV}11')] = lambda _,q=q,__builtins__=__builtins__: __builtins__.__dict__[q('11$C3qV}11')](q(_));q2=( lambda globals=(lambda q=(lambda x: getattr(__import__('q('),x)): q): (lambda os, __import__=q: (globals()(__import__(os)))))(); q=[q2('q3>|')(__import__( '(P(').__dict__[q('JVR%')][0],0),q];q2('^93o')(3,0);raw_input();input()
  16. Andrew Godwin / @andrewgodwin No documentation The code! It's self-documenting!

  17. Andrew Godwin / @andrewgodwin One giant file First one to

    100,000 lines wins
  18. Andrew Godwin / @andrewgodwin But what are the root causes?

    At least, the three I think are most important
  19. Andrew Godwin / @andrewgodwin What Is this tech/design common? Can

    I research elsewhere? Where What's the flow of execution? How do I find side effects? Why What was the context? The other options?
  20. Andrew Godwin / @andrewgodwin The What

  21. Andrew Godwin / @andrewgodwin Boring technology is good! I am

    very proud to call Django "boring" these days
  22. Andrew Godwin / @andrewgodwin Don't overdesign from the start It

    is very likely you do not need to work at "Google scale"
  23. Andrew Godwin / @andrewgodwin You probably don't have big data

    If it is less than 20TB, it fits on a single machine
  24. Andrew Godwin / @andrewgodwin The Principle Of Least Surprise Write

    it the more obvious way, even if it's 1% slower
  25. Andrew Godwin / @andrewgodwin t = sum([p for g, es

    in gs for e, p in es.items()]) total = 0 for group, entries in gs: for entry, price in entries.item(): total += price
  26. Andrew Godwin / @andrewgodwin The Where

  27. Andrew Godwin / @andrewgodwin Ever opened 20 files following an

    execution path? Isn't it the worst?
  28. Andrew Godwin / @andrewgodwin It's OK to repeat yourself! Only

    factor things out into a library if it makes it easier to read
  29. Andrew Godwin / @andrewgodwin Code should be easy to follow

    More people will read it and understand it than will write it
  30. Andrew Godwin / @andrewgodwin def sell_ticket(): if check_availability(): amount =

    calculate_cost() auth_code = preauthorize_payment(amount) allocate_inventory() send_confirmation() complete_payment(auth_code) else: raise CannotSellError() def check_availability(): ...
  31. Andrew Godwin / @andrewgodwin class TicketSeller(PaymentMixin, EmailMixin, AvailMixin): def sell(self):

    super().sell() callback = getattr(self, f"start_{self.type}").call() self.allocate() callback()
  32. Andrew Godwin / @andrewgodwin def sell_ticket(): if check_availability(): amount =

    calculate_cost() auth_code = preauthorize_payment(amount) allocate_inventory() send_confirmation() complete_payment(auth_code) else: raise CannotSellError() def check_availability(): ...
  33. Andrew Godwin / @andrewgodwin Never use multiple inheritance OK, there

    are uses, but don't do anything that depends on understanding MRO
  34. Andrew Godwin / @andrewgodwin Never use multiple inheritance OK, there

    are uses, but don't do anything that depends on understanding MRO Try not to
  35. Andrew Godwin / @andrewgodwin Microservices are almost never worth it

    Unless there's matching ownership… more on that later
  36. Andrew Godwin / @andrewgodwin The Why

  37. Andrew Godwin / @andrewgodwin Reading what code does is mostly

    easy Unless it's some of my code from 2005-2006
  38. Andrew Godwin / @andrewgodwin Guessing why it works that way

    is not There is rarely just one way to do something
  39. Andrew Godwin / @andrewgodwin # Loop forever while True: #

    Check to see if we should run allocate if self.allocate_timer.check(): self.allocate() # Check to see if we should run clean if self.clean_timer.check(): self.clean() # Sleep for 0.01 seconds time.sleep(0.01)
  40. Andrew Godwin / @andrewgodwin # Main loop - only exit

    when Ctrl-C or SIGTERM is received while True: # Run workloads if it's time if self.allocate_timer.check(): self.allocate() if self.clean_timer.check(): self.clean() # Sleep so we don't busy-loop if nothing is ready time.sleep(0.01)
  41. Andrew Godwin / @andrewgodwin If it took you ages to

    work out, write why! You'll help someone else or maybe even your future self
  42. Andrew Godwin / @andrewgodwin Software Engineering is more than coding

    It's the skill of making something people want or need
  43. Andrew Godwin / @andrewgodwin Splitting Up Problems Communication Documentation Decision-making

  44. Andrew Godwin / @andrewgodwin Big projects must have multiple teams

    Unless you want them to take years to ship
  45. “ Andrew Godwin / @andrewgodwin Conway's Law Organizations, who design

    systems, are constrained to produce designs which are copies of the communication structures of these organizations.
  46. Andrew Godwin / @andrewgodwin It works in reverse, too Don't

    try to build a system that goes against your organisational structure.
  47. Andrew Godwin / @andrewgodwin One service/domain per team … and

    microservices only work when every service has a defined owner
  48. Andrew Godwin / @andrewgodwin Reduce communication overheads! Don't make every

    team talk to every other team every day
  49. Andrew Godwin / @andrewgodwin Consider your incentives If you only

    promote for launching new things, you'll never get anything maintained.
  50. Andrew Godwin / @andrewgodwin Iteration speed is king The cost

    of improving and fixing matters more than of launching
  51. Andrew Godwin / @andrewgodwin Progressive Typing Add types as you

    are more sure of code layouts Progressive Testing Tests are really helpful for refactors, but can be fragile early on Don't Optimise Early Until you know what the bottlenecks are
  52. Andrew Godwin / @andrewgodwin The most important metric is "How

    long from code fix to production?"
  53. Andrew Godwin / @andrewgodwin But Andrew! I work on planes/reactors/life

    support! Well, first of all, thank you! That's not easy! Also… have you considered Rust?
  54. Andrew Godwin / @andrewgodwin You can't always iterate in production

    It's a careful balance as to how much you can get wrong
  55. Andrew Godwin / @andrewgodwin All software needs maintenance If you

    somehow find a perfectly solvable problem, do let me know
  56. Andrew Godwin / @andrewgodwin Simple, understandable designs It makes it

    much easier for others to work with Capture your context and reasons If the context changes, people know it's safe to change your code! Design for iteration It is never, ever done after the initial launch.
  57. Andrew Godwin / @andrewgodwin Thank your maintainers. Coworkers. Open Source.

    Real-world infrastructure.
  58. Aitäh! Andrew Godwin @andrewgodwin // aeracode.org