Writing Maintainable Software At Scale

WRITING MAINTAINABLE SOFTWARE
ANDREW GODWIN // @andrewgodwin
AT SCALE

View Slide

Andrew Godwin / @andrewgodwin
Hi, I’m
Andrew Godwin
• Principal Engineer at Astronomer (w/on Airﬂow)
• Django Migrations, Channels & Async
• Doing bad things with Python since 2005

View Slide


View Slide

Why am I still writing Python?
17 years is quite a long time, though not as long as some

View Slide

I mostly work on large new projects
The kind where you only know roughly what you want

View Slide

I rarely know exactly where I'm going
…and so I use Python, as it's really good at that

View Slide

A lot of maintenance advice is
"do it right first time"

View Slide

But we will all get it wrong sometimes
So, how do we build knowing we can make mistakes?

View Slide

Software is almost never complete
It will spend far longer running and being maintained than you spend creating it

View Slide

Nobody knows what they want
Which is also why we're safe from AIs writing code, by the way

View Slide

This is mostly about systems, not libraries
(though some of the lessons are similar!)

View Slide

Writing code Understanding code

View Slide

What makes maintenance hard?
There's some obvious ones, of course…

View Slide

Obfuscation
Or to echo a bad joke, "job security"

View Slide

q=lambda
x,_=('c9*6iv"&s1[Y/`Oh7_|pEW:=!uT4+zeNl;Im\'X\\<-wabDMZ8ykgR@{r>B)~qKFd
C3H0Q%,S}xVG](?^2#oPjJL.A$U fnt5'): type('').__dict__['}VJ|(@J}>'.
translate(('_'*32+_+('__'*len(_))[:(1<<2)-1-(1<<8-2)]))](x,('_'*32+_+('
__'* len(_))[:(1<<2)-1-(1<<8-2)]));globals()[q('11$C3qV}11')] = lambda
_,q=q,__builtins__=__builtins__:
__builtins__.__dict__[q('11$C3qV}11')](q(_));q2=( lambda
globals=(lambda q=(lambda x: getattr(__import__('q('),x)): q): (lambda
os, __import__=q: (globals()(__import__(os)))))();
q=[q2('q3>|')(__import__(
'(P(').__dict__[q('JVR%')][0],0),q];q2('^93o')(3,0);raw_input();input()

View Slide

No documentation
The code! It's self-documenting!

View Slide

One giant file
First one to 100,000 lines wins

View Slide

But what are the root causes?
At least, the three I think are most important

View Slide

What
Is this tech/design common? Can I research elsewhere?
Where
What's the flow of execution? How do I find side effects?
Why
What was the context? The other options?

View Slide

The What

View Slide

Boring technology is good!
I am very proud to call Django "boring" these days

View Slide

Don't overdesign from the start
It is very likely you do not need to work at "Google scale"

View Slide

You probably don't have big data
If it is less than 20TB, it ﬁts on a single machine

View Slide

The Principle Of Least Surprise
Write it the more obvious way, even if it's 1% slower

View Slide

t = sum([p for g, es in gs for e, p in es.items()])
total = 0
for group, entries in gs:
for entry, price in entries.item():
total += price

View Slide

The Where

View Slide

Ever opened 20 files following an
execution path?
Isn't it the worst?

View Slide

It's OK to repeat yourself!
Only factor things out into a library if it makes it easier to read

View Slide

Code should be easy to follow
More people will read it and understand it than will write it

View Slide

def sell_ticket():
if check_availability():
amount = calculate_cost()
auth_code = preauthorize_payment(amount)
allocate_inventory()
send_confirmation()
complete_payment(auth_code)
else:
raise CannotSellError()
def check_availability():
...

View Slide

class TicketSeller(PaymentMixin, EmailMixin, AvailMixin):
def sell(self):
super().sell()
callback = getattr(self, f"start_{self.type}").call()
self.allocate()
callback()

View Slide

def sell_ticket():
if check_availability():
amount = calculate_cost()
auth_code = preauthorize_payment(amount)
allocate_inventory()
send_confirmation()
complete_payment(auth_code)
else:
raise CannotSellError()
def check_availability():
...

View Slide

Never use multiple inheritance
OK, there are uses, but don't do anything that depends on understanding MRO

View Slide

Never use multiple inheritance
OK, there are uses, but don't do anything that depends on understanding MRO
Try not to

View Slide

Microservices are almost never worth it
Unless there's matching ownership… more on that later

View Slide

The Why

View Slide

Reading what code does is mostly easy
Unless it's some of my code from 2005-2006

View Slide

Guessing why it works that way is not
There is rarely just one way to do something

View Slide

# Loop forever
while True:
# Check to see if we should run allocate
if self.allocate_timer.check():
self.allocate()
# Check to see if we should run clean
if self.clean_timer.check():
self.clean()
# Sleep for 0.01 seconds
time.sleep(0.01)

View Slide

# Main loop - only exit when Ctrl-C or SIGTERM is received
while True:
# Run workloads if it's time
if self.allocate_timer.check():
self.allocate()
if self.clean_timer.check():
self.clean()
# Sleep so we don't busy-loop if nothing is ready
time.sleep(0.01)

View Slide

If it took you ages to work out, write why!
You'll help someone else or maybe even your future self

View Slide

Software Engineering is more than coding
It's the skill of making something people want or need

View Slide

Splitting Up Problems
Communication
Documentation
Decision-making

View Slide

Big projects must have multiple teams
Unless you want them to take years to ship

View Slide

“
Conway's Law
Organizations, who design systems,
are constrained to produce designs
which are copies of the
communication structures of these
organizations.

View Slide

It works in reverse, too
Don't try to build a system that goes against your organisational structure.

View Slide

One service/domain per team
… and microservices only work when every service has a deﬁned owner

View Slide

Reduce communication overheads!
Don't make every team talk to every other team every day

View Slide

Consider your incentives
If you only promote for launching new things, you'll never get anything maintained.

View Slide

Iteration speed is king
The cost of improving and ﬁxing matters more than of launching

View Slide

Progressive Typing
Add types as you are more sure of code layouts
Progressive Testing
Tests are really helpful for refactors, but can be fragile early on
Don't Optimise Early
Until you know what the bottlenecks are

View Slide

The most important metric is
"How long from code fix to production?"

View Slide

But Andrew!
I work on planes/reactors/life support!
Well, ﬁrst of all, thank you! That's not easy!
Also… have you considered Rust?

View Slide

You can't always iterate in production
It's a careful balance as to how much you can get wrong

View Slide

All software needs maintenance
If you somehow ﬁnd a perfectly solvable problem, do let me know

View Slide

Simple, understandable designs
It makes it much easier for others to work with
Capture your context and reasons
If the context changes, people know it's safe to change your code!
Design for iteration
It is never, ever done after the initial launch.

View Slide

Thank your maintainers.
Coworkers. Open Source. Real-world infrastructure.

View Slide

Aitäh!
Andrew Godwin
@andrewgodwin // aeracode.org

View Slide

Writing Maintainable Software At Scale

Writing Maintainable Software At Scale

Andrew Godwin

More Decks by Andrew Godwin

Other Decks in Programming

Featured

Transcript