[Code Pub] Washing away code smells

Does your code smell? Have a weird fragrance? It turns out code smells are a real thing and an amazing conceptualization of suboptimal design. This talk helps you identify code smells in Python. It also shows you how to wash them away by the technique of refactoring. You will learn the art of writing Pythonic, clean and maintainable code.

Code smells refer to the symptoms of problematic code design. Identifying different types of code smells is the first step to successful refactoring. I will talk through some classic examples:

- Unnecessary long and complex code
- Using mutable data structures
- Uncommunicative naming
- Coupled code

Knowing what to refactor, I will share a few learnings that lead to good quality code:
- The scout's rule: always leave the code cleaner than you found it
- Pythonic data structures: Enum, Namedtuple
- The art of naming
- DRY and the separation of concerns principle

I will also share tips on using refactoring at your company, which includes convincing your product manager, looking out for code smells during code reviews, and employing automatic tools.

Transcript

1. Washing away code smells @yennycheung
2. IMAGE Yenny Cheung Originally from Hong Kong Software Engineer at Yelp On the Biz National team In Hamburg Speaker at EuroPython, PyConDE, PyDays Vienna and Talk Python podcast ABOUT ME @yennycheung
3. IMAGE Connecting people with great local businesses @yennycheung
5. Yelpers have written 155 million reviews since 2004. Some fun facts about Yelp We have over 500 developers. We have over 300 services and our monolith yelp-main has over 3 million lines of code! We have 74 million desktop and 30 million mobile app monthly unique visitors. @yennycheung
6. Agenda for today What are code smells? How to use refactoring to wash away code smells? Tips for bringing refactoring to your company Why do we care? @yennycheung
7. What are code smells? @yennycheung
8. WHAT ARE CODE SMELLS "A code smell is a surface indication that usually corresponds to a deeper problem in the system." - Martin Fowler, author of the book “Refactoring” @yennycheung
9. Why do we care? @yennycheung
10. WHY DO WE CARE ABOUT CODE SMELLS "Let 1,000 flowers bloom. Then rip 999 of them out by the roots." - Peter Seibel, tech lead for Twitter’s Engineering Effectiveness group @yennycheung
11. IMAGE Allows code rot! WHY DO WE CARE ABOUT CODE SMELLS Builds up tech debt Makes it harder to build flexible software Decreases productivity and developer happiness Code smells when left unchecked... @yennycheung
12. Refactoring comes to our rescue @yennycheung
13. WHAT IS REFACTORING Refactoring is a changes the design of your code but not the functionality. @yennycheung
14. def get_cheese (mood, hunger, money): if mood > 3: if money == 0: return None # good mood and hungry if hunger > 4: return 'bleu' # good mood and not hungry else: return 'american' else: if mood > 4: return None if money == 0: return None else: # bad mood and hungry if hunger > 4: return 'brie' # bad mood and not hungry else: return 'mozzarella' if __name__ == "__main__" : cheese = get_cheese( 3, 5, 1) Uncommunicative naming Comments as deodorant Dead code Duplicated code Conditional complexity CODE SMELLS @yennycheung
15. def get_cheese (mood, hunger, money): """Evaluate criteria and pick cheese.""" is_happy = mood > 3 is_hungry = hunger > 4 has_money = money > 0 if not has_money: return None if is_hungry and not is_happy: return 'brie' if not is_hungry and is_happy: return 'american' if not is_hungry and not is_happy: return 'mozzarella' else: return 'bleu' if __name__ == "__main__" : cheese = get_cheese( 3, 5, 1) Better! REFACTORING Guard clauses @yennycheung
16. # Base cheese = get_cheese(3, 5, 1) # Better chosen_cheese = get_cheese(3, 5, 1) # Even better chosen_cheese = get_cheese(mood=3, hunger=5, money=1) # The Best chosen_cheese = get_cheese( mood=3, hunger=5, money=1, ) PEP8 compliant REFACTORING Keyword arguments, getting whitespaces right The styleguide of Python @yennycheung
17. The Zen of Python (PEP20) REFACTORING Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
18. Refactoring deep-dive @yennycheung
19. def get_cheese (mood, hunger, money): if mood > 3: if money == 0: return None # good mood and hungry if hunger > 4: return 'bleu' # good mood and not hungry else: return 'american' else: if mood > 4: return None if money == 0: return None else: # bad mood and hungry if hunger > 4: return 'brie' # bad mood and not hungry else: return 'mozzarella' if __name__ == "__main__" : cheese = get_cheese( 3, 5, 1) Uncommunicative naming → Name it right! Comments as deodorant → Name it right! Dead code → Remove code Duplicated code → DRY Conditional complexity → Decompose conditional into guard clauses REFACTORING TECHNIQUES @yennycheung
20. REFACTORING TECHNIQUES Refactoring Techniques Name it right! Get organized Picking the right data structure @yennycheung
21. A cure for uncommunicative naming REFACTORING TECHNIQUES > NAME IT RIGHT Name it right! Python is dynamically typed Variable, function & module naming Keyword arguments increase clarity Replace magic strings and numbers with Enums! @yennycheung
22. >>> class Mood(Enum): ... EXUBERANT = 0 ... CONTENT = 1 ... APATHETIC = 2 ... MELANCHOLIC = 3 ... >>> for mood in Mood: ... print( mood) ... Mood.EXUBERANT Mood.CONTENT Mood.APATHETIC Mood.MELANCHOLIC >>> print(Mood.EXUBERANT ) Mood.EXUBERANT >>> print(repr( Mood.EXUBERANT )) <Mood.EXUBERANT : 0> >>> my_mood_count_this_week = {} >>> my_mood_count_this_week [Mood.EXUBERANT ] = 3 >>> my_mood_count_this_week [Mood.MELANCHOLIC ] = 1 >>> my_mood_count_this_week [Mood.APATHETIC ] = 3 >>> my_mood_count_this_week {<Mood.APATHETIC : 2>: 3, <Mood.EXUBERANT : 0>: 3, <Mood.MELANCHOLIC : 3>: 1} Enum how to REFACTORING TECHNIQUES > NAME IT RIGHT > ENUM Supports iterable Enum members are hashable Explicit @yennycheung
23. REFACTORING TECHNIQUES > GET ORGANIZED Get organized A cure for long functions, classes and parameter lists Single Responsibility principle Function extraction Decompose conditionals DRY (Don’t repeat yourself) @yennycheung
24. def identify_cheese( country, smell, touch, city, year, taste, ): ... class CheeseProductionInfo(NamedTuple): country: str city: str year: str class CheeseAttributes(NamedTuple): smell: str taste: str touch: str def identify_cheese( cheese_production_info, cheese_attributes, ): ... REFACTORING TECHNIQUES > GET ORGANIZED > FIXING LONG PARAM LISTS Fixing long parameter lists Long param list NamedTuples to the rescue @yennycheung
25. REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE Picking the right data structure Dictionaries NamedTuples Lists Sets @yennycheung
26. REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE Check out the standard library, especially Itertools and Collections for handy tools! @yennycheung
27. Testing in the refactoring process @yennycheung
28. 1. Write integration / end-to-end tests for the code to be refactored Tests that your application will still behave the same 2. Refactoring 3. Write unit tests for refactored code Tests that the code is correct TESTING Testing in the refactoring process @yennycheung
29. Tips for bringing refactoring to your company @yennycheung
30. The secret weapon of code reviews ADVOCATING FOR REFACTORING Encourage refactoring when we add code and fix bugs Scout’s rule: leave it cleaner than you found it @yennycheung
31. How to convince your product manager ADVOCATING FOR REFACTORING If all things fail, abstracting out the implementation detail, adjust estimates to include refactoring and test, “this feature takes X” Break down the tasks and take maintenance into account, with refactoring, 4 weeks, otherwise 6 weeks @yennycheung
32. Automate your refactoring process ADVOCATING FOR REFACTORING Yelp uses a debt tracker: Branch Debt Example metrics: noqa count, deprecated function count, lines added to our monolith yelp-main Yelp’s open source tool: Undebt Based on pyparsing, massive find and replace tool @yennycheung
33. Takeaways from the talk What are code smells? How to use refactoring to wash away code smells? Tips for bringing refactoring to your company Why do we care? @yennycheung
34. www.yelp.com/careers/ Offices @ Hamburg London San Francisco We're Hiring!
36. Thank you! @yennycheung
37. Questions? @yennycheung
38. >>> def sum_cheese( ... cheese_counts={ ... 'bleu':0, ... 'brie':0 ... } ... ): ... cheese_counts['bleu'] += 1 ... >>> sum_cheese.__defaults__ ({'brie': 0, 'bleu': 0},) >>> sum_cheese() >>> sum_cheese.__defaults__ ({'brie': 0, 'bleu': 1},) REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE > DICTIONARIES Picking the right data structure Using dictionaries Beware that dictionaries are mutables! @yennycheung
39. REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE > NAMEDTUPLES Picking the right data structure Using NamedTuples NamedTuples are immutables >>> from typing import NamedTuple >>> class CheeseCounts(NamedTuple): ... bleu: int ... brie: int >>> CheeseCounts. __new__.__defaults__ = (0, 0) >>> print(CheeseCounts( brie=2)) CheeseCounts(bleu=0, brie=2) >>> print(CheeseCounts()) CheeseCounts(bleu=0, brie=0) @yennycheung
40. REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE > LISTS Picking the right data structure Using Lists This is very verbose def select_favorite_cheese_from_catalog ( cheese_catelog , my_favorite_cheese , ): selected_cheese = [] for cheese in cheese_catelog : if cheese in my_favorite_cheese : selected_cheese .append(cheese) return selected_cheese select_favorite_cheese_from_catalog ( cheese_catelog =[ Cheese .BLEU, Cheese .CHEDDAR, ], my_favorite_cheese =[ Cheese .TRUFFLE_BRIE, Cheese.BLEU, ], ) >>> [<Cheese.BLEU: 'Bleu'>] @yennycheung
41. REFACTORING TECHNIQUES > PICKING THE RIGHT DATA STRUCTURE > SETS Picking the right data structure Using Sets Set comparisons are great def select_favorite_cheese_from_catalog ( cheese_catelog , my_favorite_cheese , ): return ( cheese_catelog .intersection(my_favorite_cheese ) ) select_favorite_cheese_from_catalog ( cheese_catelog =set([ Cheese.BLEU, Cheese.CHEDDAR, ]), my_favorite_cheese =set([ Cheese.TRUFFLE_BRIE, Cheese.BLEU, ]), ) >>> {<Cheese.BLEU: 'Bleu'>} @yennycheung