What is Software? - Columbus Code Camp

What if DNA unlocked more than just the secrets to life? What if, hidden within our genetic code, we could find the secrets to building truly robust software?

Join Jacob as he shares a fresh perspective on our industry, where it is, and ultimately, where it could go to better support the ever changing needs of today’s competitive business environment. Together, we will take a look at both monolith and SOA architectures, and by taking time-tested patterns from life itself, explore a new way to address the concerns of modern business applications.

Join Jacob in the exploration of the true nature of software, and perhaps, even life itself!

Transcript

1. WHAT IS
   SOFTWARE?
   

   View Slide

2. JACOB MATHER
   ABOUT ME
   • Software Architect for Lease
   • Spouter of Opposing Ideas
   • Community Evangelist
   • Blog: http://jmather.com
   • Twitter: @thejmather
   • Medium: @jacobmather
   

   View Slide

3. TODAY IS HUGE FOR ME
   I HAVE FAMILY IN THE AUDIENCE!
   

   View Slide

4. WHO AM I?
   

   View Slide

5. I’VE BEEN WORKING WITH DISTRIBUTED SYSTEMS A WHILE…
   WHO AM I?
   • Started playing turned based multiplayer games on BBSs ~1988.
   • Started running my own BBSs when I was 9.
   • Built my own home network between two houses when I was 13.
   • Started #networking on Undernet IRC when I was 14.
   • Set up linux servers for my school district when I was 15.
   • Began freelance software development over the web at 16.
   • Started my own hosting and software development businesses in 2004, ran
   until 2014.
   • Started building things running at scale in the Silicon Valley in 2014.
   

   View Slide

6. I’VE BUILT A LOT OF SYSTEMS FOR A LOT OF COMPANIES…
   WHO AM I?
   

   View Slide

7. HOW I HAVE DEFINED MYSELF OVER TIME
   • I am a Programmer
   • I am a Developer
   • I am a Web Developer
   • I am a Software Engineer
   • I pretend to be a Software Engineer
   • I pretend to be a Software Engineer for sufficiently narrow definitions of
   software
   • Theoretically I pretend to be a Software Engineer for sufficiently narrow
   definitions of software
   

   View Slide

8. THE CURRENT
   STATE OF THINGS
   

   View Slide

9. MONOLITHIC VS SERVICE ORIENTED ARCHITECTURE
   • What makes a monolith a monolith
   • What makes SOA SOA?
   • Why this distinction matters
   

   View Slide

10. MONOLITH ARCHITECTURE
    VS.
    SERVICE ORIENTED ARCHITECTURE
    

    View Slide

11. MONOLITHS COMMONLY…
    GENERAL STATEMENTS ABOUT MONOLITHS
    • are self-contained in a single repository.
    • solve the solution end-to-end, providing user interface, data
    backend, and admin system all in one.
    • provide developers an ORM for interfacing with the data store
    backend.
    • house migrations for the data structure and content.
    • assume all data is “local”
    

    View Slide

12. GENERAL PROBLEMS WITH MONOLITHS
    • Improper abstraction of assumptions leads to very intertwined
    functionality that eventually becomes very resistant to change.
    • The longer code is around, the more it rots, especially when the impact
    of changes becomes difficult to reason about.
    • Too many cooks in the kitchen. Different developers with different
    backgrounds use different patterns for … EVERYTHING, and you’ll have
    to learn them all for everything you touch.
    • Migrations often use application level functions which change and break
    over time, without the migrations being updated or removed.
    • Improperly staged changes can become difficult to release, possibly even
    requiring downtime in order to ensure proper isolation.
    

    View Slide

13. SOAS COMMONLY…
    GENERAL STATEMENTS ABOUT SOA
    • are spread across multiple repositories
    • allow each service to own it’s backend data store
    • assume some data is “local” and some is “remote”
    • coordinating the network of “valid” services which cooperate
    together becomes a new challenge, especially for improperly
    staged feature releases.
    

    View Slide

14. GENERAL PROBLEMS WITH SOA
    • The data model of the business becomes distributed amongst
    multiple services and applications.
    • The complexity of managing compatible versions can become
    onerous.
    • It’s easier to accidentally break dependent pieces because without
    extra effort, the impact of changes within SOA are less visible
    than those same changes within a Monolith
    • Without a solid continuous integration implementation, managing
    releases of each independent component becomes much harder.
    

    View Slide

15. MOST SOA ISN’T SOA
    • What makes a monolith a monolith isn’t that it’s applications and
    services all rolled into one. A monolith is monolithic because it
    encapsulates multiple conceptually distinct layers within a single
    application or service.
    • If your SOA services each own their own data persistence layers, I
    believe what you have a Monolithic Service Oriented Architecture.
    • This just takes all the problems of Monolithic architecture and
    smears it across your entire engineering operation.
    

    View Slide

16. MONOLITHIC ARCHITECTURE
    • 1 Application
    • 1 Business Domain,
    belonging to the
    Application
    • Business Processes map
    to Features
    • Entire supported
    business domain
    reflected in data model.
    

    View Slide

17. MONOLITHIC SOA
    • N Applications
    • N Business Domains,
    each bound to their
    respective Applications
    • Otherwise, exactly what
    we have been doing:
    building monoliths.
    

    View Slide

18. PURE SOA
    • Applications no longer
    own the Business
    Domain
    • The Application is now
    directly associated
    with a purposeful
    documented business
    process
    • Features in the
    application are then
    built around the
    interactions required
    to perform the process
    

    View Slide

19. SO YOU WANNA FISTICUFFS, EH??
    APPLICATIONS VS ANALYTICS
    • Data for the business becomes spread across multiple applications
    • Business needs to analyze this data for many purposes
    • Capacity planning
    • Sales forecasting
    • Trend detection
    • Hypothesis validation
    • Analytics group is formed to aggregate the data.
    • Analytics is frequently broken by application data updates.
    

    View Slide

20. HOW NARROW IS MY
    DEFINITION OF
    SOFTWARE?
    

    View Slide

21. FACTS ABOUT SOFTWARE THAT HAUNT ME
    • For most development methodologies, every 1,000 lines of code, there will be
    at least 1 bug. Most reports tend to say industry average is around 15-30 bugs
    per 1,000 lines of code.
    • I have a suspicion that the vast majority of these bugs occur when we write
    code that transforms data formats. Or do anything related to date math.
    • More code = more bugs.
    • Static analysis can be as introspective as you’re willing to invest.
    • Proper isolation means you can test all scenarios before launching to
    production.
    • Production should only ever go from one known good state to another known
    good state.
    

    View Slide

22. THE QUEST FOR A DEFINITION OF SOFTWARE
    • Everyone seems happy to write a lot more code than I do.
    • Everyone seems happy to rewrite what I see as core functionality.
    • Everywhere seems to have a difficult time managing the
    complexity that change over time brings.
    • Almost every system ends up with assumptions spread across the
    code, which become more and more complex to change later.
    • Every business eventually seems to equate software releases with
    risking the stability of the production system.
    

    View Slide

23. BUILDING A DEFINITION OF SOFTWARE
    • Software is comprised primarily of two components:
    • Configuration Data
    • Execution Code
    • A software solution implementation is then composed of:
    • Configuration Data
    • Execution Code
    • Runtime Environment
    

    View Slide

24. THOUGHTS AROUND VALIDATING SOFTWARE
    • Working in games really reinforced the difference in risk between
    shipping configuration changes vs code changes.
    • Configuration can be validated, generally quickly and efficiently.
    • Code can be validated, but cost of managing that validation goes
    up exponentially with complexity as permutations increase.
    • If configuration is cheaper and easier to validate than code, then
    why do we write so little configuration and so much code?
    • Is there anything out there that we could use for a reference
    implementation, to demonstrate this pattern is robust?
    

    View Slide

25. MEANWHILE IN THE
    OUTSIDE WORLD…
    

    View Slide

26. GENETIC MUTATIONS
    • I discovered I have a (fairly
    common) genetic mutation of
    an enzyme configuration
    named MTHFR.
    • The “operation” is a
    “reduction” which is the
    exchange of a hydrogen
    (atomic weight 1) molecule
    for an oxygen (a.w. 8)
    molecule.
    

    View Slide

27. FROM DIET TO ESSENTIAL NUTRIENTS TO MOLECULES
    SO I APPLIED ENGINEERING TO HEALTH
    • Mapped essential nutrients to their most common food sources.
    • Mapped favorite foods to food sources as ingredients.
    • Followed the metabolic path of how our bodies produce energy.
    • Did 23&Me, downloaded my genome, got third party reports.
    • Came to view biology as a very advanced, robust, and simple
    software system which allows for nearly infinite complex
    expression from simple building blocks.
    

    View Slide

28. THE HUMAN BODY AS A SOFTWARE SYSTEM
    • One of the most basic biological molecular operations is the
    exchange of 1 for 8. It allows you to trade a negatively charged
    hydrogen molecule for a positively charged oxygen molecule.
    • Is it any surprise, then, that we use bits to signify 1, and bytes to
    signify 8?
    • Perhaps, since we know we project ourselves onto other people
    when we interact with them, just maybe, we project ourselves
    onto our other creative activities. Like, say, designing a computer?
    

    View Slide

29. DNA
    THE HUMAN BODY AS A SOFTWARE SYSTEM
    • DNA is made up of 4 nucleotides, we all know as T, A, C, and G.
    • These nucleotides are actually pairs, T pairs to A, and C to G.
    • This effectively mimics Raid 1 (mirrored) storage. Instead of 0, 1,
    2, and 3, think of them as 0, 0’, 1, and 1’.
    • DNA is read in codons, which are a set of three nucleotides.
    • One of those codons is stop.
    • DNA is not code it is config, because it is interpreted statically.
    

    View Slide

30. ENZYMES
    THE HUMAN BODY AS A SOFTWARE SYSTEM
    • Enzymes enable chemical reactions to happen faster than they
    would otherwise happen.
    • Enzymes are built on the fact that nitrogen (and nitrogen like
    molecules, which only have) within a compound creates a small
    “energy imbalance”. They enable a compound to temporarily hold
    on to extra electrons, which provides the “fuel” for the reactions
    the enzymes enable.
    

    View Slide

31. BIOLOGY AS A PROGRAMMING LANGUAGE
    • DNA is your Genetic Configuration.
    • Enzymes are built-in functions.
    • Mitochondria, and other cellular biological organisms are third
    party libraries.
    • Water is the runtime environment.
    • I can’t help but want to call it bio-assembler.
    

    View Slide

32. BIOLOGY AS A DISTRIBUTED SYSTEM
    • Organically, our bodies are very much like machines, following
    sets of instructions, operating on local resources.
    • We understand the body as a set of closed systems: nervous
    system, cardiovascular system, lymph system, etc…
    • We can survive for a long time with incorrect inputs because the
    failure of one system doesn’t necessarily mean failure of the entire
    machine.
    • Hormones are long distance messages, the nervous system, blood
    vessels and lymph paths are the network, the brain is the central
    hub, for monitoring, logging and coordination.
    

    View Slide

33. WHAT BIOLOGY TEACHES US
    • Software is comprised of both configuration and code, executed
    within an environment.
    • Configuration should be large, each environment should maintain it’s
    own full copy, and have validation mechanisms built in.
    • Code should be small, serve a single purpose, and assumed to be
    disposable.
    • The safest systems for handling change over time have strong
    boundaries and clear responsibilities.
    • Complex systems should be made by aggregating the functionality of
    simple systems.
    

    View Slide

34. WHAT IF WE
    RETHINK OUR JOBS?
    

    View Slide

35. BUILDING FOR CHANGE OVER TIME
    RETHINKING SOFTWARE ENGINEERING
    • Traditionally, software engineering seems to be viewed as
    “building the thing”, but the only thing we know for sure at the
    beginning of the project is that eventually, our definition of
    “thing” will no longer be congruent with our implementation.
    • Basically, the only thing we know is that eventually we’ll be wrong,
    if we aren’t already operating in some known shade of wrong.
    • If we pull back, and instead look at it as a longer play, it allows us
    to cut our concerns differently, in a way that I believe becomes
    much more robust.
    

    View Slide

36. WHAT IS AN API?
    

    View Slide

37. STRUCTURE VS INSTANCE
    WHAT IS AN API?
    • An API is a definition of
    structure. Not an
    implementation.
    • An API may have one or more
    implementations. This may be
    an SDK, or a server exposing
    the API via REST, GraphQL,
    RPC, or similar, over any
    number of protocols (like
    HTTP or HTTPS).
    

    View Slide

38. WHAT IS AN API?
    • A function name.
    • A defined structure of inputs.
    • A defined structure of outputs.
    • A process which is executed:
    • against an instance of an application.
    • within a context.
    • with defined resources.
    

    View Slide

39. HOW IS AN API ACCESSED?
    • An API will be implemented via one or more interfaces.
    • REST, JSON-RPC, and GraphQL are examples of API interfaces.
    • An API implementation will then be exposed via a protocol, such
    as HTTPS.
    

    View Slide

40. WHAT IF WE RETHINK
    OUR REQUIREMENTS?
    

    View Slide

41. THE PURPOSE OF BUSINESS APPLICATIONS
    • To enable a company to solve
    problems for their customers
    better than the customer is
    able to do with their
    resources alone.
    • We build business
    applications to help the
    business solve problems
    more efficiently.
    • The first problem a business
    encounters is tracking
    customer data.
    

    View Slide

42. BUSINESS APPLICATION REQUIREMENTS
    • A need to store, modify, and retrieve data.
    • A need to understand this data to grow the business.
    • A need to expose only certain data to certain customers.
    • A need to communicate information to customers.
    

    View Slide

43. BUSINESS APP APIS AS CAPABILITIES
    • Accept web requests
    • Validate incoming requests
    • Persist and recall data
    • Perform stateful modifications of existing data
    • Notify an action has been performed
    

    View Slide

44. A Business API broken
    down into pieces
    

    View Slide

45. How that model is then
    exposed
    

    View Slide

46. HOW CUSTOM APPLICATIONS ARE BUILT ON TOP
    

    View Slide

47. DATA
    AS A SERVICE
    

    View Slide

48. WHY I THINK DATA IS A SOLVED DOMAIN
    

    View Slide

49. WHY I THINK DATA IS A SOLVED DOMAIN
    • Business objects can be appropriately defined using an extended
    form of JSON Schema.
    • State machine definitions can be written for the business objects.
    • Objects can be collected into repositories which define
    relationships and access patterns.
    • DB Schema can be generated from the repository definitions.
    • Best practices can constrain DB Schema modifications to be safe.
    • APIs can be generated from business objects and state machines.
    

    View Slide

50. COMPELLING ARGUMENTS THAT DATA IS SOLVED
    • Model schema to table generation is solved.
    • Migration generation for table structure is solved.
    • Exposing an API in a language which has an introspect-able runtime model is solved.
    • Exposing an API via configuration, attached to an introspect-able runtime model has been
    solved.
    • Data modeling is solved VERY WELL by tools in that domain, which can be made to export
    compatible model schema definitions.
    • State machine modeling is better communicated by tools in that domain, which can be made to
    export compatible state machine schema I suspect.
    • Generating the required list of operations for a given data model is solved.
    • Componentizes a clear reusable need.
    • Keeps analytics from dealing with a fragmented business model understanding.
    

    View Slide

51. THEN IT SHOULD BE INFRASTRUCTURE.
    IF DATA IS SOLVED…
    • Data should be a first class service within your Service Oriented
    Architecture.
    • All applications should store and retrieve data the same way.
    • Data structure is then encapsulated within it’s own layer.
    • All data definitions live together, meaning half of analytics job is
    done.
    • Validation can easily detect when deprecating a field impacts
    analytics reports and block release accordingly.
    

    View Slide

52. TAKEAWAYS
    

    View Slide

53. SOFTWARE IS CODE
    & CONFIGURATION
    

    View Slide

54. MORE DESCRIPTIVE CONFIGURATION
    +
    SMALLER PIECES OF CODE
    =
    NATURALLY RESILIENT SOFTWARE
    

    View Slide

55. IS YOUR SOA REALLY SOA?
    MAYBE NOT.
    

    View Slide

56. CONSIDER DATA AS A SERVICE
    I PREDICT IT’LL BE A THING
    FIND OUT MORE AT:
    HTTPS://GITHUB.COM/APITHETIC/APITHY
    

    View Slide

57. LEAVE COMMENTS!
    HTTP://BIT.LY/CCC2017-WIS
    

    View Slide