Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What is Software? - Columbus Code Camp

What is Software? - Columbus Code Camp

What if DNA unlocked more than just the secrets to life? What if, hidden within our genetic code, we could find the secrets to building truly robust software?

Join Jacob as he shares a fresh perspective on our industry, where it is, and ultimately, where it could go to better support the ever changing needs of today’s competitive business environment. Together, we will take a look at both monolith and SOA architectures, and by taking time-tested patterns from life itself, explore a new way to address the concerns of modern business applications.

Join Jacob in the exploration of the true nature of software, and perhaps, even life itself!

Jacob Mather

October 14, 2017
Tweet

More Decks by Jacob Mather

Other Decks in Programming

Transcript

  1. WHAT IS
    SOFTWARE?

    View full-size slide

  2. JACOB MATHER
    ABOUT ME
    • Software Architect for Lease
    • Spouter of Opposing Ideas
    • Community Evangelist
    • Blog: http://jmather.com
    • Twitter: @thejmather
    • Medium: @jacobmather

    View full-size slide

  3. TODAY IS HUGE FOR ME
    I HAVE FAMILY IN THE AUDIENCE!

    View full-size slide

  4. I’VE BEEN WORKING WITH DISTRIBUTED SYSTEMS A WHILE…
    WHO AM I?
    • Started playing turned based multiplayer games on BBSs ~1988.
    • Started running my own BBSs when I was 9.
    • Built my own home network between two houses when I was 13.
    • Started #networking on Undernet IRC when I was 14.
    • Set up linux servers for my school district when I was 15.
    • Began freelance software development over the web at 16.
    • Started my own hosting and software development businesses in 2004, ran
    until 2014.
    • Started building things running at scale in the Silicon Valley in 2014.

    View full-size slide

  5. I’VE BUILT A LOT OF SYSTEMS FOR A LOT OF COMPANIES…
    WHO AM I?

    View full-size slide

  6. HOW I HAVE DEFINED MYSELF OVER TIME
    • I am a Programmer
    • I am a Developer
    • I am a Web Developer
    • I am a Software Engineer
    • I pretend to be a Software Engineer
    • I pretend to be a Software Engineer for sufficiently narrow definitions of
    software
    • Theoretically I pretend to be a Software Engineer for sufficiently narrow
    definitions of software

    View full-size slide

  7. THE CURRENT
    STATE OF THINGS

    View full-size slide

  8. MONOLITHIC VS SERVICE ORIENTED ARCHITECTURE
    • What makes a monolith a monolith
    • What makes SOA SOA?
    • Why this distinction matters

    View full-size slide

  9. MONOLITH ARCHITECTURE
    VS.
    SERVICE ORIENTED ARCHITECTURE

    View full-size slide

  10. MONOLITHS COMMONLY…
    GENERAL STATEMENTS ABOUT MONOLITHS
    • are self-contained in a single repository.
    • solve the solution end-to-end, providing user interface, data
    backend, and admin system all in one.
    • provide developers an ORM for interfacing with the data store
    backend.
    • house migrations for the data structure and content.
    • assume all data is “local”

    View full-size slide

  11. GENERAL PROBLEMS WITH MONOLITHS
    • Improper abstraction of assumptions leads to very intertwined
    functionality that eventually becomes very resistant to change.
    • The longer code is around, the more it rots, especially when the impact
    of changes becomes difficult to reason about.
    • Too many cooks in the kitchen. Different developers with different
    backgrounds use different patterns for … EVERYTHING, and you’ll have
    to learn them all for everything you touch.
    • Migrations often use application level functions which change and break
    over time, without the migrations being updated or removed.
    • Improperly staged changes can become difficult to release, possibly even
    requiring downtime in order to ensure proper isolation.

    View full-size slide

  12. SOAS COMMONLY…
    GENERAL STATEMENTS ABOUT SOA
    • are spread across multiple repositories
    • allow each service to own it’s backend data store
    • assume some data is “local” and some is “remote”
    • coordinating the network of “valid” services which cooperate
    together becomes a new challenge, especially for improperly
    staged feature releases.

    View full-size slide

  13. GENERAL PROBLEMS WITH SOA
    • The data model of the business becomes distributed amongst
    multiple services and applications.
    • The complexity of managing compatible versions can become
    onerous.
    • It’s easier to accidentally break dependent pieces because without
    extra effort, the impact of changes within SOA are less visible
    than those same changes within a Monolith
    • Without a solid continuous integration implementation, managing
    releases of each independent component becomes much harder.

    View full-size slide

  14. MOST SOA ISN’T SOA
    • What makes a monolith a monolith isn’t that it’s applications and
    services all rolled into one. A monolith is monolithic because it
    encapsulates multiple conceptually distinct layers within a single
    application or service.
    • If your SOA services each own their own data persistence layers, I
    believe what you have a Monolithic Service Oriented Architecture.
    • This just takes all the problems of Monolithic architecture and
    smears it across your entire engineering operation.

    View full-size slide

  15. MONOLITHIC ARCHITECTURE
    • 1 Application
    • 1 Business Domain,
    belonging to the
    Application
    • Business Processes map
    to Features
    • Entire supported
    business domain
    reflected in data model.

    View full-size slide

  16. MONOLITHIC SOA
    • N Applications
    • N Business Domains,
    each bound to their
    respective Applications
    • Otherwise, exactly what
    we have been doing:
    building monoliths.

    View full-size slide

  17. PURE SOA
    • Applications no longer
    own the Business
    Domain
    • The Application is now
    directly associated
    with a purposeful
    documented business
    process
    • Features in the
    application are then
    built around the
    interactions required
    to perform the process

    View full-size slide

  18. SO YOU WANNA FISTICUFFS, EH??
    APPLICATIONS VS ANALYTICS
    • Data for the business becomes spread across multiple applications
    • Business needs to analyze this data for many purposes
    • Capacity planning
    • Sales forecasting
    • Trend detection
    • Hypothesis validation
    • Analytics group is formed to aggregate the data.
    • Analytics is frequently broken by application data updates.

    View full-size slide

  19. HOW NARROW IS MY
    DEFINITION OF
    SOFTWARE?

    View full-size slide

  20. FACTS ABOUT SOFTWARE THAT HAUNT ME
    • For most development methodologies, every 1,000 lines of code, there will be
    at least 1 bug. Most reports tend to say industry average is around 15-30 bugs
    per 1,000 lines of code.
    • I have a suspicion that the vast majority of these bugs occur when we write
    code that transforms data formats. Or do anything related to date math.
    • More code = more bugs.
    • Static analysis can be as introspective as you’re willing to invest.
    • Proper isolation means you can test all scenarios before launching to
    production.
    • Production should only ever go from one known good state to another known
    good state.

    View full-size slide

  21. THE QUEST FOR A DEFINITION OF SOFTWARE
    • Everyone seems happy to write a lot more code than I do.
    • Everyone seems happy to rewrite what I see as core functionality.
    • Everywhere seems to have a difficult time managing the
    complexity that change over time brings.
    • Almost every system ends up with assumptions spread across the
    code, which become more and more complex to change later.
    • Every business eventually seems to equate software releases with
    risking the stability of the production system.

    View full-size slide

  22. BUILDING A DEFINITION OF SOFTWARE
    • Software is comprised primarily of two components:
    • Configuration Data
    • Execution Code
    • A software solution implementation is then composed of:
    • Configuration Data
    • Execution Code
    • Runtime Environment

    View full-size slide

  23. THOUGHTS AROUND VALIDATING SOFTWARE
    • Working in games really reinforced the difference in risk between
    shipping configuration changes vs code changes.
    • Configuration can be validated, generally quickly and efficiently.
    • Code can be validated, but cost of managing that validation goes
    up exponentially with complexity as permutations increase.
    • If configuration is cheaper and easier to validate than code, then
    why do we write so little configuration and so much code?
    • Is there anything out there that we could use for a reference
    implementation, to demonstrate this pattern is robust?

    View full-size slide

  24. MEANWHILE IN THE
    OUTSIDE WORLD…

    View full-size slide

  25. GENETIC MUTATIONS
    • I discovered I have a (fairly
    common) genetic mutation of
    an enzyme configuration
    named MTHFR.
    • The “operation” is a
    “reduction” which is the
    exchange of a hydrogen
    (atomic weight 1) molecule
    for an oxygen (a.w. 8)
    molecule.

    View full-size slide

  26. FROM DIET TO ESSENTIAL NUTRIENTS TO MOLECULES
    SO I APPLIED ENGINEERING TO HEALTH
    • Mapped essential nutrients to their most common food sources.
    • Mapped favorite foods to food sources as ingredients.
    • Followed the metabolic path of how our bodies produce energy.
    • Did 23&Me, downloaded my genome, got third party reports.
    • Came to view biology as a very advanced, robust, and simple
    software system which allows for nearly infinite complex
    expression from simple building blocks.

    View full-size slide

  27. THE HUMAN BODY AS A SOFTWARE SYSTEM
    • One of the most basic biological molecular operations is the
    exchange of 1 for 8. It allows you to trade a negatively charged
    hydrogen molecule for a positively charged oxygen molecule.
    • Is it any surprise, then, that we use bits to signify 1, and bytes to
    signify 8?
    • Perhaps, since we know we project ourselves onto other people
    when we interact with them, just maybe, we project ourselves
    onto our other creative activities. Like, say, designing a computer?

    View full-size slide

  28. DNA
    THE HUMAN BODY AS A SOFTWARE SYSTEM
    • DNA is made up of 4 nucleotides, we all know as T, A, C, and G.
    • These nucleotides are actually pairs, T pairs to A, and C to G.
    • This effectively mimics Raid 1 (mirrored) storage. Instead of 0, 1,
    2, and 3, think of them as 0, 0’, 1, and 1’.
    • DNA is read in codons, which are a set of three nucleotides.
    • One of those codons is stop.
    • DNA is not code it is config, because it is interpreted statically.

    View full-size slide

  29. ENZYMES
    THE HUMAN BODY AS A SOFTWARE SYSTEM
    • Enzymes enable chemical reactions to happen faster than they
    would otherwise happen.
    • Enzymes are built on the fact that nitrogen (and nitrogen like
    molecules, which only have) within a compound creates a small
    “energy imbalance”. They enable a compound to temporarily hold
    on to extra electrons, which provides the “fuel” for the reactions
    the enzymes enable.

    View full-size slide

  30. BIOLOGY AS A PROGRAMMING LANGUAGE
    • DNA is your Genetic Configuration.
    • Enzymes are built-in functions.
    • Mitochondria, and other cellular biological organisms are third
    party libraries.
    • Water is the runtime environment.
    • I can’t help but want to call it bio-assembler.

    View full-size slide

  31. BIOLOGY AS A DISTRIBUTED SYSTEM
    • Organically, our bodies are very much like machines, following
    sets of instructions, operating on local resources.
    • We understand the body as a set of closed systems: nervous
    system, cardiovascular system, lymph system, etc…
    • We can survive for a long time with incorrect inputs because the
    failure of one system doesn’t necessarily mean failure of the entire
    machine.
    • Hormones are long distance messages, the nervous system, blood
    vessels and lymph paths are the network, the brain is the central
    hub, for monitoring, logging and coordination.

    View full-size slide

  32. WHAT BIOLOGY TEACHES US
    • Software is comprised of both configuration and code, executed
    within an environment.
    • Configuration should be large, each environment should maintain it’s
    own full copy, and have validation mechanisms built in.
    • Code should be small, serve a single purpose, and assumed to be
    disposable.
    • The safest systems for handling change over time have strong
    boundaries and clear responsibilities.
    • Complex systems should be made by aggregating the functionality of
    simple systems.

    View full-size slide

  33. WHAT IF WE
    RETHINK OUR JOBS?

    View full-size slide

  34. BUILDING FOR CHANGE OVER TIME
    RETHINKING SOFTWARE ENGINEERING
    • Traditionally, software engineering seems to be viewed as
    “building the thing”, but the only thing we know for sure at the
    beginning of the project is that eventually, our definition of
    “thing” will no longer be congruent with our implementation.
    • Basically, the only thing we know is that eventually we’ll be wrong,
    if we aren’t already operating in some known shade of wrong.
    • If we pull back, and instead look at it as a longer play, it allows us
    to cut our concerns differently, in a way that I believe becomes
    much more robust.

    View full-size slide

  35. WHAT IS AN API?

    View full-size slide

  36. STRUCTURE VS INSTANCE
    WHAT IS AN API?
    • An API is a definition of
    structure. Not an
    implementation.
    • An API may have one or more
    implementations. This may be
    an SDK, or a server exposing
    the API via REST, GraphQL,
    RPC, or similar, over any
    number of protocols (like
    HTTP or HTTPS).

    View full-size slide

  37. WHAT IS AN API?
    • A function name.
    • A defined structure of inputs.
    • A defined structure of outputs.
    • A process which is executed:
    • against an instance of an application.
    • within a context.
    • with defined resources.

    View full-size slide

  38. HOW IS AN API ACCESSED?
    • An API will be implemented via one or more interfaces.
    • REST, JSON-RPC, and GraphQL are examples of API interfaces.
    • An API implementation will then be exposed via a protocol, such
    as HTTPS.

    View full-size slide

  39. WHAT IF WE RETHINK
    OUR REQUIREMENTS?

    View full-size slide

  40. THE PURPOSE OF BUSINESS APPLICATIONS
    • To enable a company to solve
    problems for their customers
    better than the customer is
    able to do with their
    resources alone.
    • We build business
    applications to help the
    business solve problems
    more efficiently.
    • The first problem a business
    encounters is tracking
    customer data.

    View full-size slide

  41. BUSINESS APPLICATION REQUIREMENTS
    • A need to store, modify, and retrieve data.
    • A need to understand this data to grow the business.
    • A need to expose only certain data to certain customers.
    • A need to communicate information to customers.

    View full-size slide

  42. BUSINESS APP APIS AS CAPABILITIES
    • Accept web requests
    • Validate incoming requests
    • Persist and recall data
    • Perform stateful modifications of existing data
    • Notify an action has been performed

    View full-size slide

  43. A Business API broken
    down into pieces

    View full-size slide

  44. How that model is then
    exposed

    View full-size slide

  45. HOW CUSTOM APPLICATIONS ARE BUILT ON TOP

    View full-size slide

  46. DATA
    AS A SERVICE

    View full-size slide

  47. WHY I THINK DATA IS A SOLVED DOMAIN

    View full-size slide

  48. WHY I THINK DATA IS A SOLVED DOMAIN
    • Business objects can be appropriately defined using an extended
    form of JSON Schema.
    • State machine definitions can be written for the business objects.
    • Objects can be collected into repositories which define
    relationships and access patterns.
    • DB Schema can be generated from the repository definitions.
    • Best practices can constrain DB Schema modifications to be safe.
    • APIs can be generated from business objects and state machines.

    View full-size slide

  49. COMPELLING ARGUMENTS THAT DATA IS SOLVED
    • Model schema to table generation is solved.
    • Migration generation for table structure is solved.
    • Exposing an API in a language which has an introspect-able runtime model is solved.
    • Exposing an API via configuration, attached to an introspect-able runtime model has been
    solved.
    • Data modeling is solved VERY WELL by tools in that domain, which can be made to export
    compatible model schema definitions.
    • State machine modeling is better communicated by tools in that domain, which can be made to
    export compatible state machine schema I suspect.
    • Generating the required list of operations for a given data model is solved.
    • Componentizes a clear reusable need.
    • Keeps analytics from dealing with a fragmented business model understanding.

    View full-size slide

  50. THEN IT SHOULD BE INFRASTRUCTURE.
    IF DATA IS SOLVED…
    • Data should be a first class service within your Service Oriented
    Architecture.
    • All applications should store and retrieve data the same way.
    • Data structure is then encapsulated within it’s own layer.
    • All data definitions live together, meaning half of analytics job is
    done.
    • Validation can easily detect when deprecating a field impacts
    analytics reports and block release accordingly.

    View full-size slide

  51. SOFTWARE IS CODE
    & CONFIGURATION

    View full-size slide

  52. MORE DESCRIPTIVE CONFIGURATION
    +
    SMALLER PIECES OF CODE
    =
    NATURALLY RESILIENT SOFTWARE

    View full-size slide

  53. IS YOUR SOA REALLY SOA?
    MAYBE NOT.

    View full-size slide

  54. CONSIDER DATA AS A SERVICE
    I PREDICT IT’LL BE A THING
    FIND OUT MORE AT:
    HTTPS://GITHUB.COM/APITHETIC/APITHY

    View full-size slide

  55. LEAVE COMMENTS!
    HTTP://BIT.LY/CCC2017-WIS

    View full-size slide