Disorder & Tolerance in Distributed Systems at Scale Rethinking intelligent resilient systems

@helenaedelson Disorder & Tolerance in Distributed Systems at Scale Rethinking
intelligent resilient systems Helena Edelson, Scale By The Bay 2017

@helenaedelson Seen In The Wild Committer/Contributor FiloDB, Akka, Spark Cassandra
Connector, Kafka Connect Cassandra, Spring Integration Helena Edelson twitter.com/helenaedelson Program Committee Member Kafka Summit 2018 Reactive Summit 2016-2017 Speaker Kafka Summit, Spark Summit (EU, NYC), Strata (NYC, SJ), QCon SF, Scala Days (EU, NYC), Reactive Summit (’16, ’17), Philly ETE, Scale by the Bay! linkedin.com/in/helenaedelson

@helenaedelson • Interdisciplinary look at how complex adaptive systems apply
to distributed systems and information engineering • Systems, intelligence and theories • Entropy, Events and Time • Rethinking adaptive systems, complexity and resilience Diﬀerent Approaches

@helenaedelson Inspired By • My scientiﬁc research before working in
tech • What I've noticed in the industry over almost two decades • Questioning how we approach distributed systems, balance and disorder Finding better ways to handle system dynamics • Creating models to predict system dynamics • Re-engineer energy ﬂows in biological systems • Slow the rate of entropy in those systems

@helenaedelson – Albert Einstein “Problems cannot be solved with the
same mind set that created them.”

@helenaedelson Intelligent Systems

@helenaedelson It's All About Information Data: much of what our
systems support and transport

@helenaedelson

@helenaedelson sys·tem • An entity comprised of interdependent elements and
subsystems • More than the sum of its parts • Has feedback loops • Deﬁned by its distinguishing edges In this talk we refer to open systems

@helenaedelson Systems Theory • Discovering how elements of a system
and its subsystems interact to produce given end states • To understand a system's dynamics • Changing one part aﬀects others in the system • Many systems-related theories developed out of this Interdisciplinary study of systems

@helenaedelson Bertalanffy proposed that Systems Theory needed a much broader,
unified approach • Transcending technical problems • Applicable to all scientific study (biology, physics...) General System Theory Was a new paradigm for scientific inquiry

@helenaedelson Complex Adaptive Systems Theory • Used to model an
array diﬀerent systems • Complex, Non-Linear Systems: how order emerges, e.g. in neural networks, galaxies, ecosystems • Self-organization - suggests living systems can migrate to a dynamic state, the ”edge of chaos” - This discipline suggests living systems migrate to a state of dynamic stability they call the "edge of chaos" or balance point. Complexity Theory

@helenaedelson Distributed Systems • With increasing scale comes increased complexity
and potential for disorder • The more moving parts in a system, the more things that can fail • In biological systems, the greater the diversity and/or complexity, the greater the overall resilience The larger the scale, the greater potential to fail

@helenaedelson The Butterly Effect Weather prediction: small causes can have
larger eﬀects

@helenaedelson Ensemble Forecasting Range of possible future states

@helenaedelson Ensemble Forecasting Wildﬁre prediction: a range of possible future
states, diﬀering initial conditions

@helenaedelson Destruction as Transformative Force Laying the foundation for next
state of energy end state = regeneration

@helenaedelson Entropy, Events And Time Order and disorder, time as
events

@helenaedelson Second Law of Thermodynamics • The law from physics
stating that entropy increases • Measures the degree of disorder of a system • The increase in entropy accounts for the irreversibility of natural processes, and the asymmetry between future and past Entropy

@helenaedelson Entropy And The Arrow Of Time "If given complete
knowledge of the universe for two instances of time, how would you solve which instance happened ﬁrst? Order Disorder Time Calculate the entropy of the two snapshots. The one with lower entropy was ﬁrst." - Muller, Richard A, The Physics of Time

@helenaedelson Future Light Cone "If the sun were to cease
to shine at this very moment, it would not aﬀect things on earth at the present time because they would be in the elsewhere of the event when the sun went out." - Stephen Hawking, A Brief History of Time, 1988 Stephen Hawking, A Brief History of Time

@helenaedelson Stephen Hawking, A Brief History of Time • Events
lie in the future light cone everywhere that is not its origin • When we look at the universe we are seeing the past

@helenaedelson Time As Derivative Of Events? Events are sequences of
things happening in time OR Time is a sequence of events

@helenaedelson –Anthony Aguirre “Maybe it’s more accurate to say that
time ﬂows as events happen. The ﬂowing of time or passage of time, is events.”

@helenaedelson Now The sense that time moves forward, in the
continual creation of new nows

@helenaedelson Biological Systems Intelligent, Adaptive, Self-Organizing Systems

@helenaedelson We Are All Hosts Virus as champion of adaptation
and co-evolution

@helenaedelson The Immune System • Exhibits a highly distributed, adaptive
and self-organizing behavior • Is a self-programming system • Inﬁnite ability to re-program itself to destroy threatening microbes • Is a self-learning system • Learns in parallel to ﬁght the many forms of virus

@helenaedelson Complexity & Resiliency From systems theory

@helenaedelson Domino Effect • Change of one can trigger change
in others • Genesis event • As elements of the system are eﬀected, they generate more events • E.g. cascading failure

@helenaedelson Evolution & Complexity At The Edge Thriving complex systems
at transition zones

@helenaedelson Self-Organization • We tend to assume that organization and
order need to be imposed by some external force. • Self-organization is the idea that this type of global organization can instead be the result of local interactions.

@helenaedelson Musk Oxen in the arctic organize to form a
circle around the young Peer to Peer Organization

@helenaedelson Self-Organization: Emergence schooling, swarming, herding

@helenaedelson Emergence Ant colonies are governed by very simple rules,
and only local interactions. Through combined activities, generate colonies that • Exhibit complex structures and behavior • Far exceed intelligence or capability of the individual • Decentralized structure to self-organizing systems • Organization is distributed over the whole system • All parts contribute equally Case Study

@helenaedelson Traditional centralized organization is relatively static model. Self-organization is
dynamic, with autonomous members densely interacting locally. Economies of scale

@helenaedelson Cyclic, Predictable Patterns & Resilience Biological systems have natural
feedback loops and strategies that enable resilience to ﬂuctuation. The Three Rs • Replication • Regeneration • Rebalance

@helenaedelson Self-Organizing Patterns Migration

@helenaedelson Annual Pattern of Movement Arctic Tern • Longest migration
on earth • Pole to pole and back every year

@helenaedelson Daily Pattern of Movement Arctic Wolves • Top of
their food chain • Operate in packs, 30+ • Pack roams its territory daily • Travel 40-100 miles per day • Follows herd food sources annually in their migration

@helenaedelson Predictable patterns in time and space that are changed
and cause change sea·sons

@helenaedelson Planetary Orbit and Axial Tilt Changes cascade to all
elements in all systems

@helenaedelson Resilient Systems & Diversity Variety of entities makes the
systems more eﬀective at absorbing change. and variations in its environment.

@helenaedelson Role Niche • Organisms role in an ecosystem •
The environment of the entity • What it consumes • How it interacts with other elements or entities • Entities role in a system • Data ingestion • Functions in the system • How it interacts with other elements or entities If the number of entities performing a necessary function in a system decrease, the system can fall into imbalance.

@helenaedelson – John Muir “When we try to pick out
anything by itself, we ﬁnd it hitched to everything else in the Universe.”

@helenaedelson Tropic Cascade A process which starts at the top
of the system or meta-system hierarchy, eventually aﬀecting all the way down to the base.

@helenaedelson – Stephen Hawking “It is a matter of common
experience that disorder will tend to increase if things are left to themselves.”

@helenaedelson Tropic Cascade Case Study A complex system in constant
change In 1926 the last wolf in Yellowstone NP in the US was eliminated. By 1994 the elk population grew to roughly 19,000.

@helenaedelson Elimination of the wolves caused a cascade of changes
through the entire ecosystem. With no natural predator, Elk consumed most of their food resources. Tropic Cascade Case Study A complex system in constant change

@helenaedelson Destabilization As elk increased • Berries for bear food
supply decreased • Bear population fell to Endangered Species levels • The coyote population increased to partially ﬁll the niche left by the wolves • Tree and plant hight and numbers decreased dramatically Absence of top predator altered the entire system

@helenaedelson Reintroduction • In 1995 14 grey wolves from Canada
were introduced to Yellowstone, after being absent for over 60 years • A year later 17 wolves were introduced • By December, 2001 their population had grown to 132 Of entities performing the primary regulating role

@helenaedelson Adaptation & Predatory Pressure Predatory pressure keeps prey on
the move so they don't use up resources in one area

@helenaedelson Regeneration Elk started to avoid parts of the park
where they were more exposed for the wolves to hunt. • Forests of aspen and willow began growing back • As bushes and grasses grew back, there were more berries • The diversity and number of birds started increasing

@helenaedelson Repopulation Trees started to grow taller again as the
elk population decreased. • Beaver, previously extinct in the region, returned • The dams beavers built provided habitat for otters and other animals and reptiles • Wolves hunted the coyote, decreasing their population 50% • The numbers of rabbits and mice were able to grow back • Which brought more red foxes, weasels, badgers • The bald eagle and hawk populations grew

@helenaedelson The Bison population began to grow back. Large Mammal
Populations Rebalanced

@helenaedelson Diversity Rebalanced Large mammals can not thrive unless diversity
in their system is also balanced

@helenaedelson Rebalance With the rebalancing of predator / prey, the
populations of many other species were again able to rebalance. • The vegetation along rivers and lakes returned • Erosion decreased • Which changed the shape of the rivers • River banks stabilized, channels narrowed • More pools of water formed • Increasing habitat for water birds and reptiles

@helenaedelson One Role can change the entire topology

@helenaedelson – Stephen Hawking “It is a matter of common
experience that disorder will tend to increase if things are left to themselves.” Self-Balancing Systems

@helenaedelson Innovation assembly line versus research

@helenaedelson Research There was a time when companies weren’t afraid
to invest in basic science. Companies still invest heavily in innovation, but the focus is practical applications rather than basic science. Research and development has become “less R, more D” - Prof. Ashish Arora, economics of technology and technical change

@helenaedelson Rate Of Innovation • Why is information technology seemingly
behind technology in scientiﬁc ﬁelds such as astrophysics, particle physics, molecular biology and behavioral neuroscience? • They have made phenomenal gains but the compute systems that network and manage them, and also capture, process, store and query those system's data has not seen the same speed in innovation.

@helenaedelson Be Experimental Gather real data vs assumption planning without
proof

@helenaedelson – Kip S. Thorne, Nobel Prize in Physics, 2017
“Huge discoveries are really the result of giant collaborations”

@helenaedelson Thank You! @helenaedelson

Disorder & Tolerance in Distributed Systems at ...

Disorder & Tolerance in Distributed Systems at Scale Rethinking intelligent resilient systems

More Decks by Helena Edelson

Other Decks in Technology

Featured

Transcript