Slide 1

Slide 1 text

MODELLING INFRASTRUCTURE WITH INVENTORY DATA HOW DOES ANSIBLE MODEL ITS INVENTORY AND HOW CAN WE MODEL OUR INFRASTRUCTURE?

Slide 2

Slide 2 text

$ WHOAMI {'name': 'Serge van Ginderachter'} {'url': 'http://ginsys.eu'} {'email': '[email protected]'} {'IM': '[email protected]'} {'twitter': '@svg'} {'IRC': 'svg'} {'Github': 'sergevanginderachter'}

Slide 3

Slide 3 text

$WORK INFRASTRUCTURE "CONSULTANT" Serge van Ginderachter Started in M$ shops in Small Business Environments ( at Belgian scale: 5-150 employees ) Followed the Jedi path and turned to the bright side of life after 2005 Ansible since +/- 18 months No experience with other cfgmgmt tools First upstream commit c o m m i t d a 9 2 c e 7 9 6 b 4 8 e c 8 0 e 3 e a d 1 c f e 9 b c b c 7 1 f 5 f c e 8 0 5 A u t h o r S e r g e v a n G i n d e r a c h t e r D a t e W e d O c t 1 0 1 9 : 3 8 : 3 0 2 0 1 2 + 0 2 0 0 f i x m i s s i n g - - l i m i t i n d o c s s i t e e x a m p l e s

Slide 4

Slide 4 text

CURRENT MAJOR PROJECT Flemish Government, one of the complex parts of Belgium http://www.milieuinfo.be: about environmental things Open Source island within the Flemish Government Enterprise upcoming stuff: "cloud" built on Ceph storage and Cloudstack Ansible since Summer 2012

Slide 5

Slide 5 text

ANSIBLE INVENTORY How can this thing be implemented? ~ / s r c / a n s i b l e / l i b / a n s i b l e $ l s i n v e n t o r y / d i r . p y e x p a n d _ h o s t s . p y g r o u p . p y h o s t . p y i n i . p y _ _ i n i t _ _ . p y s c r i p t . p y v a r s _ p l u g i n s

Slide 6

Slide 6 text

ANSIBLE.INVENTORY 9 6 e l i f o s . p a t h . e x i s t s ( h o s t _ l i s t ) : 9 7 i f o s . p a t h . i s d i r ( h o s t _ l i s t ) : 9 8 # E n s u r e b a s e d i r i s i n s i d e t h e d i r e c t o r y 9 9 s e l f . h o s t _ l i s t = o s . p a t h . j o i n ( s e l f . h o s t _ l i s t , " " ) 1 0 0 s e l f . p a r s e r = I n v e n t o r y D i r e c t o r y ( f i l e n a m e = h o s t _ l i s t ) 1 0 1 s e l f . g r o u p s = s e l f . p a r s e r . g r o u p s . v a l u e s ( ) 1 0 2 e l i f u t i l s . i s _ e x e c u t a b l e ( h o s t _ l i s t ) : 1 0 3 s e l f . p a r s e r = I n v e n t o r y S c r i p t ( f i l e n a m e = h o s t _ l i s t ) 1 0 4 s e l f . g r o u p s = s e l f . p a r s e r . g r o u p s . v a l u e s ( ) 1 0 5 e l s e : 1 0 6 s e l f . p a r s e r = I n v e n t o r y P a r s e r ( f i l e n a m e = h o s t _ l i s t ) 1 0 7 s e l f . g r o u p s = s e l f . p a r s e r . g r o u p s . v a l u e s ( ) 1 0 8 1 0 9 u t i l s . p l u g i n s . v a r s _ l o a d e r . a d d _ d i r e c t o r y ( s e l f . b a s e d i r ( ) , w i t h _ s u b d i r = T r u e )

Slide 7

Slide 7 text

ANSIBLE.INVENTORY.DIR 5 8 i f o s . p a t h . i s d i r ( f u l l p a t h ) : 5 9 p a r s e r = I n v e n t o r y D i r e c t o r y ( f i l e n a m e = f u l l p a t h ) 6 0 e l i f u t i l s . i s _ e x e c u t a b l e ( f u l l p a t h ) : 6 1 p a r s e r = I n v e n t o r y S c r i p t ( f i l e n a m e = f u l l p a t h ) 6 2 e l s e : 6 3 p a r s e r = I n v e n t o r y P a r s e r ( f i l e n a m e = f u l l p a t h ) 6 4 s e l f . p a r s e r s . a p p e n d ( p a r s e r )

Slide 8

Slide 8 text

~/SRC/ANSIBLE/EXAMPLES/HOSTS # T h i s i s t h e d e f a u l t a n s i b l e ' h o s t s ' f i l e . # U n g r o u p e d h o s t s , s p e c i f y b e f o r e a n y g r o u p h e a d e r s . g r e e n . e x a m p l e . c o m b l u e . e x a m p l e . c o m 1 9 2 . 1 6 8 . 1 0 0 . 1 1 9 2 . 1 6 8 . 1 0 0 . 1 0 # A c o l l e c t i o n o f h o s t s b e l o n g i n g t o t h e ' w e b s e r v e r s ' g r o u p [ w e b s e r v e r s ] a l p h a . e x a m p l e . o r g b e t a . e x a m p l e . o r g 1 9 2 . 1 6 8 . 1 . 1 0 0 1 9 2 . 1 6 8 . 1 . 1 1 0 # A c o l l e c t i o n o f d a t a b a s e s e r v e r s i n t h e ' d b s e r v e r s ' g r o u p [ d b s e r v e r s ] d b 0 1 . i n t r a n e t . m y d o m a i n . n e t d b 0 2 . i n t r a n e t . m y d o m a i n . n e t 1 0 . 2 5 . 1 . 5 6 1 0 . 2 5 . 1 . 5 7

Slide 9

Slide 9 text

DYNAMIC INVENTORY SCRIPTS { " d a t a b a s e s " : { " h o s t s " : [ " h o s t 1 . e x a m p l e . c o m " , " h o s t 2 . e x a m p l e . c o m " ] , " v a r s " : { " a " : t r u e } } , " w e b s e r v e r s " : [ " h o s t 2 . e x a m p l e . c o m " , " h o s t 3 . e x a m p l e . c o m " ] , " a t l a n t a " : { " h o s t s " : [ " h o s t 1 . e x a m p l e . c o m " , " h o s t 4 . e x a m p l e . c o m " , " h o s t 5 . e x a m p l e . c o m " ] , " v a r s " : { " b " : f a l s e } , " c h i l d r e n " : [ " m a r i e t t a " , " 5 p o i n t s " ] , } , " m a r i e t t a " : [ " h o s t 6 . e x a m p l e . c o m " ] , " 5 p o i n t s " : [ " h o s t 7 . e x a m p l e . c o m " ] " _ m e t a " : { " h o s t v a r s " : { " m o o c o w . e x a m p l e . c o m " : { " a s d f " : 1 2 3 4 } , " l l a m a . e x a m p l e . c o m " : { " a s d f " : 5 6 7 8 } , } } }

Slide 10

Slide 10 text

THE ANSIBLE INVENTORY IS NOT A TREE! g o o g l e g o o g l e / g c a l e n d a r g o o g l e / g c a l e n d a r / b a c k e n d g o o g l e / g c a l e n d a r / b a c k e n d / s t o r a g e 1 g o o g l e / g c a l e n d a r / b a c k e n d / s t o r a g e 2 g o o g l e / g c a l e n d a r / b a c k e n d / s t o r a g e 3 g o o g l e / g c a l e n d a r / f r o n t e n d g o o g l e / g c a l e n d a r / f r o n t e n d / w e b 1 g o o g l e / g c a l e n d a r / f r o n t e n d / w e b 2 g o o g l e / g c a l e n d a r / f r o n t e n d / w e b 3 g o o g l e / g d r i v e g o o g l e / g d r i v e / b a c k e n d g o o g l e / g d r i v e / b a c k e n d / s t o r a g e 1 g o o g l e / g d r i v e / b a c k e n d / s t o r a g e 2 g o o g l e / g d r i v e / b a c k e n d / s t o r a g e 3 g o o g l e / g d r i v e / f r o n t e n d g o o g l e / g d r i v e / f r o n t e n d / w e b 1 g o o g l e / g d r i v e / f r o n t e n d / w e b 2 g o o g l e / g d r i v e / f r o n t e n d / w e b 3 g o o g l e / g m a i l g o o g l e / g m a i l / b a c k e n d g o o g l e / g m a i l / b a c k e n d / s t o r a g e 1 g o o g l e / g m a i l / b a c k e n d / s t o r a g e 2 g o o g l e / g m a i l / b a c k e n d / s t o r a g e 3 g o o g l e / g m a i l / f r o n t e n d g o o g l e / g m a i l / f r o n t e n d / w e b 1 g o o g l e / g m a i l / f r o n t e n d / w e b 2 g o o g l e / g m a i l / f r o n t e n d / w e b 3

Slide 11

Slide 11 text

NODES CAN LIVE IN DIFFERENT GROUPS ON DIFFERENT LEVELS g o o g l e / n g i n x g o o g l e / n g i n x / w e b 1 g o o g l e / n g i n x / w e b 2 g o o g l e / n g i n x / w e b 3 g o o g l e / t o m c a t g o o g l e / t o m c a t / s t o r a g e 1 g o o g l e / t o m c a t / s t o r a g e 2 g o o g l e / t o m c a t / s t o r a g e 3

Slide 12

Slide 12 text

INTERNAL ANSIBLE INVENTORY MODEL {'web1': {'groups': ['all', 'google', 'gcalendar', 'gdrive', 'gmail', 'frontend']}} {'storage2': {'groups': ['all', 'google', 'gcalendar', 'gdrive', 'gmail', 'backend']}}

Slide 13

Slide 13 text

INTERNAL ANSIBLE INVENTORY MODEL - 2 {'web1': {'groups': [{'groupname': 'all', 'depth': 0}, {'groupname': 'google', 'depth': 1}, {'groupname': 'gcalendar', 'depth': 2}, {'groupname': 'gdrive', 'depth': 2}, {'groupname': 'gmail', 'depth': 2}, {'groupname': 'frontend', 'depth': 3}]}} {'storage2': {'groups': [{'groupname': 'all', 'depth': 0}, {'groupname': 'google', 'depth': 1}, {'groupname': 'gcalendar', 'depth': 2}, {'groupname': 'gdrive', 'depth': 2}, {'groupname': 'gmail', 'depth': 2}, {'groupname': 'frontend', 'depth': 3}]}}

Slide 14

Slide 14 text

HOW ABOUT INVENTORY VARIABLE PRECEDENCE? Ansible docs are very succinct on how inventory variables precede each other Because As all things Ansible, KEEP IT SIMPLE, they say. There is only one Empire State Building. One Mona Lisa, etc. Figure out where to define a variable, and do not make it complicated. Remember: Child groups override parent groups, and hosts always override their groups.

Slide 15

Slide 15 text

SO ANSIBLE INVENTORY IS NOT A TREE, BUT VARIABLES PRECEDE IN CHILD GROUPS? Child? Parent? Tree? 3 5 d e f a d d _ c h i l d _ g r o u p ( s e l f , g r o u p ) : 3 6 3 7 i f s e l f = = g r o u p : 3 8 r a i s e E x c e p t i o n ( " c a n n o t a d d g r o u p t o i t s e l f " ) 3 9 4 0 # d o n o t a d d i f i t i s a l r e a d y t h e r e 4 1 i f n o t g r o u p i n s e l f . c h i l d _ g r o u p s : 4 2 s e l f . c h i l d _ g r o u p s . a p p e n d ( g r o u p ) 4 3 g r o u p . d e p t h = m a x ( [ s e l f . d e p t h + 1 , g r o u p . d e p t h ] ) 4 4 g r o u p . p a r e n t _ g r o u p s . a p p e n d ( s e l f ) 4 5 s e l f . c l e a r _ h o s t s _ c a c h e ( ) When we encounter a child group, then "depth" gets a level deeper Deeper == more child is more precedence That *is* some kind of a tree, no?

Slide 16

Slide 16 text

THE PROJECT mostly a Java shop, very standardized {'75% of all virtual machines are tomcat hosts': 'ONE ROLE'} every tomcat app is a two node "cluster" behind a loadbalancer; per app one or more healthchecks per cluster typically 1 application (context), sometimes more 1 big functional application typically == several application clusters applications are part of a project, a project is part of an organisation every application has three instances in each environment ['development', 'testing', 'production'] some applications communicate with other applications through the loadbalancer

Slide 17

Slide 17 text

INVENTORY ORGANISATION a l l i n v e n t o r y | _ o r g a n i s a t i o n 1 | _ p r o j e c t 1 | _ a p p l i c a t i o n 1 | _ d e v | _ n o d e 1 | _ n o d e 2 | _ t e s t | _ . . | _ p r o d | _ . . | _ a p p l i c a t i o n 2 | _ . . | _ p r o j e c t 2 | _ . . | _ o r g a n i s a t i o n 1 | _ . .

Slide 18

Slide 18 text

OTHER GROUP TREES (WE HAVE DIFFERENT TREE'S!) | _ d e v e l o p m e n t | _ o r g a n i s a t i o n 1 - d e v | _ a p p l i c a t i o n 1 - d e v | _ t e s t i n g | _ p r o d u c t i o n OR ALSO t o m c a t | _ a p p l i c a t i o n 1 | _ a p p l i c a t i o n 2 < s o m e _ o t h e r _ s e r v e r _ r o l e _ b e s i d e s _ t o m c a t > | _ a p p l i c a t i o n 7 | _ a p p l i c a t i o n 9 < / s o m e _ o t h e r _ s e r v e r _ r o l e _ b e s i d e s _ t o m c a t > lowest groups with nodes are at different levels in different tree's

Slide 19

Slide 19 text

EXAMPLE: A SET OF REVERSE PROXIES: a l l ( 0 ) . i n f r a ( 1 ) . . r p ( 2 ) . . . r p - v o n e t - o e ( 3 ) . . . r p - v o n e t - o n ( 3 ) . . . r p - v o n e t - p r ( 3 ) . . . r p - i n n e t - o e ( 3 ) . . . r p - i n n e t - o n ( 3 ) . . . r p - i n n e t - p r ( 3 ) . . r p - v o n e t ( 2 ) . . . r p - v o n e t - o n ( 3 ) . . . r p - v o n e t - o e ( 3 ) . . . r p - v o n e t - p r ( 3 ) . . r p - i n n e t ( 2 ) . . . r p - i n n e t - o n ( 3 ) . . . r p - i n n e t - o e ( 3 ) . . . r p - i n n e t - p r ( 3 )

Slide 20

Slide 20 text

SOMETIMES DEPTH IS WRONG! . u n i s o n ( 1 ) . . g e o s e r v e r - d o v p u b ( 2 ) . . g e o s e r v e r - d o v p u b - o e ( 2 ) . . g e o s e r v e r - d o v p u b - o n ( 2 ) . . g e o s e r v e r - d o v p u b - p r ( 2 ) [geoserver-dovpub] is a tomcat application, but is *also* member of the [unison] group (to which a unison role maps) child groups are environment specific subgroups, but they get the the same depth as their parent

Slide 21

Slide 21 text

PLAYBOOK REMINDER: ANSIBLE LOOPS => WITH_ITEMS Please "deploy some apps" app1 app2 t a s k : - c o m m a n d : a 2 e n s i t e { { i t e m } } w i t h _ i t e m s : a p p l i c a t i o n s l i s t applicationslist is defined in the inventory a p p l i c a t i o n s l i s t : - a p p l i c a t i o n 1 - a p p l i c a t i o n 2 more loops? nested loops? with_nested, with_subelements, ..

Slide 22

Slide 22 text

WITHIN AN APPLICATION SERVER we have several apps and each app can have one or more healthchecks n o d e | _ s u b a p p 1 | _ h e a l t h c h e c k 1 | _ h e a l t h c h e c k 2 | _ s u b a p p 1 . . . . . .

Slide 23

Slide 23 text

DATA MODELLING SO FAR: NODES, POOLS AND HEALTHCHECKS p u b l i s h e d a p p s : - n a m e : " w e b " t y p e : " { { d e f a u l t _ a p p t y p e } } " p o r t : 8 0 8 0 l b p o r t : " { { d e f a u l t _ l b p o r t } } " m o n i t o r t y p e : " { { d e f a u l t _ m o n i t o r t y p e } } " q u o r u m : 0 m o n i t o r s : - n a m e : " { { d e f a u l t _ m o n i t o r _ a p p n a m e } } " t y p e : h t t p g e t _ p a t h : " { { d e f a u l t _ g e t _ p a t h } } " p r o t o c o l : " { { d e f a u l t _ p r o t o c o l } } " g e t _ e x t r a : " { { d e f a u l t _ g e t _ e x t r a } } " r e c e i v e : " { { d e f a u l t _ r e c e i v e _ s t r i n g } } "

Slide 24

Slide 24 text

DATA MODELLING SO FAR: NODES, POOLS AND HEALTHCHECKS - CONTINUED ( . . . p u b l i s h e d a p p s : ) - n a m e : t c p t y p e : t c p p o r t : 1 2 3 4 l b p o r t : 6 0 1 2 3 4 m o n i t o r t y p e : " { { d e f a u l t _ m o n i t o r t y p e } } " q u o r u m : 0 m o n i t o r s : - n a m e : " t c p " t y p e : t c p _ h a l f _ o p e n s e n d : " " r e c e i v e : " "

Slide 25

Slide 25 text

CREATE THE CLUSTER APPLICATION MONITORS (HEALTHCHECKS) snippet of a task: m o d u l e : . . . n a m e : > { { i t e m . 0 . m o n i t o r n a m e | d e f a u l t ( ' M O N - ' ~ i t e m . 0 . t y p e | d e f a u l t ( d e f a u l t _ a p p t y p e ) | u p p e r ~ ' - ' ~ a p p ~ ( ' - ' ~ i t e m . 0 . n a m e | d e f a u l t ( ' w e b ' ) ) | r e p l a c e ( ' - w e b ' , ' ' ) ~ ( ' - ' ~ i t e m . 1 . n a m e | d e f a u l t ( ' w e b ' ) ) | r e p l a c e ( ' - w e b ' , ' ' ) ~ ' - ' ~ z u i l ) } } s e n d : > { { ' G E T ' ~ i t e m . 1 . g e t _ p a t h ~ ' ' ~ i t e m . 1 . p r o t o c o l | d e f a u l t ( d e f a u l t _ p r o t o c o l ) ~ i t e m . 1 . g e t _ e x t r a | d e f a u l t ( ' ' ) ~ ' \ \ r \ \ n \ \ r \ \ n ' } } r e c e i v e : " { { i t e m . 1 . r e c e i v e | d e f a u l t ( d e f a u l t _ r e c e i v e _ s t r i n g ) } } " p o r t : " { { i t e m . 1 . p o r t | d e f a u l t ( 0 ) } } " w h e n : i t e m . 1 . t y p e i s d e f i n e d a n d i t e m . 1 . t y p e = = ' h t t p ' w i t h _ s u b e l e m e n t s : - p u b l i s h e d a p p s - m o n i t o r s I will need this monitor name again later, but I can't easily definite and keep it in a variable, as it comes from a loop construct

Slide 26

Slide 26 text

CREATE THE CLUSTER SERVER POOLS task snippet where we define the monitors (healthchecks) to be applied to the pool m o n i t o r s : > { % f o r m o n i n i t e m . m o n i t o r s % } { { ' / C o m m o n / ' ~ i t e m . m o n i t o r n a m e | d e f a u l t ( ' M O N - ' ~ i t e m . t y p e | d e f a u l t ( d e f a u l t _ a p p t y p e ) | u p p e r ~ ' - ' ~ a p p ~ ( ' - ' ~ i t e m . n a m e | d e f a u l t ( ' w e b ' ) ) | r e p l a c e ( ' - w e b ' , ' ' ) | ~ ( ' - ' ~ m o n . n a m e | d e f a u l t ( ' w e b ' ) ) | r e p l a c e ( ' - w e b ' , ' ' ) | ~ ' - ' ~ z u i l ) } } { % i f n o t l o o p . l a s t % } , { % e n d i f % } { % e n d f o r % } w i t h _ i t e m s : p u b l i s h e d a p p s sometimes looping within a jinja2 template gives more power, and more possibilities (and more bugs)

Slide 27

Slide 27 text

MORE LOOPS, MORE DEEPLY NESTED? TODO: I need to define proxies based on previous pools Each application needs access to a set of other applications. == An applications needs the definition of a list of other applictions to get access to Then I need to loop though that list, and then the list of publishedapps for every application... This can possibly go cross-environment...(eg we don't have separate smtp servers per environment, so dev nodes need access to smtp-production) Maybe the list of applications can differ per environment? ... THIS IS GETTING TOO COMPLEX... Or should we NOT automate all things?

Slide 28

Slide 28 text

MANAGING INVENTORY DATA updating software versions of applications in different stacks, in different environments update default java version: for all dev servers? Keep deployed version for each application in each environment seperately Nice to have = an inventory tool that keeps track of this let's you inherit defaults, but then remember them for particular setup

Slide 29

Slide 29 text

Q/A ?

Slide 30

Slide 30 text

THANKS! https://github.com/sergevanginderachter/revealjs- presentations/tree/cfgmgmtcampeu-ansible- inventory