A GLIMPSE OF ANSIBLE INTERNALS KAI 

ABOUT ME ▸ Kai Xia, Pythonista and DevOps ▸ Worked in a game company in China(NTES) for 7 years ▸ Working@ShineSolutions ▸ Handler: @xiaket {gmail,github,linkedin,etc} 

BACKGROUND ▸ Personal interest: reading source code helps ▸ Work at NTES: make Ansible work with internal CM tool ▸ Hacker: Learn what I can do about it 

AGENDA ▸ How ansible works(in general) ▸ Selected ‘How xxx works’ 

HOW ANSIBLE WORKS 

from ansible.runner import Runner results = Runner( module_name=‘ping’, module_args=‘’, pattern=‘web*’, forks=10, ).run() Run Ansible 1.x using Python 

URL: http://docs.ansible.com/ansible/latest/dev_guide/developing_api.html 

WHAT’S CHANGED? ▸ Codebase reworked ▸ Smaller classes created to meet individual needs.(Runner has 45 init parameters, that file has 1500+ lines). ▸ New features introduced 

THE CODEBASE: META ▸ Ansible: 2.3.2.0 Item Number of files Lines of code w/ modules 1535 424528 w/o modules 327 53792 

TEXT THE CODEBASE: VILLAINS 

TEXT THE CODEBASE: VILLAINS(MORE EQUAL ONES) 

HOW ANSIBLE WORKS ▸ Generate a file ▸ scp it ▸ run it Terms and conditions apply 

TEXT PHASE 1: GATHER INFORMATION ▸ Playbook, inventory, roles, become, vault, templates, command line arguments, etc. ▸ DataLoader: yaml, json and vault ▸ VariableManager: host/group/extra/magic vars, facts ▸ Inventory: Represent a static/dynamic inventory 

TEXT PHASE 2: ORGANISE TODO LIST ▸ Play: Represent a playbook ▸ Block ▸ Task data = { 'name': “A Play", 'hosts': "localhost", 'gather_facts': "no", 'tasks': [ {"action": {"module": "shell", "args": "ls"}, "register": “ls_cmd”}, {"action": {"module": "debug", “args": {"msg": "{{ ls_cmd.stdout }}"}}}, ] } play = Play.load(data, variable_manager=vm, loader=ldr) 

TEXT PHASE 3: DISPATCH & CONTROL ▸ TaskQueueManager: Director ▸ StrategyPlugin: Project manager ▸ PlayIterator: The overseer ▸ {Worker,Result}Process: The work force ▸ Task{Executor,Result}: the chainsaw used by the work force 

TEXT PHASE 4: THOSE 7 WORDS ▸ TaskExecutor will: ▸ Get the connection plugin(put_file, fetch_file, exec_command) ▸ Get shell plugin ▸ Get the action plugin/modules ▸ Generate the file 

TEXT THE WORKFLOW VariableManager DataLoader Inventory Play Block Task TaskQueueManager PlayIterator StrategyPlugin TaskExecutor ActionPlugin/Module ConnectionPlugin ShellPlugin 

SELECTED TOPICS 

TEXT SELECTED TOPICS ▸ The strategy plugin ▸ The raw module ▸ The pipelining & ssh performance 

TEXT THE STRATEGY PLUGIN ▸ Linear: get next task, queue it for all hosts, wait till this task is done, move on. ▸ Free: task was sent to hosts as fast as possible, with some fairness tweak. 

TEXT THE STRATEGY PLUGIN: COMPARED 

TEXT THE FREE STRATEGY: FAIRNESS TWEAK 3 hosts, 2 tasks, fork = 2 

TEXT THE STRATEGY PLUGIN: ONE MORE THING ▸ Inject arbitrary code before and after a task. class StrategyModule(Linear): def run(self, iterator, play_context): for block in iterator._blocks: for task in block.block: if task.get_name() == "your module name": hosts = self._inventory.get_hosts(iterator._play_.hosts) run_your_code_here(task, hosts, play_context) return Linear.run(self, iterator, play_context) 

TEXT THE RAW MODULE ▸ An action plugin in disguise ▸ TRANSFERS_FILES = False ▸ self._low_level_execute_command(self._task.args.get('_raw_p arams'), executable=executable) ▸ rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable) 

TEXT TRANSFER_FILES = FALSE? ▸ Utils: assert/debug/fail ▸ Helpers: add_host/group_by/include_vars/set_fact/ set_stats ▸ Raw stuff: raw/service/package/win_reboot ▸ Networking: telnet/wait_for_connection 

TEXT SSH PERFORMANCE ▸ control master: retain ssh socket, feature of openssh ▸ pipelining: minimize len(command), reduce disk IO, feature of ansible 

TEXT PIPELINING IN ACTION 

THANKS!