What Is Apache Airavata ? ● A framework that supports execution and management ● Of computational scientific applications and workflows ● Mainly for grid based systems but will support – Remote clusters, cloud-based systems – Can be extended via implementation ● Can be a middleware for scientific developers ● Offers an extensible architecture ● Open source Apache 2.0 license ● Written in Java, C++ 

Airavata Architecture 

Airavata Architecture ● Airavata API – Users / gateway developers can communicate with Airavata ● Orchestrator – Manages application AMQP worker queue ● Workflow interpreter – Manages submitted work flows ● Application Factory (Gfac) – Communicates with remote resources 

Airavata Architecture ● Registry – The Airavata data store ● Messaging – Publishes notifications for application statuses ● API written using Apache Thrift – Supports many languages 

Airavata AppCatalog Data Model 

Airavata AppCatalog Data Model ● Used for application registration and contains ... ● ComputeResourceModel – Abstraction of remote compute resource – Contains information about resource queues – Host addresses, job submission protocols – Useful information related to compute resource ● ApplicationModule – Remote resources contains modules / applications – Already installed in the systems – User defined applications possible 

Airavata AppCatalog Data Model ● Used for application registrationand contains ... ● ApplicationInterfaceModel – Interface which defines application I/O – Can have multiple interfaces defined ● ApplicationDeploymentModel – Bridges the compute resource and the module ● GatewayProfileModel – Abstraction of the science gateway ● GatewayPreferenceModel – Model to specify gateway compute resources preferences – Includes information like login usernames, allocation ids etc 

Airavata ExperimentCatalog Data Model 

Airavata ExperimentCatalog Data Model ● Used for application execution ● Application might use single node or work flow ● Airavata internally creates models to represent process – Process model, task model, job model ● Work flows contain a list processes ● Tasks can have many types – Input staging, job submission – Job monitoring, output staging etc ● Users can develop their own tasks for the task chain 

Airavata UI ● Has a Java based application called XBaya – Create workflows, – Submit and manage multiple applications – Works best with Airavata 0.14 ● Also has a web-based interface (PHP Gateway) – Written using Airavata PHP client library – Users can deploy it in their own systems – Use it to register applications – Run and monitor applications 

Available Books ● See “Big Data Made Easy” – Apress Jan 2015 ● See “Mastering Apache Spark” – Packt Oct 2015 ● See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” ● Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ ● Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 

Connect ● Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at – open-source-systems.blogspot.com/ ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration