Apache Airavata

This presentation gives an overview of the Apache Airavata project. It explains Apache Airavata in terms of it's architecture, data models and user interface.

Links for further information and connecting

http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/

https://nz.linkedin.com/pub/mike-frampton/20/630/385

https://open-source-systems.blogspot.com/

Transcript

1. What Is Apache Airavata ? • A framework that supports execution and management • Of computational scientific applications and workflows • Mainly for grid based systems but will support – Remote clusters, cloud-based systems – Can be extended via implementation • Can be a middleware for scientific developers • Offers an extensible architecture • Open source Apache 2.0 license • Written in Java, C++
2. Airavata Architecture
3. Airavata Architecture • Airavata API – Users / gateway developers can communicate with Airavata • Orchestrator – Manages application AMQP worker queue • Workflow interpreter – Manages submitted work flows • Application Factory (Gfac) – Communicates with remote resources
4. Airavata Architecture • Registry – The Airavata data store • Messaging – Publishes notifications for application statuses • API written using Apache Thrift – Supports many languages
5. Airavata AppCatalog Data Model
6. Airavata AppCatalog Data Model • Used for application registration and contains ... • ComputeResourceModel – Abstraction of remote compute resource – Contains information about resource queues – Host addresses, job submission protocols – Useful information related to compute resource • ApplicationModule – Remote resources contains modules / applications – Already installed in the systems – User defined applications possible
7. Airavata AppCatalog Data Model • Used for application registrationand contains ... • ApplicationInterfaceModel – Interface which defines application I/O – Can have multiple interfaces defined • ApplicationDeploymentModel – Bridges the compute resource and the module • GatewayProfileModel – Abstraction of the science gateway • GatewayPreferenceModel – Model to specify gateway compute resources preferences – Includes information like login usernames, allocation ids etc
8. Airavata ExperimentCatalog Data Model
9. Airavata ExperimentCatalog Data Model • Used for application execution • Application might use single node or work flow • Airavata internally creates models to represent process – Process model, task model, job model • Work flows contain a list processes • Tasks can have many types – Input staging, job submission – Job monitoring, output staging etc • Users can develop their own tasks for the task chain
10. Airavata UI • Has a Java based application called XBaya – Create workflows, – Submit and manage multiple applications – Works best with Airavata 0.14 • Also has a web-based interface (PHP Gateway) – Written using Airavata PHP client library – Users can deploy it in their own systems – Use it to register applications – Run and monitor applications
11. Available Books • See “Big Data Made Easy” – Apress Jan 2015 • See “Mastering Apache Spark” – Packt Oct 2015 • See “Complete Guide to Open Source Big Data Stack – “Apress Jan 2018” • Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ • Connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020
12. Connect • Feel free to connect on LinkedIn – www.linkedin.com/in/mike-frampton-38563020 • See my open source blog at – open-source-systems.blogspot.com/ • I am always interested in – New technology – Opportunities – Technology based issues – Big data integration