Integrate JBrowse REST API Framework with Adama Federation Architecture

This presentation describes the work done to integrate the JBrowse REST API Framework with the Araport.org-developed Adama Federation Architecture, enabling community developers to package their published datasets and expose them in a manner which is compatible with JBrowse.

Presented by Vivek Krishnakumar

Transcript

1. araport.org @araport Integrate JBrowse REST API with ADAMA Vivek Krishnakumar 12/03/2015
2. araport.org @araport Outline • JBrowse REST API spec • ADAMA endpoints • Mapping JBrowse to ADAMA endpoints • Extending JBrowse REST feature store • Handling Auth in JBrowse 1.x • Use Case – Integrate Chromatin States data into JBrowse via ADAMA • Summary • Challenges
3. araport.org @araport JBrowse REST API spec • Javascript based portable genome browser tool developed by GMOD community • Ships with REST feature and names store adapter • REST feature store requires access to following endpoints via HTTP GET: – (base)/stats/global – (base)/stats/region/(refseq)?start=123&end=456 – (base)/stats/regionFeatureDensities/(refseq)?start=123&end=456&basesP erBin=20000 – (base)/features/(refseq)?start=234&end=5678 • Can provide feature, sequence, and quantitative data for display • Accepted response type: application/json http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse-compatible_Web_Services
4. araport.org @araport ADAMA endpoints • Araport Data And Microservices API – Developed by WalterM and MattV • Encapsulates python code into reusable docker containers • Exposes internal logic as API endpoints: – query, generic adapters via HTTP GET • (base)/search • (base)/list – passthrough adapters via HTTP GET and POST • (base)/access • Implements token based Auth via iPlant OAuth2 https://github.com/Arabidopsis-Information-Portal/adama
5. araport.org @araport Mapping JBrowse to ADAMA endpoints • ADAMA (currently) doesn’t allow defining custom endpoints • So, we have to map the 4 JBrowse endpoints to the 2 ADAMA endpoints • Proposed endpoint mapping: JBrowse ADAMA (base)/stats/global (base)/search?q=global (base)/stats/region (base)/search?q=region (base)/stats/regionFeatureDensities (base)/search?q=regionFeatureDensities (base)/features (base)/search?q=features (base)/list?q=makeTrackListJson
6. araport.org @araport Extending JBrowse REST feature store • JBrowse/Store/SeqFeature/REST handles requests from the browser to the configured API service – Understands the 4 pre-defined endpoints • REST feature store extended based on proposed endpoint mapping • Utilize JBrowse Plugin architecture to develop Araport plugin: Araport/Store/SeqFeature/REST http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse_Plugins
7. araport.org @araport Handling Auth in JBrowse 1.x • ADAMA (currently) requires all requests to be accompanied by an Authorization Bearer $TOKEN • JBrowse 1.x does not support OAuth2 implicit flow, thus cannot generate tokens • Solution: – OAuth2 $TOKEN created with unlimited validity (-1) • $TOKEN generated by Araport anonymous user • $TOKEN scope limited to ADAMA services – Araport/Store/SeqFeature/RESTAuth hard codes $TOKEN – Araport/Store/SeqFeature/REST inherits from RESTAuth
8. araport.org @araport Use Case Integrate Chromatin States data into JBrowse via ADAMA • Published dataset representing a landscape of Arabidopsis thaliana chromatin states using 16 features, including DNA sequence, CG methylation, histone variants, and modifications • Data represents chromosomal regions (start/stop coordinates) corresponding with 9 distinct chromatin states • Published with manuscript as supplementary dataset (in XLS format) Sequeira-Mendes, J., Aragüez, I., Peiró, R., Mendez-Giraldez, R., Zhang, X., Jacobsen, S. E., ... & Gutierrez, C. (2014). The functional topography of the Arabidopsis genome is organized in a reduced number of linear motifs of chromatin states. The Plant Cell, 26(6), 2351-2366.
9. araport.org @araport • Develop ADAMA adapter to dynamically query data from desired region and represent as JBrowse compatible JSON • Code: https://github.com/Arabidopsis-Information- Portal/chromatin_states_to_jbrowse • Implementation logic: – Data transformed from 9 XLS worksheets into individual GFF3 files – Pythonic parser developed using gffutils, allowing range querying (via intermediary sqlite database) – ADAMA metadata.yml written which describes dependencies, endpoints and provenance • Data hosted with code (due to small size, well within GitHub file size limits) • ADAMA namespace created and adapter registered • Available in API explorer (search for ‘chromatin’) Use Case Integrate Chromatin States data into JBrowse via ADAMA https://jira.araport.org/browse/MINE-1078
10. araport.org @araport Integrate Chromatin States data into JBrowse via ADAMA • JBrowse tracks configured to talk to registered ADAMA endpoint • End result visible in screenshot above • Live view: https://apps.araport.org/jbrowse/?data=arabidopsis&loc=Chr3:11355449..11407858&tracks=chromatin_state_1,chro matin_state_2,chromatin_state_3,chromatin_state_4,chromatin_state_5,chromatin_state_6,chromatin_state_7,chromat in_state_8,chromatin_state_9,Araport11_gene_models
11. araport.org @araport Summary • Extended JBrowse to integrate with ADAMA endpoints • Exposed a published community dataset via ADAMA as an API • Exposed track configuration via ADAMA API • Visualized dataset in JBrowse via ADAMA API
12. araport.org @araport Challenges • Making web service calls to ADAMA requires Authorization $TOKEN • JBrowse needs to hard code the Authorization $TOKEN • Track configuration needs to be manually added to JBrowse • Scaling to larger datasets not feasible via GitHub route
13. araport.org @araport Acknowledgements • JCVI Araport Team – Chia-Yi Cheng – Agnes Chan – Chris Town • TACC Araport Team – Walter Moreira – Matthew Vaughn • Prof. Crisanto Gutierrez (CBMSO)