Integrate JBrowse REST API Framework with Adama Federation Architecture

This presentation describes the work done to integrate the JBrowse REST API Framework with the Araport.org-developed Adama Federation Architecture, enabling community developers to package their published datasets and expose them in a manner which is compatible with JBrowse.

Presented by Vivek Krishnakumar

Transcript

1. araport.org
   @araport
   Integrate JBrowse REST API
   with ADAMA
   Vivek Krishnakumar
   12/03/2015
   

   View Slide

2. araport.org
   @araport
   Outline
   • JBrowse REST API spec
   • ADAMA endpoints
   • Mapping JBrowse to ADAMA endpoints
   • Extending JBrowse REST feature store
   • Handling Auth in JBrowse 1.x
   • Use Case
   – Integrate Chromatin States data into JBrowse via
   ADAMA
   • Summary
   • Challenges
   

   View Slide

3. araport.org
   @araport
   JBrowse REST API spec
   • Javascript based portable genome browser tool
   developed by GMOD community
   • Ships with REST feature and names store adapter
   • REST feature store requires access to following
   endpoints via HTTP GET:
   – (base)/stats/global
   – (base)/stats/region/(refseq)?start=123&end=456
   – (base)/stats/regionFeatureDensities/(refseq)?start=123&end=456&basesP
   erBin=20000
   – (base)/features/(refseq)?start=234&end=5678
   • Can provide feature, sequence, and quantitative data
   for display
   • Accepted response type: application/json
   http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse-compatible_Web_Services
   

   View Slide

4. araport.org
   @araport
   ADAMA endpoints
   • Araport Data And Microservices API
   – Developed by WalterM and MattV
   • Encapsulates python code into reusable docker
   containers
   • Exposes internal logic as API endpoints:
   – query, generic adapters via HTTP GET
   • (base)/search
   • (base)/list
   – passthrough adapters via HTTP GET and POST
   • (base)/access
   • Implements token based Auth via iPlant OAuth2
   https://github.com/Arabidopsis-Information-Portal/adama
   

   View Slide

5. araport.org
   @araport
   Mapping JBrowse to ADAMA endpoints
   • ADAMA (currently) doesn’t allow defining
   custom endpoints
   • So, we have to map the 4 JBrowse endpoints
   to the 2 ADAMA endpoints
   • Proposed endpoint mapping:
   JBrowse ADAMA
   (base)/stats/global (base)/search?q=global
   (base)/stats/region (base)/search?q=region
   (base)/stats/regionFeatureDensities (base)/search?q=regionFeatureDensities
   (base)/features (base)/search?q=features
   (base)/list?q=makeTrackListJson
   

   View Slide

6. araport.org
   @araport
   Extending JBrowse REST feature store
   • JBrowse/Store/SeqFeature/REST handles
   requests from the browser to the configured
   API service
   – Understands the 4 pre-defined endpoints
   • REST feature store extended based on
   proposed endpoint mapping
   • Utilize JBrowse Plugin architecture to
   develop Araport plugin:
   Araport/Store/SeqFeature/REST
   http://gmod.org/wiki/JBrowse_Configuration_Guide#Writing_JBrowse_Plugins
   

   View Slide

7. araport.org
   @araport
   Handling Auth in JBrowse 1.x
   • ADAMA (currently) requires all requests to be
   accompanied by an Authorization Bearer $TOKEN
   • JBrowse 1.x does not support OAuth2 implicit flow,
   thus cannot generate tokens
   • Solution:
   – OAuth2 $TOKEN created with unlimited validity (-1)
   • $TOKEN generated by Araport anonymous user
   • $TOKEN scope limited to ADAMA services
   – Araport/Store/SeqFeature/RESTAuth hard codes
   $TOKEN
   – Araport/Store/SeqFeature/REST inherits from RESTAuth
   

   View Slide

8. araport.org
   @araport
   Use Case
   Integrate Chromatin States data into JBrowse via ADAMA
   • Published dataset representing a landscape of
   Arabidopsis thaliana chromatin states using 16
   features, including DNA sequence, CG
   methylation, histone variants, and modifications
   • Data represents chromosomal regions
   (start/stop coordinates) corresponding with 9
   distinct chromatin states
   • Published with manuscript as supplementary
   dataset (in XLS format)
   Sequeira-Mendes, J., Aragüez, I., Peiró, R., Mendez-Giraldez, R., Zhang, X., Jacobsen, S. E., ... & Gutierrez, C. (2014). The functional topography
   of the Arabidopsis genome is organized in a reduced number of linear motifs of chromatin states. The Plant Cell, 26(6), 2351-2366.
   

   View Slide

9. araport.org
   @araport
   • Develop ADAMA adapter to dynamically query data from
   desired region and represent as JBrowse compatible JSON
   • Code: https://github.com/Arabidopsis-Information-
   Portal/chromatin_states_to_jbrowse
   • Implementation logic:
   – Data transformed from 9 XLS worksheets into individual GFF3 files
   – Pythonic parser developed using gffutils, allowing range querying (via intermediary sqlite
   database)
   – ADAMA metadata.yml written which describes dependencies, endpoints and provenance
   • Data hosted with code (due to small size, well within GitHub
   file size limits)
   • ADAMA namespace created and adapter registered
   • Available in API explorer (search for ‘chromatin’)
   Use Case
   Integrate Chromatin States data into JBrowse via ADAMA
   https://jira.araport.org/browse/MINE-1078
   

   View Slide

10. araport.org
    @araport
    Integrate Chromatin States data into JBrowse via ADAMA
    • JBrowse tracks configured to talk to registered ADAMA endpoint
    • End result visible in screenshot above
    • Live view:
    https://apps.araport.org/jbrowse/?data=arabidopsis&loc=Chr3:11355449..11407858&tracks=chromatin_state_1,chro
    matin_state_2,chromatin_state_3,chromatin_state_4,chromatin_state_5,chromatin_state_6,chromatin_state_7,chromat
    in_state_8,chromatin_state_9,Araport11_gene_models
    

    View Slide

11. araport.org
    @araport
    Summary
    • Extended JBrowse to integrate with ADAMA
    endpoints
    • Exposed a published community dataset via
    ADAMA as an API
    • Exposed track configuration via ADAMA API
    • Visualized dataset in JBrowse via ADAMA API
    

    View Slide

12. araport.org
    @araport
    Challenges
    • Making web service calls to ADAMA requires
    Authorization $TOKEN
    • JBrowse needs to hard code the
    Authorization $TOKEN
    • Track configuration needs to be manually
    added to JBrowse
    • Scaling to larger datasets not feasible via
    GitHub route
    

    View Slide

13. araport.org
    @araport
    Acknowledgements
    • JCVI Araport Team
    – Chia-Yi Cheng
    – Agnes Chan
    – Chris Town
    • TACC Araport Team
    – Walter Moreira
    – Matthew Vaughn
    • Prof. Crisanto Gutierrez (CBMSO)
    

    View Slide