Collaborative visual analytics of radio surveys in the Big Data era

ABSTRACT
Radio survey datasets comprise an increasing number of individual observations stored as sets of multidimensional data. In large survey projects, astronomers commonly face limitations regarding: 1) interactive visual analytics of sufficiently large subsets of data; 2) synchronous and asynchronous collaboration; and 3) documentation of the discovery workflow. To support collaborative data inquiry, we present encube, a large-scale comparative visual analytics framework. Encube can utilise advanced visualization environments such as the CAVE2 (a hybrid 2D and 3D virtual reality environment powered with a 100 Tflop/s GPU-based supercomputer and 84 million pixels) for collaborative analysis of large subsets of data from radio surveys. It can also run on standard desktops, providing a capable visual analytics experience across the display ecology. Encube is composed of four primary units enabling compute-intensive processing, advanced visualisation, dynamic interaction, parallel data query, along with data management. Its modularity will make it simple to incorporate astronomical analysis packages and Virtual Observatory capabilities developed within our community. We discuss how encube builds a bridge between high-end display systems (such as CAVE2) and the classical desktop, preserving all traces of the work completed on either platform — providing a research process that can be continuous regardless of location.

Transcript

1. Collaborative visual analytics 
 of radio surveys in the Big Data era Dany Vohl | Astroinformatics 2016 | In collaboration with Christopher J. Fluke, Amr H. Hassan, 
 David G. Barnes (Monash University) and Virginia A. Kilborn @danyvohl
2. Background Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Meidt, Rand & Merrifield, 2009. ApJ. doi:10.1088/0004-637X/702/1/277
3. Background Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Large-Scale Spectral-Cube Surveys • large # of spectral-cubes, each with large # of sources • e.g. Apertif: 20,000 spectral-cubes, each containing ~100 sources
 (Verheijen et al. 2009, Punzo et al. 2015) ASKAP Cube Dimensions 
 6144 x 6144 x 16384
 APERTIF Cube Dimensions 
 2048 x 2048 (spatial) x 16384 (spectral)
 0.25 TB
4. Background Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Visualisation & 
 Analysis Common limitations of large surveys Synchronous & async.
 collaboration Documentation of 
 discovery workflow
5. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 DOI 10.7717/peerj-cs.88 arXiv:1610.00760
6. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Interaction Unit Panel 1 Panel 2 Panel 3 Panel 4 Display Units Manager Unit Process-Render Units Process data / Render visualisation Metadata server / scheduler / Workflow serialization control / query / visualise visualise data cubes Input/output layer Process layer Panel 1 Panel 2 Panel 3 Panel 4 Panel 1 Panel 2 Panel 3 Panel 4 … … Simple Instruction Multiple Visualisation
7. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016
8. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 • Part immersive visualization environment • 8-meter diameter working area, • 320 degree panoramic display system; • 80 stereo-capable displays • arranged in 20 four-panel columns, • 84 million pixels • Part supercomputer • 40 GPUs • ∼100 TFLOP/s of integrated 
 GPU-based processing power Hybrid 2D/3D virtual reality environment for immersive simulation and information analysis.
9. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Data: The HI Nearby Galaxy Survey (THINGS, Walter et al. 2008 ) Simple Instruction Multiple Visualisations
10. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Flux (Jy) P(Flux) Simple Instruction Multiple Visualisations
11. encube — Collaborative visual analytics Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Simple Instruction Multiple Visualisations
12. Workflow serialisation a key to synchronous and asynchronous collaboration Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Documentation of 
 discovery workflow Synchronous & 
 asynchronous collaboration
13. On-going work Integration with GraphTIVA and the Cloud Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 Rendered the whole southern sky 
 from 387 cubes (HIPASS survey) Data from Russell Jurek, ATNF 2013 :
 Hassan, Fluke, Barnes & Kilborn 
 (Swinburne University of Technology) 3D Visualisation with GraphTIVA • 96 GPUs cluster • 0.5 TB Cube • 7–10 frames per second
14. On-going work Integration with GraphTIVA and the Cloud Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016
15. Final Thoughts Dany Vohl | Astroinformatics 2016 | 22 Oct. 2016 • The CAVE2 permits new approaches to, and applications of, visual analytics
 • It offers great potential to accelerate the discovery process 
 in the era of large-scale spectral-cube surveys. 
 • encube provides a best of both worlds approach through support of high-end, collaborative visualisation in the CAVE2 while also supporting desktop-based analysis and discovery. This work was enabled and supported by the Monash Immersive Visualisation Platform (http://monash.edu/mivp). Thanks to CaveHD team. Thanks to the Astronomical Society of Australia for their travel financial support. DOI 10.7717/peerj-cs.88 arXiv:1610.00760