Seminar on network analysis & the cosmic web

"Seminar on network analysis & the cosmic web", Queen's University Belfast, Wed 30th November 2016

Transcript

1. Dark Energy Survey 1 Donnacha Kirk & Lorne Whiteway DARK ENERGY SURVEY Network Analysis of the Cosmic Web
2. Overview • The Cosmic Web • Dark Energy Survey • Decomposition Methods • The Network Approach • The Validation Problem. • Uses of the Cosmic Web
3. Cosmologists assume the Universe is homogenous and isotropic. Homogeneity & Isotropy
5. 2.73K
6. Daniela Saadeh 1605.07178
7. Homogeneity and Isotropy Gravity Expansion
8. Gravity Expansion Homogeneity and Isotropy
9. Primordial anisotropies seed structure formation. Gravitational collapse and cosmic expansion produce the cool, structured late-Universe that we observe.
10. Primordial anisotropies seed structure formation. Gravitational collapse and cosmic expansion produce the cool, structured late-Universe that we observe.
11. Universe was 0.2 Gyr old
12. Universe was 1 Gyr old
13. Universe was 4.7 Gyrs old
14. Today (13.6 Gyr)
16. • 2 point spatial correlation function. • Average of temperature at pairs of points separated by some angular distance. Temperature Power Spectrum
17. • Cl_TT Temperature Power Spectrum
18. (figure from Max Tegmark) If the initial fluctuations are a Gaussian random field, we only need to know the power spectrum and the cosmological parameters to describe the ICs DIFFERENT PROBES OF THE MASS POWER SPECTRUM • 2 point spatial correlation function. • Based on the relative separation of pairs of galaxies. Matter Power Spectrum
19. The Dark Energy Survey Overlap with the South Pole Telescope Survey (SPT) DARK ENERGY SURVEY • Probe origin of Cosmic Acceleration: Distance vs. redshift Growth of Structure • Two multicolor surveys: 300 M galaxies over 5000 s.d. grizY to 24th mag 3500 supernovae (30 sq deg) • New camera for CTIO Blanco 4m telescope Facility instrument - 570 Mpixels - 3 deg2 FOV - Facility instrument • Five-year Survey started Aug. 31, 2013 525 nights (Sept.-Feb.) - Science Verification (SV) - Nov 2012-Feb 2013 -Year 3 started August 2015 DECam on the CTIO Blanco 4m www.darkenergysurvey.org www.darkenergydetectives.org
20. The Dark Energy Survey Overlap with the South Pole Telescope Survey (SPT) DARK ENERGY SURVEY • Probe origin of Cosmic Acceleration: Distance vs. redshift Growth of Structure • Two multicolor surveys: 300 M galaxies over 5000 s.d. grizY to 24th mag 3500 supernovae (30 sq deg) • New camera for CTIO Blanco 4m telescope Facility instrument - 570 Mpixels - 3 deg2 FOV - Facility instrument • Five-year Survey started Aug. 31, 2013 525 nights (Sept.-Feb.) - Science Verification (SV) - Nov 2012-Feb 2013 -Year 3 started August 2015 DECam on the CTIO Blanco 4m www.darkenergysurvey.org www.darkenergydetectives.org
21. 21 Dark Energy Survey Collaboration Fermilab, UIUC/NCSA, University of Chicago, LBNL, NOAO, University of Michigan, University of Pennsylvania, Argonne National Lab, Ohio State University, Santa-Cruz/SLAC/Stanford, Texas A&M Brazil Consortium UK Consortium: UCL, Cambridge, Edinburgh, Nottingham, Portsmouth, Sussex Spain Consortium: CIEMAT, IEEC, IFAE CTIO Ludwig-Maximilians Universität LM U ETH Zurich ~300 scientists US support from DOE+NSF Australia
22. 22 Dark Energy Survey Collaboration Fermilab, UIUC/NCSA, University of Chicago, LBNL, NOAO, University of Michigan, University of Pennsylvania, Argonne National Lab, Ohio State University, Santa-Cruz/SLAC/Stanford, Texas A&M Brazil Consortium UK Consortium: UCL, Cambridge, Edinburgh, Nottingham, Portsmouth, Sussex Spain Consortium: CIEMAT, IEEC, IFAE CTIO Ludwig-Maximilians Universität LM U ETH Zurich ~300 scientists US support from DOE+NSF Australia CTIO
25. DES Timeline • Project start 2003 • Research & Development 2004-8 • DECam Construction 2008-11 • Installation 2012 • First Light Sept. 2012 • Commissioning Sept-Oct. 2012 • Science Verification Nov. 2012-Feb. 2013 • First Season (Year 1) Aug. 2013-Feb. 2014 • Y2: Aug. 2014-Feb. 2015, Y3: Aug. 2015-Feb. 2016 • Fourth Season (Y4) Aug. 2016-Feb. 2017 • Five 105-night seasons
26. DES SV image of a deep SN field
27. First Images Fornax Cluster of Galaxies First Light on Sept. 12, 2012
28. First Images Fornax Cluster of Galaxies First Light on Sept. 12, 2012 Galaxy NGC 1365 in Fornax image from a single CCD
29. • Becker et al. 2015 (arXiv:1507.05360) • Five tomographic bins. Galaxy clustering, photo-z and systematics in DES-SV 13 Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ 0.1 1 0.00 0.01 0.02 0.03 0.04 0.05 0.06 q @degD wHqL Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.01 0.1 0.01 0.1 0.2 < zphot < 0.4 Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ 0.1 1 0.00 0.02 0.04 0.06 0.08 q @degD wHqL Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.01 0.1 0.01 0.1 0.4 < zphot < 0.6 Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.1 1 0.00 0.02 0.04 0.06 0.08 0.10 q @degD wHqL Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.01 0.1 0.01 0.1 0.6 < zphot < 0.8 Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.1 1 0.00 0.02 0.04 0.06 0.08 q @degD wHqL Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.01 0.1 0.1 0.5 0.8 < zphot < 1.0 Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ 0.01 0.1 1 0.0 0.1 0.2 0.3 q @degD wHqL Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê 0.01 0.05 0.1 1.0 < zphot < 1.2 Weak Lensing 2pt Measurement
30. • DES Collaboration 2015 (arXiv:1507.05552) • Consistent with Planck 2015 cosmology constraints. • Robust to choice of shape catalogue, modelling of systematics. Weak Lensing Cosmology Paper
31. g et al. Hong et al. • Powerful though they are, 2pt statistics are blind to non-Gaussian features of the density field. • Need to exploit higher order moments. 2pt Statistics are Inadequate
32. icting nonlinear density PDFs from ICs Non-linear Collapse à Deviations from Gaussianity
33. icting nonlinear density PDFs from ICs (cosmic(web:(voids,(walls,(filaments,(nodes 3 How(is(the(cosmic(web(woven?( How(do(structures(grow(in(the(Universe? gravity Gaussian(primordial(fluctua5ons( expansion ~ r ~ r ~ r1 ~ r1 ~ r2 ~ r2 ✓ ⇠3 (~ r1,~ r2 ) = ⇠3 (r1, r2, ✓) 4 2 2 4 x 0.1 0.2 0.3 0.4 P x (Gaussian(PDF Effect$of$non)lineari2es$ $on$the$PDF G ( x ) = e x 2 / 2 p 2 ⇡ S3 (PDF:(probability(for(this(random(variable(x(to( take(a(given(value) peaks(become(denser Clerkin’16(( (DES) Non-linear Collapse à Deviations from Gaussianity
34. Difficulty – not obvious in optical, biased tracers
35. Weak Lensing Mass Maps • Vikram et al. 2015 (arXiv:1504.03002) & Chang et al. 2015 (arXiv:1505.01871) • Kaiser-Squires reconstruction: relate the Fourier transform of the shear to that of the convergence.
36. • Vikram et al. 2015 (arXiv:1504.03002) & Chang et al. 2015 (arXiv:1505.01871) • Kaiser-Squires reconstruction: relate the Fourier transform of the shear to that of the convergence. Weak Lensing Mass Maps
37. • Vikram et al. 2015 (arXiv:1504.03002) & Chang et al. 2015 (arXiv:1505.01871) • Kaiser-Squires reconstruction: relate the Fourier transform of the shear to that of the convergence. • Shows correlation with RedMapper galaxy clusters. Weak Lensing Mass Maps
38. Take galaxies as biased tracers, smooth to find density field.
39. The Cosmic Web • Define environment by the form of the tidal tensor: structure 1 2 3 clusters/knots + + + filaments - + + sheets/walls - - + voids - - - structure categories defined by the gravitational tidal field tensor, using the ei ns correspond to positive (negative) eigenvalues. e was employed to characterise the alignment of satellites in terms of the c s and projected shapes, with both quantities being accessible also observat in the gravitational field generated by the matter d motion of the test particles; see Equation (12). The fo h directions gravitational forces contract (positive ei ts. For eigenvalues 1 < 2 < 3 , one can thus defin genvector corresponding to 1 specifies the directio also Figure 2). For sheet-like structures, the eigen of the sheet. Clusters, or knots of the cosmic web ons of the tidal tensors, and these ellipsoids tend t ical (see the argument in ? ). Figure 7 shows an exam cult to identify. Candidates for galaxy clusters c on and then be confirmed if redshift information is
40. Still very challenging: • Details of smoothing can have an effect. • Multiscale Distribution. • No clear defined boundaries. • Orders of magnitude variation in the density field. • Very difficult with photometric redshifts.
41. • Can we do without smoothing? • Try to decompose the cosmic web at the level of the marked point process? i.e. galaxy locations. • Network analysis provides tools for describing points (nodes) and their connections (edges). This work follows on from the initial publication by Hong & Dey (arXiv 1504.00006). Network Approaches
42. • A network is a set of nodes and edges (which may be directed and/or weighted). • Network analysis: look for interesting network statistics. Examples: <edges per node>; distribution of edges per node; Erdős number; PageRank; centrality measures. Bridges of Konigsberg Networks
43. system, it is the w steps owever, ary for ystems us sys- time of omains, A rout- r tables cture of process revious es. This ors at a f paths gives a nomous ms. As ver the routers dges as P-based n at the measure d from class C nets are tively a nternet, n recent Figure 2.3: The structure of the Internet at the level of autonomous systems. (See Plate III for color version.) The vertices in this netvvork representation of the Internet are autonomous systems and the edges show the routes taken by data traveling between them. This figure is different from Fig. 1.1, which shows the netvvork at the level of class C sub- nets. The picture was created by Hal Burch and Bill Cheswick. Patent(s) pending and Copyright Lumeta Corporation 2009. Reproduced with permission. The Internet Networks
44. Networks Hamlet
45. Travelling Salesman Problem Networks
46. Networks Leopold Bloom’s puzzle: “Good puzzle would be to cross Dublin without passing a pub” Rory McCann, 2014 https://www.theguardian.com/books/2011/jun/17/joyce-puzzle-dublin-passing-pub
47. Network Measures Centrality measures assign a score to each node; this score is high if the node is ‘central’. Various flavours:
48. Centrality measures assign a score to each node; this score is high if the node is ‘central’. Various flavours: • Degree centrality: dc(a) is the number of edges at a. Example: flu jab candidates. DC Network Measures
49. Centrality measures assign a score to each node; this score is high if the node is ‘central’. Various flavours: • Degree centrality: dc(a) is the number of edges at a. Example: flu jab candidates. • Betweenness centrality: bc(a) is the probability, for randomly selected nodes b and c, that a is on the shortest path from b to c. Example: motorway café. DC BC Network Measures
50. Centrality measures assign a score to each node; this score is high if the node is ‘central’. Various flavours: • Degree centrality: dc(a) is the number of edges at a. Example: flu jab candidates. • Betweenness centrality: bc(a) is the probability, for randomly selected nodes b and c, that a is on the shortest path from b to c. Example: motorway café. • Closeness centrality: cl(a) is the average of the shortest path lengths from a to other nodes. Example: Amazon warehouses. DC BC CC Network Measures
51. By Tapiocozzo - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=39064835 Degree centrality Betweenness centrality Closeness centrality e.g. Social media ‘influencers’ e.g. Motorway service stations e.g. Amazon warehouses Network Measures
52. By Tapiocozzo - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=39064835 Degree centrality Betweenness centrality Closeness centrality e.g. Social media ‘influencers’ e.g. Motorway service stations e.g. Amazon warehouses • Given a 3D catalogue of galaxy positions, form a network N with: ◦ one node per galaxy, ◦ An edge between two galaxies if and only if their separation is less than some linking length L. • Then discard the catalogue and do network analysis on N. • Hong-Dey: ◦ High dc indicates cluster membership; ◦ High bc/dc indicates filament membership. Network Measures
53. H&D Results
54. Toy Model (1) Two clusters, joined by a lower-density filament. This is a 2D projection of a 3D catalogue.
55. Toy Model (2) Red dots: top 10% by dc; these trace out the clusters. ✔ Blue dots: top 10% by bc/dc. Blue lines: “minimal spanning tree” for blue dots. Here the linking length is ‘too small’ and hence the filament doesn’t appear in the network as a unified structure. ✖
56. Toy Model (3) With increased linking length we ‘find the filament’, and the blue line now traces out the filament’s spine. ✔
57. Uniform background plus four clusters and one filament. The filament is not so obvious. Less obvious toy model
58. Less obvious toy model With a linking length of 1.27 (corresponding to 7 neighbours on average) the filament is found. ✔ Here the blue dots are the top 3% by bc/dc.
59. Sloan Great Wall
60. Sloan Great Wall
61. Sanity check: methods succeed with obvious large features. Sloan Great Wall
62. Real data: Slice from DES Y1A1 data, using photo-z. Linking length = 11.04 h-1 Mpc (corresponding to 6 neighbours on average). Here red = 10% top dc; blue = 10% top bc/dc.
63. Limitations/Extensions • Fold in ‘non-geometric’ information into the network e.g. colour, morphology, velocity… • A more fundamentally probabilistic method would be nice. • Need to understand quantitively and qualitatively how the method deals with masks. • Most of all: how do we know our result is in any way meaningful?
64. • Multiple methods, very different outputs. • To what extent to any of them describe the ‘true’ cosmic web structure? • Internal approach: Score methods’ agreement wrt each other. • External approach: Try to evaluate a method wrt some ‘objective’ metric. The Validation Problem
65. Internal Approach: Photometric Redshifts • DES is an example of a photometric survey. • DECam takes multiple exposures of the same object in different frequency filters.
66. • DES is an example of a photometric survey. • DECam takes multiple exposures of the same object in different frequency filters. • Use the intensity in each filter as a very low resolution spectrum to determine redshift. Internal Approach: Photometric Redshifts
67. • DES is an example of a photometric survey. • DECam takes multiple exposures of the same object in different frequency filters. • Use the intensity in each filter as a very low resolution spectrum to determine redshift. Internal Approach: Photometric Redshifts
68. Impact on Cosmic Web Distribution
69. Impact on Cosmic Web Distribution
70. Define a Summary Statistic
71. Pop 1 Pop 2 Define a Summary Statistic
72. Pop 1 Pop 2 Define some volume around each point. Mutual overlap of the volumes associated with each population defines ‘agreement score’ Finding the same ‘feature’ scores well, even when no objects in common. Define a Summary Statistic
73. Define a Summary Statistic
74. Pop 1 Pop 2 Define a Summary Statistic
75. Pop 1 Pop 2 Disjoint‘features’ still score but the level of relative agreement is apparent. Define a Summary Statistic
87. Impact of Photometry on the Cosmic Web • Significant structure retained even at full photo-z scatter. • Summary statistic requires tuning. • How do we know the spec-z ‘structure’ was truly present? • Can we find model- independent tests of our cosmic web decomposition?