Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

A data-scientific noise-removal method for effi...

A data-scientific noise-removal method for efficient submillimeter spectroscopy with single-dish telescopes / 2021-08-18

For submillimeter spectroscopy with ground-based single-dish telescopes, removing noises from the Earth's atmosphere and instruments is essential. For this purpose, we present a new noise-removal method based on a data-scientific approach. The key technique is statistical matrix decomposition that automatically separates astronomical emission lines from noises in fast-sampled (1-10 Hz) time-series spectra. Unlike the conventional position-switching (PSW) method, the new method does not perform direct subtraction between on-source and off-source spectra. As a result, it improves the observation sensitivity by a factor of √2. It enables to reduce baseline ripples on the data, helping to improve the sensitivity as well.

We developed two different observing methods to utilize the new method. Frequency modulation system and observations with the Nobeyama 45 m telescope (Taniguchi et al. 2019) demonstrated both improvements in sensitivity and observation efficiency because the noise removal can be performed without off-source measurements. Fast-sampled PSW observations with the LMT 50 m telescope (Taniguchi et al. 2021 in press) demonstrated the improvement in sensitivity only by offline processes. Finally we address the application of the method to deep spectroscopy driven by the future large submillimeter single-dish telescopes (e.g., LST, AtLAST), where PSW observations by mechanical antenna or mirror drive are difficult to achieve.

Akio Taniguchi

August 18, 2021
Tweet

More Decks by Akio Taniguchi

Other Decks in Science

Transcript

  1. A data-scienti fi c noise-removal method for ef fi cient

    submillimeter spectroscopy with single-dish telescopes Akio TaniguchiʢPostdoc, Astrophysics lab, Nagoya University) 2021-08-18 ALMA-J Seminar Based on: Taniguchi+19 PASJ (arxiv:1911.02574) and Taniguchi+21 AJ in press (arxiv:2107.06290)
  2. 涯㖑㔳խ傈劤⴨䃊խ⚥鿇㖑倯 h濶鿇牂 Self introduction 2 || Research activities • 2013

    - 2018: Kohno lab, IoA, Mitaka, University of Tokyo (Ph.D.) • 2018 - 2021: Astrophysical lab, Nagoya University (Postdoc) || Recent works and research interests • DESHIMA - an ultra-wideband on-chip spectrometer • Application of data-scienti fi c methods to data / observations '96-'09 '09-'13 '90-'96 '13-'18 '18-'21 BS, MS, Ph.D Postdoc (DESHIMA) Home town
  3. Self introduction: DESHIMA - an ultra-wideband submm spectrometer 3 ||

    DESHIMA 1.0 • A fi rst-light experiment of on-chip spectrometer with ~50-GHz (322 - 377 GHz) bandwidth || Commissioning campaign on the ASTE 10-m • Detection of molecular line and continuum emission from bright astronomical objects • Excellent agreement between on-sky measurements, lab measurements, and the design DESHIMA DESHIMA x
  4. Self introduction: DESHIMA - an ultra-wideband submm spectrometer 3 ||

    DESHIMA 1.0 • A fi rst-light experiment of on-chip spectrometer with ~50-GHz (322 - 377 GHz) bandwidth || Commissioning campaign on the ASTE 10-m • Detection of molecular line and continuum emission from bright astronomical objects • Excellent agreement between on-sky measurements, lab measurements, and the design Antenna Filters MKIDs DESHIMA DESHIMA x
  5. Self introduction: DESHIMA - an ultra-wideband submm spectrometer 4 ||

    DESHIMA 1.0 • A fi rst-light experiment of on-chip spectrometer with ~50-GHz (322 - 377 GHz) bandwidth || Commissioning campaign on the ASTE 10-m • Detection of molecular line and continuum emission from bright astronomical objects • Excellent agreement between on-sky measurements, lab measurements, and the design DESHIMA DESHIMA readout signal line signal antenna astronomical signal line lens filter MKID aluminium absorber NbTiN ground plane 49 channels sky signal (line frequency) readout signal (5.1-6.0 GHz) sky signal (other frequencies) sky signal & readout signal filter of line frequency a b 49 channels CO(3-2) redshift z 0.02 0 (ALMA) Antenna Filters MKIDs Endo et al. 2019b Redshifted CO (3-2) from an extra-galaxy VV114 45 GHz (49 channels)
  6. Self introduction: DESHIMA - an ultra-wideband submm spectrometer 5 ||

    DESHIMA 1.0 • A fi rst-light experiment of on-chip spectrometer with ~50-GHz (322 - 377 GHz) bandwidth || Commissioning campaign on the ASTE 10-m • Detection of molecular line and continuum emission from bright astronomical objects • Excellent agreement between on-sky measurements, lab measurements, and the design CO(3-2) 345.8 GHz HCN(4-3) 354.5 GHz HCO+(4-3) 356.7 GHz a b c d CH3OH SO 5 arcmin~0.6 pc CO HCN HCO + e g f Flux density (Jy) CO(3-2) 345.8 GHz HCN(4-3) 354.5 GHz a b c d 5 arcmin~0.6 pc e g f Antenna Filters MKIDs DESHIMA DESHIMA Orion KL region Endo et al. 2019b 40 GHz
  7. Self introduction: Large submillimeter telescope (LST) 6 || Wide fi

    eld-of-view, frequency coverage, and high cadence surveys with the large (D~50 m) submillimeter single-dish telescopes (e.g., LST, AtLAST) Submm Transients Nearby Galaxies Astrochemistry ? Super-massive Black Holes Galactic Plane Deep Extragalactic/Cosmological Survey Clusters of Galaxies Magellanic Clouds Planetary atmosphere Unknown Unknowns WIDE-FIELD SPECTROSCOPIC/POLARIMETRIC IMAGING HIGH-CADENCE SUBMM VLBI WIDE-FIELD HIGH-CADENCE IMAGING WIDE-FIELD MULTI-CHROIC IMAGING AREA CADENCE FREQUENCY INSTANTANEOUS IMAGING/SPECTROSCOPY © Y. Tamura
  8. This talk: data scienti fi c methods towards LST 7

    || Getting new knowledge • from small (or incomplete) data (sparse modeling for super-resolution images) • from (extremely) big data (classi fi cation of celestial objects by machine learning) || Handling "big data" • Ef fi cient detection or identi fi cation (background-foreground separation) • Automation (queue observations, pipeline processing, database) particular, the raw reconstructed images in Figure 6 clearly show that smooth edges in the ground-truth images, which are attributed to a smooth transition in the emissivity and opacity of the plasma in the accretion flow, are much better reconstructed with ℓ1 +TSV regularization. As a consequence of this, the TSV term comes reproduces a much clearer shadow feature in the reconstructed images. For the Free-fall model, the size of the black hole shadow is larger in the ℓ1 +TSV image that the isoTV term and gets closer to the ground truth than the isoTV term. For sub-Keplerian and Keplerian models, the black hole shadow is visible in the ℓ1 +TSV images but is mostly obscured (except for the darker funnel region) in the ℓ1 +isoTV methods. The appearance of the reconstructed images indicates that ℓ1 +TSV regularization is justified based on a more physically reasonable assumption and is therefore more suitable to image the objects seen in many astronomical observations. In the following subsections, we evaluate the images more quantita- tively with the image fidelity metrics described in Section 4. 5.3. NRMSE Analysis and Optimal Beam Sizes In Figure 7, we evaluate the NRMSE metric on the image domain and its gradient domain over various spatial scales, as in previous work (Chael et al. 2016; Akiyama et al. 2017a, 2017b). The black curves represent the ideal NRMSE curves between the original (unconvolved) ground-truth image and the ground-truth image after convolution with a Gaussian beam scaled to each resolution on the horizontal axis. These curves represent the highest fidelity available at a given resolution, as would be provided by an algorithm that reconstructs the image Figure 5. Ground-truth image (left-most) and images reconstructed with CS-CLEAN (second from left), ℓ1 +isoTV (second from right), and ℓ1 +TSV (right-most) regularization. All reconstructed images are convolved with elliptical Gaussian beams represented by the yellow ellipses, for which the size corresponds to the optimal resolution determined with the image-domain NRMSE curve in Figure 7 (see Section 5.3). The same transfer function is adopted for four images of each model (i.e., on each row). 9 The Astrophysical Journal, 858:56 (14pp), 2018 May 1 Kuramochi et al. the rank and cardinality of the low-rank and sparse matrices, respectively. For the low-rank matrix L, we check the distribution of singular values, applying SVD, as shown in the main panel of Figure 2. The matrix L is expressed as = L UDVT , where U and V are orthogonal matrices, and D is a diagonal one. Then, we set the rank of the low-rank matrix by setting zeros for the singular values at indices larger than the rank. Within the sparse matrix S, the transient events can be easily extracted because these events are innately sparse. The time variation of the sky background can be monitored by checking the noise matrix G. After the data matrix M has been decomposed into the three matrices L, S, and G, further data processing is necessary, because otherwise the data size would be three times larger than that of the original data. The low-rank matrix L is easily compressed into three small matrices, as shown in Figure 1. For the sparse matrix S, the frames that contain a transient event(s) must be preserved, movie data, due to the speed of computation and memory consumption. We have rewritten the GoDec code in C++, utilizing the OpenBLAS5 and LAPACK6 libraries. We use Quick Select, instead of full sorting, to select non-zero elements for a sparse matrix in the GoDec algorithm. 3. APPLICATION OF THE PROPOSED METHOD We used the movie data set of a CMOS sensor for 400 frames obtained with the Tomo-e PM in 2015 December, which contains some transient events lasting for a short duration (Ohsawa et al. 2016). Panels (a) and (e) of Figure 3 show the subarray images with 300×300 pixels in two different time frames, which contained a transient point source and a meteor, respectively. We applied the decomposition to the data by setting r=10 and = ´ k 1 108. Panels (b)–(d) and Figure 3. Example decomposition images for movie data of the Tomo-e Gozen from two frames (top and bottom rows). Original (denoted as the matrix M in the main text), low-rank (L), sparse (S), and noise (G) images are shown in the four columns from left to right, respectively. A transient point source appears near the center of the image at the time frame of the top row, as spotted in the original image (a), in contrast to (e), which was taken in a different frame (bottom row), and as clearly visible in the sparse one (c), in contrast to (g). On the other hand, a line, which is a light trail caused by a meteor, is seen in the second time frame (bottom row), as in the original image (e) and the sparse one (g). These transient events are not recognized in the low-rank images (b) and (f). The noise images (d) and (h) do not contain any noticeable patterns. The Astrophysical Journal, 835:1 (5pp), 2017 January 20 Morii et al. Galaxy Zoo: Classifying Galaxies with Crowdsourcing and Active Learning Low-rank + sparse decomposition (Morii+2017) Sparse modeling (Kuramochi+2018, ...)
  9. This talk: data scienti fi c methods towards LST 8

    || Data science as a new research fi eld of instrumentation • improvement of sensitivity per unit time without developing hardware || Future application of data-scienti fi c methods • essential for handling "big data" from large submillimeter telescopes ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon Sensitivity (lower is better)
  10. This talk: data scienti fi c methods towards LST 8

    || Data science as a new research fi eld of instrumentation • improvement of sensitivity per unit time without developing hardware || Future application of data-scienti fi c methods • essential for handling "big data" from large submillimeter telescopes Site environment (atmospheric transmission) ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon Sensitivity (lower is better)
  11. This talk: data scienti fi c methods towards LST 8

    || Data science as a new research fi eld of instrumentation • improvement of sensitivity per unit time without developing hardware || Future application of data-scienti fi c methods • essential for handling "big data" from large submillimeter telescopes Instruments (larger dish, more pixels, lower noise, ...) Site environment (atmospheric transmission) ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon Sensitivity (lower is better)
  12. This talk: data scienti fi c methods towards LST 8

    || Data science as a new research fi eld of instrumentation • improvement of sensitivity per unit time without developing hardware || Future application of data-scienti fi c methods • essential for handling "big data" from large submillimeter telescopes Instruments (larger dish, more pixels, lower noise, ...) Site environment (atmospheric transmission) ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon Data science • lower the noise factor (√2 → 1) • improve the on- source fraction (ηon = 0.5 → 1) ΔS = 2 kB Tsys Aeff Npixel Δν tobs ηon Sensitivity (lower is better)
  13. Contents 9 • Noise removal has a room for sensitivity

    improvement • Signal separation by statistical methods Data science and noise-removal methods Observations with noise-removal methods • Frequency modulation: off-point-less observations • Fast-sampled position-switching: of fl ine processing Future application of data-scienti fi c methods • Big-data from the future large single-dish telescopes • Noise removal for ultra-wideband instruments
  14. Sky-removal is crucial in ground-based telescopes 10 || Observation setup

    of a typical submillimeter single-dish telescope dusty star-forming galaxy (target) ASTE 10-m DESHIMA Chajnantor, Chile (alt. 4860 m) atmosphere Earth's astronomical signal atmospheric noise
  15. Issues on sky-removal using the position-switching (PSW) 11 || Transmission

    of sky has dependence on frequency and time • ON-OFF position-switching (PSW) is simple but dif fi cult to go deeper • Baseline wiggles often occur, noise level gets worse by a factor of √2 || Removing the atmospheric emission from observed data • Essential for getting pure, transmission-corrected astronomical signals Tsky (ν) = T⋆ a (ν) ηatm (ν, t) + Tatm [1 − ηatm (ν, t)] observable want to get transmission (should be removed) → Intensity frequency atmosphere + signal (ON) Intensity only atmosphere (OFF) frequency astronomical signal Intensity frequency
  16. Issues on sky-removal using the position-switching (PSW) 11 || Transmission

    of sky has dependence on frequency and time • ON-OFF position-switching (PSW) is simple but dif fi cult to go deeper • Baseline wiggles often occur, noise level gets worse by a factor of √2 || Removing the atmospheric emission from observed data • Essential for getting pure, transmission-corrected astronomical signals Tsky (ν) = T⋆ a (ν) ηatm (ν, t) + Tatm [1 − ηatm (ν, t)] observable want to get transmission (should be removed) → Intensity frequency atmosphere + signal (ON) Intensity only atmosphere (OFF) frequency astronomical signal Intensity frequency
  17. Data-scienti fi c noise removal methods 12 || Step 1:

    continuous time-series spectra at high sampling rate • Perfect (Nyquist) sampling of the time variation of the atmosphere (~1 Hz) || Step 2: modulation of astronomical signals on time-series spectra • Making emission lines uncorrelated between spectra 㱻 atmosphere (correlated) || Step 3: separation of astronomical signals by statistical methods • Modeled atmospheric emission is removed → no √2 worsening in sensitivity! Freq. channels Observation time ON OFF ON OFF ON OFF OFF Continuous time-series spectra (proposed)
  18. Data-scienti fi c noise removal methods 12 || Step 1:

    continuous time-series spectra at high sampling rate • Perfect (Nyquist) sampling of the time variation of the atmosphere (~1 Hz) || Step 2: modulation of astronomical signals on time-series spectra • Making emission lines uncorrelated between spectra 㱻 atmosphere (correlated) || Step 3: separation of astronomical signals by statistical methods • Modeled atmospheric emission is removed → no √2 worsening in sensitivity! Freq. channels Observation time ON OFF ON OFF ON OFF OFF Continuous time-series spectra (proposed) Position switching (PSW) spectra (conventional)
  19. Data-scienti fi c noise removal methods 13 || Step 1:

    continuous time-series spectra at high sampling rate • Perfect (Nyquist) sampling of the time variation of the atmosphere (~1 Hz) || Step 2: modulation of astronomical signals on time-series spectra • Making emission lines uncorrelated between spectra 㱻 atmosphere (correlated) || Step 3: separation of astronomical signals by statistical methods • Modeled atmospheric emission is removed → no √2 worsening in sensitivity! spectrometer channels observation time spectrometer channels observation time sky position 1st LO freq. Fast-sampled position-switching Frequency modulation (FM)
  20. Data-scienti fi c noise removal methods 13 || Step 1:

    continuous time-series spectra at high sampling rate • Perfect (Nyquist) sampling of the time variation of the atmosphere (~1 Hz) || Step 2: modulation of astronomical signals on time-series spectra • Making emission lines uncorrelated between spectra 㱻 atmosphere (correlated) || Step 3: separation of astronomical signals by statistical methods • Modeled atmospheric emission is removed → no √2 worsening in sensitivity! spectrometer channels observation time spectrometer channels observation time sky position 1st LO freq. Fast-sampled position-switching Frequency modulation (FM)
  21. Signal separation is well-studied in signal processing 14 || Prior

    information about signal and background is often used • smoothness: adjacent pixels in an image should have similar values • sparseness: the number of signal-containing pixels should be very small • low-rankness: the small number of eigenvectors should represent an image || Time-series spectra can be considered a monochromatic image • atmospheric emission → low-rank; astronomical signals → sparse Separation of people from a movie using low-rank and sparse priors (Zhou and Tao 2011) → + +
  22. Signal separation is well-studied in signal processing 14 || Prior

    information about signal and background is often used • smoothness: adjacent pixels in an image should have similar values • sparseness: the number of signal-containing pixels should be very small • low-rankness: the small number of eigenvectors should represent an image || Time-series spectra can be considered a monochromatic image • atmospheric emission → low-rank; astronomical signals → sparse Separation of people from a movie using low-rank and sparse priors (Zhou and Tao 2011) → + +
  23. Signal separation is well-studied in signal processing 15 || Prior

    information about signal and background is often used • smoothness: adjacent pixels in an image should have similar values • sparseness: the number of signal-containing pixels should be very small • low-rankness: the small number of eigenvectors should represent an image || Time-series spectra can be considered a monochromatic image • atmospheric emission → low-rank; astronomical signals → sparse TC STDC Proposed (TV) Proposed (SV) FBCP-MP Incomplete IALM LTVNN boon Barbara Facade House Lena Pep Original Damaged Restored Restoration of an image using smooth and low-rank priors (Yokota, Zhao, and Cichocki 2016) → →
  24. ena Peppers Giant Wasabi Sailboat Signal separation is well-studied in

    signal processing 16 || Prior information about signal and background is often used • smoothness: adjacent pixels in an image should have similar values • sparseness: the number of signal-containing pixels should be very small • low-rankness: the small number of eigenvectors should represent an image || Time-series spectra can be considered a monochromatic image • atmospheric emission → low-rank; astronomical signals → sparse Original Damaged Restored Restoration of an image using smooth and low-rank priors (Yokota, Zhao, and Cichocki 2016) → →
  25. Signal separation is well-studied in signal processing 17 || Prior

    information about signal and background is often used • smoothness: adjacent pixels in an image should have similar values • sparseness: the number of signal-containing pixels should be very small • low-rankness: the small number of eigenvectors should represent an image || Time-series spectra can be considered a monochromatic image • atmospheric emission → low-rank; astronomical signals → sparse Separation of emission lines from data using low-rank and sparse priors (Taniguchi et al. 2021) Atmosphere Emission lines Photon noise Observed data
  26. Mini summary: data-scienti fi c noise-removal methods 18 || Data-scienti

    fi c noise-removal methods • signal processing: separation of emission lines from time-series spectra • advantages: √2-times better sensitivity, stable baseline, easy application || Requests for observations arising from the methods • fast spectrum sampling: ~1 sample/s (faster than switching interval) • fast frequency-setup change: only for frequency modulation observations Separation of emission lines from data using low-rank and sparse priors (Taniguchi et al. 2021) Atmosphere Emission lines Photon noise Observed data
  27. Contents 19 • Noise removal has a room for sensitivity

    improvement • Fast-sampled position-switching: of fl ine processing Future application of data-scienti fi c methods • Big-data from the future large single-dish telescopes • Noise removal for ultra-wideband instruments
  28. FMLO: frequency modulation observations 20 || Frequency modulation (FM) observing

    method • FMLO: FM is achieved by changing 1st local-oscillator (LO) frequency at 10 Hz • signal separation using low-rank prior (by principal component analysis; PCA) || No off-source position observations • high on-source fraction of ~90% → 3-4 times more ef fi cient than PSW • no contamination from "line emission at the OFF-point" if exists https://youtu.be/UbJu1q_3HTw →
  29. FMLO: frequency modulation observations 20 || Frequency modulation (FM) observing

    method • FMLO: FM is achieved by changing 1st local-oscillator (LO) frequency at 10 Hz • signal separation using low-rank prior (by principal component analysis; PCA) || No off-source position observations • high on-source fraction of ~90% → 3-4 times more ef fi cient than PSW • no contamination from "line emission at the OFF-point" if exists https://youtu.be/UbJu1q_3HTw →
  30. FMLO: frequency modulation observations 20 || Frequency modulation (FM) observing

    method • FMLO: FM is achieved by changing 1st local-oscillator (LO) frequency at 10 Hz • signal separation using low-rank prior (by principal component analysis; PCA) || No off-source position observations • high on-source fraction of ~90% → 3-4 times more ef fi cient than PSW • no contamination from "line emission at the OFF-point" if exists https://youtu.be/UbJu1q_3HTw →
  31. FMLO: frequency modulation observations 21 || Frequency modulation (FM) observing

    method • FMLO: FM is achieved by changing 1st local-oscillator (LO) frequency at 10 Hz • signal separation using low-rank prior (by principal component analysis; PCA) || No off-source position observations • high on-source fraction of ~90% → 3-4 times more ef fi cient than PSW • no contamination from "line emission at the OFF-point" if exists Observation time O N O F F O F F O N O F F O N ɾɾɾ PSW ON-OFF moving time Comparison of on-source time between PSW and FMLO methods
  32. FMLO: frequency modulation observations 21 || Frequency modulation (FM) observing

    method • FMLO: FM is achieved by changing 1st local-oscillator (LO) frequency at 10 Hz • signal separation using low-rank prior (by principal component analysis; PCA) || No off-source position observations • high on-source fraction of ~90% → 3-4 times more ef fi cient than PSW • no contamination from "line emission at the OFF-point" if exists Observation time O N O F F O F F O N O F F O N ɾɾɾ PSW FMLO O N Comparison of on-source time between PSW and FMLO methods
  33. FMLO: iterative signal separation using PCA 22 1st iteration 2nd

    iteration Tcal ̂ Tcor ̂ Tast ̂ Enc Tcal − ̂ Tast ̂ Tcor ̂ Tast ̂ Enc … flow of estimation matrix operation t νIF subtraction subtraction Emission line partially remains in the 1st iteration Emission line is almost removed in the 2nd iteration
  34. FMLO: observation with the Nobeyama 45m telescope 23 || FMLO

    and PSW are consistent in both spectrum and map • atmospheric ozone lines are detected but can be removed || FMLO is 3.0x (point source) or 1.2x (map) more ef fi cient than PSW • FMLO observation goes 1.7x (point source) or 1.1x (map) deeper during a period FMLO Conventional OTF CS TA * (K) Obs. frequency (GHz) 98.0 98.4 98.2 97.6 97.8 97.4 97.2 0 2 4 6 CH3OH SO2 + 34SO OCS Spectrum within r<30" of Orion KL (Taniguchi et al. 2019)
  35. FMLO: observation with the Nobeyama 45m telescope 24 || FMLO

    and PSW are consistent in both spectrum and map • atmospheric ozone lines are detected but can be removed || FMLO is 3.0x (point source) or 1.2x (map) more ef fi cient than PSW • FMLO observation goes 1.7x (point source) or 1.1x (map) deeper during a period Ori-KL Ori-KL Conventional OTF FMLO Taniguchi et al. 2020a (PASJ) Integrated intensity maps of CS (2-1) around Orion-KL (Taniguchi et al. 2019)
  36. fPSW: fast-sampled position-switching observations 25 || Fast-sampled position-switching observing method

    • fast (~1 sample/s) spectrum sampling during a PSW observation • signal separation using low-rank and sparse priors by GoDec (Zhou and Tao 2011) || √2-times better sensitivity by of fl ine signal processing • No additional systems are required (can be applied to the past observations!)
  37. fPSW: fast-sampled position-switching observations 25 || Fast-sampled position-switching observing method

    • fast (~1 sample/s) spectrum sampling during a PSW observation • signal separation using low-rank and sparse priors by GoDec (Zhou and Tao 2011) || √2-times better sensitivity by of fl ine signal processing • No additional systems are required (can be applied to the past observations!)
  38. fPSW: observation with the LMT 50m telescope 26 || Observation

    of a high-z galaxy's CO (4-3) line with LMT / B4R • sampling rate: 1 Hz, on-source time: 10 s x 30, off-source time: 10 s x 30 || 1.67 (> √2) times better sensitivity was achieved compared to PSW • Both (1) no direct ON-OFF subtraction and (2) removal of baseline wiggles Conventional position-switch data-reduction Proposed new data-reduction using GoDec 1.67-times improvement!
  39. fPSW: observation with the LMT 50m telescope 27 || Observation

    of a high-z galaxy's CO (4-3) line with LMT / B4R • sampling rate: 1 Hz, on-source time: 10 s x 30, off-source time: 10 s x 30 || 1.67 (> √2) times better sensitivity was achieved compared to PSW • Both (1) no direct ON-OFF subtraction and (2) removal of baseline wiggles ON-OFF subtraction only ON-OFF + linear baseline fi t Proposed new method
  40. Mini summary: new observing methods for noise removal 28 PSW

    Frequency modulation Fast-sampled PSW Statistical method direct subtraction iterative PCA GoDec Real-time sky-removal ✔︎ ✔︎ On-source fraction < 0.5 > 0.9 ~ 0.5 Sensitivity improvement compared to PSW 1 2 √2 No need upgrading instruments? - ✔︎ Free from off-source contamination? - ✔︎ Limitations? - continuum, broad line emission continuum, line forest
  41. Contents 29 • Noise removal has a room for sensitivity

    improvement fi c methods • Big-data from the future large single-dish telescopes • Noise removal for ultra-wideband instruments
  42. Large submillimeter telescope (LST) 30 || Wide fi eld-of-view, frequency

    coverage, and high cadence surveys with the large (D~50 m) submillimeter single-dish telescopes (e.g., LST, AtLAST) Submm Transients Nearby Galaxies Astrochemistry ? Super-massive Black Holes Galactic Plane Deep Extragalactic/Cosmological Survey Clusters of Galaxies Magellanic Clouds Planetary atmosphere Unknown Unknowns WIDE-FIELD SPECTROSCOPIC/POLARIMETRIC IMAGING HIGH-CADENCE SUBMM VLBI WIDE-FIELD HIGH-CADENCE IMAGING WIDE-FIELD MULTI-CHROIC IMAGING AREA CADENCE FREQUENCY INSTANTANEOUS IMAGING/SPECTROSCOPY © Y. Tamura
  43. Big data from large submm single-dish telescopes 31 || Application

    of data science in astronomy is inevitable! • Petabyte (PB) class data in optical-to-infrared telescopes in 2020 • (Sub)millimeter telescopes may output terabyte (TB) class data as well ... Figure 4. Detector counts of several field leading millimetre or submillimetre-wave direct-detection instruments, using bolometers, transition edge sensors, or kinetic inductance detectors. The data were compiled from an amalgam of publications and websites, and are shown here solely to illustrate a trend. The best-fit log-linear relation implies the number of detectors can increase by an order of magnitude roughly every seven years, reaching the megapixel regime circa 2032. Our projection for a wide-field, Cassegrain-mounted first-generation camera, which we refer to generically as ‘AtLAST Cam,’ is marked with a red star. Acronyms are defined in Table A in the appendix. performance) operational goal for each instrument, and develop the technical requirements to meet that goal. We will mature this further during the design study, but at the moment we consider the following operational goals as driving: 1) full band spectral mapping of extended sources (more than a few arcminutes) as the driving goal for the high resolution spectrometer, 2) detection of high redshift galaxies for the continuum camera, and 3) redshift determination of detected galaxies for the low resolution spectrometer. Broadly, these three driving goals determine the frequency band allocation within the limited feed horn count of the high resolution spectrometer and limited focal plane area of the continuum camera, and also optimise the spectral resolution # of pixels in continuum cameras neutrino observatories, which will produce tens of events per hour (Reitze 2019). This will require developing the cyberinfrastructure needed to combine several large-area follow-up surveys (i.e., LSST and ZTF) with real-time alerts (LIGO/Virgo, IceCube, and LISA) and analysis software tools. The white papers above provide concrete examples of how large data sets will be vital to make progress in specific science areas spanning astrophysics. Moreover, in an additional series of 6 science white papers, Fabbiano et al. (2019) emphasize that many paradigm-shifting discoveries in the 2020s will not be made through well-formulated hypotheses based on knowledge of the time, but rather by an exploratory discovery approach enabled by new telescopes and instrumentation, as well as by high-quality data products in easily accessible and interoperable science archives. Figure 1. The 2020s and beyond will see large increases in data volumes. Approximate expected data volumes in terabytes of selected astronomical observational facilities and surveys are shown as a function of time. Symbols are plotted at the (expected) end of operations. ​Ongoing surveys as of this writing are plotted in 2019 with an arrow. The current size of major data centers are shown on the right axis. 5 KATANA on LST 
 (~1.5M detectors) Estimate of the total data size Klaassen+2020 Desai et al. 2019 1 PB 1 TB 100000 pixels
  44. DESHIMA 2.0: upgrading to cover instantaneous 220-GHz bandwidth 32 ||

    Ready for detection of the atomic carbon lines from bright high-z galaxies 2.0 1.0 [CII] fl ux from a galaxy with LFIR = 5 x 1013 Lsun assuming 5σ detection in an 8 hr observation DESHIMA 1.0 detectable frequency- fl ux space DESHIMA 2.0 detectable frequency- fl ux space! goal baseline assuming 5σ detection in an 8 hr observation
  45. DESHIMA 2.0: ultra-wideband noise removal method 33 Alle-Jan van der

    Veen (Circuits and Systems Group) A. Endo (Terahertz Sensing Group) Stefanie. Brackenhoff A. Taniguchi || Challenge to ultra-wideband (UWB) x long-integration spectroscopy • Separation of astronomical signals from observed data that contain intense sky emission • Low-rank and additive matrix separationʢ : Brackenhoff et al. in prepʣ • Developing and verifying the method using simulated data for DESHIMA 2.0
  46. DESHIMA 2.0: ultra-wideband noise removal method 34 || End-to-end system

    noise simulator (TiEMPO; Huijten+2020) • Simulated time-series spectra of various observation modes can be made || Novel data-scienti fi c method to estimate signals (SPLITTER; Brackenhoff+) • "Detection" of line and continuum emission with a √2 times better sensitivity Simulated noise budgets of DESHIMA 2.0 Noise (W Hz-1/2)
  47. DESHIMA 2.0: ultra-wideband noise removal method 34 || End-to-end system

    noise simulator (TiEMPO; Huijten+2020) • Simulated time-series spectra of various observation modes can be made || Novel data-scienti fi c method to estimate signals (SPLITTER; Brackenhoff+) • "Detection" of line and continuum emission with a √2 times better sensitivity Simulated noise budgets of DESHIMA 2.0 Noise (W Hz-1/2)
  48. DESHIMA 2.0: ultra-wideband noise removal method 35 || End-to-end system

    noise simulator (TiEMPO; Huijten+2020) • Simulated time-series spectra of various observation modes can be made || Novel data-scienti fi c method to estimate signals (SPLITTER; Brackenhoff+) • "Detection" of line and continuum emission with a √2 times better sensitivity Detection of simulated line and continuum emission using SPLITTER molecular/atomic lines
  49. Summary 36 • Noise removal has a room for sensitivity

    improvement • Signal separation by statistical methods Data science and noise-removal methods Observations with noise-removal methods • Frequency modulation: off-point-less observations • Fast-sampled position-switching: of fl ine processing Future application of data-scienti fi c methods • Big-data from the future large single-dish telescopes • Noise removal for ultra-wideband instruments
  50. Summary 37 • Noise removal has a room for sensitivity

    improvement • Signal separation by statistical methods Data science and noise-removal methods Observations with noise-removal methods • Frequency modulation → arxiv:1911.02574 • Fast-sampled position-switching → arxiv:2107.06290 Future application of data-scienti fi c methods • Big-data from the future large single-dish telescopes • Noise removal for ultra-wideband instruments