shampoo for SHAMU: Digital Holographic Microscopy for Astrobiology

NASA’s 2006 Solar System Exploration Roadmap and 2008 Astrobiology Road map elevate
Saturn’s icy moon Enceladus to the ranks of Titan and Europa as a top-priority icy world worthy
of study for astrobiology. Enceladus exhibits cryovolcanism which presents an opportunity for a future orbiter or lander to directly sample ice that originated in Enceladus’ warm interior. Cassini mass spectrometry of Enceladus’ plumes reveals organic matter, begging the question: how can we differentiate between biosignatures – the effects of metabolic processes on an environment, for example – and signs of extant life in the solar system?


In my research rotation project, I was privileged to join a team of interdisciplinary experts on microbial motility and holographic microscopy who seek to test the hypothesis: “is microbial motility an unambiguous biosignature of extant life?” We know that organisms on Earth have innumerable reasons to move; for example, organisms move towards nutrients, away from toxins, towards heat, towards light. Perhaps life on worlds like Enceladus would evolve motility as well. One advantage of searching for microbial motility, rather than searching for ATP or DNA, is that the motility hypothesis is agnostic about the chemistry of the organisms that are moving, so long as the organisms move.


We are testing this hypothesis by studying motile terrestrial microbes with SHAMU – or the “Submersible Holographic Astrobiology Microscope with Ultraresolution” led by PI Jay Nadeau, (Portland State University). SHAMU is a digital holographic microscope (DHM) with ~1 micron resolution. The hardware required is compact, requires comparatively little energy and no moving parts, which makes it a good candidate for future space exploration missions. 


Digital holographic microscopes use interferometry to measure perturbations imparted on the phase and amplitude of a laser beam by small specimens – on scales similar to the wavelength of the laser (300 nm). The holograms recorded by the DHM encode the full phase and amplitude of the light at every position in the fluid sample chamber, allowing us to locate the 3D positions of specimens in each hologram. Time-series holography allows us to measure the full 3D velocities of the specimens as they move with bulk flow in the fluid, and as they swim. Unambiguously detecting this difference between bulk flow and swimming – algorithmically, and with minimal human intervention – is the near-term goal of the SHAMU project.


One of the primary obstacles to achieving this goal is a computational bottleneck. Each hologram recorded by the microscope must be processed in order to recover the phase and amplitude information – this process is called hologram reconstruction. When I began working with the SHAMU team, all reconstructions were completed with proprietary, “black-box” software (called Koala Acquisition & Analysis by LyncéeTec), for which the group had a limited number of licenses. It could take several days for the software to finish reconstructions for an hour’s worth of microscope observations. 


My contribution to the SHAMU project was to create the first open source numerical holographic reconstruction toolkit in Python, which could be used alongside or in place of the proprietary software. There are several advantages to creating our own software pipeline: (1) we know what the software is doing at each step in the reconstruction process, (2) we can tweak the reconstruction algorithm to suit our motility observations, (3) the free software is platform-independent and can be used on as many machines as you have access to (no license necessary), (4) despite being written in Python, which is generally thought of as a “slow” language, our pipeline is more efficient than the alternative package with limited licenses, because you can run our free software with many reconstruction jobs in parallel, greatly reducing the wall clock time for a given reconstruction.  


Many of the tasks necessary for numerical reconstruction of digital holograms are performed manually by microscope users. For example, the user might be required to create a real-image mask in Fourier-space, and to select background regions of a reconstructed image, which are used to train aberration correction routines. These steps are redundant if the holographic microscope configuration that doesn't change significantly from one observation to the next, and if the area of the image occupied by specimens is typically small compared to the background – as is the case when working with unresolved specimens. The software I developed takes advantage of these opportunities to automate the reconstruction and aberration correction processes. 


The software package, dubbed “shampoo”, has been adopted by the SHAMU group, and I continue to lead code development and maintenance in collaboration with J. Kent Wallace and Santos Fregoso (NASA JPL). We are in the early stages of writing up a paper for submission to the journal Applied Optics on the many design decisions that went into our custom-tailored reconstruction algorithm. The source code is freely available on GitHub*. 


In this project I learned so much that it’s hard to summarize concisely, thanks to the guidance of my advisor Professor Jody Deming, her graduate student Max Showalter, and the rest of the SHAMU team.

* Source code: https://github.com/bmorris3/shampoo

Transcript

1. RV Thomas G Thompson, 2014
2. RV Thomas G Thompson, 2014 SBU, 2006
3. shampoo for SHAMU: Digital Holographic Microscopy for Astrobiology with Gordon Max Showalter, UW Chris Lindensmith, JPL J. Kent Wallace, JPL Kurt Liewer, JPL Manuel Bedrossian, Caltech Brett Morris Prof. Jody Deming (UW) Prof. Jay Nadeau (Portland State)
5. NASA’s Europa Clipper Mission: launch in 2020’s
6. NASA/ESA/L. Roth/SWRI/University of Cologne
8. Organics Detection at Enceladus Sample Collection Gas chromatograph and mass spectrometer Waite Jr et al. 2006
9. Organics Detection at Enceladus Sample Collection Gas chromatograph and mass spectrometer LETHAL PROCESS Waite Jr et al. 2006
10. Life Detection at Europa Sample Collection Gas chromatograph and mass spectrometer Non-lethal, in situ life detection experiment SHAMU: Waite Jr et al. 2006
11. SHAMU: Submersible Holographic Astrobiology Microscope with Ultraresolution Can we use microbial motility as a biosignature?
12. SHAMU: Submersible Holographic Astrobiology Microscope with Ultraresolution Motility as a biosignature
13. SHAMU: Submersible Holographic Astrobiology Microscope with Ultraresolution Motility as a biosignature
14. Taxis: A Universal Biosignature?
15. Taxis: A Universal Biosignature?
16. Taxis: A Universal Biosignature? Nutrients
17. Taxis: A Universal Biosignature? Nutrients Toxins
18. Taxis: A Universal Biosignature? Nutrients Toxins Taxis
19. Goals:
20. Goals: 0) Use digital holographic microscopes to observe microscopic samples of water from Earth
21. Goals: 0) Use digital holographic microscopes to observe microscopic samples of water from Earth
22. Goals: 0) Use digital holographic microscopes to observe microscopic samples of water from Earth 1) Use software to turn holograms into human- interpretable images
23. Goals: 0) Use digital holographic microscopes to observe microscopic samples of water from Earth 1) Use software to turn holograms into human- interpretable images 2) Identify and track specimens in space and time to detect motility
24. Challenges: 0) What is digital holographic microscopy? 1) The software for processing DHM data is proprietary and expensive 2) Identifying and tracking specimens by hand is expensive
25. SHAMU: Submersible Holographic Astrobiology Microscope with Ultraresolution
26. Mach–Zehnder interferometer Colomb et al. 2006
27. Detour: Interferometry (actually LIGO)
28. Detour: Interferometry (actually LIGO)
29. Mach–Zehnder interferometer Colomb et al. 2006
30. Example Hologram US Air Force Resolution Target
31. SHAMU raw image or hologram h=
32. SHAMU raw image or hologram Interference Fringe Pattern h=
33. Fourier transform of the impulse response function: L(x, y) = exp ✓ i 2⇡ P2(x, y) ◆ G(n, m) = exp 0 B B @ i 2⇡d v u u t 1 2 n + N 2 x 2 2 d 2 N 2 x 2 2 ⇣ m + N 2 y 2 2 d ⌘2 N 2 y 2 1 C C A Introduce numerical parametric lens: Reconstructed wave: (⇠, ⌘) = F 1{F(ˆ h · L) · G} Numerical Hologram Reconstruction (don’t panic)
34. Fourier transform of the impulse response function: L(x, y) = exp ✓ i 2⇡ P2(x, y) ◆ G(n, m) = exp 0 B B @ i 2⇡d v u u t 1 2 n + N 2 x 2 2 d 2 N 2 x 2 2 ⇣ m + N 2 y 2 2 d ⌘2 N 2 y 2 1 C C A Introduce numerical parametric lens: Reconstructed wave: (⇠, ⌘) = F 1{F(ˆ h · L) · G} Numerical Hologram Reconstruction (don’t panic) 𝝀 = wavelength of laser (resolution ~1 μm) d = distance to specimen
 (one image = multiple foci) 𝚪 = complex waveform (phase and amplitude)
35. Numerical Hologram Reconstruction (don’t panic) Reconstruct at different distances = adjust focus
36. Numerical Hologram Reconstruction (don’t panic) Reconstruct at different distances = adjust focus
37. Re ( (⇠, ⌘)) Im ( (⇠, ⌘)) Intensity Phase Numerical Hologram Reconstruction (don’t panic)
38. Re ( (⇠, ⌘)) Intensity Numerical Hologram Reconstruction: “ultra”-resolution 5 μm
39. Challenges: 0) What is digital holographic microscopy? 1) The software for processing DHM data is proprietary and expensive 2) Identifying and tracking specimens by hand is expensive
40. Numerical Hologram Reconstruction: shampoo
41. Numerical Hologram Reconstruction: shampoo
42. # Set input file, propagation distance:
 hologram_path = 'data/USAF_test.tif'
 propagation_distance = 0.03685 # m
 
 # Construct the hologram object from the raw TIF file
 from shampoo import Hologram
 h = Hologram.from_tif(hologram_path) 
 # Reconstruct the wave
 wave = h.reconstruct(propagation_distance)
 
 # Plot the result
 import matplotlib.pyplot as plt
 wave.plot()
 plt.close() Numerical Hologram Reconstruction: shampoo
43. • Image sharpness metrics (std. dev.) • Fails for small specimen/ background ratio • Identify region of interest, then find min/max in phase/amplitude at each z • Only works for pure phase/ amplitude objects and vibrio, for example, seems to be neither Dubois 2014 Numerical Hologram Reconstruction: Autofocus
44. Autofocus is messy.
45. Challenges: 0) What is digital holographic microscopy? 1) The software for processing DHM data is proprietary and expensive 2) Identifying and tracking specimens by hand is expensive
46. Numerical Hologram Reconstruction Vibrio, through time
47. Numerical Hologram Reconstruction Vibrio, through time
48. Computer vision: convolution-based 2D centroiding
49. Computer vision: convolution-based 2D centroiding
50. Tracks with Machine Learning How do we group the positions into tracks?
51. Tracks with Machine Learning How do we group the positions into tracks?
52. Tracks with Machine Learning Minimum Spanning Tree • Link each point to its nearest neighbor until all points are linked • Choose a height from the top of the tree to truncate into groups
53. Tracks with Machine Learning Minimum Spanning Tree • Dimensions for distance calculation need not be spatial: • Radius, index of refraction, time, etc.
54. Identify Tracks with Machine Learning Color corresponds to track labels from the minimum spanning tree classifier.
55. Identify Tracks with Machine Learning Color corresponds to track labels from the minimum spanning tree classifier.
56. Pioneered by David Grier’s lab at NYU 1. Model the scattering due to a sphere with parameters (x, y, z) position, index of refraction n, radius r 2. Fit scattering profile for those parameters 34H Model I translated Grier’s Lorenz-Mie code to Python, sped it up with a just-in-time compiler (numba) Typical difference between Python and IDL models < 1 ppm Lorenz-Mie Scattering
57. In practice, vibrio and 34H (pictured left) are not perfect fits for the assumptions that go into Mie scattering: • Particle is similar in size to the wavelength (cells may be larger) • Particle is a sphere (vibrio and 34H are very non-spherical) 34H Model Lorenz-Mie Scattering
58. That’s why the 34H signal is asymmetric and rapidly decaying – asymmetric ringing interferes with itself. Modeling each particle as two identical, offset particles produces more realistic model (below) 34H One particle model 34H Two particle model Residuals Lorenz-Mie Scattering
59. What I learned: • Computer vision • Machine learning • Interferometry (with applications to astronomical coronagraphy!) • Experimental design
60. Thanks! • Professor Jody Deming • Max Showalter • Jay Nadeau and Chris Lindensmith • J. Kent Wallace
61. Future work: Writing!