Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Spectrometry, Data Acquisition, and 2D Image Processing

Bruce Ravel
January 07, 2014

Spectrometry, Data Acquisition, and 2D Image Processing

Overview of a high energy resolution fluorescence detection experiment using a bent Laue analyzer and a Pilatus area detector.

Bruce Ravel

January 07, 2014
Tweet

More Decks by Bruce Ravel

Other Decks in Science

Transcript

  1. BLA Measurement Improving on the art Take-away
    Spectrometry, Data Acquisition, and 2D Image Processing
    Bruce Ravel
    Synchrotron Methods Group
    Materials Measurement Laboratory
    National Institute of Standards and Technology
    ISS Weekly Meeting, 11 April, 2012
    1 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  2. BLA Measurement Improving on the art Take-away
    Acknowledgements
    Jeremy Kropf, Argonne Jeremy developed and built the spectrometer and contributed to
    the image processing development
    Gordon Southam, Chris Omelan, Jeremiah Shuster, University of Western Ontario Sample
    prep and help running the experiment
    Matt Newville, University of Chicago Epics help and author of Epics Channel Access for
    Python
    Everything shown here happened at MRCAT, 10ID, APS.
    2 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  3. BLA Measurement Improving on the art Take-away
    A common XAS science problem ...
    Tracking the evolution of a chemical
    transformation is a common sort of
    problem for an XAS beamline.
    In this case, we see the reduction of
    a gold chloride solution to metallic
    gold over the course of many hours.
    Questions
    What is the rate constant?
    Is there an intermediate species?
    3 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing
    M. Lengke et el., Mechanisms of Gold Bioaccumulation by Filamentous
    Cyanobacteria from Gold(III)-Chloride Complex, Environ. Sci. Technol.
    40(20) p. 6304-6309. (2006), DOI: 10.1021/es061040r

    View full-size slide

  4. BLA Measurement Improving on the art Take-away
    ... and its solution
    We solve this by testing linear combinations of measured standards against the
    data. The linear combination that best fits the data is assumed to be the
    constituents of the data.
    In this case, we can model the data as a combination of gold metal, gold
    chloride solution, and a species with a gold-sulphur bond.
    The problem
    Within the statistical limitations of the data,
    we could not distinguish between the various
    sulfur species.
    4 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  5. BLA Measurement Improving on the art Take-away
    XANES at higher energy resolution
    A likely solution to the sulphur species problem
    is high energy resolution fluorescence detection,
    or “HERFD”. (Yes, it really is called HERFD....)
    In HERFD, we use some kind of crystal
    spectrometer to measure energy resolution
    below the core-hole lifetime, which is the
    physical effect dominating the energy resolution
    in a conventional measurement.
    We hope that the enhancement of the features
    enables better distinction between similar
    chemical species.
    5 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  6. BLA Measurement Improving on the art Take-away
    The bent Laue spectrometer
    The analyzer is a thin silicon wafer
    bent over an aluminum frame milled into
    an approximation of a log-spiral.
    This is the right shape for preserving
    the divergence of the beam while using
    diffraction from the crystal planes for
    wavelength discrimination.
    The Pilatus is used for energy
    resolution – photons with different
    wavelengths hit different places on the
    Pilatus.
    6 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing
    The romesco and the plot are public domain images taken from the
    Wikipedia page “Logorithmic Spiral”. The shell: “This Wikipedia and
    Wikimedia Commons image is from the user Chris73 and is freely
    available at http://commons.wikimedia.org/wiki/...
    File:NautilusCutawayLogarithmicSpiral.jpg under the Creative
    Commons cc-by-sa 3.0 license.”

    View full-size slide

  7. BLA Measurement Improving on the art Take-away
    Why bent Laue spectrometry?
    The main reason
    Jeremy has this spectrometer.
    Pros
    Fairly efficient for emission
    energies above 9 keV
    One wafer can cover a large
    energy range (by changing the
    angle)
    Good solution for high energy
    edges
    Very easy to align
    Can be multiplexed for better
    throughput
    Except for the Pilatus, fairly
    inexpensive
    Cons
    Energy resolution limited to
    about 1.5 eV to 2 eV, at best
    Not suitable for transition
    metals which have core-hole
    lifetimes smaller than the
    intrinsic resolution of the
    instrument
    Fairly small solid angle
    subtended per analyzer
    Each analyzer needs its own
    Pilatus
    7 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  8. BLA Measurement Improving on the art Take-away
    The HERFD measurement
    Optics This is an XAS scan, so we moved the monochromator
    with coordinated motion of the undulator
    Focusing We used the functional equivalent of a KB mirror pair,
    which was a non-standard beamline component
    Motors Sample stage + three degrees of freedom for the
    spectrometer (linear rowards sample, crystal rotation,
    rotation of Pilatus arm)
    Sample Solution samples in plastic frames, enclosed with Kapton
    tape
    Scan file File containing columns of I0
    , It
    , ROI from the SDD, some
    areal ROIs from camserver, & a few miscellaneous
    columns. Also a column of metadata!
    Images One image for each energy pointin the XANES scan +
    10-15 images for energy calibration
    8 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  9. BLA Measurement Improving on the art Take-away
    The HERFD measurement
    Optics This is an XAS scan, so we moved the monochromator
    with coordinated motion of the undulator
    Focusing We used the functional equivalent of a KB mirror pair,
    which was a non-standard beamline component
    Motors Sample stage + three degrees of freedom for the
    spectrometer (linear rowards sample, crystal rotation,
    rotation of Pilatus arm)
    Sample Solution samples in plastic frames, enclosed with Kapton
    tape
    Scan file File containing columns of I0
    , It
    , ROI from the SDD, some
    areal ROIs from camserver, & a few miscellaneous
    columns. Also a column of metadata!
    Images One image for each energy pointin the XANES scan +
    10-15 images for energy calibration
    Surprise! Severe radiation damage required moving the sample
    stage in ˆ
    z synchronously during image exposure
    8 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  10. BLA Measurement Improving on the art Take-away
    One energy point
    At each energy, the Pilatus sees something that looks like this:
    This image:
    1 is a tiff file
    2 has 487 × 195 pixels, each containing a signed 32 bit integer
    3 has several bad pixels
    4 is almost 400 kB
    5 somehow represents µ(E) for that value of E
    6 and is in need of further processing
    How do we turn that image into a data point?
    9 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  11. BLA Measurement Improving on the art Take-away
    Energy calibration
    Take a sequence of snapshots with the incident energy in a range
    around the Au Kα1 peak (9713 eV).
    9697 eV
    9705 eV
    9713 eV
    9699 eV
    9707 eV
    9715 eV
    9701 eV
    9709 eV
    9717 eV
    9703 eV
    9711 eV
    9719 eV
    Energy disperses across the face of the Pilatus
    This allows us to assign particular emission energies to particular
    pixels. This gives us good energy resolution.
    10 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  12. BLA Measurement Improving on the art Take-away
    Mask creation
    Here’s another view of the energy
    dispersion.

    I developed a simple algorithm for removing
    bad pixels, scrubbing stray pixels, and filling
    in the gaps.
    This results in a simple “AND” mask. The
    scalar-multiplication of the mask and the
    image is summed over all pixels. This sum is
    the value of µ at that energy.
    So, = 1,292,800 counts (in 20 sec).
    11 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  13. BLA Measurement Improving on the art Take-away
    Putting the pieces together
    A scan, then, consists of
    1 A column data file with scalars and metadata
    2 1 tiff image at each energy point – we were measuring about 100 energy
    points with 20 second integration and about 3 seconds of overhead per
    point, for about 40 minutes per scan.
    3 12 tiff images at incident energies around the emission peak, again with 20
    second integrations, for about 3 more minutes per sample.
    4 In total, each measurement required 47 Mb (uncompressed – gzip
    compresses by 4.48 to 10.5 Mb) of diskspace and 45 minutes of time.
    5 Various efficiencies in spectrometer design could multiply diskspace by N
    and divide time by N to 2N (N = number of analyzers).
    12 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  14. BLA Measurement Improving on the art Take-away
    Data and metadata
    The column data file, Aufoil1.001
    # HERFD scan on Aufoil1
    # ----------------------------------
    # energy time ring_current i0 it ifl ir roi1 roi2 roi3 roi4 tif
    11850.000 20 98.6063252127 1387833 721744 466592 1420285 18 49 16 51 1
    11860.000 20 98.5954416527 1372787 715385 473045 1404234 26 40 19 69 2
    11870.000 20 98.5859850127 1370129 714683 480828 1402733 32 38 19 62 3
    ... etc ...
    energy point tiff files Aufoil1 00001.tif through Aufoil1 00112.tif
    elastic tiff files Aufoil1 elastic 9697 00001.tif through
    Aufoil1 elastic 9719 00001.tif
    13 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  15. BLA Measurement Improving on the art Take-away
    Data and metadata
    The column data file, Aufoil1.001
    # HERFD scan on Aufoil1
    # ----------------------------------
    # energy time ring_current i0 it ifl ir roi1 roi2 roi3 roi4 tif
    11850.000 20 98.6063252127 1387833 721744 466592 1420285 18 49 16 51 1
    11860.000 20 98.5954416527 1372787 715385 473045 1404234 26 40 19 69 2
    11870.000 20 98.5859850127 1370129 714683 480828 1402733 32 38 19 62 3
    ... etc ...
    energy point tiff files Aufoil1 00001.tif through Aufoil1 00112.tif
    elastic tiff files Aufoil1 elastic 9697 00001.tif through
    Aufoil1 elastic 9719 00001.tif
    1 All files have a common stub (Aufoil1)
    2 The final column in the data file identifies the energy point tiff file (and is
    a number)
    3 Processing the energy point tiff file requires a particular elastic tiff file
    13 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  16. BLA Measurement Improving on the art Take-away
    Data and metadata
    The column data file, Aufoil1.001
    # HERFD scan on Aufoil1
    # ----------------------------------
    # energy time ring_current i0 it ifl ir roi1 roi2 roi3 roi4 tif
    11850.000 20 98.6063252127 1387833 721744 466592 1420285 18 49 16 51 1
    11860.000 20 98.5954416527 1372787 715385 473045 1404234 26 40 19 69 2
    11870.000 20 98.5859850127 1370129 714683 480828 1402733 32 38 19 62 3
    ... etc ...
    energy point tiff files Aufoil1 00001.tif through Aufoil1 00112.tif
    elastic tiff files Aufoil1 elastic 9697 00001.tif through
    Aufoil1 elastic 9719 00001.tif
    1 All files have a common stub (Aufoil1)
    2 The final column in the data file identifies the energy point tiff file (and is
    a number)
    3 Processing the energy point tiff file requires a particular elastic tiff file
    This is ridonkulously fragile!
    13 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  17. BLA Measurement Improving on the art Take-away
    Post-processing the data
    Mask creation + processing 113 data points takes 45 seconds on my 4
    year old Dell. In 3 days, we measured 31 possibly-useful samples.
    14 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  18. BLA Measurement Improving on the art Take-away
    In-line image processing
    Processing the whole measurement – 45 seconds
    bla herfd -e=9713 -c=config.ini Aufoil1
    15 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  19. BLA Measurement Improving on the art Take-away
    In-line image processing
    Processing the whole measurement – 45 seconds
    bla herfd -e=9713 -c=config.ini Aufoil1
    Processing just the mask – 15 seconds
    bla mask -e=9713 -c=config.ini Aufoil1
    Each elastic mask can be computed synchronously during the
    following exposure.
    15 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  20. BLA Measurement Improving on the art Take-away
    In-line image processing
    Processing the whole measurement – 45 seconds
    bla herfd -e=9713 -c=config.ini Aufoil1
    Processing just the mask – 15 seconds
    bla mask -e=9713 -c=config.ini Aufoil1
    Each elastic mask can be computed synchronously during the
    following exposure.
    Processing just one point, given a mask – 1 second
    bla point -e=9713 -c=config.ini -p=17 Aufoil1
    Each data point is computed fast enough for in-scan display.
    15 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  21. BLA Measurement Improving on the art Take-away
    In-line image processing
    Processing the whole measurement – 45 seconds
    bla herfd -e=9713 -c=config.ini Aufoil1
    Processing just the mask – 15 seconds
    bla mask -e=9713 -c=config.ini Aufoil1
    Each elastic mask can be computed synchronously during the
    following exposure.
    Processing just one point, given a mask – 1 second
    bla point -e=9713 -c=config.ini -p=17 Aufoil1
    Each data point is computed fast enough for in-scan display.
    Post-processing is only necessary if the user wants to experiment with
    the mask creation algorithm.
    15 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  22. BLA Measurement Improving on the art Take-away
    The wonder of MX
    MRCAT uses a little-used control system called
    MX. MX runs most, but not quite all, of the
    beamline.
    16 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  23. BLA Measurement Improving on the art Take-away
    The wonder of MX
    16 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing
    It has ... ummm ... quirks and some missing
    features. For example, no one present knew how
    to make MX run a motor synchronously quite in
    the way we needed.

    View full-size slide

  24. BLA Measurement Improving on the art Take-away
    The wonder of MX
    The fundamental problem with MX is the same
    as with spec and some other control solutions.
    It’s quirky and suffers from fundamentally
    poor language design.
    It’s not open source.
    It’s incomplete and adding new features
    requires interaction with a person whose
    priorities probably do not match those of
    the beamline scientist.
    Thus MX is a rather hostile environment for the
    clever, creative, motivated beamline scientist.
    16 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  25. BLA Measurement Improving on the art Take-away
    The wonder of MX
    The fundamental problem with MX is the same
    as with spec and some other control solutions.
    It’s quirky and suffers from fundamentally
    poor language design.
    It’s not open source.
    It’s incomplete and adding new features
    requires interaction with a person whose
    priorities probably do not match those of
    the beamline scientist.
    Thus MX is a rather hostile environment for the
    clever, creative, motivated beamline scientist.
    Our solution
    We wrote a python script. The python Mx
    bindings moved the mono and read the scalars.
    Matt Newville’s EPICS bindings talked to the
    Pilatus and controlled the synchronous motor
    motion.
    16 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  26. BLA Measurement Improving on the art Take-away
    Tackling the metadata problem
    Our solution to munging an ASCII file and ∼ 125 tiff images into a XANES
    spectrum involved rigid assumptions about file names. One innocuous change to
    the DAQ python script or one case change in a file name and the
    postprocessing fails.
    The fundamental problem is the reliance on file names and the file system.
    Fortunately better technology exists:
    Hierarchical file HDF5 is a popular solution in some circles. Even a zip file
    would be a defensible solution (consider Open Document Foundation
    formats). HDF5 uses tags to encode the relationships. A zip file
    would require a combination of convention and encapsolated
    documentation.
    Database Data, images, and metadata can be stored in a relational database.
    Reconstuction of the data depends upon a documented table layout.
    17 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  27. BLA Measurement Improving on the art Take-away
    Tackling the metadata problem (2)
    In either case, images should be
    1 collected from the server
    2 compressed (integer tiffs are very compressable)
    3 stored directly in the hierarchical file or the database
    An argument for a database would be
    for better consideration of questions
    that cross-cut many measurements. In
    this example, what is the cause of the
    glitch at about 12045 that appears in
    most data sets?
    18 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  28. BLA Measurement Improving on the art Take-away
    The s32 tiff problem
    The extended tiff standard allows for a mind-boggling variety of data types
    stored at each pixel. Signed 32 bit integers are a perfectly reasonable data
    type that virtually no existing software supports.
    ImageJ reads s32 by not using libtiff. Consequently everyone using a
    Pilatus also uses ImageJ.
    Libtiff, out of the box, does not do the s32. Any viewing application that
    uses libtiff does not read Pilatus images.
    The Windows desktop image viewer does not read s32 tiffs.
    ImageMagick and GraphicsMagick both require a special build to handle
    s32. I was able to compile IM to do so on linux only, but not GM.
    Imager, a Perl extension, flattens s32 numbers into an u24 format. Because
    our pixel values are small and positive definate, this is acceptible.
    My solution
    I use Imager (I also played around with ImageMagick) to import the image
    and immediately stuff it into a Perl Data Language object. All subsequent
    work is done with PDL.
    19 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  29. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  30. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  31. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  32. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  33. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  34. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide

  35. BLA Measurement Improving on the art Take-away
    The take away messages
    The control system must allow for synchronous motion of arbitrary motors.
    We lacked a control framework that is complete, well supported,
    linguistically robust, and accessible to the motivated beamline scientist or
    expert user.
    Prototyping control and acquisition applications at the beamline is
    important.
    Standard, maintained, supported libraries for whatever wonky image format
    the detectors spew out are essential. It must be easy to compile and/or
    install on all the platforms.
    We lacked an image processing framework that can be plugged into a
    measurement loop in interesting ways and used to post-process data.
    Hierarchical files or well-structured databases are superior vehicles to
    named files strewn about the network on various file systems.
    Open Source is good for science. Closed source is bad for science.
    20 / 20
    Spectrometry, Data Acquisition, and 2D Image Processing

    View full-size slide