Managing XAS data across scientific disciplines, across synchrotron facilities, and across decades
My talk for the 2013 SLAC Users' Meeting. I begin with a discussion of why old-school XAFS remains relevant to 21st century, 3rd-generation synchrotron science, the discuss the importance of data format standards to the future of XAFS.
Managing XAS data across scientific disciplines, across synchrotron facilities, and across decades Bruce Ravel Synchrotron Science Group, Materials Measurement Science Division Materials Measurement Laboratory National Institute of Standards and Technology & Local Contact, Beamline X23A2 National Synchrotron Light Source SLAC Users’ Meeting October 1, 2013 Managing XAS data 1 / 24
Copyright This document is copyright c 2010-2013 Bruce Ravel. This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA. You are free: to Share — to copy, distribute, and transmit the work to Remix — to adapt the work to make commercial use of the work Under the following conditions: Attribution – You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). Share Alike – If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. With the understanidng that: Waiver – Any of the above conditions can be waived if you get permission from the copyright holder. Public Domain – Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license. Other Rights – In no way are any of the following rights affected by the license: Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations; The author’s moral rights; Rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights. Notice – For any reuse or distribution, you must make clear to others the license terms of this work. This is a human-readable summary of the Legal Code (the full license). Managing XAS data 2 / 24
Eventually all of these happened And almost 5 dozen others... Today Dale, Ed, and Ferrel’s clever little idea is one of the core competencies of synchrotron science. Managing XAS data 5 / 24
The last 2 years at X23A2 Field visits Nuclear materials 11 Catalysis 8 Batteries 7 Electronic materials 4 Chemical analysis 1 Environmental science 7 Materials science 10 Unique groups 19 PRT time has been used for instrument development and cultural heritage studies, as well as many of the science areas in the GU list. At NSLS, we also have the Synchrotron Catalysis Consortium (3 beamlines), 3 more general purpose hard X-ray XAS, 1 bioXAS, 1 tender X-ray, 2 soft X-ray, and 2 microprobes. Managing XAS data 6 / 24
XAS is used by virtually all scientific disciplines Here’s a somewhat trite way of expressing importance. XAS is used in at least 1400∗ publications each year. This indicates our collective success is 1 evangelizing the technique 2 training scientists in many fields to rely upon it for their research Managing XAS data 7 / 24 ∗ This is certainly a low-ball estimate!
Bigger things: time-resolved studies Energy dispersive XAS and quick-XAS are two ways of doing time-resolved XAS with 10 ms to 10 s time resolution. Both approaches require specially- equipped beamlines and both focus on the dynamics of the system. Managing XAS data 8 / 24 Data from W.A. Caliebe et al., HASYLAB Annual Report (2006) pp. 283-284; EDE schematic from SPring-8 press release, 30 April, 2009; QXAS schematic from SLS SuperXAS beamline webpage.
Bigger things: combinatorial screening Here is a way of performing multiple, concurrent XAS measurements. 0 0.5 1 1.5 2 8320 8340 8360 8380 8400 8420 8440 Normalized Absorption Energy (eV) 0 0.5 1 1.5 2 8320 8340 8360 8380 8400 8420 8440 Normalized Absorption Energy (eV) 0 0.5 1 1.5 2 8320 8340 8360 8380 8400 8420 8440 Normalized Absorption Energy (eV) 0 0.5 1 1.5 2 8320 8340 8360 8380 8400 8420 8440 Normalized Absorption Energy (eV) 400°C in N 2 500°C in N 2 400°C 5% steam in air 500°C 5% steam in air By scaling this concept to the wide swath from a wiggler and using a slew scanning mono, 100s of samples could be screened per hour. Managing XAS data 9 / 24 B. Ravel, et al, J. Synchrotron Rad. (2010) 17, pp. 380-385 DOI: 10.1107/S0909049510006230 S.R. Bare, et al, Phys. Chem. Chem. Phys. (2010) 12, pp. 7702-7711 DOI: 10.1039/B926621F
Fancier things: imaging and µXAS Here is an extraordinary XRF map of a metal hyperaccumulating plant that also forms star-shaped, inorganic nodules on its leaves. While the image is itself a great result, we end up measuring XAS spectra with the microbeam. Managing XAS data 11 / 24 R. Tappero, et al., New Phytologist 175:4 (2007) pp 641-654 DOI:10.1016/10.1111/j.1469-8137.2007.02134.x
Fancier things: DAFS With coordinated motion between monochromator and goniometer, DAFS measures the height of a diffraction peak with respect to energy through the resonant energy of an atom in the crystal. In the end, we extract a site-specific χ(k) function which is analyzed like normal EXAFS. Managing XAS data 12 / 24 B. Ravel et al., Phys. Rev. B 60 (1999) pp 778-785. DOI:10.1103/PhysRevB.60.778
Fancier things: NIXS Here is NIXS data from a non-resonant inelastic scattering measurement on CaZrTi2O7 from 20ID at APS. Again, a XANES spectrum comes from this elaborate experiment. Managing XAS data 13 / 24 Lerix-I instrument: G. Seidler et al.; Data: Thesis of D. Reid, University of Sheffield
Common issues The “bigger” things each involve large volumes of XAS data The “fancy” things each involve XAS data extracted from a larger, multispectral data set We also have persistent problems even with small data volumes and simple experiments: Data archaeology (have you ever tried to extract data from the Ferrel Lytle archive at IIT?) Moving data from the beamline to the data analysis package Sharing data between different analysis packages Submitting supplemental data with a publication Building web and other data-centric applications (such as editable archives of standards) Managing XAS data 14 / 24
IUCr Commission on XAFS working group Four of us – , , , ∗ – were tasked with defining proposals for data format standards for use with XAS. We came up∗∗ with concepts for: 1 A text format to encapsolate a single spectrum 2 A hierarchical format to encapsolate multispectral data 3 A database format for large data ensembles of XAS and other data Managing XAS data 15 / 24 ∗ Armando is surprisingly good at keeping his photo off the ’net. ∗∗ B. Ravel et al, J. Synchrotron Rad. (2012) 19, pp. 869-874 DOI: 10.1107/S0909049512036886
Problems with beamline formats They require additional processing in order to display µ(E), including Conversion to energy Dead-time or other corrections Ambiguous metadata, for instance How is the beamline identified? What consitutes a user comment? What describes the condition of the source or the beamline? XAS data analysis software and other plotting software may have difficulty importing and interpreting the data This data is probably not appropriate for submission to a journal as supplemental material Data interchange A standard for the interchange of µ(E) data would address most of these concerns. Managing XAS data 17 / 24
Goals of a data interchange format The smallest unit of currency is the µ(E) spectrum. 1 Be easy for a human to read. Be easy for a computer to read. 2 Establish a common language for transferring data between XAS experimenters, data analysis packages, web applications, journals and anything else that needs to process XAS data, thus enhancing the user experience. 3 Increase the relevance and longevity of experimental data by reducing the amount of data archaeology future interpretations of that data will require. 4 Provide a mechanism for extracting and preserving a single XAS or XAS-like data set from a multispectral experiment or from a complex data structure. 5 Be a building block for hierarchical or database data structures. Managing XAS data 18 / 24
XDI: XAS Data Interchange XDI is an ad hoc format loosely based on the format of e-mail and structured in a way that looks like a familiar column data file. # XDI/1.0 MX/2.0 # Beamline.name: APS 10ID # Beamline.edge-energy: 7112.00 # Beamline.d-spacing: 3.1356 # Ring.energy: 7.00 # Source.type: undulator a # Source.undulator-harmonic: 1 # Time.start: 2005-03-08T20:08:57 # Optics.crystal: Si 111 # Optics.harmonic-rejection: flat Rh-coated mirror # Column.1: energy eV # Column.2: mu # Column.3: i0 # MX.Num-regions: 1 # MX.SRB: 6900 # MX.SRSS: 0.5 # MX.SPP: 0.1 # MX.Settling-time: 0 # MX.Offsets: 11408.00 11328.00 13200.00 10774.00 # MX.Gains: 8.00 7.00 7.00 9.00 #/// # Fe K-edge, Lepidocrocite powder on kapton tape, RT # 4 layers of tape # exafs, 20 invang #--- # energy mcs3 mcs4 6899.9609 -1.3070486 149013.70 6900.1421 -1.3006104 144864.70 6900.5449 -1.3033816 132978.70 6900.9678 -1.3059724 125444.70 6901.3806 -1.3107085 121324.70 (....etc....) Managing XAS data 19 / 24
XDI: XAS Data Interchange Metadata specific to a beamline, a data acquisition system, or a data processing program are specified by “extension headers”. These use the same format as standard metadata headers, but with a domain specific “namespace”. # XDI/1.0 MX/2.0 # Beamline.name: APS 10ID # Beamline.edge-energy: 7112.00 # Beamline.d-spacing: 3.1356 # Ring.energy: 7.00 # Source.type: undulator a # Source.undulator-harmonic: 1 # Time.start: 2005-03-08T20:08:57 # Optics.crystal: Si 111 # Optics.harmonic-rejection: flat Rh-coated mirror # Column.1: energy eV # Column.2: mu # Column.3: i0 # MX.Num-regions: 1 # MX.SRB: 6900 # MX.SRSS: 0.5 # MX.SPP: 0.1 # MX.Settling-time: 0 # MX.Offsets: 11408.00 11328.00 13200.00 10774.00 # MX.Gains: 8.00 7.00 7.00 9.00 #/// # Fe K-edge, Lepidocrocite powder on kapton tape, RT # 4 layers of tape # exafs, 20 invang #--- # energy mcs3 mcs4 6899.9609 -1.3070486 149013.70 6900.1421 -1.3006104 144864.70 6900.5449 -1.3033816 132978.70 6900.9678 -1.3059724 125444.70 6901.3806 -1.3107085 121324.70 (....etc....) Managing XAS data 19 / 24
Raw data v. processed data Strictly speaking, XDI is an interchange format for µ(E) data. XDI is meant to encapsolate the merged spectrum, not necessarily the raw data that gets merged. That said, for raw data with a few handfuls of scalars, XDI would be a fine beamline format. Managing XAS data 20 / 24
XDI timeline Phase 1. Define the specification (mostly done) Phase 2. Define the metadata library (partially done) Phase 3. Write an I/O library in C with bidings in common languages (C, Python, & Perl mostly done) Phase 4. Encourage its wide adoption? Phase 5. Profit Managing XAS data 21 / 24
Other data types: hierarchical A hierarchical data file, like HDF5, can be thought of as an encapsolated file system, where “folders” are measurements and “files” are data arrays or metadata scalars. This is perfect for Multispectral data such as an XRF image with XAS, XES, and/or XRD measurements at (one|many|all) points Capturing the path through data processing and analysis software (like ) Software for this HDF5 file would do single-scan XAS I/O using XDI. Managing XAS data 22 / 24
Other data types: database A database emphasizes the relations among its contents, i.e. all Cu K edge data all data from NSLS X23A2 all data measured on Tuesday July 17, 2007 at an Asian synchrotron all anatase data measured at elevated temperature using Si 311 crystals This is great for Your personal collection of data The save format for a program like A collection of standards XDI is the single-scan I/O format for the database software Managing XAS data 23 / 24
Join in! some clown Matt Newville Jim Hester V. Armando Sol´ e (NIST/NSLS) (U Chicago/APS) (ANSTO) (ERSF) Gerd Wellenreuther Chris Chantler Edmund Welter Darren Dale (DESY) (U Melbourne) (DESY) (CHESS) Github page and wiki https://github.com/XraySpectroscopy/XAS-Data-Interchange Mailing list http://millenia.cars.aps.anl.gov/mailman/listinfo/xasformat For a copy of this talk https://speakerdeck.com/bruceravel Managing XAS data 24 / 24
bruceravel
dropout009
tagtag
miura55
parrales
stevenshorrock
sansandsoc
line_developers_tw2
elix
hidekatsu_izuno
mkimura
roy29fuku
koukyo1994
gr2m
bkeepers
matthewcrist
jeffersonlam
62gerente
hannesfritz
speakerdeck
lauravandoore
geoffreycrofte
chriscoyier
malarkey
geeforr
