An update on the transients detection and classification pipeline Evert Rol University of Amsterdam 

TRAP: an end to end pipeline • Relatively simple, but we can go from raw data to classified transients • For practical purposes, it also incorporates the imaging steps (flagging, calibration, imaging), but does not have to. • So far, only Martin Bell and I have run it... • Documentation online: http://docs.transientskp.org/trap 

Pipeline • flagging • calibration • imaging • source extraction • database storage • source association • transient detection • feature extraction • classification Uses the default LOFAR pipeline framework Steps: Imaging pipeline TKP Python library 

TKP library • General code clean up • Improved independence between subpackages (eg, the sourcefinder can now more easily be run standalone) • Updated and extended (unit) tests • Nightly build on local (A'dam) machines, CEP1 and CEP2 (includes automated tests) • Updated and extended documentation: http:// docs.transientskp.org/tkp 

From images to transients pipeline Severe disconnect in the form of lacking metadata: • Images produced by the imager come with only very, very minimal header information. Missing are eg observation times. source extraction images imager imaging pipeline transients pipeline 

From images to transients pipeline Severe disconnect in the form of lacking metadata: • Images produced by the imager come with only very, very minimal header information. Missing are eg observation times. Hack: source extraction images imager MS set FITS update header find original data imaging pipeline transients pipeline 

From images to transients pipeline Severe disconnect in the form of lacking metadata: • Images produced by the imager come with only very, very minimal header information. Missing are eg observation times. Looking forward to HDF5! source extraction images imager imaging pipeline transients pipeline 

Source extraction • Has not changed over the past time • Being looked at improvements • Does rely on a decent image quality from the imaging pipeline 

Database storage, source association and transient detection • All this happens inside the database itself (SQL). See also Bart Scheers' talk • No matching with existing catalogues • No cross-frequency matching • Transient detection may fail when transient goes below background 

Database storage, source association and transient • Transient detection may fail when transient goes below background • As recently tested by Martin Bell 0 5 10 0 0.5 1 1.5 Time Flux Real Lightcurve Detection level (5m) Current Lightcurve 0 5 10 0 0.5 1 1.5 Time Flux Real Lightcurve Detection level (5m) Current Lightcurve 0 5 10 0 0.5 1 1.5 Time Flux Real Lightcurve Detection level (5m) Current Lightcurve 0 5 10 0 0.5 1 1.5 Time Flux Real Lightcurve Detection level (5m) Current Lightcurve 

Classification • Feature extraction • Default statistics • No periodicity search yet • Needs more: what to look for? • Classification • Manual decision tree exists. Lack of decisions • Clustering algorithms being implemented. Lack of training sample • Tests with existing data sets (optical) 

Classification 

Classification 

Future • Near: • Documentation update • Source association and transient detection improvement • Classification training set, and decision tree update • Far: • Different classification algorithms • Use of multi-frequency data (association, classification) • Speed tests • Response to and from other event sources 

Classification 

Manual decision tree implementation class SlowTransient(ClassifiedTransient): """Slow transient""" def test_duration(self): if self.duration > 1e6: return 0.9 def test_variability(self): if self.variability > 1e4: return 0.9 class MainBranch(Branch): """Starting point for manual classification tree""" def test1(self): return [SubBranch1] def test2(self): return [SubBranch2] class SubBranch1(Branch): def test1(self): return [SlowTransient]