The LOFAR Transients Pipeline

Astronomy and Computing

Volumn 11, Issue PA, 2015, Pages 25-48

  Swinbank, John D ^a,b   Staley, Tim D ^c   Molenaar, Gijs J ^b,d   Rol, Evert ^b   Rowlinson, Antonia ^e   Scheers, Bart ^b,f   Spreeuw, Hanno ^g   Bell, Martin E ^e,h   Broderick, Jess W ^c,i   Carbone, Dario ^b   Garsden, Hugh ^j   van der Horst, Alexander J ^b   Law, Casey J ^k   Wise, Michael ^l   Breton, Rene P ^i,m   Cendes, Yvette ^b   Corbel, Stéphane ^j,n   Eislöffel, Jochen ^o   Falcke, Heino ^l,p   Fender, Rob ^c   Grießmeier, Jean Mathias ⁿ   Hessels, Jason W T ^b,l   Stappers, Benjamin W ^m   Stewart, Adam J ^c   Wijers, Ralph A M J ^b   Wijnands, Rudy ^b   Zarka, Philippe ^q   more..

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c UNIVERSITY OF OXFORD   (United Kingdom)

^d RHODES UNIVERSITY   (South Africa)

^e CSIRO   (Australia)

^f CWI   (Netherlands)

^g NETHERLANDS CANCER INSTITUTE   (Netherlands)

^h UNIVERSITY OF SYDNEY   (Australia)

ⁱ UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^j DIF   (France)

^k UNIVERSITY OF CALIFORNIA   (United States)

^l ASTRON   (Netherlands)

^m UNIVERSITY OF MANCHESTER   (United Kingdom)

ⁿ UNIVERSITÉ D'ORLÉANS   (France)

^o Thuringer Landessternwarte   (Germany)

^p RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^q UNIVERSITÉ PARIS DIDEROT   (France)

more..

Author keywords

Astronomical databases; Astronomical transients; Methods: data analysis; Techniques: image processing; Time domain astrophysics

Indexed keywords

ASTROPHYSICS; IMAGE ANALYSIS; INTERFEROMETERS; PIPELINES; RADIO ASTRONOMY;

ASTRONOMICAL DATABASE; ASTRONOMICAL SURVEYS; DEVELOPMENT METHODOLOGY; LARGE DATA VOLUMES; METHODS:DATA ANALYSIS; RADIO INTERFEROMETERS; TECHNIQUES: IMAGE PROCESSING; TIME DOMAIN;

TIME DOMAIN ANALYSIS;

EID: 84926435533     PISSN: 22131337     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ascom.2015.03.002     Document Type: Article

References (112)

1. Alard C., Lupton R.H. A method for optimal image subtraction. Astrophys. J. 1998, 503:325-331.
2. Allan, A., Denny, R.B., 2009. VOEvent transport protocol, Version 1.1. Note, International Virtual Observatory Alliance, August.
3. Amdahl G.M. Validity of the single processor approach to achieving large scale computing capabilities. Proceedings of the April 18-20, 1967, Spring Joint Computer Conference. AFIPS'67 (Spring) 1967, 483-485. ACM, New York, NY, USA.
4. Anderson G.E., et al. Probing the bright radio flare and afterglow of GRB 130427A with the Arcminute Microkelvin Imager. Mon. Not. R. Astron. Soc. 2014, 440:2059-2065.
5. Astropy: a community Python package for astronomy. Astronom. Astrophys. 2013, 558:A33. Astropy Collaboration.
6. Atwood W.B., et al. The large area telescope on the fermi gamma-ray space telescope mission. Astrophys. J. 2009, 697:1071-1102.
7. Bannister K.W., et al. A 22-yr southern sky survey for transient and variable radio sources using the Molonglo Observatory Synthesis Telescope. Mon. Not. R. Astron. Soc. 2011, 412:634-664.
8. Banyer J., Murphy T. VAST-a real-time pipeline for detecting radio transients and variables on the Australian SKA Pathfinder (ASKAP) telescope. Astronomical Society of the Pacific Conference Series 2012, vol. 461:725. The VAST Collaboration. P. Ballester (Ed.).
9. Becker R.H., White R.L., Helfand D.J. The VLA's FIRST survey. Astronomical Society of the Pacific Conference Series 1994, vol. 61:165. D.R. Crabtree, R.J. Hanisch, J. Barnes (Eds.).
10. Bell M.E., et al. An automated archival Very Large Array transients survey. Mon. Not. R. Astron. Soc. 2011, 415:2-10.
11. Bell M.E., et al. A survey for transients and variables with the Murchison Widefield Array 32-tile prototype at 154 MHz. Mon. Not. R. Astron. Soc. 2014, 438:352-367.
12. Benjamini Y., Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B Stat. Methodol. 1995, 57:289-300.
13. Bertin E., Arnouts S. SExtractor: software for source extraction. Astron. Astrophys. Suppl. Ser. 1996, 117:393-404.
14. Bolch B.W. More on unbiased estimation of the standard deviation. Amer. Statist. 1968, 22:27.
15. Booth R.S., Jonas J.L. An overview of the MeerKAT project. Afr. Skies 2012, 16:101.
16. Bower G.C., et al. The Allen telescope Array Pi GHz sky survey II. Daily and monthly monitoring for transients and variability in the Boötes field. Astrophys. J. 2011, 739:76.
17. Broderick, J., et al. (in preparation).
18. Brown T.M., et al. Las Cumbres observatory global telescope network. Publ. Astron. Soc. Pac. 2013, 125:1031-1055.
19. Carbone, D., et al. PySE: software for extracting sources from LOFAR radio telescope images (in preparation).
20. Carbone, D., et al. New methods to constrain the radio transient rate: results from a survey of four fields with LOFAR (submitted for publication),. arxiv:1411.7928.
21. Codd E.F. A relational model of data for large shared data banks. Commun. ACM 1970, 13:377-387.
22. Coenen T., et al. The LOFAR pilot surveys for pulsars and fast radio transients. Astronom. Astrophys. 2014, 570:A60.
23. Cohen A.S., et al. The VLA low-frequency sky survey. Astron. J. 2007, 134:1245-1262.
24. Condon J.J. Errors in elliptical Gaussian fits. Publ. Astron. Soc. Pac. 1997, 109:166-172.
25. Condon J.J., et al. The NRAO VLA sky survey. Astron. J. 1998, 115:1693-1716.
26. Croft S., et al. The Allen Telescope Array twenty-centimeter survey-a 700-square-degree, multi-epoch radio data set. II. Individual epoch transient statistics. Astrophys. J. 2011, 731:34.
27. de Bruyn A.G., Bernardi G., Lofar Eor-Team The first deep WSRT 150 MHz full polarization observations. Astron. Soc. Pac. Conf. Ser. 2009, vol. 407:3. D.J. Saikia, D.A. Green, Y. Gupta, T. Venturi (Eds.).
28. Denneau L., et al. The Pan-STARRS moving object processing system. Publ. Astron. Soc. Pac. 2013, 125:357-395.
29. Dent W.A. Quasi-stellar sources: variation in the radio emission of 3C 273. Science 1965, 148:1458-1460.
30. de Ruiter H.R., Willis A.G., Arp H.C. A Westerbork 1415 MHz survey of background radio sources. II-optical identifications with deep IIIA-J plates. Astron. Astrophys. Suppl. Ser. 1977, 28:211-293.
31. Dewdney, P., et al. 2010. SKA Memo 130: SKA phase 1: preliminary system description.
32. Drake A.J., et al. First results from the catalina real-time transient survey. Astrophys. J. 2009, 696:870-884.
33. Fender R.P., Bell M.E. Radio transients: an antediluvian review. Bull. Astron. Soc. India 2011, 39:315-332.
34. Fender, R., et al. 2006. The LOFAR transients key project. In: Proceedings of the VI Microquasar Workshop: Microquasars and Beyond. PoS(MQW6)014.
35. Fender, R., et al. The LOFAR radio sky monitor (in preparation).
36. Fisher R. Dispersion on a sphere. Proc. R. Soc. Lond. Ser. A 1953, 219:295-305.
37. Fishman, G.J., 1992. BATSE-the burst and transient source experiment on the Gamma Ray Observatory. pp. 265-272.
38. Frail D.A., et al. The radio afterglow from the γ-ray burst of 8 May 1997. Nature 1997, 389:261-263.
39. Frail D.A., et al. A revised view of the transient radio sky. Astrophys. J. 2012, 747:70.
40. Frail D.A., Kulkarni S.R., Bloom J.S. An outburst of relativistic particles from the soft γ-ray repeater SGR1900+14. Nature 1999, 398:127-129.
41. Freund J.E., Williams F.J. Outline of Basic Statistics: Dictionary and Formulas 2010, Courier Corporation.
42. Gaensler B.M., Hunstead R.W. Long-term monitoring of Molonglo calibrators. Publ. Astron. Soc. Aust. 2000, 17:72-82.
43. Gautschi W. Error Function and Fresnel Integrals 1972, Dover.
44. Gehrels N., et al. The swift gamma-ray burst mission. Astrophys. J. 2004, 611:1005-1020.
45. Graham, M., et al. 2014. Employing simpletimeseries for representing time series, version 1.0. Note, International Virtual Observatory Alliance. May.
46. Gray J., Nieto-Santisteban M.A., Szalay A.S. The zones algorithm for finding points-near-a-point or cross-matching spatial datasets. MSR TR 2006 52 2006, Microsoft and The Johns Hopkins University.
47. Gregory P.C., et al. The nature of the first Cygnus X-3 radio outburst. Nat. Phys. Sci. 1972, 239:114-117.
48. Greisen E.W. AIPS, the VLA, and the VLBA. Information Handling in Astronomy-Historical Vistas. Vol. 285 2003, 109.
49. Hancock P.J., et al. Compact continuum source finding for next generation radio surveys. Mon. Not. R. Astron. Soc. 2012, 422:1812-1824.
50. Harmon B.A., et al. The burst and transient source experiment earth occultation technique. Astrophys. J. Suppl. Ser. 2002, 138:149-183.
51. Heald, G., et al. 2010. Progress with the LOFAR imaging pipeline. In: Proceedings of the ISKAF2010 Science Meeting. PoS(ISKAF2010)057. August.
52. Heald G., et al. LOFAR: recent imaging results and future prospects. J. Astrophys. Astron. 2011, 32:589-598.
53. Heald, G., et al. The LOFAR Multifrequency Snapshot Sky Survey (MSSS): 1. Survey description and first results. Astronom. Astrophys. submitted for publication.
54. Hendricks W.A., Robey K.W. The sampling distribution of the coefficient of variation. Ann. Math. Statist. 1936, 7(3):129-132.
55. Högbom J.A. Aperture synthesis with a non-regular distribution of interferometer baselines. Astron. Astrophys. Suppl. Ser. 1974, 15:417-426.
56. Hopkins A.M., et al. A new source detection algorithm using the false-discovery rate. Astron. J. 2002, 123:1086-1094.
57. Hopkins A.M., et al. The phoenix deep survey: the 1.4 GHz microJansky catalog. Astron. J. 2003, 125:465-477.
58. Hopkins, A.M., et al. The ASKAP/EMU source finding data challenge (in preparation).
59. Hunter J.D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 2007, 9(3):90-95.
60. Idreos S., et al. MonetDB: two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 2012, 35(1):40-45.
61. Ivezić, Ž., et al. 2014. LSST: from science drivers to reference design and anticipated data products, version 3.1. arxiv:0805.2366.
62. Jenet F.A., Gil J. Using the intensity modulation index to test pulsar radio emission models. Astrophys. J. Lett. 2003, 596(2):L215.
63. Kent W. A simple guide to five normal forms in relational database theory. Commun. ACM 1983, 26(2):120-125.
64. Kesteven M.J.L., Bridle A.H., Brandie G.W. Variability of extragalactic sources at 2.7 GHz. III-the nature of the variations in different source classes. Astron. J. 1977, 82:541-556.
65. Koposov S., Bartunov O. Q3C, Quad Tree Cube-the new sky-indexing concept for huge astronomical catalogues and its realization for main astronomical queries (cone search and xmatch) in Open Source Database PostgreSQL. Astronomical Society of the Pacific Conference Series 2006, vol. 351:735. C. Gabriel, C. Arviset, D. Ponz, S. Enrique (Eds.).
66. Lande R. On comparing coefficients of variation. Syst. Biol. 1977, 26(2):214-217.
67. Law N.M., et al. The Palomar Transient factory: system overview, performance, and first results. Publ. Astron. Soc. Pac. 2009, 121:1395-1408.
68. Law C.J., et al. Millisecond imaging of radio transients with the pocket correlator. Astrophys. J. 2011, 742:12.
69. Levan A.J., et al. An extremely luminous panchromatic outburst from the nucleus of a distant galaxy. Science 2011, 333:199-202.
70. Levine A.M., et al. First results from the all-sky monitor on the Rossi X-ray timing explorer. Astrophys. J. 1996, 469:L33.
71. Loose G.M. LOFAR self-calibration using a blackboard software architecture. Astronomical Society of the Pacific Conference Series 2008, Vol. 394:91. R.W. Argyle, P.S. Bunclark, J.R. Lewis (Eds.).
72. Lyubarsky Y. A model for fast extragalactic radio bursts. Mon. Not. R. Astron. Soc. 2014, 442:L9-L13.
73. McDowell, J., et al. 2015. IVOA spectral data model, Version 2.0. Proposed Recommendation, International Virtual Observatory Alliance. February.
74. McKay A.T. Distribution of the coefficient of variation and the extended "t" distribution. J. Roy. Statist. Soc. 1932, 95(4):695-698.
75. Meegan C., et al. The fermi gamma-ray burst monitor. Astrophys. J. 2009, 702:791-804.
76. Mooley K.P., et al. Sensitive search for radio variables and transients in the extended chandra deep field south. Astrophys. J. 2013, 768:165.
77. Moré J.J. The Levenberg-Marquardt algorithm: implementation and theory. Lecture Notes in Mathematics 1977, vol. 630:105-116. G.A. Watson (Ed.).
78. Murphy T., et al. VAST: an ASKAP survey for variables and slow transients. Publ. Astron. Soc. Aust. 2013, 30:6.
79. Narayan R. The physics of pulsar scintillation. Philos. Trans. R. Soc. Lond. Ser. A 1992, 341:151-165.
80. Nijboer, R.J., Pandey-Pommier, M., de Bruyn, A.G., 2009. LOFAR imaging capabilities and system sensitivity. Memo 113, Square Kilometre Array Project.
81. O'Mullane, W., et al. 2005. Batch is back: CasJobs, serving multi-TB data on the Web. February,. arxiv:cs/0502072.
82. Pérez F., Granger B.E. IPython: a system for interactive scientific computing. Comput. Sci. Eng. 2007, 9(3):21-29.
83. Prasad, P., Wijnholds, S.J., 2012. AARTFAAC: towards a 24×7, all-sky monitor for LOFAR. In: Proceedings of 'New Windows on Transients across the Universe': A Discussion Meeting of the Royal Society, London, May.
84. Rau A., et al. Exploring the optical transient sky with the Palomar transient factory. Publ. Astron. Soc. Pac. 2009, 121:1334-1351.
85. Rengelink R.B., et al. The Westerbork Northern Sky Survey (WENSS). I. A 570 square degree mini-survey around the North Ecliptic Pole. Astron. Astrophys. Suppl. Ser. 1997, 124:259-280.
86. Rickett B.J. Radio propagation through the turbulent interstellar plasma. Annu. Rev. Astron. Astrophys. 1990, 28:561-605.
87. Rowlinson, A.R., et al. Methodology for optimising the LOFAR Transients Pipeline (in preparation).
88. Rutledge R.E., Brunner R.J., Prince T.A., Lonsdale C. XID: cross-association of ROSAT/Bright source catalog x-ray sources with USNO A-2 optical point sources. Astrophys. J. Suppl. Ser. 2000, 131:335-353.
89. Sagiv A., Waxman E. Collective processes in relativistic plasma and their implications for gamma-ray burst afterglows. Astrophys. J. 2002, 574:861-872.
90. Scheers L.H.A. Transient and variable radio sources in the LOFAR sky 2011, (Ph.D. thesis), University of Amsterdam, URL: http://hdl.handle.net/11245/1.328956.
91. Scheers, L.H.A., et al. LOFAR transients key project archival database (in preparation).
92. Seaman, R., et al. 2011. Sky event reporting metadata, version 2.0. Recommendation, International Virtual Observatory Alliance. July.
93. Spreeuw J.N. Search and detection of low frequency radio transients 2010, (Ph.D. thesis), University of Amsterdam, URL: http://hdl.handle.net/11245/1.324881.
94. Staley T.D., et al. Automated rapid follow-up of Swift gamma-ray burst alerts at 15 GHz with the AMI Large Array. Mon. Not. R. Astron. Soc. 2013, 428:3114-3120.
95. Stappers B.W., et al. Observing pulsars and fast transients with LOFAR. Astronom. Astrophys. 2011, 530:A80.
96. Stewart, A., et al. LOFAR NCP transient search (in preparation).
97. Sutherland W., Saunders W. On the likelihood ratio for source identification. Mon. Not. R. Astron. Soc. 1992, 259:4130.
98. Swinbank J. The LOFAR transients pipeline. Astronomical Society of the Pacific Conference Series 2011, vol. 442:313. I.N. Evans, A. Accomazzi, D.J. Mink, A.H. Rots (Eds.).
99. Swinbank J.D. Comet: a VOEvent broker. Astron. Comput. 2014, 7:12-26.
100. Szalay A., et al. Indexing the sphere with the Hierarchical Triangular Mesh. MSR TR 2005 123 2006, The John Hopkins University, Microsoft Research and CERN.
101. Taylor G.B., et al. First light for the first station of the long wavelength array. Journal of Astronomical Instrumentation 2012, 1:50004.
102. Thyagarajan N., et al. Variable and transient radio sources in the FIRST survey. Astrophys. J. 2011, 742:49.
103. Tingay S.J., et al. The Murchison Widefield Array: the square kilometre array precursor at low radio frequencies. Publ. Astron. Soc. Aust. 2013, 30:7.
104. Usov V.V., Katz J.I. Low frequency radio pulses from gamma-ray bursts?. Astronom. Astrophys. 2000, 364:655-659.
105. van der Laan H. A model for variable extragalactic radio sources. Nature 1966, 211:1131.
106. van der Tol S., Jeffs B.D., van der Veen A.-J. Self-calibration for the LOFAR radio astronomical array. IEEE Trans. Signal Process. 2007, 55:4497-4510.
107. van Haarlem M.P., et al. LOFAR: the LOw-frequency ARray. Astronom. Astrophys. 2013, 556:A2.
108. Welch J., et al. The Allen Telescope Array: the first widefield, panchromatic, snapshot radio camera for radio astronomy and SETI. IEEE Proc. 2009, 97:1438-1447.
109. Whitaker J.C. The Electronics Handbook 1996, CRC Press.
110. Williams, R.D., Barthelmy, S.D., Denny, R.B., Graham, M.J., Swinbank, J., 2012. Responding to the event deluge. Vol. 8448 of SPIE Conf. Ser.
111. Williams P.K.G., et al. ASGARD: a large survey for slow galactic radio transients. I. Overview and first results. Astrophys. J. 2013, 762:85.
112. Wolter K.M. An investigation of some estimators of variance for systematic sampling. J. Amer. Statist. Assoc. 1984, 79(388):781-790. URL: http://www.jstor.org/stable/2288708.