Big Data Analytics for Earth Sciences: the EarthServer approach

International Journal of Digital Earth

Volumn 9, Issue 1, 2016, Pages 3-29

  Baumann, Peter ^a,b   Mazzetti, Paolo ^c   Ungar, Joachim ^d   Barbera, Roberto ^e,f,g   Barboni, Damiano ^h   Beccati, Alan ^a   Bigagli, Lorenzo ^c   Boldrini, Enrico ^c   Bruno, Riccardo ^e,f   Calanducci, Antonio ^f   Campalani, Piero ^a   Clements, Oliver ⁱ   Dumitru, Alex ^a   Grant, Mike ⁱ   Herzig, Pasquale ^j   Kakaletris, George ^k   Laxton, John ^l   Koltsida, Panagiota ^k   Lipskoch, Kinga ^a   Mahdiraji, Alireza Rezaei ^a   Mantovani, Simone ^h   Merticariu, Vlad ^a   Messina, Antonio ^m   Misev, Dimitar ^a   Natali, Stefano ^h   Nativi, Stefano ^c   Oosthoek, Jelmer ^a   Pappalardo, Marco ^m   Passmore, James ^l   Rossi, Angelo Pio ^a   Rundo, Francesco ^e   Sen, Marcus ^l   Sorbera, Vittorio ^e   Sullivan, Don ⁿ   Torrisi, Mario ^f   Trovato, Leonardo ^m   Veratelli, Maria Grazia ^h   Wagner, Sebastian ^j   more..

^a JACOBS UNIVERSITY BREMEN   (Germany)

^b Rasdaman GmbH   (Germany)

^c CNR   (Italy)

^d EOX IT Services GmbH   (Austria)

^e Consorzio COMETA ^*  (Italy)

^f INFN SEZIONE DI CATANIA   (Italy)

^g UNIVERSITY OF CATANIA   (Italy)

^h LAB srl   (Italy)

ⁱ PLYMOUTH MARINE LABORATORY   (United Kingdom)

^j FRAUNHOFER IGD   (Germany)

^k INSTITUTE FOR THE MANAGEMENT OF INFORMATION SYSTEMS   (Greece)

^l BRITISH GEOLOGICAL SURVEY   (United Kingdom)

^m Bioos srl   (Italy)

ⁿ NASA AMES RESEARCH CENTER   (United States)

more..

Author keywords

array databases; big data; Big Data Analytics; Earth Sciences; interoperability; standards

Indexed keywords

BIG DATA; DATA ANALYTICS; DIGITAL STORAGE; EARTH SCIENCES; INTEROPERABILITY; LARGE DATASET; MARINE APPLICATIONS; QUERY LANGUAGES; QUERY PROCESSING; STANDARDS; STORAGE AS A SERVICE (STAAS);

COMPUTING CAPABILITY; DATABASE TECHNOLOGY; EARTH SCIENCE DATA; ENVIRONMENTAL SCIENCE; GEO-SPATIAL INFORMATIONS; SCIENTIFIC COMMUNITY; SERVICE INTERACTION; TECHNIQUES AND TOOLS;

ADVANCED ANALYTICS;

COMPLEXITY; COMPUTER SIMULATION; DATA PROCESSING; DATA SET; NUMERICAL MODEL; VISUALIZATION;

EID: 84958866882     PISSN: 17538947     EISSN: 17538955     Source Type: Journal    
DOI: 10.1080/17538947.2014.1003106     Document Type: Article

References (71)

1. A.Aiordachioaie, Andrei, and P.BaumannPeter. 2010. “PetaScope: An Open-source Implementation of the OGC WCS Geo Service Standards Suite.” In Proceedings of the 22nd International Conference on Scientific and Statistical Database Management, edited by Michael, Gertz, and Bertram Ludäscher, 160–168. Heidelberg: Springer.
2. ARDA Project. 2012. AMGA – Overview. Aprile 16. http://amga.web.cern.ch/amga/.
3. Array DBMS.” 2014. Wikipedia, Agosto 20. http://en.wikipedia.org/wiki/Array_DBMS.
4. P.Bargellini, Pier, C.SimonettaCheli, Y.-L.DesnosYves-Louis, B.GrecoBruno, V.Guidettiveronica, M.P.GiorgioMarchetti Pier, C.ComparettoCarmen, S.NativiStefano, and G.SawyerGeoff. 2013. Big Data from Space – Event Report. Frascati: ESA. http://www.congrexprojects.com/docs/default-source/13c10_docs/13c10_event_report.pdf?sfvrsn=2.
5. P.Baumann, Peter. 1994. “Management of Multidimensional Discrete Data.” The VLDB Journal 3 (4): 401–444. doi:10.1007/BF01231603.
6. P.Baumann, Peter. 2008. Web Coverage Processing Service (WCPS) Implementation Specification, OGC 08-068. Version 1.0. OGC. http://www.opengeospatial.org/standards/.
7. P.Baumann, 2009. “Array Databases and Raster Data Management.” In Encyclopedia of Database Systems, edited by L. Liu, and T. Özsu. Heidelberg: Springer. www.opengeospatial.org/standards/.
8. P.Baumann, Peter. 2010. “The OGC Web Coverage Processing Service (WCPS) Standard.” Geoinformatica 14 (4): 447–479. doi:10.1007/s10707-009-0087-2.
9. P.Baumann, Peter. 2012a. “OGC GML Application Schema – Coverages.” OGC 09-146r2. Version 1.0. OGC.
10. P.Baumann, Peter. 2012b. “OGC® WCS 2.0 Interface Standard – Core.” OGC 09-110r4. Version 2.0. OGC.
11. P.Baumann, Peter, P.CampalaniPiero, J.YuJinsongdi, and D.MisevDimitar. 2012. “Finding My CRS: A Systematic Way of Identifying CRSs.” In Proceedings of the 20th International Conference on Advances in Geographic Information Systems, 71–78. Redondo Beach, CA: ACM, ISBN 978-1-4503-1697-2.
12. P.Baumann, Peter, B.HoweBill, K.OrsbornKjell, and S.StefanovaSilvia. 2011. “EDBT/ICDT Workshop on Array Databases.” Proceedings of 2011 EDBT/ICDT Workshop on Array Databases, Uppsala, Sweden.
13. P.Baumann, Peter, and S.MeisslStephan. 2010. OGC WCS 2.0 Application Profile Earth Observation. OGC 10-140. Version 1.0. OGC.
14. BGS. 2013. Geology Service Home Page. British Geological Survey. Accessed September 1, 2013. http://earthserver.bgs.ac.uk/.
15. R.Bruno, Riccardo, and R.BarberaRoberto. 2013. “The EarthServer Project and Its Lighthouse Applications: Exploiting Science Gateways and Mobile Clients for Big Earth Data Analysis.” In Proceedings International Symposium on Grids and Clouds (ISGC). Taipei: Academia Sinica.
16. F.Cantini, Federico, A.Pio RossiAngelo, R.OroseiRoberto, P.BaumannPeter, D.MisevDimitar, J.OosthoekJelmer, A.BeccatiAlan, P.CampalaniPiero, and V.UnnitanVikram. 2014. MARSIS Data and Simulation Exploited Using Array Databases: PlanetServer/EarthServer for Sounding Radars. Vienna: Geophysical Research Abstracts.
17. O.Clements, Oliver, and P.WalkerPeter. 2013. EarthServer Project Deliverable D240.12. Plymouth: PML.
18. D3. 2013. Data-Driven Documents. Accessed 2014. http://d3js.org/.
19. B.Domenico, Ben. 2010. OGC CF-netCDF Core and Extensions Primer. OGC 10-091r1. Version 1.0. OGC.
20. B.Domenico, Ben. 2011. OGC Network Common Data form (NetCDF) Core Encoding Standard Version 1.0. OGC. http://portal.opengeospatial.org/files/?artifact_id=43732.
21. B.Domenico, Ben. 2012. OGC Network Common Data form (NetCDF) NetCDF Enhanced Data Model Extension Standard. OGC. https://portal.opengeospatial.org/files/?artifact_id=50294.
22. B.Domenico, Ben, and S.NativiStefano. 2013. CF-netCDF3 Data Model Extension Standard. OGC. https://portal.opengeospatial.org/files/?artifact_id=51908.
23. A.Dumitru, V.Merticariu, P.Baumann. 2014. Exploring Cloud Opportunities from an Array Database Perspective. Snowbird, USA: Proceeding of the ACM SIGMOD Workshop on Data analytics in the Cloud (DanaC’2014).
24. EOX. 2013. CryoService Home Page. EOX. Accessed September 1, 2013. http://earthserver.eox.at.
25. ESA. 2014. Multi-sensor Evolution Analysis. 28 Aprile. http://wiki.services.eoportal.org/tiki-index.php?page=Multi-sensor+Evolution+Analysis.
26. R.P.Gupta, 2003. Remote Sensing Geology. Berlin: Springer.
27. K.Gwinner, Klaus, F.ScholtenFrank, F.PreuskerFrank, S.ElgnerStephan, T.RoatschThomas, M.SpiegelMichael, R.SchmidtRalph, J.OberstJurgen, R.JaumannRalf, and C.HeipkeChristian. 2010. “Topography of Mars from Global Mapping by HRSC High-resolution Digital Terrain Models and Orthoimages: Characteristics and Performance.” Earth and Planetary Science Letters 294 (3): 506–519. doi:10.1016/j.epsl.2009.11.007.
28. D.J.Heather, David J, M.BarthelemyMaud, N.ManaudNicolas, S.MartinezSanta, M.SzumlasMarek, J.L.VazquezJose Luis, and P.OsunaPedro. 2013. “ESA's Planetary Science Archive: Status, Activities and Plans.” European Planetary Science Congress. http://www.epsc2013.eu/.
29. P.Herzig, Pasquale, M.EnglertMichael, S.WagnerSebastian, and Y.JungYvonne. 2013. “X3D-EarthBrowser: Visualize Our Earth in Your Web Browser.” Web3D’13 Proceedings of the 18th International Conference on 3D Web Technology. San Sebastian: ACM New York.
30. T.Hey, Tony, and A.E.TrefethenAnne E. 2002. “The UK e-Science Core Programme and the Grid.” International Journal for e-Learning Security (IJeLS) 1 (1/2): 1017–1031.
31. M.Hirtl, Marcus, S.MantovaniSimone, B.C.KrügerBernd C., G.TriebnigGerhard, C.FlandorferClaudia, and M.BottoniMaurizio. 2014. “Improvement of Air Quality Forecasts with Satellite and Ground Based Particulate Matter Observations.” Atmospheric Environment 1: 20–27.
32. N.Houghton, Nigel. 2013. “ESA Reprocessing. A Service Based Approach.” Big Data from Space. Frascati. http://www.congrexprojects.com/docs/default-source/13c10_docs/session-2a.zip.
33. E.Hovmöller, Ernest. 1949. “The Trough-and-Ridge Diagram.” Tellus 1 (2): 62–66.
34. A.Howard, Andrew, B.HattonBill, F.ReitsmaFemke, and K.LawrieKenneth. 2009. “Developing a Geoscience Knowledge Framework for a National Geological Survey Organisation.” Computers and Geosciences 35 (4): 820–835.
35. M.Iacono-Manno, Marcello, M.FargettaMarco, R.BarberaRoberto, A.FalzoneAlberto, G.AndronicoGiuseppe, S.MonforteSalvatore, A.MuoioAnnamaria,. 2010. “The Sicilian Grid Infrastructure for High Performance Computing.” International Journal of Distributed Systems and Technologies 1: 40–54.
36. ISO. 2005. Geographic Information – Schema for Coverage Geometry and Functions. ISO 19123:2005.
37. Jacobs University Bremen and Rasdaman GmbH. 2013. Rasdaman Project, Standards and Demonstrator Page. Accessed September 15, 2013. http://standards.rasdaman.org/.
38. R.Jaumann, Ralf, G.NeukumGerhard, T.BehnkeThomas, T.C.DuxburyThomas C., K.EichentopfKarin, J.FlohrerJoachim, S.van GasseltStephan,. 2007. “The High-resolution Stereo Camera (HRSC) Experiment on Mars Express: Instrument Aspects and Experiment Conduct from Interplanetary Cruise Through the Nominal Mission.” Planetary and Space Science 55 (7): 928–952.
39. G.Kakaletris, George, P.KoltsidaPanagiota, T.PerperisThanassis, and P.BaumannPeter. 2013. EarthServer Project Deliverable, Query Language Pilot, D330.12. EarthServer Project.
40. M.Kersten, Y.Zhang, M.Ivanova, A.N.Nesand N. 2011. “SciQL, a Query Language for Science Applications.” Proceedings of the Workshop on Array Databases, Uppsala.
41. D.Laney, Doug. 2001. 3D Data Management. Controlling Data Volume, Velocity, and Variety in Application Delivery Strategy. Stamford: META Group.
42. T.Lankester, Thomas. 2009. OpenGIS Web Map Services – Profile for EO Products. OGC 07-063r1.
43. J.Laxton, John, M.SenMarcus, J.PassmoreJames, and S.BurdenSimon. 2013. EarthServer Project Deliverable D250.11, OGC 07-063r1. Nottingham: BGS.
44. M.C.Malin, Michael C., J.F.BellJames F., B.A.CantorBruce A., M.A.CaplingerMichael A., W.M.CalvinWendy M., R.T.ClancyRobert Todd, K.S.EdgettKenneth S.,. 2007. “Context Camera Investigation on Board the Mars Reconnaissance Orbiter.” Journal of Geophysical Research: Planets 112: 2156–2202. doi:10.1029/2006JE002808.
45. S.Mantovani, Simone, D.BarboniDamiano, and S.NataliStefano. 2013. EarthServer Project Deliverable D230.12. Ferrara: MEEO.
46. P.Mazzetti, Paolo, A.Pio RossiAngelo, O.ClementsOliver, S.MantovaniSimone, S.NataliStefano, M.G.VeratelliMaria Grazia, J.UngarJoachim, and J.LaxtonJohn. 2013. Community Reporting. EarthServer Deliverable D120.21.
47. P.Mazzetti, Paolo, J.LaxtonJohn, M.G.VeratelliMaria Grazia, and M.GrantMike. 2012. Community Reporting. EarthServer Deliverable D120.21.
48. S.K.McMahon, Susan K. 1996. “Overview of the Planetary Data System.” Planetary and Space Science 44 (1): 3–12.
49. MEEO. 2013. Climate Data Service Home Page. MEEO. Accessed September 1, 2013. http://earthserver.services.meeo.it/.
50. G.Merticariu, George. 2014. “Keeping 1001 Nodes Busy.” BSc diss., Bremen: Jacobs University.
51. D.Misev, Dimitar, M.RusuMihaela, and P.BaumannPeter. 2012. “A Semantic Resolver for Coordinate Reference Systems.” Proceedings of 11th International Symposium on Web and Wireless Geographical Information Systems (W2GIS), 47–56. Naples: Springer.
52. A.A.Monaghan, S.L.B.Arkley, K.Whitbread, and M.McCormac. 2014. Clyde Superficial Deposits and Bedrock Models Released to the ASK Network 2014: A Guide for Users. Version 3. Nottingham: British Geological Survey.
53. S.L.Murchie, Scott L., R.E.ArvidsonRaymond E., P.BediniPeter, K.Beisser, J.P.Bibring, J.Bishop, J.Boldt,. 2007. “Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) on Mars Reconnaissance Orbiter (MRO).” Journal of Geophysical Research: Planets 112 (5): 2156–2202. doi:10.1029/2006JE002682.
54. OGC. 2007. Abstract Specification Topic 6: Schema for Coverage Geometry and Functions. OGC 07-011. Version 7.0. OGC.
55. J.H.P.Oosthoek, J.Flahaut, A.P.Rossi, P.Baumann, D.Misev, P.Campalani, and V.Unnithan. 2013. “PlanetServer: Innovative Approaches for the Online Analysis of Hyperspectral.” Advances in Space Research 23 (12): 1858–1871. doi:10.1016/j.asr.2013.07.002.
56. Oracle. n.d. Accessed June 10, 2014. http://www.oracle.com.
57. Paragon. n.d. PostGIS. Accessed June 10, 2014. http://www.postgis.org.
58. M.Petitdidier, Monique, R.CossuRoberto, P.MazzettiPaolo, P.FoxPeter, H.SchwichtenbergHorst, and W.S.de CerffWim Som. 2009. “Grid in Earth Sciences.” Earth Science Informatics 2: 1–3.
59. PML. 2013a. Ocean Data Service Home Page. PML. Accessed September 1, 2013. http://earthserver.pml.ac.uk/portal.
60. PML. 2013b. WCPS How to Pages (EarthServer: Ocean Data Service). Accessed September 14, 2013. http://earthserver.pml.ac.uk/portal/how_to/.
61. Rasdaman.” 2013. Wikipedia, December 29. http://en.wikipedia.org/wiki/Rasdaman.
62. Rasdaman GmbH. 2013. Rasdaman Query Language Guide. 8.4 ed., 91. Bremen: Rasdaman GmbH.
63. A.P.Rossi, Angelo Pio, and J.OosthoekJelmer. 2013. EarthServer Project Deliverable D260.12. Bremen: Jacobs University Bremen.
64. S.Sarawagi, Sunita, and M.StonebrakerMichael. 1994. “Efficient Organization of Large Multidimensional Arrays.” Proceedings of the 11th IEEE ICDE Conference, Hannover, Germany, 328–336.
65. SciDB. n.d. SciDB. Accessed June 10, 2014. http://www.scidb.org.
66. Square. 2012. Cubism.js. https://square.github.io/cubism/.
67. Teradata. n.d. Accessed June 10, 2014. http://www.teradata.com.
68. UCAR. n.d. Network Common Data Form (NetCDF). Accessed March 27, 2014. http://www.unidata.ucar.edu/software/netcdf/.
69. J.Ungar, Joachim. 2013. EarthServer Project Deliverable D220.12. Wien: EOX.
70. W3C. 2014. “XQuery 3.0: An XML Query Language.” http://www.w3.org/TR/2014/REC-xquery-30-20140408/.
71. X3DOM. n.d. x3dom Homepage. http://www.x3dom.org.