W15: Big Data: The Challenges Presented by WCS 2.0

Lead Presenter: Ian Edwards (Met Office)

with Richard Hattersley (Met Office) , Mark Hedley (Met Office) , Ed Campbell (Met Office) , Peter Killick (Met Office)

Hallward Library: LG101 - Sept. 18, 2013, 2 p.m. to Sept. 18, 2013, 6 p.m.

In this workshop we will introduce IRIS, along with interesting multi-dimensional datasets, and show how "big data" can be served up using your existing open source software stack using GDAL, QGIS, GeoServer and OpenLayers.

The free and open source Iris python package, developed by the UK Met Office, allows users to work with large multi-dimensional datasets such as (but not limited to) those found in the fields of weather and climate science. Iris builds on the semantics and data model from the Climate and Forecasting conventions for NetCDF (CF-NetCDF) which exist to define the metadata within NetCDF files in order to provide a definitive description of each of the data variables including their spatial and temporal properties. This enables users of data from different sources to decide which quantities are comparable and to build applications with powerful extraction, regridding, and display capabilities [1]. In this workshop we will introduce this new open source library, along with interesting multi-dimensional datasets, and show how "big data" can be served up using your existing open source software stack using GDAL, QGIS, GeoServer and OpenLayers. We will then investigate the challenges of OGC WCS 2.0 looking into the core functionality, such as subsetting, that all WCS implementations must provide when accessing coverages[2]. The ability to provide data sets of three, four, and higher-dimensions represents a significant expansion of the capabilities of web coverage services. We will demonstrate how the CF conventions cater for multidimensional data, and how Iris can provide a managing interface to provide NetCDF datasets. We will investigate the opportunities and challenges of using NetCDF as a transport mechanism and consider how current server technologies will be able to deliver the breath of WCS 2.0 services. Iris is available under LGPL, allowing the geospatial community a great flexibility in the use of this software. More details can be found at www.scitools.org.uk Reference: [1] http://cf-pcmdi.llnl.gov/ [2] http://www.ogcnetwork.net/wcs