WCS Scheduler

The WCS-Scheduler is a more or less simple java application, which observes the folder structure and uploads new files to the Geoserver.

Technologies used:

For the conversion from the raw Geonetcast stream data the program makes use of the GDal data library and from Ilwis, also using the MSG driver from Ilwis. For the purpose of uploading the data to the Geoserver we used the REST API, which is available as a plugin for the geoserver. For publishing via REST we used cURL.

How to set the Scheduler up:

The Scheduler consists of three files, the .bat-file, the .jar-file and a config file in xml. In order to set up the service we need to edit the .bat- and the xml-file.

This is how the .bat looks like:

C:\Users\Administrator\GNC\ilwis\Extensions\Geonetcast-Toolbox\MSGDataRetriever\gdal_translate.exe --config GDAL_CACHEMAX 30 -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL" -co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:4326 MSG(%1\,%2%3%4%5,(1,2,3,4,5,6,7,8,9,10,11),N,B,1,1) C:\Users\Administrator\ GNC\Upload\%6.tif C:\Users\Administrator\GNC\curl\curl -v -XPUT --basic --user user:password -H "Content-type: text/plain" -d "file:C:\Users\Administrator\ GNC\Upload\%6.tif" \ http://localhost:8080/geoserver/rest/workspaces/acme/coveragestores/%6/external.geotiff?configure=first\coverageName=%6

What needs to be changed:

• The path to the Ilwis-directory, so that it links to the correct gdal_translate.exe
• The path to the ‘upload’-folder, this is the folder where all the converted files are put, so that they can be uploaded to the Geoserver.
• The path to the curl.exe, and
• In the same command the path to the in 2. Specified upload folder
• Also we have to change ‘user:password’ to the geoservers correct credentials
• If your geoserver has a different address change this,too!
• Change ‘acme’ into the name of your Geoserver Workspace.

In the properties.xml:

• Change the msg_hrit.folder to the path of the Geonet cast stream data
• And the msg_hrit.script to the path of the above mentioned .bat-file

Now you should be ready to go!

Requirements:

• Version of Geoserver above 2.0.2 installed and running, with REST Config extension installed
• cURL installed
• an installation of Ilwis available
• the scheduler works up to now ONLY with MSG2_HRIT data

How does the scheduler work?

The java program itself just iterates through the file structure every few seconds and checks for new files. The file Structure of the incoming MSG files should look like this:

• Path: \MSG2_HRIT\2010\06\15
• There are different files for each channel like: H-000-MSG2__-MSG2________-WV_062___-000008___-201007151800-C_
• But there are also the prologue and the epilogue files
  • Prologue: H-000-MSG2__-MSG2________-_________-PRO______-201007151800-__
  • Epilogue: H-000-MSG2__-MSG2________-_________-EPI______-201007151200-__
• First the PRO-file, then the data and then lastly the EPI-file gets send, so only if a EPI file arrives, the scheduler starts the .bat-file with the appropriate parameters
• The .bat-file starts the conversion with gdal:
  • gdal_translate.exe --config GDAL_CACHEMAX 30 -of GTiff -co "PROFILE=GeoTIFF" -co "INTERLEAVE=PIXEL" -co "COMPRESS=LZW" -co "TILED=YES" -a_srs EPSG:4326 MSG(%1\,%2%3%4%5,(1,2,3,4,5,6,7,8,9,10,11),N,B,1,1) \%6.tif
  • the values with a ‘%’ in front of them are parameters, which are given from the java program: 1 Raw data absolute path; %2 Year; %3 Month; %4 Day; %5 Time; %6 Store data
  • EPSG:4326 defines the reference system
  • ,(1,2,3,4,5,6,7,8,9,10,11) this are the different layers which should be taken into account
• The .bat also triggers the upload command via cURL:
  • curl -v -XPUT --basic --user user:password -H "Content-type: text/plain" -d "file: %6.tif" \ http://localhost:8080/geoserver/rest/workspaces/acme/coveragestores/%6/external.geotiff?configure=first\coverageName=%6
  • see this for reference: http://docs.geoserver.org/stable/en/user/extensions/rest/rest-config-examples-curl.html

-- MartinDueren - 19 Jul 2010