Matlab and Octave in the 52°North Web Geoprocessing Framework

Idea

Support Matlab and Octave as processing backends from the WPS. They are powerful and widely-spread analysis environments with a large user base.

Applications/Projects

• INTAMAP2 based on http://uncertws.aston.ac.uk/madhu/
• https://github.com/autermann/matlab-wps

How to do it

The are several ways we can integrate a scripting language into the 52°North WPS - framework.

Annotations

Decouple the annotation mechanism from WPS4R and use in-script annotations for Octave as well. This effectively results in something like WPS4Octave and WPS4Matlab. We parse the script in Java to generate the process metadata and then pass the calls in the script to Matlab or Octave.

Matlab or Octave can run remotely or on the same machine for this.

Existing resources:

• matlab-connector (GPL3, UncertWeb/Richard Jones),The matlab-connector library enables function execution on a remote MATLAB instance. JSON is used as a platform and language independent.
• JavaOctave: https://kenai.com/projects/javaoctave/pages/Home resp. http://sourceforge.net/projects/javaoctave/ (old) new: https://kenai.com/projects/javaoctave/sources/source-code-repository/show
• ps-examle (UncertWeb)
• ...

Estimate: potentially high, but would have similarly easy integration for script developers.

Thoughts: I prefer the highest resusability of existing scripts. Though, we should assess the libs in the first place. -- MatthesRieke - 2013-06-24

Configuration File

Implemented by https://github.com/autermann/matlab-wps

Thoughts: Easy configuration file, no changes to existing scripts.

Command line interface

Calling full scripts from the command line. This could best be wrapped in a MovingCode package which effectively results in having "MC Octave/MC Matlab", respectively OctaveCLIProcessor and MatlabCLIProcessor similar to OctaveCLIProcessor similar to PythonCLIProcessor. The process metadata is written by hand and a wrapper script might be needed to handle the input and output data.

Existing resources:

• http://www.gnu.org/software/octave/doc/interpreter/Invoking-Octave-from-the-Command-Line.html

Estimate: potentially low, quickly done for specific scripts, but process not as automated.

Thoughts: Clean with low overheads, but might reduce reusability of scripts (users have to provide process metadata, e.g. as WPS ProcessDescription). -- MatthesRieke - 2013-06-24

Hard-coded

Java processes call Matlab/Octave methods. Similar to the ps example we effectively do all input and output handling in Java and only pass on simple types/data to Octave for the calculation.

Estimate: Not really nice... too inflexible, people must write Java and Matlab/Octave.

Thoughts: Veto. -- MatthesRieke - 2013-06-24

Issues

...

Use Cases

INTAMAP2 - ASTON / GeoViQua

What would be nice is to have something like the WPS4R but for Octave. This fits into GeoViQua in WP3: A domain scientist writes Matlab/Octave regularly and would like to have some method that he could annotate my Matlab/Octave function with, and these would then be exposed as WPS / Geoprocessing Service framework services, this gives me a way to quickly provide methods to compute quality indicators.

Use case: Collocated data from GECAaaS is forwarded in a WPS service chain to another WPS that calculates quality indicators using an Octave script.

INTAMAP2 is available here: http://uncertws.aston.ac.uk/madhu/. The service interface uses Richard Jones' processing service framework, not WPS, and connects to Octave (see intamap_Madhu.pptx). The whole thing is very simple in the end, once you know the right libraries to use.

Code is provided under Apache 2.0 license:

• client.zip
• ps-example.zip
• Octave code: gpmlab-octave.zip
  • Attached is a version of GPMLab which runs in Octave 3.2.4.
  • You need to first compile the mex files (see README). There is a function, at the root, called build_emulator.m. This builds a multi_output emulator (integrating out all hyperparameters but length scales, using Matern3 kernel). You need to pass in xtrn, ytrn and xpred. It returns the mean and std dev of ypred as matrices (1 column per output). See the test_build_emulator.m script for an example usage.
• Java interface: InterpolateOctave.java