You are here: Wiki>Geostatistics Web>WPS4R>WPS4RDocumentation (21 Nov 2016, DanielNuest)Edit Attach

WPS4R Documentation


General documentation page for using and installing WPS4R based on the 52°North WebProcessingService. For typical use cases please also see the WPS4R tutorials.

ALERT! WPS4R is unstable and under development. The current release of the 52°North WPS might not provide the most stable functionality. Also, the tutorials in this Wiki are potentially outdated. Please get in touch on the community mailing list if you want to use WPS4R! Thanks for your understanding. ALERT!

Using WPS4R

Annotate a Script

For working with R scripts, any text editor can be used. But it is easier to use GUIs with syntax highlighting, such as: In order to expose a R script as a WPS process, you will have to annotate the script. These annotations are described in the following.

Syntax and semantics

There are three types of annotations, which might occur anywhere in the script: wps.in, wps.out and wps.des. Arguments are passed either sequential, as unordered key-value pairs or as composite of sequence and key-value pairs. The wps.des-annotation occurs only once in a script. Every input / output variable refers to one annotation and vice versa. Mandatory arguments for each wps.in- and wps.out-annotation are "identifier" and "type". For R, annotations simply appear as comments, so each line has to start with '#'.

Description

wps.des: id, title, abstract;

wps.des - annotations must occure once in a script. They provide general information for the process description.

Optional fields: version

Resources

wps.resource: resourceFile;

The files are saved, perserving the relative path structure, into the process working directory. Files are loaded from the directory specified in the R_resources configuration paramter (see below).

Example

#wps.resource: test/dummy1.txt, test/dummy2.png;

Input

wps.in: id, type, title, abstract, value = null, minOccurs = 1, maxOccurs = 1>;

wps.in - annotations occur once for each input variable. They provide information for the process description as well as for parsing the input. Before the script is executed, the WPS will initialize a variable of the declared name ("id"), which either holds a literal value or a file URI (path as string), which points to a data file.

Examples

wps.in: variable1, integer, this is variable1, further description;

wps.in: variable2, string, abstract = further description, value = "abc";

Output

wps.out: id, type, title, abstract;

A wps.out - annotation usually occurs once in a script and denotes the output variable. Similar to the wps.in annotation it provides information for the process description and data handling. The corresponding output variable has to be initialized from within the script.

Annotation Parameters

The following parameters are used within annotations.

Attribute Type Annotation Mandatory Description
abstract String wps.des,
wps.in,
wps.out
No Equals Abstract-arguments of process description
identifier String wps.des, wps.in, wps.out Yes Equals Identifier-arguments of process description
Id in wps.des currently does not affect anything, cause retrieved from filename or upload form
minOccurs Integer(0 or 1) wps.in No Equals minOccurs-arguments of process description; Set to 0 if a default value was given, otherwise 1
maxOccurs Integer wps.in No Equals maxOccurs-arguments of process description
Multiple inputs for one id are currently not supported
title String wps.des, wps.in, wps.out No Equals Title-arguments of process description
type String wps.in, wps.out Yes Type argument for input or output according to the list in section "Supported input/output types"
value Any value(to be read from R) wps.in No Set a default literal value to a process input; this means minOccurs=0, maxOccurs=1

Script example

# wps.des: simpleExample, title = A Simple WPS Process, 
# abstract = Example Calculation with R; 
# wps.in: input, integer; 
# wps.out: output, double; 

# calculate something... variable "input" doesn't have to be initialized
output = runif(1)*input

Supported Input and Output types

The following data types and mime types are currently supported by WPS4R. Default encoding is UTF-8. Note that every complex input has to be imported from within R, therefore you got to rely on packages like "rgdal", for example, if you want to import a shape file, use:

readOGR(<inputID>, sub(".shp", "", <inputID>))

Literal data

Data type Annotation key Input Output
double double Yes Yes
integer integer Yes Yes
string string Yes Yes
boolean boolean Yes Yes

Complex data

Raster

Data type Annotation key Input Output
application/dbase dbf Yes Yes
application/geotiff geotiff Yes Yes
application/x-geotiff geotiff_x Yes Yes
application/img img Yes Yes
application/x-erdas-hfa img_x Yes Yes
application/netcdf netcdf Yes Yes
application/x-netcdf netcdf_x Yes Yes
application/remap remap Yes Yes

Image

Data type Annotation key Input Output
image/geotiff geotiff_image Yes Yes
image/gif gif Yes Yes
image/jpeg jpeg Yes Yes
image/png png Yes Yes
imgage/tiff tiff Yes Yes

Vector

Data type Annotation key Input Output
application/dgn dgn Yes Yes
application/shp shp Yes Yes
application/vnd.google-earth.kml kml Yes Yes
application/x-zipped-shp shp_x Yes Yes

Text

Data type Annotation key Input Output
text/plain text Yes Yes
text/xml xml Yes Yes

Installation and Configuration

Configuration parameters

The following parameters can be configured from the admin console:
  • R_Rserve_Host (default: localhost)
  • R_Rserve_Port (default: 6311)
  • R_Rserve_User and R_Rserve_Password for authenticated connection to Rserve (only if Rserve requires login)
  • R_enableBatchStart - try to start Rserve on the local machine
  • R_utilsScriptDirectory - set the directory that utils scripts are loaded from. This can be a directory relative to the webapp, a full path, or a relative path to a *file* within the classpath (no folders are supported on the classpath and files from the classpath are copied to a temp file so that R can load them).
  • R_wdName - the path for the manually set work directory or base directory in conjuction with the strategy manualbasedir.
  • R_resourceDirectory - the (relative) path to a directory with resources that can be requested in scripts (default: R/resources)
  • R_scriptDirectory - one (or several delimited by ";") folder where R scripts are located and loaded from (default: R/scripts)
  • R_cacheProcesses - if deactivated, the scripts files are loaded every time the algorithm is called, for development (default: true)
  • R_session_memoryLimit - set the R session memory limit (default: 1000)
  • R_wdStrategy - influences WPS4R on choosing the R working directory for each process run. Possible values:
    1. default:
      • Local: R uses the temporary WPS workdirectory within the tomcat folder.
      • Remote: R uses the default workdirectory depending on the setup of R and Rserve, and creates a randomly named subdirectory as workdir for each process run.
    2. preset: The used working directory completely relies on the configuration of R and Rserve, and a randomly named subdirectory as workdir for each process run.
    3. temporary: R will create a temporary folder if running on a remote machine. If it runs on localhost it uses the existing temporary working directory of the WPS.
    4. manual: A custom path (relative or absolute filepath as value) to be used as working directory.
    5. manualbasedir: Use a custom path as the base directory for creating randomly named working directories for each process run. The path (relative or absolute) must be configured in the parameter R_wdName.
  • R_datatypeConfig (default: R/R_Datatype.conf): Location of a config file were you may add costum data types that WPS4R should handle (see below).

Create and Manage Process

  1. How to upload processes
    1. Open Web Admin Console in your browser (e.g. http://<service host>:<service port>/wps/webAdmin/index.jsp)
    2. Click "Upload R Script"
    3. Choose an annotated R script (Process id will be org.n52.wps.server.r.[id]). Up to WPS release 3.2.0, the process id is derived from the filename, recent versions derive it from the wps.des annotation inside the script
    4. BETA: Optionally type an alternative process name into the submit form; process id will be org.n52.wps.server.r.[process name]
    5. Click "submit", process will be submitted and added to the LocalRAlgorithmRepository
  2. How to manage processes
    1. Open Web Admin Console in Browser
    2. Go to "Algorithm Repositories", scroll down name "LocalRAlgorithmRepository"
    3. Among the properties are the registered processes. They got the name "Algorithm" and a value which is the process identifier
    4. BETA: Delete processes by clicking "x" * , activate / deactivate them with the checkbox
    5. Click "Save and Activate configuration" to make changes take effect
  3. Handle errors
    • If the script is causes errors that prevent the process from being executed, the repository view will display errors with icons beneath the listed processes. Click on these icons to display error messages:
      • A questionmark "?" appears if the process not available for any reason
      • An exclamation mark "!" appears, if the script contains invalid content, for instance annotations with wrong syntax.
    • Note that not all errors can be detected before the process is executed. It is recommended to test R scripts locally before uploading them to the server.
    • The execute response also contains errors and warnings passed by R. Warnings can be retrieved from the "warnings" - output that features every R process.
    • Flawed scripts can be replaced by re-uploading a script with same filename.

GML Support

Using type = shp_x or type = application/x-zipped-shp as an attribute to input and output annotation enables internal conversions from various GML formats to shapefile and backwarts. Hence the default input / output format will be shapefiles, but also GML occurs in the list of supported data formats of the process description. Hence if GML is feed to an input, a GML parser will generate a shapefile and pass it to R. Vice versa, the WPS generates GML from a shapefile if GML is the requested output format.

Add costum data types

To add costum data types (currently only file based), do the following:
  1. Open the config file, add an entry to thecomma separated list e.g rdata, application/rData, file rdata will be the key used within the R-script. application/rData will be the mimetype that occurs in the proccess description, file is a hint for wps4r on how to handle the data type.
  2. From the admin console, configure the GenrericFileParser and GenericFileGenerator so that they support the data format as well. In this example, it is one entry for each with mimetype "application/rData" and encoding "default" or "base64"
Topic revision: r20 - 21 Nov 2016, DanielNuest
Legal Notice | Privacy Statement


Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Wiki? Send feedback