You are here: Wiki>Geostatistics Web>WPS4R>WPS4RDevelopment>WPS4RBacklog (02 Oct 2013, DanielNuest)Edit Attach

WPS4R Backlog

Legend

Priorities: 1 = highest priority, n = lower priorities

Costs: "Felt" development costs, relative, no actual estimation neccessary, going from led-aqua = "low" to led-aqualed-aqualed-aqualed-aqua = "very high"

Statuses: unchecked = open task, checked = somebody is working on the task, DONE the task is "done". We have a very loose definition of done: "Contributor did some tests and it worked."

Group: Can be used freely to associate items together, but is optional.

Items for Version 1.1 of the Backend

Group Item PrioritySorted ascending Costs Contributor/Status
feature As a user I want to use the INTAMAP functionalities based on WPS4R. 1 led-aqua led-aqua led-aqua unchecked
doc As a user I want the the documentation to be up to date, especially the tutorials. All documentation must be checked and partially updated. Transfer backend documentation to Wiki from here to here 1 led-aqua led-aqua unchecked
dev Fix warninigs in integration tests (issue with RServe), email by MH. 1 led-aqua led-aqua unchecked
dev As a developer I want the integration tests to run on the build server (using remote RServe?) 3 led-aqua led-aqua led-aqua unchecked
doc As a user I want a demonstration of the capabilities with spatial input and output data, best based on INTAMAP functionality and common input formats, also demonstrating the data conversion facilities of the WPS, e.g. shp input to INTAMAP process with GML output. 4 led-aqua unchecked

Items for later Version

Group Item PrioritySorted ascending Costs Contributor/Status
  As a script developer I want to be able to send different log statements to the runtime environment (as part of a wps4R extension package) which are added with log levels ERROR, INFO and DEBUG to the log file of the WPS.     unchecked
  As a developer I want to use/develop/test the annotation parsing mechanism standalone as well, e.g. for integration into MovingCode. 1 led-aqua led-aqua unchecked
  As a new user I want a very short (no scrolling) description of WPS4R in website R-backend description and more details moved to wiki pages. 1 led-aqua unchecked
  Check before running a script if all required R packages are installed. If not do not attempt to start the script but throw an error (option: autoinstall?). 1 led-aqua unchecked
 

As an adminstrator I want to ensure that no harmful code is executed in my runtime environments based on a simple blacklist of commands.

1 led-aqua led-aqua led-aqua unchecked
  As a user I want to be informed about annotation validation errors then I call a process. 1 led-aqua unchecked
  Check https://wiki.52north.org/bin/view/Geostatistics/WPS4RDevelopment#WPS4R_Source_Setup and update the tutorial (https://wiki.52north.org/bin/view/Processing/TutorialBackendR) with new configuration options. 2 led-aqua led-aqua unchecked
  As a user I want to be notified about missing dependencies (by scanning the require and library statements of my script) before the process is started. 2 led-aqua led-aqua unchecked
  As a user I want to validate R scripts locally as part of a wps4R R extension package. 2 led-aqualed-aqua %ICON{led-aqua} unchecked
  As a script developer I want to run the same script that works in WPS4R in a moving code package. 2 led-aqua led-aqua unchecked
  Include sessionInfo() as text in every execute response 3 led-aqua unchecked
  As a user I want to upload several versions of scripts and have this included in provenance information documents. 3 led-aqua led-aqua unchecked
  Create WPS4RExamples and WPS4RAnnotations wiki page. 4 led-aqua led-aqua unchecked
  As a script developer I want to submit new R script using the transactional WPS interface (WPS-T) 4 led-aqua unchecked
  As a user I want to use extended INTAMAP functionalities with a temporal component. 5 led-aqua led-aqua unchecked
  As an adminstrator I want to ensure that no harmful code is executed in my runtime environments based on a sandbox. 5 led-aqua led-aqua led-aqua unchecked
  As an administrator I want to run WPS4R in a public cloud environment (Microsoft, Amazon, ...) 5 led-aqua led-aqua led-aqua unchecked
  As a user I want to use a generic WPS4RforSweave process so that I can upload a Sweave file and get pdf/dvi/latex as output. Potentially the last one could be used as input for the first two proceses using a process chain. I want to run this process from an wps4R extension package and have a conveience function that let's me check if the process is finished yet and if so it download the output file. 5 led-aqua led-aqua led-aqua unchecked
 

As a user I want to run asynchronous R processes so that I...

  • can query the status of a process.
  • can abort a process.

7 led-aqualed-aqua led-aqua led-aqua unchecked
  As a script developer I want to use an R API (as part of a wps4R extension package) to communicate the progess of the process to the runtime environment. 8 led-aqua led-aqua unchecked
  As an administrator I want to install R scripts interactively in the web admin backend. Or even have a full console there. 10 led-aqualed-aqua unchecked
  As a user I want to see the R version and available packages in the capabilities file of the WPS. This should be done via a generic mechanism in the process repository based on the ows:AbstractMetaData element. 10 led-aqua led-aqua unchecked
 

As a user I want to outsource parts of my R script by upload RData files to a WPS and getting RData files back. The wps4R package should allow me seamless integration of these resources.

  1. R Datentypenals input und output unterstützen, siehe http://geoprocessing.forum.52north.org/problem-with-annotating-the-R-script-td4025097.html [Ist: etwa 5-6 Stunden]
    • Konfigurationsdatei Datentypen (mapping extension <> mimetype)
    • .RData (damit geht ja dann quasi "alles") als Input [4 Stunden]
    • .RData als output [4 Stunden]
    • Demo mit spatial class gewrappt in .RData (and dadurch auch die davon abgeleiteten SpatialPoints, SpatialPolygons, SpatialLines, SpatialGrid und deren ~DataFrame Varianten) [4 Stunden]
      • application/rData+Spatial und application/rData+SpatialPoints und application/rData+SpatialPolygons > Benennung der Variablen
      • Beispielskript für Points und auf WPS4RExamples basierend auf Zeilen 19, 26, 27, 28 von http://www.asdar-book.org/book/geos_mod.R aus dem ASDAR Buch (bitte verlinken im Beispiel) [1 Stunde]
        • Inputs:
          • SpatialPointsDataFrame (in Code: meuse), sollte im Beispiel dann einen generischeren Namen bekommen (z.B. inputSPDF). Dies gilt auch für die Zwischenergebnisse, genauer gesagt zn.lm dass besser linearModel oder so heißen sollte.
          • x-Koordinatenname als Text (in Code: "x")
          • y-Koordinatenname als Text (in Code: "y")
          • linear model als Text (hier: "log(zinc)~sqrt(dist)")
        • Outputs:
          • SpatialPointsDataFrame mit linear model (in Code: meuse)
          • Text: Feldname für Modelergebnis (in Code, und vermutlich auch im Beispiel: fitted.s)
          • Text: Feldname für Modelresiduen (in Code, und vermutlich auch im Beispiel: residuals)
10 led-aqua led-aqua unchecked
Gen. As a user I want to upload/manage (CRUD) resources for specific processes using a process, or using specific operations (e.g. from Spacebel WPS?!) 11 led-aqua led-aqua unchecked
  As a user I want to use compiled R code (http://dirk.eddelbuettel.com/blog/2011/04/12/) to increase performance - maybe this could even be applied automatically in WPS4R? 15 led-aqua led-aqua led-aqua unchecked

Other Tentative Tasks and Ideas

  • Research about security mechanisms in INTAMAP, RWeb and OpenCPU [DN]
  • WPS4R for open, transparent climate science might be a great usecase: http://www.easterbrook.ca/steve/?p=2865
  • wps4R client extension package for R [DN]
    • test annotated scripts offline
    • upload scripts
    • wpsSessionInfo()
    • wpsResourceByName()
  • Possible features based on User Stories
    • O&M Parser for R [4] (could be re-used in sos4R)
    • WPS client in R which also allows local execution of scripts (via boolean flag, or using a wrapper function that contains inputs and returns a list with outputs) [6, 7]
    • Automatic deployment of processes from R (simple switch between local and remote execution) [1, 2, 5]
    • errors and warnings in execute response, including tracebacks [8]
    • 1:n > one maintainer (script uploader), many users [1, 2, 5, 8]
    • n:m > many users who can upload scripts: WPS user management to allow script upload, but not service configuration [3, 6]
    • multiple processing sessions using one script (distinct environments)
    • A framework or environment which allows to share WPS processes in the Web [1, 5, 6, 8]
      • moving code paradigm, app-store model > dependencies must be included in annotations (documented, "tested with versions")
    • Compatibility assurance / testing WPS clients for WebGIS with WPS4R [1, 8]
    • documentation of tested package versions of a script (metadata) [1, 3, 5, 7]
    • downloading whole scripts and the package versions it was tested/deployed with [1, 7, 8]
      • should it be possible to disallow this? Intelectual property on an algorithm?
      • should there be annotations that specifiy (when uploading) which package versions were used, and what should the service response if the version do not match the ones installed?
    • process sandbox (handle things like conflicting package versions?) [3, 5]
  • Could WPS4R help in reproducible research?

Metadata

  • Topic created by: DanielNuest
  • Topic created on: 2013-04-14
Topic revision: r7 - 02 Oct 2013, DanielNuest
Legal Notice | Privacy Statement


This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Wiki? Send feedback