Open Sensor Search Software Development

General information, goals and roadmap can be found on the page OpenSensorSearch.

Architecture and Features (under development!)

Architecture

• Solr versus Elasticsearch: https://docs.google.com/spreadsheet/ccc?key=0AqZCsXqvHsQfdDd1SEsyQ3pna0dzR3JWT1NiVFFHaXc#gid=0

Libraries to use

• FEST and mockito
• Spring MVC
• Rhino (for Javascript because of Sandbox feature), see chapter 3.4, 4, 5 of "Javascript on server" - book
• use Json-schema: http://json-schema.org/

Things to consider

• What level will the RESTful API be? Read http://www.slideshare.net/cappelaere/rest-level-5-a-trek-to-the-summit
• Feature developments are weighted with the 80/20 rule in mind: http://en.wikipedia.org/wiki/Pareto_principle
• solr just supports locations, not bounding boxes, which is fine (using centers of bboxes)
• ...

Modules (TBD)

• binding
  • opensearch
  • rest
  • sir-kvp
  • sir-pox
• coding
  • json
  • (json-schema?)
  • ows
  • sir
  • sensorML
  • kml
  • atom
  • rss
  • ebrim
• convert
  • sml-ebrim
  • json-sml
  • sml-json
• core
  • api
  • oauth
  • test
• db
  • lucene
  • (mongo?)
• harvest
  • sos100
  • sos20
  • ioos
  • xively
  • thingspeak
• operations
  • core (GetCapabilities, DescribeSensor, SearchSensor, the RESTful equivalents)
  • transactional (InsertSensorInfo, DeleteSensorInfo, UpdateSensorDescription, HarvestService, their RESTful equivalents)
  • status (GetSensorStatus, InsertSensorStatus)
  • catalog (ConnectoToCatalog, DisconnetFromCatalog)
  • transform (SensorML to ebRIM, ...)
• webapp
  • admin
  • client

Resources

• /sensors
• /phenomena
• /services
• /stations (by using to Sensor Web Client's server component?)

Development Process

• Agile development with Scrum
• TDD with all tests following the F.I.R.S.T. principles: fast, isolates, repeatable, self-validating, timely.
  • Unit tests for day-to-day developments
  • Before accepting pull requests into the main repository, integration tests have to run successfully

Product Backlog

Backlog is managed on easybacklog.com > contact DanielNuest to get access.

Development Version

Basics:

• We use the fork & pull development model.
• "No bugfix without a test."
• Bugs are managed with GitHub issues.
• We use the 52°North Eclipse code formatter, see Documentation

Get a local development version in Eclipse

• Fork the repo on GitHub.
• Clone the fork into you local repository.
• Add the local repository to Eclipse, e.g. using e-Git.
• Go to the working directory, right click and import as "general project", click Finish.
• After the import, right click on the project and Configure > Convert to Maven Project
• (Alternatively:) New > Project > Select Wizard "Checkout Maven Projects from SCM"

Building with Maven

Build the project

mvn clean install

Notice File

A notice file is mandatory for all projects licenced under Apache 2.0, so we use this handy plugin to create one for us. Important: This only works automatically for Maven dependencies!

• Check if the notice file is up-to-date, which will also check if there are missing licenses for Java libraries: mvn notice:check
• Generate the notice file: mvn notice:generate

When you add a new library or dependency, run the command below and see if it is added to the file LICENSE in the project root. If not you have to manually add the information to the file src/main/resources/license-mappings.xml.

License Headers

The licence header plugin can be annoying at times, but it keeps the code alsways in a good state, therefore it always runs on every build.

• Check which files miss the header: mvn license:check
• Add the headers to these files: mvn license:format

If you include external libs as files don't forget to add their path/names to the exclusion patterns!

Show Gitlog

The gitlog plugin is used to create a CHANGES file containing all the information that was changed between versions. This should motivate you to write reasonable commit messages!

• Show the output of gitlog in command line: mvn gitlog:show

The gitlog is not yet included the default build process but will be once we add the proper assembly configurations.

Run Integration Tests

1. Configure and Start Solr
   • See http://wiki.apache.org/solr/SolrInstall for full documentation.
   • You can either install solr to your existing servlet container, or just download it and run it from the command line.
   • Deploy to servlet container
     • Follow the instructions here
   • Run from the command line
     • Download and extract Solr: http://lucene.apache.org/solr/downloads.html
     • Got to directory <unpack-destination>/solr-4.x.x/example/solr/collection1/conf/
     • Replace the contents of the file schema.xml with this file from the OpenSensorSearch project: OpenSensorSearch/52n-sir/src/main/resources/xml/SolrSchema.xml
     • Go to the folder <unpack-destination>/solr-4.x.x/example/, run the jar file with : java -jar start.jar, for more instructions see <unpack-destination>/solr-4.x.x/example/README.txt
2. Configure and Start PostgreSQL
   • See https://wiki.52north.org/bin/view/SensorWeb/SirTutorial#Create_Database
   • Create new database based on PostGIS 2.0 template (e.g. with pgAdmin)
   • Create table structure using OpenSensorSearch/52n-sir/src/main/resources/datamodel_sir.sql
   • Change the connection settings in OpenSensorSearch/52n-sir/src/main/webapp/WEB-INF/db.properties=
3. Open a console in the project root directory and make sure Maven is on the command line
4. Run mvn verify
5. ...

Dummy Data Generators

TBDocumented: the old one with SensorML, the new one by Mohammad