SOS 5.x Documentation

---

Quick Links

About

How to install

How to Develop

---

Build status

Jenkins:
Travis:

---

Introduction

Scope

This document describes the architecture and the install process of the 52°North Sensor Observation Service (SOS).

Changes compared to previous versions

The 52°North SOS 5.x is an improved implementation of the 52°North SOS. Here are the biggest changes from the previous version 4.x:

• Use Arctic Sea
  • Extracted code from the previous SOS 4.x
  • Binding (iceland)
  • De-/Encoding (svalbard)
  • OGC OWS service framework (iceland)
  • OGC entity API (shetland)
  • Settings (faroe)
  • Utilities (janmayen)
• new database model: Support SOS, Sensor Web Server Helgoland, Sensor Web Server STA, simplification, performance
• Spring framework for bean loading

Some Words on OGC SOS

The information about the OGC SOS specifications are described in SensorObservationServiceDocumentation

The OGC SOS version 2.0 specification defines a core and five extensions (in SOS 1.0.0 called profiles). The current release implements these operations:

Extension	Operation	Description	KVP	SOAP	POX	JSON	EXI
Core	GetCapbilities	for requesting a self-description of the service
Core	GetObservation	for requesting the pure sensor data encoded in Observation&Measurements 2.0 (O&M 2.0).
Core	DescribeSensor	for requesting information about the sensor itself, encoded in a Sensor Model Language (SensorML) instance document.
Enhanced Operations	GetFeatureOfInterest	for requesting the features hosted by this SOS instance
Enhanced Operations	GetObservationById	for requesting a single observation instance by its id	/
Result Handling	InsertResultTemplate	for inserting a result template for later easy observation retrieval
Result Handling	InsertResult	for inserting results for a before registered result template
Result Handling	GetResultTemplate	for requesting a template of the result structure returned by a GetResult request
Result Handling	GetResult	for requesting results without observation metadata and information about the result structure
Transactional	InsertSensor	for adding a new sensor
Transactional	InsertObservation	for adding a new observation
Transactional	UpdateSensorDescription	for updating the description of an already inserted sensor
Transactional	DeleteSensor	for deleting a sensor/procedure and all related offerings, observations
Data Availability	GetDataAvailability	for requesting the data availability for certain configurations (not specified in the SOS 2.0)
Delete Observation	DeleteObservation	for deleting one single observation identified by id (not specified in the SOS 2.0)
Insert Feature	InsertFeatureOfInterest	for inserting feature of interest into the SOS (not specified in the SOS 2.0)

= implemented = not implemented = not specified

Bindings

[A binding] describe[s] how SOS 2.0 clients and servers can communicate with each other. (OGC-006)

• KVP - Key Value Pair encoding of requests via HTTP GET.
• SOAP - SOAP based encoding of all extensions via HTTP POST.
• POX - XML based encoding of all extensions via HTTP POST ( without SOAP wrapping).
• JSON - JSON based encoding of all extensions via HTTP POST. The JSON implementation is based as much as possible on the abstract definitions of the specifications but it is not yet specified and not all features are supported.
• EXI - Efficient XML Interchange (EXI) Format 1.0 based encoding of all extensions via HTTP POST and POX encoded request. SOAP requests are currently not supported via the EXI binding.

Architecture

This image describes the basic architecture ot the 52°North SOS 5.x.

Click to enlarge.

Installation

Requirements

This tutorial is based upon the following software which have to be downloaded and installed.

• Java runtime environment (JRE) 8.0 or higher (or JDK 8.0 or higher if you want to build the SOS from source): Download
• Java Servlet-API 2.5 or higher compatible application server. E.g.:
  • Tomcat 8 or higher: Download
    • Do not use Tomcat version 8.0.8 because is a known bug in these releases whereby the SOS installation fails. Please use an newer Tomcat release.
    • Tomcat 10 does not yet work as Tomcat has changed from javax to jakarta!
  • Jetty 6 or higher Download
  • Glassfish 3 or higher [[http://glassfish.java.net/public/downloadsindex.html#top][Dowsos_architecture.pngnload]]
• Running Database Managment System
• Apache Maven 3 or higher (if you want to build the SOS from source): Download

Supported Database Managment System

• PostgreSQL/PostGIS
  • PostgreSQL 9 or higher: Download
  • pgAdmin3 if you need to create a new database: Download
  • PostGIS 2.0 or higher: Download
• Oracle
  • Oracle 9 or higher: Information
• MySQL (currently only when Building from Source)
  • MySQL 5.6 or higher: Information
  • Community Edition: Information and Download and for Windows-Users Download for Windows
• Microsoft SQL Server:
  • Microsoft SQL Server 2012: Information

A1) Download the SOS WAR-File

If you want to build the SOS from source you can skip this point and proceed with B1) Building from Source.

Download package including the WAR file:

• 52°North SOS latest version

The 52°North SOS includes the Sensor Web Server REST-API and the Sensor Web Viewer Helgoland..

Unzip the package and browse to UNZIPPED_PACKAGE/bin/target where the WAR FILE is located.

B1) Building from Source

If you want to use the binary distribution of the SOS you can skip this step and proceed with 2) Installing the Webapp.

Make shure you have all required Software installed.

• Checkout the sources directly from our subversion repository
  • Checkout the sources directly from our subversion repository: git clone https://github.com/52North/SOS SOS
  • or checkout the latest code from the main development line: git clone https://github.com/52North/SOS/tree/develop SOS
• change to the directory where you've just checked out the source code
• run mvn package to build the webapp.
• copy webapp/target/52n-sos-webapp.war to a directory of your choice
  • Or the bundle can be found in her webapp-bundle/target/52n-sos-webapp.war

C1) Building a Preconfigured Service

If you don't want the built-in installer you also can configure the SOS while building it:

• To define the database connection settings copy conf/datasource.properties.template to conf/datasource.properties and adjust it to your needs.
  • typically only the user name/password and URL properties have to be changed. While user name and password are quite self explanatory the URL is constructed like the following: jdbc:postgresql://$HOST:$PORT/$DATABASE_NAME
• To adjust service settings open the copy the file db/default_settings.sql and edit to your needs and recreate conf/default_settings.db (e.g sqlite3 misc/conf/default_settings.db -init misc/db/default_settings.sql).
• To build the project run mvn package -Pconfigure-datasource,use-default-settings[,additional-profiles]
  • to choose a different file than conf/datasource.properties pass it to Maven by specifying -DdatabaseConfigurationFile=ABSOLUTE_PATH_TO_FILE at the command line
  • to choose a different file than conf/default_settings.db pass it to Maven by specifying -DsqliteSettingDatabase=ABSOLUTE_PATH_TO_FILE at the command line
  • any profile of the standard build process (see above)
• After building the preconfigured webapp is located at webapp/target/52n-sos-webapp.war

D1) Using a docker image

Docker images are automatically deployed to Docker Hub.

The configuration is located in `/etc/sos`. The directory is per default a volume and when no other directory is mounted (see also the [Docker documentation](https://docs.docker.com/storage/) for pre-populated volumes) a default configuration using H2 is used. This default installation uses the admin credentials `admin:admin`. If you want to install the SOS, you have to mount a empty directory instead of the default volume.

The cache file is located in /var/lib/jetty/webapps/ROOT/tmp. The directory is per default a volume.

A custom Helgoland settings.json can be mounted to /var/lib/jetty/webapps/ROOT/static/client/helgoland/settings.json. This is needed, e.g when the external URL of the SOS differs from http://localhost:8080/.

Examples

Default:

 docker run -p 8080:8080 52north/sos:latest

With a local configuration folder:

 docker run -p 8080:8080 -v ./config:/etc/sos 52north/sos:latest

Remote debugging enabled:

 docker run -p 8080:8080 -p 8000:8000 -e 'JAVA_OPTIONS=-Xdebug -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n' 52north/sos:latest

2) Installing the Webapp

Deploy the war file

There are two options how to deploy the 52n-sos-webapp.war file:

1. Via Tomcat manager

• Navigate to the Tomcat Manager, e.g. http://localhost:8080/manager/html
  • By default the file upload size is limited to 50mb. See here how to increase the limit.
• Scroll down to the section “WAR file to deploy” and select the WAR file which you've build or downloaded in the previous step.
• Click on “deploy”. The new service instance will be shown in the application list.

2. Via copying the file to the webapps folder

• Copy your 52n-sos-webapp.war file into <TOMCAT_BASE>/webapps and start the Tomcat server. For Linux instance:
  

  sudo service tomcat8 start

Datasource configuration

On the datasource page of the installation wizard you can select the database managment system which you would like to use.

The datasources are provided as default and as Custoom (Core) where custom core shoult be used if you have a custom database model and use modified hibernate mapping files.

DBMS	Default	Custom (Core)	Comment
H2/GeoDB			provided as file based and in memory
MySQL/MariaDB			-
Oracle Spatial			-
PostgreSQL/PostGIS			-
SQL Server			-

Datasource settings

• Database Configuration
  • User Name: database user name
  • Password: dabase user password
  • Database: database name
  • Host: host name or URL to the database,e.g. localhost
  • Database port: the port of the database
• Advanced Database Configuration
  • Schema: the schema/catalog of the database
  • Database concept: Select from dropdown
    • Simple database model (creates only the minimum required database tables. No transactional operations support!)
    • Transactional database model (default)
    • eReporting database model (is an extended transactional database model)
  • Database extension: Select from dropdown
    • Default database model (not additional tables, default)
    • Extended model to support Samplings/MeasuringPrograms (addtitional tables to support samplings and measuring programs)
  • Feature concept: Select from dropdown
    • Default feature concept (not additional tables, default)
    • Extended feature concept (additional tables to support WaterML MontiforingPoint. Not yet transactional supported!)
  • Provided JDBC drive: should the SOS should not deregister the JDBC driver during the shutdown (e.g. provided by Tomcat)
  • Minimum ConnectionPool size: the minimum size of available database connections
  • Maximum ConnectionPool size: the maximum size of available database connections
  • Batch size: database insert/update batch size
  • Datasource time zone: define the time zone of the datasource to ensure time is always queried in the defined time zone. Default is UTC.
  • Datasource time string format: define the time string format of the datasource to ensure time is always parsed in the defined time format
• Actions
  • Create tables: this will create the necessary tables in the database.
  • Delete existing tables: this will delete all existing tables in the database.
  • Force updating existing tables: this will update all existing tables in the database if needed.

PostgreSQL installation

• Make sure your Tomcat and PostgreSQL are running.
• Create a new PostgreSQL database with e.g. pgAdmin3 using the PostGIS template created during the PostGIS installation.
• Navigate to the Tomcat Manager, e.g. http://localhost:8080/manager/html
• Scroll down to the section “WAR file to deploy” and select the WAR file which you've build or downloaded in the previous step.
• Click on “deploy”. The new service instance will be shown in the application list.
• Click on the service instance to navigate to your newly deployed SOS.
• Setup SOS server
  • Now you have 52N SOS running in your Tomcat server. To access it, open this URL in your browser:

    http://localhost:8080/52n-sos-webapp

    You will find the 52N SOS home page with the following message:

    You first have to complete the installation process! Click here to start it.
            

• Follow the steps on the screen to configure your SOS instance (you don't have to do this if you've build the preconfigured webapp). More information about the settings can be found in the Configure/Administrate the SOS section of this page.
  • If you have installed the preconfigured webapp change the administrator password using the admin GUI. The default password is password and the default user name is admin.

Oracle installation

We assume that an Oracle 11g or higher instance is already running and accessible through the network.

1. Create an Oracle user

We need to create an Oracle user with the following configuration:

• Connect permission
• Create table permission
• Create sequence permission
• Unlimited tablespace
• Quota unlimited on users

To create a sos user with this configuration, you can use the following sqlplus commands:

> create user sos identified by *password*;
> alter user sos identified by *password* quota unlimited on users;
> grant connect, create table, create sequence to sos;
> grant unlimited tablespace to sos;

2. Get the Oracle Instant Client (OCI)

52n SOS supports either thin (pure java) or OCI (more performant, platform specific) Oracle 11g drivers. The OCI driver is recommended for production.

Get the thin (pure java) JDBC driver from Oracle's website: https://www.oracle.com/database/technologies/appdev/jdbc-downloads.html

Get the platform specific Oracle Instant Client ( OCI) from Oracle's website: https://www.oracle.com/database/technologies/instant-client/downloads.html

Note: If the full Oracle 11g has been installed in the same machine, the Instant Client is already there.

3. Install and configure Tomcat

We'll need Tomcat 8 or above. Download it from Tomcat's website, or use the package manager from your Operating System.

1. Make sure that Tomcat will have access to the OCI driver files.

   • LD_LIBRARY_PATH (unix) or PATH (windows) environment variable should point to the location where Oracle Instant Client is installed.
   • The ojdbc8.jar file in this same location should be included in tomcat's classpath. For instance, make a symbolic link to it in <TOMCAT_BASE>/lib.
     • Windows since Vista with OCI installed under C:\ and Apache Tomcat 7. You need administration rights
       • mklink "c:\Program Files\Apache Software Foundation\Tomcat 8.0\lib\ojdbc8.jar" c:\instantclient_12_1\ojdbc8.jar

4. First run: setup SOS server

Now you have 52N SOS running in your Tomcat server. To access it, open this URL in your browser:

http://localhost:8080/52n-sos-webapp

You will find the 52N SOS home page with the following message:

You first have to complete the installation process! Click here to start it.

Click here, press Start and select Oracle Spatial from the drop-down list. A database configuration form will appear below. Fill it with your database settings. For example, for the "sos" user created above, these will be the configuration values:

• Database Configuration
  • User Name: sos
  • Password: sos
  • Database: sos
  • Host: localhost
  • Database port: 1521
• Advanced Database Configuration
  • Schema: sos
  • Database concept: Select from dropdown
    • Old concept
    • Series concept (default)
    • eReporting concept (extended Series concept)
  • Provided JDBC drive: Checked
  • Minimum ConnectionPool size: 10
  • Maximum ConnectionPool size: 30
  • Batch size: 20
  • Datasource time zone: +00:00
  • Datasource time string format:
• Actions
  • Create tables: Checked
  • Delete existing tables: Unchecked
  • Force updating existing tables: Unchecked

Click Next and fill the Settings. For testing purposes it is possible to simply skip this process by clicking Next again.

Finally, set a user name and a password for the 52N SOS administrator and click Install.

Microsoft SQL Server installation

We assume that an Microsoft SQL Server 2012 instance and the Apache Tomcat áre already running and accessible through the network.

1. Create an Microsoft SQL Server user

We need to create an Microsoft SQL Server user with permissions to create a new database and to write into the new database.

2. First run: setup SOS server

Now you have 52N SOS running in your Tomcat server. To access it, open this URL in your browser:

http://localhost:8080/52n-sos-webapp

You will find the 52N SOS home page with the following message:

You first have to complete the installation process! Click here to start it.

Click here, press Start and select SQL Server from the drop-down list. A database configuration form will appear below. Fill it with your database settings. For example, for the "sos" user created above, these will be the configuration values:

• Database Configuration
  • User Name: sos
  • Password: sos
  • SQL Server instance: SQLEXPRESS
  • Database: sos
  • Host: localhost
  • Database port: 1433
• Advanced Database Configuration
  • Schema: dbo
  • Database concept: Select from dropdown
    • Old concept
    • Series concept (default)
    • eReporting concept (extended Series concept)
  • Provided JDBC drive: Checked
  • Minimum ConnectionPool size: 10
  • Maximum ConnectionPool size: 30
  • Batch size: 20
  • Datasource time zone: +00:00
  • Datasource time string format:
• Actions
  • Create tables: Checked
  • Delete existing tables: Unchecked
  • Force updating existing tables: Unchecked

Click Next and fill the Settings. For testing purposes it is possible to simply skip this process by clicking Next again.

Finally, set a user name and a password for the 52N SOS administrator and click Install.

Configure/Administrate the SOS

This section contains only the default settings of the SOS which can be supplemented by settings of an extensions.

Installation

During the installation you can configure the SOS by changing the following setting. These settings can be changed later with the administration interface.

• Informative settings
  • Service Provier: change the provider information (Capabilities) of this service
  • Service Identification: change the identification infomation (Capabilities) of this service.
• other
  • Transactional Security : Simple security for transactional operations
  • Service: Service relevant settings like URL, cache update interval, ...
  • Miscellaneous: Not assignable settings like separators, exception output
  • CRS: Default EPSG codes, axis order of the stored geometries, supports the datasource spatial queries.
  • RESTful Binding: Configure the RESTful binding
  • Procedure Description: Automatic procedure description generation and enrichment, e.g. featureOfInterest, discovery profile
  • EXI Bindind: Configure the EXI binding
  • Streaming : En-/disable XML response
  • eReporting: Define prefixes, global namespace for AQD e-Reporting support
  • FlexibleIdentifier: En-/disable flexible identifier
  • I18N: Multilingualism defintion
  • INSPIRE: Define metadata for SOS as INSPIRE download service

Administration

With the SOS administration interface you can change the settings you've made during the installation and you can make additional settings, e.g. logging configuration, supported bindings. To access the administration interface go to [SOS_URL]/admin, e.g. http://localhost:8080/52n-sos-webapp/admin.

On the administration interface start page you can perform a reloading of the Capabilities cache (internal cache) or select the folloing submenues:

• Datasource Maintenance: Get some raw SQL queries, clear the database or delete deleted observaitions (marked by DeleteObservation operation)
• Cache summary: Get some statistics about the internal cache, e.g. number of procedure or temporal and spatial bboxes.
• Reset: Resets the datasource configuration.
• Settings: Change the settings you have made during the installation process.

The Settings submenue contains further submenues which are briefly explained:

• Logging: Change the log level or show the latest log statements
• Operations: En-/Disable supported operations
• Encodings: En-/Disable supported encodings for observations and procedures
• Bindings: En-/Disable supported bindigs, see Bindings above
• Extensions: En-/Disable supported extensions for ExtendedCapabilities or Offering extensions, e.g. see the INSPIRE
• Datasource: See the datasource configuration
• Procedure Descriptions: Change or delete procedure descriptions
• Capabilities Settings: Add/Change/remove static Capabilities, Capabilities and Offering extensions.
• Observable Properties: Rename ObservableProperties.
• I18N Settings: Create/Update/Delete multilingual data, see Multilingual support

Detailed information about the administration interface: SensorObservationServiceIVAdministratorInterface

Transactional operations

To avoid the unwanted manipulation of SOS data via the transactional operations in the default 52°North SOS, we have disabled the transactional operations and enabled the Transactional Security by default.

This affects the following operations:

• InsertSensor
• InsertObservation
• InsertResultTemplate
• InsertResult
• UpdateSensorDescription
• DeleteSensor
• DeleteObservation
• InsertFeatureOfInterest

The transactional operations can be enabled via the admin GUI of the 52°North SOS. Go to the Operations submenue of the Admin -> Settings in the 52°North SOS web interface.

The Transactional Security is enabled by default and the Transactional allowed IPs is set to _127.0.0.1.

Updating the SOS

Updating the SOS from 4.x instance is not supported because there are some major changes in the new database model. The new database model contains several improvments which are not compatible with the old model.

Save the settings

The 52°North SOS 5.x offers the possibility to export the settings.

• Open a browser and enter the 52°North SOS 5.x URL, e.g. http://localhost:8080/52n-sos-webapp
• Click the Admin tab and enter username and password
• Move to the Admin tab and select Settings
• Click the Export Settings button
• A new browser tab opens with the JSON encoded settings, if the browser supports JSON files
• Save the JSON file.

NOTE: The database settings are currently not exported. But there are plans to add this in the future.

Undeploy the old instance

There are two ways to undeploy the old instance:

1. Via the Apache Tomcat HTML Manager by selecting undeploy
2. Stopping the Aache Tomcat Server and delete the war file and the folder from [TOMCAT_HOME]/webapps

Deploy the new instance

Deploy the 52°North SOS 5.x war file as described in the Installation section.

On the installation page with the title 52°North SOS Installation Wizard you can find at the bottom the heading Upload a previous configuration file.

Here you can select the previously exported settings file to import the settings into the new instance.

Database model changes from 4.x to 5.x

There are major changes in the database model for SOS 5.x, therefore it is required to use a new database for SOS 5.x.

Here are some of changes on the database model:

• series table was renamed to dataset
• featureOfInterest table was renamed to feature
• observableproperty table was renamed to phenomenon
• dataset differs by phenomenon, procedure, offering, featureOfInterest, category and platform
• observation table contains simple values (numeric, count, category, text, ...)
• observationType, featureOfInterestType and procedureDescriptionFormat was merged to format table
• one parameter table with relation tables for observation, feature, dataset
• support for OGC Sensor Things API

More information about the new database model and the differences in the concepts are described here.

Further documention

Further documentation of the SOS features and functionality can be found here SensorObservationServiceDocumentation

Frequently Asked Questions

See SosFAQ