Alexander Kmoch, GSoC weekly report notes

Estimated timeline / milestones

April 23 – May 21: Before the official coding time, Community bonding time (1 month)
  • Immerse in overall SOS server architecture, source and data model
  • Get In-depth view on data acquisition from backend and translation to actual O&M output
  • Learn more about the O&M encoding and the WaterML2.0 encoding schema
  • Get more familiar with XSLT and XMLBeans paradigm and "52n-sos-WML2" branch
ML1 (May 21): Get familiar with working environment, SOS software architecture, OGC standards
  • more or less achieved smile We grow with our challenges
  • conceptually visualise data flow in SOS for better understanding - asap -- AlexanderKmoch - 2012-05-21
May 21 – June 18: Coding time begin (1 month)
  • Implement/Update WaterML2.0 schema to latest OGC HydroDWG state
  • as WaterML is basd on O&M 2.0 update data model to support SOS2.0 and O&M 2.0 if not already done in trunk (merge changes from SOS-trunk?)
  • check with "52n-sos-WML2" to recent SOS version(?), at least regarding the encoding scheme
  • The WaterML is probably too heavy-weight for the beginning, would postpone to ML4, as the CSV encoding is a simpler start, and the SOS/O&M main encodings are already in place, just minor changes to support more flexibility -- AlexanderKmoch - 2012-06-04
  • find decision how the client selects encoding (check back with SOSAdministrator)
  • How and where to put the interface, if not using XMLBeans to convert DAO data to encode response (e.g. CSV or GeoJSON)? Probably defer ResultModel and ResponseMode logic out of the DAO
ML2 (June 18) : Architecture decision

June 18 – July 9: Coding time to mid-term evaluation (1 month)
  • derive abstraction / changeable response encoding architecture
  • start decoupling / encapsulating encoding from SOS
  • implement handy encoding plugin sourcing (at least via file sytem/read archive), based on SOS Administrator project, find it via property
ML3 (July 2): Have one simple not XML based encoding working (In collaboration with SOS Administrator already as kind of plugin)

July 9 – August 20: Coding time to pencils down (1 month)
  • coding coding coding
  • finish coding, putting the pieces together
  • thorough test (test classes, JUnit in parallel if possible), build and make release shippable (check with SOSAdministrator)
  • Code beautify and documentation (hopefully also done in parallel)
ML4 (July 16): make sure SOS standard encodings are implemented properly (have them as decoupled plugins / .jars, too?)
ML5 (July 30): have WaterML2.0 working as plugin
ML6 (August 13): cleanup code, minor changes to ensure robustness, finalise integration with SOS Administrator

Finally from 21.08.

  • demo server deployment test failed, http://gsoc.dev.52north.org/52nSOSv3.5.0.ex/sos
  • tagging failed (no access on svn/tag), but the adjusted/polished to be deployed and tagged version is in SVN
  • some documentation updates
  • blog article, TBD around wednesday

Week 14.08. - 20.08.

STATUS: TODO (finish GSoC):

Week 07.08. - 13.08.

STATUS: TODO (tracker from last to this week):
  • prepare demo
  • continue WaterML2 encoder
PROBLEMS/ISSUES:

Week 31.07. - 06.08.

STATUS: TODO (tracker from last to this week):
  • prepare demo and demo data
  • continue WaterML2 encoder for both SOS version
PROBLEMS/ISSUES:
  • Namespace beautifying (numbered ns1-nsX .. instead of om:, sos: , ... wml2)

Week 24.07. - 30.07.

STATUS:

  • some discussion and re-factoring of the code base
  • fresh trunk check-out with re-factored "good" code put back into exchangeable-encodings branch -- AlexanderKmoch - 2012-07-27
  • local maven WaterML2.0 jar from schema created (thanks Henning)
  • prepared a SOS server in the web, not yet data in it, not yet latest source (but simple CSV)
  • working on WML2Encoder for SOS v2.0
TODO (tracker from last to this week):
  • prepare demo and demo data
PROBLEMS/ISSUES:
  • XMLBeans and WaterML and how to add the elements in the right form and order is still quite challenging -- AlexanderKmoch - 2012-07-27

Week 17.07. - 23.07.

STATUS:

  • re-wrote the encoding source in a fresh SOS trunk check-out, including the input from Henning and Carsten (Key Class, one Encoder Map)
  • SOS.java would now respect contentType based on Encoder-provided value
  • arrived at 52°North office in Münster smile
TODO (tracker from last to this week):
  • WaterML2fromWaterML2 demonstration branch">https://svn.52north.org/svn/swe/main/SOS/Service/branches/52n-sos-WML2/ and HydroDWG (e.g. ask Carsten or Peter for latest schema)
  • continue documentation for enc-plugin devlopers (apparently no write rights in sensornet twiki)
  • waterml libraries / schema inclusion
  • prepare demo and demo data (Horizons RC) for online demo KW27 (this week), would be nice to have SensorML process for manual observations
PROBLEMS/ISSUES:
  • fresh trunk check-out somehow needs to be put back into exchangaeble-encodings branch

Week 10.07. - 16.07.

STATUS:

    • SOS Version 2.0.0 SOAP response experimented, tried to check (e.g. responseFormat) if XML to be expected, read about MIME SOAP Attachement, but not really the way,
    • played around with ideas from Carten and Henning regarding structuring the Encoder Map
    • read about XMLBeans and WaterML2.0 schema contens in general
TODO (tracker from last to this week):
  • WaterML2fromWaterML2 demonstration branch">https://svn.52north.org/svn/swe/main/SOS/Service/branches/52n-sos-WML2/ and HydroDWG (e.g. ask Carsten or Peter for latest schema)
  • continue documentation for enc-plugin devlopers (apparently no write rights in sensornet twiki)
  • waterml libraries / schema inclusion
  • prepare demo and demo data (Horizons RC) for online demo KW27 (this week), would be nice to have SensorML process for manual observations
PROBLEMS/ISSUES:
  • entangled myself in several construction sites, tried to start/change too much

Week 03.07. - 09.07.

STATUS:

  • SOS Version 2.0.0 SOAP response problem with bytearray output stream?! general nullpointerexception with SOS 2.0 AND given responseformat,
    not yet solved -- AlexanderKmoch - 2012-07-08
  • org.n52.sos.SOS: doSoapResponse - org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog. -- AlexanderKmoch - 2012-07-08
  • acquired apparently latest WaterML2.0 schema
  • read about XMLBeans and WaterML2.0 schema contens in general -- AlexanderKmoch - 2012-07-09
TODO (tracker from last to this week):
  • WaterML2 fromWaterML2 demonstration branch">https://svn.52north.org/svn/swe/main/SOS/Service/branches/52n-sos-WML2/ and HydroDWG (e.g. ask Carsten or Peter for latest schema)
  • continue documentation for enc-plugin devlopers (apparently no write rights in sensornet twiki)
  • waterml libraries / schema inclusion
  • prepare demo and demo data (Horizons RC) for online demo KW27 (this week), would be nice to have SensorML process for manual observations

TODO for next week:
  • would be nice to have SensorML process for manual observations
  • waterml
PROBLEMS/ISSUES:
  • GI_Forum and AGIT conferneces in Salzburg and some summer school obligations

Week 26.06. - 02.07.

STATUS:

  • refactored GenericObservationEncoder instantiation properly for SOS Version 1 and 2 -- AlexanderKmoch - 2012-07-02
  • merged from trunk - finally solved -- AlexanderKmoch - 2012-07-01 (looks like the hierarchical procedure thing for Shane works at least partially after merge even with the CSV simple demo encoder smile )
  • apparently only one service loader interface per operation and sos version, otherwise doesn't load
TODO (tracker from last to this week):

  • WaterML2 fromWaterML2 demonstration branch">https://svn.52north.org/svn/swe/main/SOS/Service/branches/52n-sos-WML2/ and HydroDWG (e.g. ask Carsten or Peter for latest schema)
  • continue documentation for enc-plugin devlopers (apparently no write rights in sensornet twiki)
  • waterml in SOS 1.0 response format "text/xml;subtype="waterml/2.0.0quot;", and in SOS 2.0 response format "http://www.opengis.net/waterml/2.0"
  • waterml libraries / schema inclusion

TODO for next week:

  • continue waterml encoder
  • prepare demo and demo data (Horizons RC) for online demo KW27, would be nice to have SensorML process for manual observations
PROBLEMS/ISSUES:

  • SOS Version 2.0.0 SOAP response problem with bytearray output stream?! general nullpointerexception with SOS 2.0 AND given responseformat
  • apparently I can't create SosEncodingPluginDevelopment in the SensorWeb TWiki
  • "Error while encoding SensorML description from stored SensorML encoded sensor description with XMLBeans: error: Unexpected element: CDATA with parent sensor
    solved, more merge updates and database new -- AlexanderKmoch - 2012-07-02

Week 18.06. - 25.06.

STATUS:

  • SWE code formatter in use and read QM guidelines
  • refactor IObsTextEncoder as general encoder (with ByteArrayOutput) -- AlexanderKmoch - 2012-06-21
  • Encoder for SOS Version 2 constructor adjusted, so now generic instantiation
  • merged from trunk, but the latest changes interfere with same classes, so quite some effort to integrate
  • started documentation (right now, just some notes in an extra topic in the standard SOS howto delivered with the source
TODO (tracker from last to this week):

  • advance ServiceLoader and SosConfigurator and Observation Listener for getVersion, getOperation and getResponseFormat, just to check in ServiceLoader, but apparently better to have each responseFormat map for one combination of (SOS service version, operation)
  • start WaterML2 starting from WaterML2 demonstration branch">https://svn.52north.org/svn/swe/main/SOS/Service/branches/52n-sos-WML2/ and HydroDWG (e.g. ask Carsten or Peter for latest schema)
  • merge latest 3.5 SOS trunk
  • start documentation for enc-plugin devlopers (ask Daniel or Carsten for template)

TODO for next week:

  • continue/start waterml encoder
  • sorting of observations etc before output, adjusted/later, finally responsibility of enc-plugin developer -- AlexanderKmoch - 2012-06-22
  • prepare demo and demo data (Horizons RC) for online demo KW27, would be nice to have SensorML process for manual observations
  • start preparing blog entry for 3rd July
PROBLEMS/ISSUES:

  • SOS Version 2.0.0 SOAP response problem with bytearray output stream?!
  • merge with the larger last changes (e.g. hierarchical procedures) is not as easy anymore, messed up my working copy -- AlexanderKmoch - 2012-06-23

Week 11.06. - 17.06.

STATUS:

  • need some classes instead (resp. implementing the interfaces) of interfaces for Responses, which then can be extended? still not yet really CSVResponse outside of 52n-sos-core working, resolved -- AlexanderKmoch - 2012-06-14
  • created ASosResponse class in sos-core with ByteArrayOutputStream
  • one generic response is ok, maybe Encoders need GenericInterface only, too (like IGenericOberservationEncoder returning something, that can be read bytes[] from)
  • CSVEncoder (in sos-coding-csv, extends IObsTextEncoder from sos-core ... ) creates String CSVList and writes the bytes in the response (within ObservationListener)
  • ServiceLoader does work now, resolved -- AlexanderKmoch - 2012-06-16
    created META-INF/resources/services/org.52n.sos.encode.IObsTextEncoder and added one line: org.52n.sos.encode.CSVEncoder
    doing the a test load in SosConfigurator
    ServiceLoader.load(IObsTextEncoder.class)
  • committed working state to SVN -- AlexanderKmoch - 2012-06-16
  • starts with and without the coding-csv.jar
TODO (tracker from last to this week):

  • general architecture decision, ServiceLoader and ext. enc. modules extending some interfaces and abstract classes to be self contained, ok
  • ServiceLoader works now, ok
  • nice CSV output, basic output from example/test.sql, leaned on IOOS/NOAA, ok (some more test material would be good)

TODO for next week:

  • more error proof code, more checks
  • sorting of observations etc before output
  • demo online, Daniel already asked for it in KW27 -- AlexanderKmoch - 2012-06-16
  • include include a library for csv writing?
  • merge sos trunk code again and refactor a bit
PROBLEMS/ISSUES:

  • on monday 18th I am going to Uni Salzburg, meeting supervisor etc, report might be a bit later in the evening smile
  • some design questions:
    - zip only for SosVersion.OMEncoder possible
    - How SosVersions to be treated in extra Encoders?
    - how do composite phenomena look like? more test material?
    - how does
    quality look like? example?

Week 04.06. - 10.06.

STATUS:

  • regarding the discussion about providing interfaces (or maybe kind of API?)
  • ISosReponse
<-- IObsXmlResponse (wrapping the XMLBeans)
<-- IObsTextResponse (this and all above are in core)

  • I played around with AbstractClasses, as the interfaces only declare methods to be Implemented instead of intantiable classes, that in return must be accessible and instantiated in the Listeners, got a bit confused about that
TODO (tracker from last to this week):

  • get code working again - ongoing, based on containment and modularity design aspects
  • learn about ServiceLoader for dynamic module registration
  • CSVObservationResponse, leaned on IOOS CSV obs spec, not much progressed on this

TODO for next week (apparently already this week):

  • get ServiceLoader working
  • try separate adding of encoding-csv jar into "stndard" SOS webapps folder and see if it works
  • CSVObservationResponse, leaned on IOOS CSV obs spec
PROBLEMS/ISSUES:

  • the flight home from NZ
  • need some abstact classes instead (resp. implementing the interfaces) of interfaces for Responses, which then can be extended?

Week 28.05. - 03.06.

STATUS:

  • week starts, found a proper starting point
  • compile and deploy works, changed to Linux and configured Maven settings.xml regarding [[ https://wiki.52north.org/bin/view/SensorWeb/SosEclipseJavaProject#Create_server_settings_and_profi][Deploy Tomcat]] and compile on Shell outside Eclipse as this error also happens under Linux, so apparently Eclipse related
  • added 52n-sos-coding-csv module (maven)
  • created org.n52.sos.encode.IObsTextEncoder (interface, as a general text-based response encoder, all other suitable encoders already are XMLBeans based) and an org.n52.sos.encode.impl.CSVEncoder that implements IObsTextEncoder
  • created org.n52.sos.resp.CSVObservationResponse (implements ISosResponse)
  • manual test call in GetObservationListener delivered already an artificial CSV file (not yet parsed from SosObservationCollection, manual headers)
  • manually registered "text/csv" as responseFormat in SosConstants
  • select ResponseFormat "text/csv" in getObservation request and pass it through SOS and in getObservationListener call CSVEncoder from SosConfigurator and create Response works
  • manually registered CSVEncoder in SosConfigurator

TODO (tracker from last to this week):

  • get development environment working again (maven, compile, SVN, Eclipse) - ok -- AlexanderKmoch - 2012-05-31
  • try to regularly merge trunk into this branch - ok for this week -- AlexanderKmoch - 2012-06-02
  • test (manually) idea, of implementing existing Interfaces in separate Encoding module -- AlexanderKmoch - 2012-06-04
  • blog entry - published -- AlexanderKmoch - 2012-06-01
TODO for next week:

  • CSVEncoding basic demo, iterating over SosObservationCollection to get general feeling, for typical Encoding issues, how to tansform data structure actually into an encoding
  • deliver basic CSV containing observationid, time, procedure, offering, phenomenon, value, foi
  • needs to get the thing working again that the manually registered Encoder is chosen based on requested
  • get comfortable with how the Encoder is registered in the SosConfigurator and what "select" mechanisms could work
  • not sure, if I can figure that out already, but some necessary values are based on constants, but to have additional values (especially responseFormats and encoder-classes) this must be enabled to evaluate on dynamical basis (String[] Encoders/responseFormats or Map <SOSVERSION, Map<responseFormat, EncoderImpl>>or similar), test manually, if such a structure could work, should be changeable at runtime, for a SOS version, have valid
  • go for more containment (all structures that are needed additionally should somehow move into the extra module)
  • general architecture decision, from the main RoadMap

PROBLEMS/ISSUES:

  • stupid compile/maven issues and keeping up with trunk changes cost quite additional time
  • next weekend/next week flight home from NZ on weekend, preparations, driving taupo - Auckland and jetlag, report not in time
  • some changes I could not yet completely encapsulate in the extra encoding module, there is actually only the Encoder, which can be "found" via the SosConfigurator, but I have the CSVObservationResponse only working under 52n-sos-core (org.n52.sos.resp, instead of 52n-sos-coding-csv)
  • apparently, it still is better (or easier?) for the architecture to have something like 52-sos-coding-csv and -csv-v2, probably useless with csv but might come handy for GeoJSON or other complex formats

Week 21.05.-27.05.

STATUS:
  • dev environment is working, IDE, SVN, debugging - partially only
  • thought about architecture - actually the different response encoding backends are called from the Listeners, so regarding the requested SOS 1.0/2.0 versions now basically the respective O&M XMLBean is used, but ISosResponse just needs an actual String object
  • if CSV or GeoJSON or another simple encoding would have been requested (via necessary SOS1.0/2.0 e.g. GetObservation request), it would be named in the ResponseFormat and comes a list - so the Listener creating the response does not necessarily return an XML object anymore
  • checked with mentor, if and how SOS trunk updates should be merged regularly into my branch - done, svn merge works, but (see probs)... -- AlexanderKmoch - 2012-05-25
sos-sequence-diag-idea.png


PROBLEMS:
  • I think the learning and production curve rises with time
  • Maven, Eclipse, read-only filesystem issues ... I think, I've come across such a problem some years ago, but don't know, how to solve ... re Windows Search/Indexer?
    For some reasons calling "mvn run config" in Eclipse or now for "svn team commit" the task abort, because for some unknown reason some places within the workspace suddenly are write-protected
    "mvn install" from command line works fine, so I used that as a work-around, but "svn commit" from within Windows (use Tortoise 1.7) could give problems with working copy format?!
    contact mentor -- AlexanderKmoch - 2012-05-25
  • finally changed to Linux, Subversive+SVNkit, at least Maven and SVN integration in Eclipse working again, recognised that one mvn compile error might be caused by recent project updates in SOS-trunk?! Check with mentor again -- AlexanderKmoch - 2012-05-27
  • changes in encoding routines in trunk and the wish to incorporate trunk changes might need a different workflow, build SOS-trunk separately before including changes... -- AlexanderKmoch - 2012-05-28

Metadata

Topic attachments
I Attachment Action Size Date Who Comment
sos-sequence-diag-idea.pngpng sos-sequence-diag-idea.png manage 31 K 28 May 2012 - 23:32 AlexanderKmoch SOS simplified internal sequence diagram
Topic revision: r32 - 28 Aug 2012 07:48:56, DanielNuest - This page was cached on 26 Aug 2016 - 23:52.

This site is powered by FoswikiCopyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Wiki? Send feedback