You are here: Wiki>Projects Web>GSoC>GSoC2020Projects>GSoC2020TrajectoryAnalyticsToolbox (24 Aug 2020, johnmurzaku)Edit Attach

GSoC2020 Project: Trajectory Analytics Toolbox

Student: John Murzaku

Mentor: Dr. Benedikt Graeler (b.graeler@52north.org)

Introduction

I will be creating a general purpose trajectory analytics toolbox in R called traviz. This project will be using the existing R package trajectories and will be adding sf and sftime support. It will add the features mentioned above and use the Track data structure. To demonstrate functionality of the toolbox and perform tests on the package, enviroCar data will be used. In order to collect enviroCar data, the envirocar-py package will be used. The main functionality of the toolbox will include the following:
  • Analysis of trajectory data
  • Visualization of trajectory data (i.e. space-time cubes, animations, heat maps, hot-spot clusters)
  • Aggregation and subsetting of trajectory data

Weekly Report

Week 1: June 1st to June 8th

Status:
  • Collected enviroCar data using envirocar-py to use for testing and function implementation
  • Converted enviroCar data to sf and then to LINESTRING. Visualized the trajectories in Munster using ggplot2
  • Created a data pipeline that takes enviroCar data, converts it to sf format, and then fits it into the Track data structure
  • Began work on sf functionality in Track data structure by allowing Track to accept sf formatted data
  • Started working on raster data pipeline. Implemented raster aggregation to a region of interest, time of interest, desired resolution, and values of interests
  • Created raster visualizations of speeds of vehicles at Munster, Germany during rush hour (16:00 - 18:00) on a user selected day
  • Learned about CRS and how raster works in R
  • Began implementation of generalized raster aggregation functions
Problems:
  • How do I deal with NA values in raster data? This is affecting the visualizations a little bit because it has blank spots where a car does not have a recorded speed for example
  • How do I deal with CRS in the raster data? At the moment I am changing the CRS manually but this seems to be inefficient
  • Need to fully understand the Track data structure more in order to effectively add sf support
Next week tasks:
  • Finish implementation of generalized raster aggregation functions. Write documentation, tests, and demos
  • Create raster visualization functions to users desired aggregation and resolution. Write documentation, tests, and demos
Daily log:

Week 2: June 8th to June 15th

Status:
  • Created Github repository of traviz (https://github.com/JamMurz/traviz)
  • Created and implemented generalized functions for conversion to sf, LINESTRING, raster, and stars formats
  • Created and implemented sf to Track function
  • Added functionality for sf and raster aggregation to region of interest and time of interest
  • Created function for inverse distance weighted interpolation of raster data
  • Created preliminary function for clustering trajectories
  • Created preliminary function for finding trajectory intersection density
  • Began experimenting with animating trajectory data from point to point using gganiminate
  • Began experimenting with kernel density estimates for trajectories to find intersection hot spots
Problems:
  • Some of the functions appear to be too specific for enviroCar data. I will need to generalize them more to fit most trajectory data. I am thinking of requiring the package to only accept sf and sftime data
  • gganimate takes a long time to animate point to point time measurements of individual trajectories. I need to limit trajectory animations to one at a time or find another package
Next week tasks:
  • Finalize current functions and make them more general purpose
  • Start working on trajectory hot spot analysis tools
Daily log: Week 3: June 15th to June 22nd

Status:
  • Added the following functions to traviz:
    • Trajectory clustering by Frechet distance
    • Trajectory heatmap that returns kernel density heatmap
    • Trajectory value heatmap to find density of values (i.e. where speeds are higher or CO2 values are higher in a city)

  • Began implementation of trajectory animation function
  • Began implementation of using kernel density estimation and quadrat for heatmap
  • Created a generic class sfTrack and sfTracks to store sf trajectories
  • Added the following generic methods to sfTrack:
    • Plotting individual or multiple trajectories with ggplot2
    • Finding distance between two tracks
    • Coercing sfTrack to trajectories::Track
    • Transforming projection of sfTrack
    • Printing sfTrack

Problems:
  • Animations still take too long(even for individual short trajectories) and requires a lot of computational power. This could not scale well for every user. I am using moveVis for this
  • The sfTrack class does not yet take connections into account like trajectories does
Next week tasks:
  • Finalize generic methods for sfTrack and sfTracks
  • Convert all current methods in traviz-methods to work with sfTrack and sfTracks
  • Update Shiny app periodically as new methods to traviz are added
Daily log: Week 4: June 22nd to June 29th

Status:
  • Added functionality for generic sf/sp/trajectories methods to the sfTrack and sfTracks classes. These include
    • st_bbox
    • st_length
    • st_intersections with ggplot2 visualization and optional zoom parameter
    • stcube from trajectories
  • Added coercion from sfTracks to data.frame
  • Began rewriting of geodata_to_sf for general use cases
  • traviz is now able to be installed and all CRAN checks pass. If interested in testing, you can use the following command to install it:
    • devtools::install_github("JamMurz/traviz")
  • Added roxygen documentation to all methods
  • Created a Shiny application of traviz with the following methods:
    • Interactive trajectory plotting (from a subset of enviroCar trajectories in Munster, Germany)
    • Interactive trajectory intersection plotting
    • Rasterization of data with value and resolution of choice. Functionality for subsetting based off latitude and longitude included also
    • Heatmap of value with resolution and value parameters included
    • Quadrat plot of trajectories
    • Frechet distance clusters of trajectories
Problems:
  • The traviz-methods are still not ported to sfTrack and sfTracks
  • There are errors when subsetting the rasterized values off time. I will need to add fixes to the sf_to_rasterize function
Next week tasks:
  • Port all traviz-methods to sfTrack and sfTracks
  • Add methods for hotspot analysis using Getis-Ord
  • Fix the time errors in the sf_to_rasterize function
Daily log: Link to Shiny application: https://jammurz.shinyapps.io/traviz-shiny/?_ga=2.238498951.665889296.1593375892-939092745.1593375892

Week 5: June 29th to July 6th

Status:
  • Added Getis-Ord visualization method to traviz-methods
  • Added sfTrack and sfTracks support to traviz-methods through coercion to data frame and accepting sfTrack as a parameter
  • Added aggregation by time to sfTrack
  • Added functionality for making a 24 hour heatmap plot in density_heatmap function
  • Began creation of individual segment analysis tools
  • Added aggregation by weekday
  • A lot of time was spent debugging this week and fixing current functions for more general support or to fix potential problems. The following was fixed:
    • Bugs in Shiny application due to traviz updates
    • geodata_to_sf convertor
    • To and from time aggregation in sf_to_rasterize. This was fixed by using the tibbletime library for easier time manipulation
    • The constructor for sfTrack created errors when coercing from data frame to sfTrack. This was fixed
    • sf_intersection inheritance was fixed
Problems:
  • The 24 hour heatmap plot returns errors when there are NA's in the data. A fix for this could be removing all NA's but that will not tell the whole story of the data
Next week tasks:
  • Finish individual segment analysis tools
    • The segment analysis tools will aggregate all trajectory data on a given weekday and return visualization and analysis of this data
  • Add better visualization to Getis-Ord method
Daily Log: Week 6: July 6th to July 13th

Status:
  • Finished segment analysis tools
    • Added aggregation by weekday and plotting of aggregation by weekday
    • Added aggregation by hour and plotting of aggregation by hour
    • Added density plot of hour and weekday aggregation
    • Added optional value parameter to these functions to show point value density
  • Extended space time cube from trajectories to create a point value space time cube
    • This now works for both sfTrack and sfTracks objects
    • Contains an optional parameter to view map overlay as done in trajectories::stcube
  • Began intersection cube method to display intersections of trajectories in a space time cube
  • Added vscube method to showcase trajectory values in geographical space
  • Added time series plot visualization methods to traviz-methods for sfTrack
Problems:
  • The intersection cube is having problems with displaying the point and time of intersection due to standarization of time
  • The intersection cube might have to be dropped as it does not provide useful information on the tracks intersection
Next week tasks:
  • Attempt d3 integration to traviz for clearer visualizations using r2d3
Daily log: Week 7: July 13th to July 20th

Status:
  • Began attempt of d3 integration into traviz
  • Read d3 tutorials and learned Javascript syntax
  • My goal is to use d3 to animate a sfTrack on Leaflet
    • At the moment, the animation method for traviz works well. However, it is computationallty intensive and may require a fast computer. Using d3 would make it less computationally intensive and more visually appealing at the same time
  • Started d3 integration but ran into many bugs and errors with r2d3 library and htmltools
  • Added geoJSON convertor to sfTrack and sfTracks. Leaflet and d3 accept only geoJSON format
  • d3 integration proved to be very time consuming and requires a lot of time to produce good results, therefore next week I will shift my focus and try to add d3 integration in the final weeks of GSoC
Problems:
  • Many errors in d3 integration due to beginner knowledge in d3 and Javascript
  • Having problems importing Leaflet into d3
Next week tasks:
  • Switch focus to polishing, documenting, and testing package functions
  • Write introductory vignettes showcasing how to use traviz
  • Clean and debug the functions and methods created so far
Week 8: July 20th to July 27th

Status:
  • Created two introductory vignettes to traviz
    • Vignette 1: Point data analysis and visualization using traviz
    • Vignette 2: Trajectory line analysis and visualization using sfTrack/sfTracks classes in traviz
  • Vignette creation led to debugging in major functions of traviz. The following was changed and fixed:
    • Aggregation errors in plot_hour and plot_weekday functions
    • Plot displaying NA in plot_hour_density and plot_day_density
    • Fixed dependency issues and data format issues
    • Added optional parameter to geodata_to_sf making sf trajectory data generalizable
    • Fixed sfTrack constructor to accept multiple types of unique identifiers
  • Created first demo on GPS trajectories dataset from UC Irvine showing the following:
    • Data conversion to sf
    • Using the sfTrack/sfTracks class
    • Visualizing this specific dataset in multiple ways
  • Found package for ST-DBSCAN algorithm created recently called stdbscanr
  • Forked and got merged pull request to stdbscanr fixing namespace and dependency errors
  • Began implementation of ST-DBSCAN on traviz
Problems:
  • Need 1-2 more demos to display package functionality. The first demo did not contain point value analysis
Next week tasks:
  • Finish implementation of ST-DBSCAN on sfTrack/sfTracks and add to traviz-methods
  • Create 1 or 2 more demos with open trajectory data
Daily Log: Week 9: July 27th to August 3rd

Status:
  • Added ST-DBSCAN to traviz-methods
  • Added optional ST-DBSCAN parameter to cluster functions
  • Created demo of trajectory data with enviroCar data
Problems:
  • None
Next week tasks:
  • Begin finishing touches on package. This will include:
    • Passing all CRAN checks
    • Writing tests for needed functions
    • Finalize current functions and make sure there are no bugs
  • These final weeks of GSoC will be spent finishing the package and getting it ready for deployment
Week 10: August 3rd to August 10th

Status:
  • Began test file for sfTrack and sfTracks
    • I will be following a similar testing methodology and procedure as trajectories in my tests
    • Tests will show check creation of a class and if coercion to other classes works as desired
    • Writing these tests resulted in bug fixes
  • Fixed bugs in constructor and coercion to other classes
  • Fixing problems in package and local environment to pass all CRAN checks
    • The package runs, builds, and loads properly but there are still a few CRAN warnings. These need to be fixed
Problems:
  • CRAN warnings need to all be solved
Next week tasks:
  • Finish test file
  • Continue debugging and adding fixes to functions that have bugs
  • Fix and solve reasons for all CRAN warnings
Week 11: August 10th to August 17th

Status:
  • Finished test file for sfTrack and sfTracks
    • Test file tests creation of class and successful coercion of class to other classes
  • Added coercion method to sf for sfTrack and sfTracks
  • Fixed bug in sfTrack constructor by generalizing unique identifier
  • Began cleaning package to pass CRAN checks
Problems:
  • Vignette 2 takes a long time to load due to testing of stcube and having to load rgl onto the vignette
Next week tasks:
  • Finalize package
  • Update Shiny
Week 12: August 17th to August 24th

Status:
  • Finalized package
    • Added license
    • Updated README to include far more detail and show off what traviz can do
    • Generated PDF manual of functions
    • Finished initial tests
  • Finished final blog post
Problems:
  • None
Tags:
create new tag
, view all tags
Edit  | Attach | Print version | History: r13 < r12 < r11 < r10 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r13 - 24 Aug 2020, johnmurzaku
Legal Notice | Privacy Statement

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Wiki? Send feedback