Monday, 17 June 2013

Final Product

1.     Introductory Product Information

1.1 Name of the product

The intention of the ANDS-funded Marine Virtual laboratory Information System (MARVLIS) is to develop a suite of tools creating value-added products from model and observational data which can be either used by themselves, or used as a ‘plug-in’ library to Marine Virtual laboratory (MARVL), thereby giving MARVL the capability to move from being a research tool for specialists to a tool with a much broader spectrum of users. It includes a clone of Australian Ocean Data Network (AODN),   CSIRO CONNIE tool, Sensor Messaging Gateway and geospatial information portal.

1.2 The primary users of your product

In the Derwent Estuary, the community can now easily access information previously only available on request. The Environment Agency can see ways in which MARVLIS can be used for routine ‘Public Health’ reporting and is assessing options for utilising MARVLIS to speed up, and enhance, future ‘State of the Derwent’ reporting. At present, because of restricted access to the DEP databases data feeds to MARVLIS occurred manually; however, because of the development within MARVLIS of the SMG feeder tools, should these access restictions be removed, then automatic feeds of data to MARVLIS are immediately possible. For MARVL, new user communities can see opportunities from the added library of software tools.

1.3 Problem to solve

For several years the Derwent Estuary has been the subject of local research monitoring of the marine ecosystem in support of the long-term health of the estuary, as impacted upon by historical and ongoing human activity and potential new developments in aquaculture operations (see the Derwent Estuary Program (DEP), http://www.derwentestuary.org.au/, and the CSIRO/UTAS/State Govt program INFORMD, http://www.emg.cmar.csiro.au/www/en/emg/projects/INFORMD.html). These activities have not had a publicly visible on-line vehicle for the data, results and information they have produced. The Derwent Estuary is the focal point for the ‘Early Activity’ of MARVL because of these underlying activities and availablity of models and observations which can readily be used to demonstrate the utility of MARVL. After consultation with the DEP it was agreed that the development of value-added products to provide fast and comprehensive information on the state of the Estuary would be of benefit and could, in the short term, provide the public with information about beach and bay water quality and, in the longer term, offer a mechanism to streamline the 5-year ‘State of the Derwent’ reporting (see http://www.derwentestuary.org.au/state-of-the-derwent-report/).  The tools developed to provide these products could easily be compiled into a library for addition to MARVL.

2.     Instructional Product Information

2.1 What end users said?

The utility of the MARVLIS portal was explored through requested feedback from the DEP partners, who, for the first time, were able to consider the value of seeing their data in the public domain (other than as a table of numbers – the Saturday text recreational water quality report in the local newspaper – or in the State of the Derwent report). The feedback was synthesised by the Director of the DEP and described in the letter of endorsement for the project (see attached document). In summary, the DEP’s response is:

 a) Public health layer:  the layer is useful and easy to navigate, and allows users to track recreational water quality at a given beach or bay over the past six swimming seasons. It also allows users to compare and contrast water quality between beaches and bays across the  system as a whole, and put these results within the context of state and national public health guideleines. The DEP and our council partners would be open to continued use and development of this layer;
     b)  Ecosystem health layer: This layer is useful and easy to navigate, and allows the user to track ambient water quality for a range of parameters at a specific monitoring site throughout the Derwent over a six year period. The sample spatial maps and modeling output would also be useful features for managers. The ability to compare observational and modeled data is also a valuable feature, which offers ‘reality check’ for both the modelled and the observational monitoring program. The DEP and our partners would be open to continued use and development of this layer for some water quality parameters.

2.2 Links to the actual product

2.3 Source code


2.4 User Guide


3.  Product (or Product Components) Re-usability Information

 The Sensor messaging Gateway (SMG) acts as a process engine and broker between the source of data and the publication of that data or any products derived from that data. Formally, this type of system is described as a Message Oriented Middle-ware stream processing engine.

SMG gives the data providers and users the ability to apply complex chains of processing, transforms, alerting, aggregation, filtering, routing, etc. to their streaming data. It allows reuse of these complex modules across any data source entering the engine. It provides a lightweight scripting API for writing sources, processors, timertasks and sinks and is compact enough to run in embedded mode on some sensors and small devices.

SMG communicates over networks / internet using Java Messaging System (JMS) making it easy for users to get data from the field to their systems, users, publication channels and sinks (web sockets may be available in the future too). Importantly it allows automation and reuse of most of the processing and publication tasks that are usually rebuilt for each new data provider and sink.

Connie 2 uses archived currents from oceanographic models and particle tracking techniques to estimate connectivity statistics from user-specified source regions (or to user-specified sink regions). A range of physical and biological behaviors can be specified including vertical migration, horizontal propulsion or swimming (user-specified random or constant velocity), and floatation or surface slick formation (user-specified windage). Ongoing developments include tracking the cumulative exposure of individual particles to environmental conditions (e.g. temperature, salinity, chlorophyll) and integration with other scientific and management tools (e.g. ecosystem models, reserve design tools). Further information can be obtained from http://www.csiro.au/connie2/.  

4. Contextual Product Information

4.1 Licensing of final product

1) Sensor Messaging Gateway (SMG): the feeder scripts to format the DEP data for ingestion into the   
2) SMG is available under CCBY v3.0 AU; the SMG engine is licenced under CSIRO terms and conditions.
3) The CONNIE interface is licenced under GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007; the CONNIE engine is licenced under CSIRO terms and conditions.
4) The AODN geospatial portal and MARVLIS additions are licenced under the GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007
5) The MARVLIS software library of data product tools is available under licence CCBY v3.0 AU.

4.2 Sustainability


1) All the software and data components will be located onto UTAS servers.

2) The MARVLIS portal will be maintained in the public domain for 12 months by eMII/AODN, with support from CRIS and MARVL funding.

3) Negotiations are underway with DEP to establish a mechanism for the delivery of new data – the aspiration is to have the ‘Public health’ data updated weekly, the ‘Ecosystem health’ data at least annually. New data will be added to the database by UTAS staff.
4) If the EPA database holding the DEP data can be placed in the public domain, automated retrieval of data using the SMG can be implemented
5) If new data is added, new metadata records will be created by UTAS staff and uploaded to the RDA

Testing and feedback from users

1. The Application

The intention of the ANDS-funded MARVLIS is to develop a suite of tools creating value-added products from model and observational data which can be either used by themselves or used as a ‘plug-in’ library to MARVL, thereby giving MARVL the capability to move from being a research tool for specialists to a tool with a much broader spectrum of users. The portal can be accessed from http://marvlis.aodn.org.au/marvlis/.

2. Who are the Users?

The users for the application can be marine managers, users of modelling study, researchers and Derwent Estuary Program partners and public users. Marine managers can use the application to run risk management and to access and evaluate the public health and environmental water quality for Derwent Estuary. Users of modelling study and researchers can use this application to learn and run models and to compare model outputs to observational data. Derwent Estuary Program partners in this case are particularly interested in, for the first time, considering the value of seeing their data in the public domain (other than as a table of numbers – the Saturday text recreational water quality report in the local newspaper – or in the State of the Derwent report. Public users now can have easy access to the water quality data and run risk estimate through the application.

3. Testing Methods and Findings

Derwent Estuary Program partners did user acceptance testing and provided valuable feedback. In summary, the DEP’s response is:

 a) Public health layer: the layer is useful and easy to navigate, and allows users to track recreational water quality at a given beach or bay over the past six swimming seasons. It also allows users to compare and contrast water quality between beaches and bays across the system as a whole, and put these results within the context of state and national public health guidelines. The DEP and our council partners would be open to continued use and development of this layer;

 b) Ecosystem health layer: This layer is useful and easy to navigate, and allows the user to track ambient water quality for a range of parameters at a specific monitoring site throughout the Derwent over a six year period. The sample spatial maps and modeling output would also be useful features for managers. The ability to compare observational and modeled data is also a valuable feature, which offers ‘reality check’ for both the modelled and the observational monitoring program. The DEP and our partners would be open to continued use and development of this layer for some water quality parameters.

4. Lessons Learned

1) When engaging with traditionally conservative data users/providers work harder, earlier in the program, to secure appropriate access to the data.
2) Work harder to ensure all parties involved are fully on the same page, and working towards the same deadlines
3) Set an earlier deadline for working systems so that engagement with stakeholders can be more interactive