Edserplo — BODC's visualisation program

The following is a technical summary of the capabilities of Edserplo — BODC's visualisation and tidal processing tool.

Background

Until 2005, BODC employed a suite of visualisation programs to quickly assess the validity of marine data. To process data collected by the UK Tide Gauge Network we developed Edteva — a program for editing, visualisation and analysis of tidal data. Skilled data scientists used the programs to screen and flag oceanographic data. Flagging is a process where suspect data are highlighted without changing the actual data values.

These programs consisted of 60,000 lines of Fortran and C++ code and depended on Silicon Graphics, Inc (SGI) hardware and software. In 2003, we began to develop a Linux and Windows-compatible replacement.

The new software had to

• Have similar functionality to the original four programs — reducing learning overheads for existing users, minimising development time and providing a benchmark for programmers.
• Respond as fast, or faster than, the software it was replacing.
• Be Windows compatible — to accommodate BODC's data scientists.
• Utilise a mainstream programming language — to safeguard future software support.

An initial prototype indicated that Java provided the facilities and performance required and Edserplo became operational in 2005.

Data types

Edserplo can present any data conforming to the BODC series model, which have pressure, depth, altitude and/or time as independent variables. Typical data types include

• CTD (Conductivity Temperature Depth).
• Underway — continuous measurements of sea surface data (e.g. salinity, temperature, attenuance, chlorophyll, nutrients), meteorology, navigation and bathymetry.
• Radiosonde.
• XBT (eXpendable BathyThermograph).
• Tide gauge.
• Current meter.
• ADCP (Acoustic Doppler Current Profiler) — static and shipborne.
• Wave statistics (Hs, Tz).
• One-dimensional wave spectra.
• Two-dimensional wave spectra.
• Thermistor chain.
• HF Radar surface currents.
• Other moored instruments.

Introduction to Edserplo

Edserplo consists of a suite of presentation pages. Each page is displayed on the screen in its own window and provides the user with the ability to perform a specific data screening or data processing function.

Special factors relating to the processing of sea level data means that there are dedicated pages for this purpose. However, the software is by no means limited to this particular task. These dedicated pages are identified by (E) below and are described in the processing tidal data section.

Graphical presentation pages

All the pages support panning, zooming and superposition.

• Time series with inset track map
• Profile (CTD) page
• Thermistor chain or ADCP (TCAD)
• Wave spectrum
• Wave histogram
• HF Radar surface current (OSCR)
• Current meter scatter plot
• X-Y scatter plot
• Port time series (E)

Selection and other control pages

• Series page
• Series location map
• Port page (E)
• Output page (E)
• Tidal analysis page (E)

Running Edserplo

To invoke the program, the user must provide individual data file names or a driver file containing a list of data files. Preferred display settings can be saved in a settings file.

Input and output functionality

Data file formats

Currently, Edserplo supports thirteen file formats. They include

• A format for multi-series
• Two formats where the data values themselves are updateable
• Four writeable formats

All formats originate from BODC with one exception, the National Oceanography Centre (NOC) format PSTAR.

Edserplo has been constructed to allow the addition of further formats without changing the kernel of the program. Each file format is supported through a dedicated class; each class extends the BODC's abstract class. Edserplo automatically recognises each input file format. Additional format classes may be added by a Java-savvy end user following instructions in the user guide.

Data parameters

Data file parameters are normally defined using the BODC parameter dictionary. Each parameter is assigned a unique 8-character code. To enable easy comparison of like parameters codes can be aliased by inserting information into the driver file.

The definition for a particular parameter code may be obtained as hover text (tooltip) via a database connection on the appropriate pages. Edserplo can operate without tooltips if the database is not available.

Generating derived quantities

Edserplo has the functionality to generate derived quantities. For example

• Eastings and Northings for wind and current data — generated through user supplied information in the driver file.
• Tide and residuals for UK tide gauge data — automatic generation using information stored in a harmonic constants library.

For each transformation, an appropriate mapping has to be defined between the flags in the derived channels and those in the input channels. This may be done by defining a precedence hierarchy amongst the possible flags.

Tidal analysis and statistics

The UK Tide Gauge Network data require tidal analyses to be retrieved from a library of harmonic constants. However, Edserplo also has the capability of generating a tidal analysis from the data loaded. These analyses may be output to a database and stored for future use. Tidal statistics, including mean sea level, surges and extremes can also be generated and stored in a database.

File output

When edits are limited to flag channels the input data files may be overwritten — if the file format permits. The user may toggle between read and write mode on the series page and opt to save as required. The user is prompted on exit if a file has been edited and a save has not been performed.

After data value edits are performed during tidal processing, the data are written out to a protected 'dump' file. The dump file retains port, channel and other setting information as well as the tidal data set and can be accessed during the users next Edserplo session. This procedure allows for the editing of data values within a series without changing the files from which they were derived. Additionally, newly generated concatenated series may be output as files from the output page.

Creating hardcopy

A screen capture of each page can be generated as a PNG file. This can be useful to communicate issues to data originators.

The series page

The series page lists series cross-referenced against their parameters, scales, bases and colours. It allows the user to select or deselect series and parameters of interest for display on subsequent pages.

The display colour for each series and the colour, scale, base and axis for each parameter may be defined directly within the series page or imported from a settings file.

If the settings are not user-defined, Edserplo computes the scales and bases from the assessed data limits for each parameter, randomly selects a colour and sets the display axis to one.

The user may toggle between read and write mode and save data files or setting files as required.

The series location map

It can be useful to see information about the location of data series collected at fixed positions. For example, neighbouring series for current meter moorings can be checked against each other. The series location map provides this information. Series locations are displayed on a map and selections can be made. The chosen series are mirrored in the series page.

The same technique can also be used to select CTD series recorded in the course of a research cruise. In this case the position is not recorded in the individual series but is obtained from a cruise database.

The time series page

Using the settings from the series page, the chosen series and parameters are displayed on the time series page. The parameter base setting is subtracted from the data value before multiplication by the parameter scale to define a pixel distance from the associated axis.

The data can be examined as

• All selected series
• One series at a time
• All selected parameters for a series
• One parameter at a time
• For two-dimensional data — one bin at a time

Series, parameters and bins can be cycled through by using keyboard commands. There are differential options to contrast parameters from different series or bins within a series. Bin data are stacked at a separation chosen by the user.

The display can be panned and zoomed horizontally via the keyboard. The time axis graduation and associated annotation can go from months to seconds. A vertical axis can also be set up by the user.

Data values can be displayed in a cycle window tied to the current series, the current pointer position and, for two-dimensional data, the current bin. The current point in the series is selected by its proximity to the mouse's X coordinate.

Editing can occur on the time series page but is limited to modifying flags. Flags can be toggled individually by the middle mouse button or collectively by defining a box.

The track map

A large part of BODC's workload is associated with processing data recorded while a ship is in transit, where multiple sensors may make recordings every few seconds. Therefore, it is useful to be able plot the ship's position on a track map. The time series page supports this functionality.

Currently the map is limited to a world coastline at two resolutions but in future it should be possible to include GEBCO bathymetry. Nonetheless the system as it stands can cope with data at vastly different scales (contrast the Solent with the whole of the North and South Atlantic) without seriously slowing down the associated time series display. The current ship position is clearly identified on the map.

The profile (CTD) page

The profile page or CTD page is predominantly used in BODC for screening CTD (Conductivity, Temperature, Depth) data. However, it can support any data where the independent variable of interest is pressure, depth or altitude.

These variables may not be, and frequently are not, strictly monotonic within a series. Therefore, using the proximity of the mouse's X coordinate to determine the current point (as used for time series) is not necessarily helpful. It could, for instance, take you between points which are not adjacent in time.

The solution is to chop the series up into 'casts' which can be thought to be predominately monotonic. The segmentation into casts is performed automatically during invocation. Information supplied by the user in the driver file ensures that ship movement and other factors do not lead to a plethora of miniscule casts.

Using the settings from the series page, the chosen series, cast and parameters are displayed on the profile page. The Y axis normally increases down the screen but can be flipped, by an option, for a height variable. The zoom orientation is vertical.

The editing of flags is permitted via the profile page. Flags can be toggled individually by the middle mouse button or collectively by defining a box.

Scatter plots

Edserplo supports two types of scatter plot, the Current meter scatter plot and the X-Y scatter plot, both of which are used with the time series page. As with all plots in Edserplo, panning and zooming are standard and both scatter plots support outlier chasing.

Outlier chasing allows points selected (by defining a box) on the scatter plot to be highlighted on the time series page. Each point can be visited in turn and flagged on the time series page if required.

Current meter scatter plot

To enable the current meter scatter plot, Easting and Northing channels must exist either as parameters or as derived quantities generated at invocation. Up to eight series can be displayed to allow comparison between current meter series from the same mooring. The X and Y scales are always locked together.

The selected series can be superimposed and the order of plotting can be reversed, so that smaller plots are not obliterated by larger ones. The plot also displays

• A compass rose, which can be rotated to assess the dominant axis direction in the tidal ellipse.
• A circle, which can be adjusted to gauge maximum and minimum currents.

An extension to support two-dimensional (ADCP) data is still to be completed.

X-Y scatter plot

Any two parameters selected via the series page may be plotted on the X-Y scatter plot. If necessary, the X and Y scales can be adjusted independently.

Thermistor chain or ADCP (TCAD) page

Any two-dimensional data selected via the series page, with time as one independent variable and depth or pressure as the other, can be presented as a series of profiles on the TCAD page.

The zoom axis is vertical. The number of profiles presented at one time is user-adjustable and there is a differential mode to distinguish the current profile from its neighbours.

This page can be locked together with the time series page so movement between cycles on one page translates to movement between cycles on the other. Flag editing is permitted on the TCAD or the time series page.

Wave data

Edserplo supports two pages for assessing wave data. The Wave histogram for wave statistics (significant wave height — Hs and zero-crossing period — Tz) and Wave spectrum page. Data flags may be edited directly on the spectrum page and indirectly via the histogram page. Both presentations are used in conjunction with the time series page.

Wave histogram

The wave histogram may used for an immediate assessment of the 'shape' of the data. A double histogram of cells 0.5 metres by 0.5 seconds is presented with Hs on the Y axis. The page supports five double histograms derived from the data series. This includes four seasonal and a fifth (a year) featuring the seasons altogether.

Curves associated with the theoretical wave steepness limit may also be displayed on the page. Any data values that fall above or to the left of these steepness curves are not feasible and should be flagged. Providing certain conditions are fulfilled flagging can be done by highlighting a cell and using the update command. By locking the page to the time series page you can check which data points will be flagged.

Wave spectrum

A wave series may include spectral data. For example, a wave buoy’s motion can be monitored for 15 minutes every three hours and the results Fourier-transformed to yield a series of power spectra. The wave spectrum page plots spectral (including directional spectra) channels against frequency. The page functionality mimics that of the TCAD page with one exception — the zoom orientation and the independent axis are horizontal.

HF Radar surface currents — the OSCR page

The Ocean Surface Current Radar (OSCR) system is used for monitoring surface currents synoptically across an area, normally in coastal waters. OSCR data is inherently noisy, as ships can produce many spurious returns, so a handy mechanism for flagging out such data is required. Other systems, collecting similar data, can also be presented on the OSCR page.

The OSCR page is used in connection with the time series page. A set of cells is defined by a set of latitudes and longitudes and these points are displayed with the associated current vector against a background map. The positioning information for the cells is integral to the series. The page can be zoomed and panned in the usual way and cells to be edited can be defined by a box.

Processing tidal data

BODC is responsible for processing UK National Tide Gauge Network data and Edserplo has been designed with this in mind. Along with its other functions, Edserplo allows users to

• Process overlapping data
• Visually assess continuity between series
• Edit data and not just flags
• Aggregate data files into month and year files
• Generate tides and residuals on a port-by-port basis
• Produce statistics
• Perform tidal analysis

To run Edserplo in tidal processing mode, the user must provide a driver file containing a list of data file names with port/channel mapping and a 'dump' file. The presence of a dump file is required to switch Edserplo to tidal processing mode. Preferred display settings may be loaded from a settings file.

The port page

The port page has a similar functionality to the series page. It lists the ports of interest, cross-referenced against channel names, scales, bases and colours. There are normally several series associated with each port. Although series can be edited, the files from which they derive are not altered, the data are retained after each invocation in the 'dump' file. Data can be added to the dump file via a driver file.

The port time series page

The port time series display page looks very similar to that of the time series however there are many subtle and not so subtle differences between the two. One peculiarity is that base and scale are defined for each port and channel combination because tidal ranges differ by an order of magnitude around the UK.

In addition to flag editing, the user may perform data value editing. Editing includes data substitution, data replacement by interpolation, gap filling, linear time stretching or shrinking and constant offset adjustment.

The output page

Having selected ports and parameters on the port page, the user goes to the output page to generate new files. The user selects

• The format
• The time span
• The filename template
• The file segmentation scheme (year, month, none, etc)
• The filtering mechanism to be employed (if necessary)

Output can only be produced if the overlaps between the series involved have already been flagged for deletion.

The output page is also used to create tidal analyses (Doodson harmonic analysis), generate tidal statistics and save deleted series in the dump file. The deletion takes effect on exit when the dump file is updated. A security mechanism ensures that only one person can update a specified dump file at any one time.

The tidal analysis page

Tidal analyses are reviewed on the tidal analysis page. It is possible to compare selected subsets of constituents of various analyses in an interleaved fashion. Tidal analysis is performed using a Java version of NOC's TIRA analysis program.