Data processing

The data processing and quality control takes place in two phases: an automated phase and a manual phase. Most of the quality control and processing of the near real-time data is carried out in an automated way using specifically developed software. The delayed mode data will have also undergone manual checks whereby a Data Scientist will have performed a visual check; undertaken parameter inter-comparisons; and cross-checked the data against published climatologies. Their findings and actions will have been fully documented.

When defining the near real-time quality control steps, care was taken to follow standards already introduced by other international projects. These include the Argo, the Global Ocean Surface Underway Data (GOSUD), and the Global Temperature and Salinity Profile Project (GTSPP) real-time quality control procedures.

A complete list of the quality control and processing procedures applied to CARBON-OPS data, prior to being loaded in the BODC CARBON-OPS database, is documented below.The items marked with ^* are planned for version II of the automated software.

Near real-time quality control procedures

1. Impossible date and time
2. Impossible location
3. Impossible ship speed
4. Global range
5. Data spikes^*
6. Flat record^*
7. Temperature comparisons^*
8. xCO2 versus standard^*

Quality control flags

Near real-time processing procedures

1. Data reduction
2. Calculation of salinity
3. Correction of pCO₂ to dry air
4. Calibration of raw pCO₂ against gas standards
5. Conversion of x(CO₂)_dry,cal to p(CO₂)_dry,cal
6. Calibration of the equilibrator's temperature sensors
7. Correction for water vapour pressure
8. Correction for in-situ temperature
9. Correction of fugacity

References

Near real-time quality control procedures

1. Impossible date and time

1. Check that month values are between 1 and 12.
2. Check that number of days in month does not exceed 29 for February and 31 for all other months. Note: this check will become month specific in Version II of the software.
3. Check that the hour values are between 0 and 23.
4. Check that the minute values are between 0 and 59.
5. Check that the second values are between 0 and 59.
6. Check that the date is between 01/10/2005 00:00 and current date (sysdate).

Action: failed records are not loaded to the database but are diverted to a log file for manual checks.

2. Impossible location

1. Check that latitude is in the range -90 and +90.
2. Check that longitude is in the range -180 and +180.
3. Check for position on land by comparison against a bathymetry file identical to that used by the Argo programme.

Action: failed records are loaded into the database but flagged with a suspect 'M' flag by the software pending manual correction.

3. Impossible ship speed

The check uses successive ship positions to calculate the distance travelled between two data cycles. When the resulting speed exceeds 10 m per second, an alarm message is sent by email to a data scientist who will check the records and manually flag or correct the erroneous value. This check enables the software to detect spikes in the ship’s position or time channels.

4. Global range

The check uses limits for parameters as set up in the BODC parameter dictionary. Checks are carried out on all incoming parameters labelled with a BODC parameter code.

This check has two goals:

1. Identifying impossible values which are outside the maximum allocated space for a particular parameter (LFM).
2. Identifying impossible values which are outside the minimum and maximum values defined in the BODC parameter dictionary for that particular parameter.

The limit and absent data values currently used for each of the input parameters is described in the parameters document (69 KB).

Action (1): input values are replaced by their absent data values as defined in BODC parameter dictionary and the flag value for that parameter record is set to 'N' (null).

Action (2): values outside the minimum and maximum range are left unchanged but the flag value for that parameter record is set to 'M' (suspect).

5. Data spikes

Checks for spikes will be performed on all temperature channels, and on the salinity, atmospheric pressure, and atmospheric CO₂ channels. Planned for version II of the automated software.

6. Flat record

Planned for version II of the automated software.

7. Temperature comparisons

Checks will include two-by-two comparisons of various temperature channels: upper and lower equilibrator temperature values, hull and thermosalinograph, hull and equilibrator, and equilibrator and thermosalinograph. Planned for version II of the automated software.

8. xCO2 versus standard

Checks will look at whether the xCO2 values measured in air and in sea water are within the range of the measured standard values. Planned for version II of the automated software.

 

Quality control flags

The CARBON-OPS data use a combination of the BODC and the International Ocean Carbon Coordination Project (IOCCP) quality control flags.

BODC quality control flags

The following single character qualifying flags may be associated with one or more individual parameters within a data cycle:

Flag	Description
Blank	Unqualified
<	Below detection limit
>	In excess of quoted value
H	Extrapolated value
L	Improbable value – originator's quality control
M	Improbable value – BODC quality control
O	Improbable value – user quality control
R	Replacement value
T	Interpolated value
U	Uncalibrated

IOCCP quality control flags

In addition to BODC quality control flags, CARBON-OPS uses the system of sub-flag recommended by IOCCP to provide additional information on questionable measurements. The flag values range from 1 to 10 with the following definitions

1. Outside of the CO₂ gas standards range
2. Questionable/interpolated sea surface temperature (SST)
3. Questionable equilibrator temperature (EQU T)
4. Anomalous (EQU T - SST) (± 1oC)
5. Questionable sea surface salinity
6. Questionable pressure
7. Low equilibrator gas flow
8. Questionable air value
9. Interpolated standards
10. Other (see metadata)

The automated quality control procedures currently in place do not use the full range of IOCCP quality control flags. This will be implemented progressively as the project develops.

 

Near real-time processing procedures

1. Data reduction

BODC receives the raw data in the form of comma-separated values (CSV) files generated from the raw pCO₂ database held at the Plymouth Marine Laboratory (PML). The raw data consists of eight separate files

1. Ship/cruise identifier
2. Equilibrator measurements
3. Marine air measurements
4. Gas standard measurements
5. Ship's ancillary data
6. Ship's navigation and meteorology data
7. Alarms
8. Messages

The underway p(CO₂) analytical cycle is composed of a sequence of analyses comprising an equilibrator air measurement, a marine air measurement, and a gas standard measurement. In addition, the sequence also includes the logging of ancillary and navigation and meteorology data from the ship’s instrumentation.

In the raw data files, each measurement is associated with its own time and position corresponding to the time the measurement was made within the analytical sequence.

Having received and archived the original data, the first task for the automated processing software is to reduce the input data to one data cycle per analytical cycle.

The reduction is based on the equilibrator measurement; its time stamp is used as the time stamp for the entire analytical cycle. The analytical is given a unique cycle identifier and all the other measurements for the same analytical cycle are linked to the equilibrator's time stamp without interpolation.

In the event of a missing equilibrator measurement, the software takes the next available measurement in the rotation to use as the time stamp for the entire cycle.

2. Calculation of salinity

The calculation of salinity is based on conductivity and the concomitant temperature measurement in the thermosalinograph (TSG) manifold. If either measurement is missing or flagged suspect by the automated quality control then salinity is not derived.

3. Correction of pCO₂ to dry air

The raw pCO₂ measurements from the equilibrator, marine air and the three standards are corrected to dry air based on the formula adapted from DOE (1994) SOP5.

p(CO₂)_{dry, uncal} = p(CO₂)_{meas, uncal} / (1 – (p(H₂O)_meas/1000))

where

• p(CO₂)_{meas, uncal} is the raw p(CO₂) data in parts per million as measured by the LICOR, prior to calibration against standard gases.
• p(H₂O)_meas is the measured H₂O content of a seawater sample of a given salinity and temperature in parts per thousand.

Corrected values will not be generated if any of the input parameters are missing or have been flagged suspect by the automated quality control.

4. Calibration of raw pCO₂ against gas standards

The raw pCO₂ values measured by the LICOR and corrected to dry air (p(CO₂)_{dry, uncal}) are calibrated against gas standards held in gas cylinders onboard the ship. Each standard has known CO₂ concentrations determined against National Oceanic and Atmospheric Administration (NOAA) standards. There are three standards: a CO₂-free gas standard, a standard with a nominal CO₂ concentration of 250 ppm and a standard with a nominal CO₂ concentration of 450 ppm.

As only one standard is measured during an analytical cycle, the standards are measured alternately, so each is measured every third cycle. Prior to performing the pCO2 calibration it is necessary, in order to have a value for each of the three standards, to interpolate or extrapolate the standard measurements.

Interpolation is carried out between successive measured values coming from the same gas cylinders and separated by less than 12 hours. For the first and last measurement within a batch of data, the missing standard values are derived by extrapolation from the nearest measurement for that cylinder within a six hour period. If a standard is not available under either of these conditions, the calibrated pCO₂ value is not derived.

Linear regression of p(CO₂)_{dry, uncal} against the known x(CO₂) values for the three standards is then used to provide slope and intercept, which are then applied to the p(CO₂)_{dry, uncal} values to correct them to x(CO₂)_{dry, cal}.

Note – linear regression against x(CO₂) removes the pressure signal from the CO₂ measurements. It would be possible to divide p(CO₂)_{dry, uncal} by pressure to obtain x(CO₂)_{dry, uncal} before undertaking linear regression to x(CO₂)_{dry, cal} but because the linear regression effectively does this anyway, it makes this extra step redundant.

In situations when the calibrated concentration value of the standard is not yet available, the software uses the nominal concentration instead and flags any derived value with a 'U' flag to indicate that the value is 'Uncalibrated'.

5. Conversion of x(CO₂)_dry,cal to p(CO₂)_dry,cal

This is done using the standard formula:

p(CO₂)_{dry, cal} = x(CO₂)_{dry, cal} . p

where

p is the barometric pressure, nominally at sea level, expressed in units of atmosphere. We use the atmospheric pressure measured by the ship's meteorological instruments at the height of the instruments (generally 5 - 10 m above sea level). The pressure is not currently corrected to sea level.If the ship's atmospheric pressure is missing or flagged suspect then the processing software uses the pressure measured by the LICOR during the marine air measurement.

The conversion will not take place if the atmospheric pressure measured by the ship's instrumention or by the LICOR are missing or have been flagged suspect.

6. Calibration of the equilibrator's temperature sensors

The data from the two platinum resistance thermometers located at the top and bottom of the equilibrator are dynamically calibrated, corrected for ice-point error and scaled to the ITS-90 temperature scale prior to loading in the BODC CARBON-OPS database. Calibration of the instruments is performed at the Plymouth Marine Laboratory every six months or as soon as possible thereafter. The calibration coefficients are then sent to BODC where they are entered in the CARBON-OPS database and applied dynamically to the incoming data.

7. Correction for water vapour pressure

Correction for water vapour pressure is carried out using the equations of Weiss & Price (1980), which are valid over the temperature range 0 to 40°C (273-313K) and salinity range 0-40 PSU.

For equilibrator seawater measurements, the water vapour pressure in the equilibrator sample in atmosphere units (p(H₂O)_equ) is first calculated as follows:

ln p(H₂O)_equ = 24.4543 – 67.4509 (100/T_equ) – 4.8489 ln (T_equ/100) – 0.000544 S

where

• S is the salinity (PSU)
• T_equ is equilibrator water temperature in degrees K

For the pCO₂ system used in CARBON-OPS, the water temperature is measured at the top (inflow) and at the bottom (outflow) of the equilibrator by two Platinum Resistance thermometers. The software takes the mean of the two values in °C if both values are good; if only one is good then this is the value that will be used. It then converts the value to degrees K to obtain T_equ.

For marine air measurements, the water vapour pressure (p(H₂O)_sst) over the sea surface is determined using the in-situ sea surface temperature measured by the ship's instruments converted to degrees K, and the sea surface salinity.

ln p(H₂O)_sst = 24.4543 – 67.4509 (100/T_s) – 4.8489 ln (T_s/100) – 0.000544 S

where

• S is the salinity (PSU)
• T_s is the sea surface water temperature measured at the ship's hull in degrees K

The water vapour pressure values are then used to correct p(CO₂)_{dry, cal} to 100% humidity using the standard formula as given in DOE (1994) SOP 5:

p(CO₂)_{wet, cal} = p(CO₂)_{dry, cal} . (1 – p(H₂O)_equ)

where

p(H₂O) is the vapour pressure in units of atmosphere over a seawater sample of a given salinity at the temperature of equilibration for the equilibrator sample or at the sea surface temperature for the marine air sample.

8. Correction for in-situ temperature

The sea water sample pCO₂ is then corrected to in-situ temperature as per DOE (1994) SOP 5, using the constant of 0.0423 determined experimentally by Takahashi et al (1993)

p(CO₂, T_s)_{wet, cal} = p(CO₂, T_equ)_{wet, cal} . e^{(0.0423(Ts-Tequ))}

9. Correction of fugacity

The calculation of the fugacity ƒ(CO₂) includes the following steps

a) Starting with x(CO₂)_{dry, cal} at T_equ, calculate at x(CO₂) at SST using Takahashi et al. (1993)

x(CO₂, T_s)_{dry, cal} = x(CO₂, T_equ)_{dry, cal} . e^{(0.0423(Ts-Tequ))}

b) Calculate at 100% humidity

x(CO₂, T_s)_{wet, cal} = x(CO₂, T_s)_{dry, cal} . (p – p(H₂O)_sst ) /p

where

p is the atmospheric pressure in Pascals and p(H₂O)_sst is the water vapour pressure at SST in Pascals.

p(H₂O)_sst in atmosphere units is calculated from Weiss & Price (1980) using the sea surface temperature measured at the hull (T_s) in degrees K.

ln p(H₂O)_sst = 24.4543 – 67.4509 (100/T_s) – 4.8489 ln (T_s/100) – 0.000544 S

p(H₂O)_sst is converted to Pascals by multiplying the value by 101325.

c) Calculate the virial coefficients (δ, β, γ) using Weiss (1974).

δ = (57.7 - 0.118 . T_s) . 10^-6
β = (-1636.75 + 12.0408 . T_s - 0.0327957 . T_s² + 3.16528 . 10^-5 . T_s³) . 10^-6
γ₁ = (β + 2δ . (1-(x(CO₂, T_s)_{wet, cal} . 10^-6))²), calculated for both sea and air x(CO₂)
γ₂ = (p . 100)/(R . T_s), where R is the gas constant 8.3144
γ_total = exp (γ₁ . γ₂), calculated for both sea and air x(CO₂)

d) Calculate the fugacity

ƒ(CO₂) = γ_total . p(CO₂, T_s)_{wet, cal}

References

DOE (1994) Standard Operating Procedures (SOP) 5: Determination of p(CO2) in air that is in equilibrium with a continuous stream of sea water.
In: Handbook of methods for the analysis of the various parameters of the carbon dioxide system in sea water - Version 2, A. G. Dickson & C. Goyet, eds. ORNL/CDIAC-74.

Takahashi T, Olafsson J, Goddard J, Chipman DW, Sutherland SC (1993) Seasonal variation of CO2 and nutrients in the high-latitude surface oceans: A comparative study.
Global Biogeochemical Cycles, 7, 843-878.

Weiss RF (1974) Carbon dioxide in water and seawater: the solubility of a non-ideal gas.
Marine Chemistry 2:203-215.

Weiss RF, Price BA (1980) Nitrous oxide solubility in water and seawater.
Marine Chemistry 8:347-359.