Creating a BGF from a Commercial Fermentation System

Whats this vignette is about

This vignette is one of three, that focus the topic of how to get data into a BGF. In particular, it deals with a certain type of BGF, which is created from a raw report file of a commercial available fermentation system.

If your interested in the general concepts of the bgfanalyzer package or in how to build or visualize a BGF, it is suggested to read vignette("Introducing-BGF",package="bgfanalyzer") first.

The ‘AMPTSV2’-type BGF

Background

Many lab-scale fermentation system’s exist. One quite popular system in biogas industry research labs is the AMPTS II distributed by the Swedish company BPC instruments AB (Lund, Sweden).

Due to its popularity, this system has also gained the attention of the R community already, as several online tutorials exist, that deal with processing data generated by this system. In fact, even the authors of the CRAN biogas package refer to it.

A dedicated workflow, that deals with AMPTS II generated data was also integrated into the bgfanalyzer package, hoping to standardize the data analysis workflow among AMPTS II-users in the current and future R community.

The commercial fermentation system

The commercial fermentation system (CFS hereafter) known as AMPTS II allows to run up to fifteen fermentations in parallel. It features automated stirring, CO2 removal from exhaust gas, and records the cumulative exhaust gas volume for each fermentation reactor. The process temperature of all fermentation reactors is adjusted by the same external thermostat.

The CFS Workflow

Many of the R workflows existing online start with pre-processing the external file generated by the CFS with third party software, so that it can be imported into R. For the bgfanalyzer package this is NOT the case.

Imagine, our CFS produced a report file name ‘AMPTSV2.csv’. Do not open this file and if you do, don’t save any changes. Spreadsheet software like Excel or libreoffice calc overwrite the default formatting when the saving the file as ‘.csv’ again.

Creating a BGF from the native ‘.csv’ file is as simple as this:

# load library
library(bgfanalyzer)

# load the example reactor layout
RL <- LabscaleBiogasLayout

# inspect reactor layout
RL
#>  [1] "Blank"     "Blank"     "Cellulose" "S1 ctrl"   "S1 ctrl"   "S1 7d"    
#>  [7] "S1 7d"     "S1 4d"     "S1 4d"     "S2 ctrl"   "S2 ctrl"   "S2 4d"    
#> [13] "S2 4d"     "S2 6d"     "S2 6d"

# import the CFS data to a BGF
myBGF <- from_AMPTSV2_report(ReactorLayout = RL, 
                             BlankLabel = "Blank",
                             name = "CFS data",
                             ProcessTemp = 42,
                             InocToSubRatio = 2,
                             path = system.file("extdata","AMPTSV2.csv",package="bgfanalyzer"))

# inspect BGF
myBGF
#> 'CFS data' - a BGF with 15 fermentation(s)
#> 
#> 
#> $ExpParam: 11 experimental paramerters
#> $metaData: 16 meta variables
#> $BioGasData: 735 observations of 7 fermentation variables
#> 
#> 
#>                   yield    sd_yield production sd_production time_production
#> Blank     -2.997602e-14  2.88499567     75.325    5.26794552             1.0
#> Cellulose  2.899017e+01          NA    182.260            NA             3.0
#> S1 ctrl    6.188975e+02  9.87799079    700.350    0.77781746             3.0
#> S1 7d      2.618791e+02  9.49000623    359.790    6.49124025             3.0
#> S1 4d      1.087374e+03 11.21296479    644.655   20.73944189             3.0
#> S2 ctrl    7.781660e+01  0.02079726    119.845    0.04949747             1.0
#> S2 4d      4.011575e+02  2.21973211    110.935    0.21920310             2.0
#> S2 6d      1.335888e+02 42.32196395    128.480   25.54069694             4.5
#>           sd_time_production
#> Blank               0.000000
#> Cellulose                 NA
#> S1 ctrl             0.000000
#> S1 7d               0.000000
#> S1 4d               0.000000
#> S2 ctrl             0.000000
#> S2 4d               0.000000
#> S2 6d               3.535534

Et voila, a BGF is created!

The function from_AMPTSV2_report() requires four arguments to be set and will then build the BGF from the external file. These required arguments are ‘ReactorLayout’, ‘ProcessTemp’, ‘InocToSubRatio’ and ‘path’.

But what exactly is from_AMPTSV2_report() doing? This will be the topic of section ‘Behind the scenes’ of this vignette. Besides, the BGF created via from_AMPTSV2_report() is suitable for data visualization using bgf_plot() and related functions.

BGF subsetting

From the reactor layout of the created BGF we can see that there were two substrates tested (S1 and S2). We can build a subset from the BGF that only contains data of either one of the two substrates using the function subset_BGF():

# inspect reactor layout of BGF
get_ReactorLayout(myBGF) # Layout 4, 5, 6, 7, 8, 9 have 'S1' in their layout, 10, 11, 12, 13, 14, 15 have 'S2'
#>  [1] Blank     Blank     Cellulose S1 ctrl   S1 ctrl   S1 7d     S1 7d    
#>  [8] S1 4d     S1 4d     S2 ctrl   S2 ctrl   S2 4d     S2 4d     S2 6d    
#> [15] S2 6d    
#> Levels: Blank Cellulose S1 ctrl S1 7d S1 4d S2 ctrl S2 4d S2 6d

# create a subset BGF by selecting individual reactors
S1_subsetBGF <- subset_BGF(myBGF,
                           reactor = c("R1","R2","R4","R5","R6","R7","R8","R9"), # <<-- # specify reactors by their ID in 'BioGasData$reactor' (= rownames of 'metaData'-layer)
                           name = "S1 subset") # <<-- # this line creates a new 'ExpParam$name'

# create a subset BGF by selecting individual reactor layout
S2_subsetBGF <- subset_BGF(myBGF,
                           layout = c("Blank","S2 ctrl","S2 4d","S2 6d"), # <<-- # specify reactors by their 'metaData$Layout' value
                           name = "S2 subset") # <<-- # this line creates a new 'ExpParam$name'

Let’s check the sub setting results by plotting the yield:

# original yield boxplot
bgf_plot(myBGF,type = "yield_box")


# S1 yield boxplot
bgf_plot(S1_subsetBGF,type = "yield_box")


# S2 yield boxplot
bgf_plot(S2_subsetBGF,type = "yield_box")

As we can see, the subsets only contain the data of one substrate (and of ‘Blank’ fermentations).

Behind the scenes

Before proceeding, let’s first briefly have a look at the ExpParam$MeasurementType of our BGF created via from_AMPTSV2_report():

# print the measurement type of the BGF
myBGF$ExpParam$MeasurementType
#> [1] "AMPTSV2"

Step 1: Object creation

Every BGF has a ‘MeasurementType’ slot in its ExpParam-layer, which is the place meant to store the type of data acquisition. The string ‘AMPTSV2’ indicates that these measurements were acquired with the CFS and that the BGF object itself was created via from_AMPTSV2_report().

# we take the same specifications as for the cal to from_AMPTSV2_report()
# the first that happens when from_AMPTSV2_report() is called is a call of BGF()
altBGF <- BGF(ReactorLayout = RL,
              BlankLabel = "Blank",
              name = "Step-by-step CFS",
              ProcessTemp = 42,
              InocToSubRatio = 2,
              MeasurementType = "AMPTSV2") # <<-- # The 'MeasurementType' is set during object creation

A new BGF was created, but at the moment it does not contain data. This will change now through a call to add_bmp_measurement().

Step 2: Adding data

Internally, this function will call read_raw_AMPTSV2_report(path) to build a list from the external file stored at path. Next, this list is added to the BGF by successively calling add_ExpPara(), add_Exp_Setup() and sort_AMOTSV2_reactors().

So:

# adding data to alternative BGF
altBGF2 <- add_bmp_measurement(x = altBGF,
                              path = system.file("extdata","AMPTSV2.csv",package="bgfanalyzer"),
                              mode = altBGF$ExpParam$MeasurementType)

# inspect result
altBGF2
#> 'Step-by-step CFS' - a BGF with 15 fermentation(s)
#> 
#> 
#> $ExpParam: 11 experimental paramerters
#> $metaData: 10 meta variables
#> $BioGasData: 735 observations of 7 fermentation variables
#> 
#> 
#> A '$yield' is not calculated yet

will yield in the same BGF as:

# import the external file to an R list
ExFile <- read_raw_AMPTSV2_report(system.file("extdata","AMPTSV2.csv",package="bgfanalyzer"))

# add data from 'ExFile' to 'ExpParam'-layer of altBGF  
altBGF <- add_ExpPara(x = altBGF,rawReport = ExFile)
altBGF <- add_ExpSetup(x = altBGF,rawReport = ExFile)
altBGF <- sort_AMPTSV2_reactors(x = altBGF,rawReport = ExFile)

# inspect result
altBGF
#> 'Step-by-step CFS' - a BGF with 15 fermentation(s)
#> 
#> 
#> $ExpParam: 11 experimental paramerters
#> $metaData: 10 meta variables
#> $BioGasData: 735 observations of 7 fermentation variables
#> 
#> 
#> A '$yield' is not calculated yet

These objects look almost like ready-to-use BGF’s, and even simple plotting works.

# try to plot altBGF
plot(altBGF)

However, using bgf_plot() will yield in awkward results:

# try to plot altBGF
bgf_plot(altBGF,type = "product")
#> `geom_line()`: Each group consists of only one observation.
#> ℹ Do you need to adjust the group aesthetic?

Step 3: Data correction

This is due to R interpreting the data imported via read_raw_AMPTSV2_report() is interpreted to be type character, as each element in the original file is quoted. Thus, it is necessary to convert certain columns to type numeric:

# convert columns in 'BioGasData'-layer to numeric data type
altBGF <- cols_to_numeric(altBGF)

The function cols_to_numeric() targets BioGasData$time, BioGasData$product, and BioGasData$production if only a BGF is specified and returns the same BGF with converted data types at the targeted locations.

Let’s try bgf_plot() again:

# try to plot altBGF again
bgf_plot(altBGF,type = "product")
#> Warning: Removed 11 rows containing missing values or values outside the scale range
#> (`geom_line()`).

As we can see, bgf_plot() now works correctly. However, there are some NA’s in BioGasData$product die to the way the original CFS report is generated. These NA’s are removed by close_gaps() when creating a BGF with from_AMPTSV2_report().

# Close gaps in 'BioGasData$product'
altBGF <- close_gaps(altBGF)

# See the plot now
bgf_plot(altBGF,type = "product")

The NA’s we’re replace by the last valid BioGasData$product value for each fermentation. All product curves have the same length now.

The CFS also calculates its own ‘flow’ values from the recorded exhaust gas measurements of each of its reactors. These ‘flow’ values are transferred to BioGasData$production during object creation with from_AMPTSV2_report().

However, they also have NA‘s as the CFS fails to calculate a flow if the recorded exhaust gas volume of a fermentation does not increase anymore. If the volume does not increase, the ’production’, e.g. the difference between current and previous volume is zero. Thus, NA’s in BioGasData$production are be corrected as follows:

# correct NA's in BioGasData$production
altBGF$BioGasData$production[grep(T,is.na(altBGF$BioGasData$production))] <- 0

Step 4: net gas calculation

Now, BioGasData$product and BioGasData$production values were transferred from the CFS-generated report to the BGF. Next, BioGasData$netGas is calculated by calling netGas().

# calculate the netGas
altBGF <- netGas(altBGF)

This function first calculates a mean value of all BioGasData$product values from fermentations marked as ‘Blank’ in metaData$Blank at each BioGasData$time. The resulting vector contains a ‘Blank’ gas volume for each time. This vector is multiplied with a fermentation specific factor and afterwards subtracted from each corresponding BioGasData$product value of every fermentation. This results in a fermentation specific net gas volume - a exhaust gas volume corrected by the amount of gas produced from the ‘Blank’ share in each reactor.

The fermentation specific factor is calculated based on a mass balance calculated from values stored at metaData$Inoculum VS/COD amount [g]. Every BGF created from a CFS-generated report has a metaData$Inoculum VS/COD amount [g].

The net gas is calculated by the following equation:

Step 5: calculate relative production

The BioGasData$rel_production is calculated by relative_production():

# calculate the relative production
altBGF <- relative_production(altBGF)

The value stored at BioGasData$rel_production resembles the amount of BioGasData$product of the final value of BioGasData$product at a BioGasData$time. It is calculated per fermentation and will sum to 100% in each.

Step 6: yield calculation and summary

Finally, BioGasData$yield is calculated via calc_yield():

# calculate yield
altBGF <- calc_yield(altBGF)

And a yield summary in the metaData-layer at the final BioGasData$time is generated by summarize_yield():

# create a yield summary
altBGF <- summarize_yield(altBGF)

# inspect the final BGF
altBGF
#> 'Step-by-step CFS' - a BGF with 15 fermentation(s)
#> 
#> 
#> $ExpParam: 11 experimental paramerters
#> $metaData: 16 meta variables
#> $BioGasData: 735 observations of 7 fermentation variables
#> 
#> 
#>                   yield    sd_yield production sd_production time_production
#> Blank     -2.997602e-14  2.88499567     75.325    5.26794552             1.0
#> Cellulose  2.899017e+01          NA    182.260            NA             3.0
#> S1 ctrl    6.188975e+02  9.87799079    700.350    0.77781746             3.0
#> S1 7d      2.618791e+02  9.49000623    359.790    6.49124025             3.0
#> S1 4d      1.087374e+03 11.21296479    644.655   20.73944189             3.0
#> S2 ctrl    7.781660e+01  0.02079726    119.845    0.04949747             1.0
#> S2 4d      4.011575e+02  2.21973211    110.935    0.21920310             2.0
#> S2 6d      1.335888e+02 42.32196395    128.480   25.54069694             4.5
#>           sd_time_production
#> Blank               0.000000
#> Cellulose                 NA
#> S1 ctrl             0.000000
#> S1 7d               0.000000
#> S1 4d               0.000000
#> S2 ctrl             0.000000
#> S2 4d               0.000000
#> S2 6d               3.535534

Besides its ExpParam$name, this BGF is exactly the same as the one created via from_AMPTSV2_report(). The general advice, however, is to use from_AMPTSV2_report() when creating a BGF from CFS-generated reports, instead of executing the six step workflow by hand.