--- title: "Creating a `BGF` from a Commercial Fermentation System" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Creating a `BGF` from a Commercial Fermentation System} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ## Whats this vignette is about This vignette is one of three, that focus the topic of how to get data into a `BGF`. In particular, it deals with a certain type of `BGF`, which is created from a raw report file of a commercial available fermentation system. If your interested in the general concepts of the `bgfanalyzer` package or in how to build or visualize a `BGF`, it is suggested to read `vignette("Introducing-BGF",package="bgfanalyzer")` first. ## The 'AMPTSV2'-type `BGF` ### Background Many lab-scale fermentation system's exist. One quite popular system in biogas industry research labs is the AMPTS II distributed by the Swedish company BPC instruments AB (Lund, Sweden). Due to its popularity, this system has also gained the attention of the R community already, as several online tutorials exist, that deal with processing data generated by this system. In fact, even the authors of the CRAN `biogas` package refer to it. A dedicated workflow, that deals with AMPTS II generated data was also integrated into the `bgfanalyzer` package, hoping to standardize the data analysis workflow among AMPTS II-users in the current and future R community. ### The commercial fermentation system The commercial fermentation system (**CFS** hereafter) known as AMPTS II allows to run up to fifteen fermentations in parallel. It features automated stirring, CO~2~ removal from exhaust gas, and records the cumulative exhaust gas volume for each fermentation reactor. The process temperature of all fermentation reactors is adjusted by the same external thermostat. ### The CFS Workflow Many of the R workflows existing online start with pre-processing the external file generated by the CFS with third party software, so that it can be imported into R. For the `bgfanalyzer` package this is NOT the case. Imagine, our CFS produced a report file name 'AMPTSV2.csv'. Do not open this file and if you do, don't save any changes. Spreadsheet software like `Excel` or `libreoffice calc` overwrite the default formatting when the saving the file as '.csv' again. Creating a `BGF` from the native '.csv' file is as simple as this: ```{r from_AMPTSV2,warning=FALSE} # load library library(bgfanalyzer) # load the example reactor layout RL <- LabscaleBiogasLayout # inspect reactor layout RL # import the CFS data to a BGF myBGF <- from_AMPTSV2_report(ReactorLayout = RL, BlankLabel = "Blank", name = "CFS data", ProcessTemp = 42, InocToSubRatio = 2, path = system.file("extdata","AMPTSV2.csv",package="bgfanalyzer")) # inspect BGF myBGF ``` Et voila, a `BGF` is created! The function `from_AMPTSV2_report()` requires four arguments to be set and will then build the `BGF` from the external file. These required arguments are 'ReactorLayout', 'ProcessTemp', 'InocToSubRatio' and 'path'. But what exactly is `from_AMPTSV2_report()` doing? This will be the topic of section 'Behind the scenes' of this vignette. Besides, the `BGF` created via `from_AMPTSV2_report()` is suitable for data visualization using `bgf_plot()` and related functions. #### `BGF` subsetting From the reactor layout of the created `BGF` we can see that there were two substrates tested (S1 and S2). We can build a subset from the `BGF` that only contains data of either one of the two substrates using the function `subset_BGF()`: ```{r subset} # inspect reactor layout of BGF get_ReactorLayout(myBGF) # Layout 4, 5, 6, 7, 8, 9 have 'S1' in their layout, 10, 11, 12, 13, 14, 15 have 'S2' # create a subset BGF by selecting individual reactors S1_subsetBGF <- subset_BGF(myBGF, reactor = c("R1","R2","R4","R5","R6","R7","R8","R9"), # <<-- # specify reactors by their ID in 'BioGasData$reactor' (= rownames of 'metaData'-layer) name = "S1 subset") # <<-- # this line creates a new 'ExpParam$name' # create a subset BGF by selecting individual reactor layout S2_subsetBGF <- subset_BGF(myBGF, layout = c("Blank","S2 ctrl","S2 4d","S2 6d"), # <<-- # specify reactors by their 'metaData$Layout' value name = "S2 subset") # <<-- # this line creates a new 'ExpParam$name' ``` Let's check the sub setting results by plotting the yield: ```{r sub-setting-result,fig.width=5} # original yield boxplot bgf_plot(myBGF,type = "yield_box") # S1 yield boxplot bgf_plot(S1_subsetBGF,type = "yield_box") # S2 yield boxplot bgf_plot(S2_subsetBGF,type = "yield_box") ``` As we can see, the subsets only contain the data of one substrate (and of 'Blank' fermentations). ## Behind the scenes Before proceeding, let's first briefly have a look at the `ExpParam$MeasurementType` of our `BGF` created via `from_AMPTSV2_report()`: ```{r MeasurementType} # print the measurement type of the BGF myBGF$ExpParam$MeasurementType ``` ### Step 1: Object creation Every BGF has a 'MeasurementType' slot in its `ExpParam`-layer, which is the place meant to store the type of data acquisition. The string 'AMPTSV2' indicates that these measurements were acquired with the CFS and that the `BGF` object itself was created via `from_AMPTSV2_report()`. ```{r Alternative-BGF} # we take the same specifications as for the cal to from_AMPTSV2_report() # the first that happens when from_AMPTSV2_report() is called is a call of BGF() altBGF <- BGF(ReactorLayout = RL, BlankLabel = "Blank", name = "Step-by-step CFS", ProcessTemp = 42, InocToSubRatio = 2, MeasurementType = "AMPTSV2") # <<-- # The 'MeasurementType' is set during object creation ``` A new `BGF` was created, but at the moment it does not contain data. This will change now through a call to `add_bmp_measurement()`. ### Step 2: Adding data Internally, this function will call `read_raw_AMPTSV2_report(path)` to build a list from the external file stored at `path`. Next, this list is added to the `BGF` by successively calling `add_ExpPara()`, `add_Exp_Setup()` and `sort_AMOTSV2_reactors()`. So: ```{r add_bmp_measurement} # adding data to alternative BGF altBGF2 <- add_bmp_measurement(x = altBGF, path = system.file("extdata","AMPTSV2.csv",package="bgfanalyzer"), mode = altBGF$ExpParam$MeasurementType) # inspect result altBGF2 ``` will yield in the same `BGF` as: ```{r add_bmp_measurement-details} # import the external file to an R list ExFile <- read_raw_AMPTSV2_report(system.file("extdata","AMPTSV2.csv",package="bgfanalyzer")) # add data from 'ExFile' to 'ExpParam'-layer of altBGF altBGF <- add_ExpPara(x = altBGF,rawReport = ExFile) altBGF <- add_ExpSetup(x = altBGF,rawReport = ExFile) altBGF <- sort_AMPTSV2_reactors(x = altBGF,rawReport = ExFile) # inspect result altBGF ``` These objects look almost like ready-to-use `BGF`'s, and even simple plotting works. ```{r plot-works,fig.width=7,fig.height=5} # try to plot altBGF plot(altBGF) ``` However, using `bgf_plot()` will yield in awkward results: ```{r bgf_plot-Error,fig.width=7,fig.height=5} # try to plot altBGF bgf_plot(altBGF,type = "product") ``` ### Step 3: Data correction This is due to R interpreting the data imported via `read_raw_AMPTSV2_report()` is interpreted to be type `character`, as each element in the original file is quoted. Thus, it is necessary to convert certain columns to type `numeric`: ```{r cols_to_num} # convert columns in 'BioGasData'-layer to numeric data type altBGF <- cols_to_numeric(altBGF) ``` The function `cols_to_numeric()` targets `BioGasData$time`, `BioGasData$product`, and `BioGasData$production` if only a `BGF` is specified and returns the same `BGF` with converted data types at the targeted locations. Let's try `bgf_plot()` again: ```{r bgf_plot_now_working,fig.width=7,fig.height=5} # try to plot altBGF again bgf_plot(altBGF,type = "product") ``` As we can see, `bgf_plot()` now works correctly. However, there are some `NA`'s in `BioGasData$product` die to the way the original CFS report is generated. These `NA`'s are removed by `close_gaps()` when creating a `BGF` with `from_AMPTSV2_report()`. ```{r close_gaps,fig.width=7,fig.height=5} # Close gaps in 'BioGasData$product' altBGF <- close_gaps(altBGF) # See the plot now bgf_plot(altBGF,type = "product") ``` The `NA`'s we're replace by the last valid `BioGasData$product` value for each fermentation. All product curves have the same length now. The CFS also calculates its own 'flow' values from the recorded exhaust gas measurements of each of its reactors. These 'flow' values are transferred to `BioGasData$production` during object creation with `from_AMPTSV2_report()`. However, they also have `NA`'s as the CFS fails to calculate a flow if the recorded exhaust gas volume of a fermentation does not increase anymore. If the volume does not increase, the 'production', e.g. the difference between current and previous volume is zero. Thus, `NA`'s in `BioGasData$production` are be corrected as follows: ```{r correct-production} # correct NA's in BioGasData$production altBGF$BioGasData$production[grep(T,is.na(altBGF$BioGasData$production))] <- 0 ``` ### Step 4: net gas calculation Now, `BioGasData$product` and `BioGasData$production` values were transferred from the CFS-generated report to the `BGF`. Next, `BioGasData$netGas` is calculated by calling `netGas()`. ```{r netGas} # calculate the netGas altBGF <- netGas(altBGF) ``` This function first calculates a mean value of all `BioGasData$product` values from fermentations marked as 'Blank' in `metaData$Blank` at each `BioGasData$time`. The resulting vector contains a 'Blank' gas volume for each time. This vector is multiplied with a fermentation specific factor and afterwards subtracted from each corresponding `BioGasData$product` value of every fermentation. This results in a fermentation specific net gas volume - a exhaust gas volume corrected by the amount of gas produced from the 'Blank' share in each reactor. The fermentation specific factor is calculated based on a mass balance calculated from values stored at ` `r paste0("metaData$",names(altBGF$metaData)[7])` `. Every `BGF` created from a CFS-generated report has a ` `r paste0("metaData$",names(altBGF$metaData)[7])` `. The net gas is calculated by the following equation: ### Step 5: calculate relative production The `BioGasData$rel_production` is calculated by `relative_production()`: ```{r rel_production} # calculate the relative production altBGF <- relative_production(altBGF) ``` The value stored at `BioGasData$rel_production` resembles the amount of `BioGasData$product` of the final value of `BioGasData$product` at a `BioGasData$time`. It is calculated per fermentation and will sum to 100\% in each. ### Step 6: yield calculation and summary Finally, `BioGasData$yield` is calculated via `calc_yield()`: ```{r calc_yield} # calculate yield altBGF <- calc_yield(altBGF) ``` And a yield summary in the `metaData`-layer at the final `BioGasData$time` is generated by `summarize_yield()`: ```{r summarize_yield,warning=FALSE} # create a yield summary altBGF <- summarize_yield(altBGF) # inspect the final BGF altBGF ``` Besides its `ExpParam$name`, this `BGF` is exactly the same as the one created via `from_AMPTSV2_report()`. The general advice, however, is to use `from_AMPTSV2_report()` when creating a `BGF` from CFS-generated reports, instead of executing the six step workflow by hand.