In this document we will describe the usage of the multiverse code block. The multiverse code block is an alternative to the R code block which will parse any code run in it directly in the multiverse, and reduces the need for auxiliary functions such as (inside). In addition, the default analysis in the multiverse will be executed in the environment in which the multiverse has been declared, which would usually be the global environment. In that case the results of the default analysis can be inspected directly in an R code block, similar to what an user would do if they were running a single universe analysis.
We demonstrate how the two methods are equivalent, and result in the same multiverse.
In this example, we will use the data which compares different modalities for physical visualizations. See (frequentist-multiverse-analysis) for more details.
As with any multiverse analysis, we first need to define the
multiverse object. This step is the same for both methods. We will
define two separate objects for illustrating the two different ways of
adding code to the multiverse for analysis. In the multiverse analysis
in this vignette, we will perform a log transformation on the duration
variable of the dataset using both the inside
function as
well as the multiverse code block
The idea of the inside function is to allow us to write code to be executed within the multiverse and not directly in R, thus allowing us to make use of a flexible syntax for declaring the different possible analysis combinations within the multiverse. The inside function takes two arguments:
{
The expression passed into the multiverse is not executed directly,
allowing us to parse and expand the expression provided by the user has
declared using our branch
and parameters
, into
their corresponding analysis combinations.
Although the inside function is the only way to add code to the multiverse in an RScript, RMarkdown notebooks allow us the opportunity to use different language engines (not just limited to R). This flexibility also provides an opportunity to write multiverse code directly into a code block, instead of using auxillary functions.
The language associated with a code block is provided by the first
argument: ```{r}
Here the first argument is r
hence the code in the associated block will be executed in R. To convert
a code block to execute in multiverse, change the first argument to
multiverse
. Thus the code block would be:
```{multiverse}
To execute a code block in multiverse, the user needs to provide two
additional arguments: label
and inside
.
The label is a unique identifier for the code block, and each code
block in the same document must have a different label. However, the
label argument does not need to be explicitly specified (as is the case
with an R code block). Therefore,
```{multiverse, label=default-m-1 ...}
,
```{multiverse, default-m-1 ...}
and
```{multiverse default-m-1 ...}
are all equivalent.
The inside argument takes in only multiverse objects, and is used to
indicate the multiverse object which will the code inside the code block
will be associated with. Thus declaring a multiverse code block would
be: ```{multiverse, default-m-1, inside = M}
We provide the ability to declare multiverse code block as an AddIn in RStudio. Users can click on AddIns toolbar menu in RStudio (see the image below). This would create a multiverse code block at the location of the cursor in the document.
Alternately, users can insert a multiverse We also allow users to create a keyboard shortcut to declare a multiverse code block inside a RMarkdown document. This can be done through the following steps:
If you are experiencing issues creating a keyboard shortcut for code
blocks, or if you are experiencing a situation where the shortcut needs
to be re-created everytime you open a new RStudio session, please refer
to the Debugging Keyboard Shortcuts
section at the end of
this document.
The declaration of the chunk below is (this gets hidden when we knit
the document):
```{multiverse default-m-1, inside = M_block}
```{multiverse default-m-1, inside = M_block}
data_transform <- branch(data_transform,
"log-transformed" ~ log,
"untransformed" ~ identity
)
duration <- do.call(data_transform, list(userlogs$duration))
```
The result of this code block will be identical to the result using
inside()
. We first compare whether the multiverse table is
generated properly for both the multiverse objects.
## # A tibble: 2 × 6
## .universe data_transform .parameter_assignment .code .results .errors
## <int> <chr> <list> <list> <list> <list>
## 1 1 log-transformed <named list [1]> <named list> <env> <lgl>
## 2 2 untransformed <named list [1]> <named list> <env> <lgl>
## # A tibble: 2 × 6
## .universe data_transform .parameter_assignment .code .results .errors
## <int> <chr> <list> <list> <list> <list>
## 1 1 log-transformed <named list [1]> <named list> <env> <lgl>
## 2 2 untransformed <named list [1]> <named list> <env> <lgl>
As you can see above, both the methods yield the same multiverse
table. Next, we inspect the .code
column of the multiverse
object. This column contains the code used to generate each analysis
combination in the multiverse. The only differences here arise from how
the expressions are stored. The multiverse code block creates a named
list, whereas the inside function creates a unnamed list for each row of
this column.
Below is the output from the first multiverse object
(M_inside
), which uses the inside function:
## [[1]]
## [[1]]$`1`
## {
## data_transform <- log
## duration <- do.call(data_transform, list(userlogs$duration))
## }
##
##
## [[2]]
## [[2]]$`1`
## {
## data_transform <- identity
## duration <- do.call(data_transform, list(userlogs$duration))
## }
This is the output from the second multiverse object
(M_block
), which uses the multiverse code block:
## [[1]]
## [[1]]$`default-m-1`
## {
## data_transform <- log
## duration <- do.call(data_transform, list(userlogs$duration))
## }
##
##
## [[2]]
## [[2]]$`default-m-1`
## {
## data_transform <- identity
## duration <- do.call(data_transform, list(userlogs$duration))
## }
During interactive use with RStudio, multiverse code blocks will work very similarly to a R code block. It will only execute the default analysis (i.e. the analysis path obtained by combining the first option of each parameter) in the global environment. Any output that is generated will have the formatting that output of R code blocks do.
When a document is knit, currently all analysis paths are executed but the document only shows the analysis path corresponding to the default analysis. We are currently developing interactive features which will allow the author to implement interactivity into the rendered HTML document; this will also allow readers to interact with the analysis and investigate the robustness of the implemented analysis themselves.
Currently we do not support knitting of RMarkdown documents with multiverse code blocks to PDF documents.
One issue that I have encountered in the past, is that I had to
create the keyboard shortcut everytime I opened a new RStudio session.
As discussed here, RStudio keybindings are saves as JSON files in the
directory ~/.R/rstudio/keybindings/
, but this directory was
missing for me.
Check if the directory exists for you: 1. Open Terminal 2. Enter:
cd ~/.config/rstudio/keybindings
If it returns an error such as
cd: no such file or directory
, this means that you do not
have the directory. The fix is to create this directory which can be
done through the following steps:
mkdir ~/.config/rstudio/keybindings/
If it returns “Permission denied” you might have to run the command
as the superuser in the Terminal:
sudo mkdir ~/.config/rstudio/keybindings/
This will request
you to enter your login password.