# Vignette 3: Accessing Interim Objects from Cluster Estimation

# Overview

The `find_clusters()` function is the main entry point for this package, and under typical conditions, the function either returns a message indicating that no clusters were found, or returns an object of class “clusters”, which is a two element list containing 1) the clusters found and 2) information about the locations in each of those clusters. However, callers can pass `return_interim=True` to obtain the interim or intermediate objects that are estimated under the hood when `find_clusters()` is called.

## Functions called by `find_clusters()`

`find_clusters()` is a wrapper function around a number of other package functions, and it calls the below functions, in order. When the user passes `interim_results=TRUE` the outputs of each of these functions are stored separately within an expanded output list.

Table: Functions called by `find_clusters()`
Function | Output |
--- | --- | ---
`generate_case_grids()` : this function take the case counts and the detect date (see `?generate_case_grids` for more options) and separates out the counts by baseline and test date interval, and estimates cumulative sums of cases by location. | An object of class `CaseGrids`, which is a list of items: | - baseline_counts_by_location: a dataframe of counts by location during the baseline interval | | - case_grid: a frame of cases during the test period, with reverse cumulative counts within location, by date | | - case_grid_totals_by_date: a frame of the reverse cumulative sum of counts over all locations, by date | | - test_cases: a frame of cases during the test period, by and location | | - detect_date: the detect date passed to this function | | - baseline_total: the total number of cases over all locations and dates |
`gen_nearby_case_info()` : This function takes an object of class `CaseGrids` distance matrix, and a distance threshold, and estimates, for each location, the estimated cumulative sum of cases in nearby locations. Two frames for these estimates are provided: one for test period and one for baseline | An object of class `NearbyClusterGrids`, which is a list of two data frames, including: “baseline”, holding the nearby information for baseline counts “test”, holding the nearby information for the test interval |
`generate_observed_expected()`: This function takes objects of classes `NearbyClusterGrids` and `CaseGrids` and derives the observed and expected calculation for all target locations | An object of class `ObservedExpectedGrid` , which is a frame that contains the observed and expected value for all target locations. |
`add_spline_threshold()`: this function ingests an object of class `ObservedExpectedGrid` and used either an internal or user provided spline lookup table to filter the `ObservedExpectedGrid` by those that meet the spline classification threshold | An object of class `ClusterAlertTable` which is a dataframe of all target candidate clusters that are reduced to those meeting the spline threshold |
`compress_clusters_fast()`: This function is a faster version of the deprecated (but accessible `compress_clusters()`. Both this function and the deprecated version take the `ClusterAlertTable` from the previous step and find only the most significant, non-overlapping clusters from the large table of candidates | An object of class `clusters`, which is a named list of two items: The first item is a data frame named “cluster_alert_table”, which holds one row per cluster, including a column “id” which enables linking to the list in the second item (see below) The second item is a data frame named “clust_loc_list” which is a list of cluster centers, indexed by values “id” in the first item (see above) and containing a vector of locations that are included in that cluster. |
`add_location_counts()`: this function ingests an object of type `clusters` and obtains location counts and sums for all locations within the clusters | Returns an object of type `clusters` which has a similar structure as above, but adds total number of locations to the first item (i.e. the table of identified clusters) and converts the second item from a list to a data frame that contains the sums for all locations within the clusters. |

See help via `?<function>` for further details on each of these above functions, including optional and required parameters and examples of how to use them.

## Example Usage:

Note: we set `return_interim` to `TRUE`

    result <- find_clusters(
      cases = example_count_data,
      distance_matrix = county_distance_matrix("OH")[["distance_matrix"]],
      detect_date = example_count_data[, max(date)],
      distance_limit = 50,
      return_interim = TRUE
    )

    # objects included in the output
    for (n in names(result)) {
      cat(
        "Name: ",
        n,
        "\nClass: ",
        paste(class(result[[n]]), collapse = " "),
        "\n\n",
        sep = ""
      )
    }
    #> Name: case_grid_info
    #> Class: list CaseGrids
    #>
    #> Name: nearby_case_info
    #> Class: list NearbyClusterGrids
    #>
    #> Name: obs_expected_frame
    #> Class: data.table data.frame ObservedExpectedGrid
    #>
    #> Name: obs_expected_frame_with_spline
    #> Class: data.table data.frame ClusterAlertTable
    #>
    #> Name: compressed_clusters
    #> Class: list clusters
    #>
    #> Name: result
    #> Class: list clusters