---
title: "Loading and automatic trimming of density profiles"
author: "Luka Krajnc"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Loading and automatic trimming of density profiles}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# Introduction
The use of resistance drilling for density assessment is becoming more and more used across different disciplines. This R package provides a way to do the density analysis in R by loading the measurements, trimming them automatically or by hand. It is also possible to remove trends from profiles and measure ring widths. Please note that this package was written with a focus on forestry, where density profiles are usually obtained on living standing trees. Other uses are also possible, wherever you are using density profiles.

Let's load the library to see what it can do:
```{r setup}
library(densitr)
```


# Loading resistance drilling density profiles
In the current version of **densitr**, only files (*.dpa) created by Rinntech Resistograph® devices are supported.  It was tested to work on files produced by R650-RC Resistograph®. Other file types are currently not supported, they might be added later on. Contributions and pull requests are always welcome.

Never try to load *.dpa files directly from the Resistograph device into R, always copy them first onto your computer and load them from there. Try at your own risk. This vignette uses some example density profiles recorded using an Rinntech Resistograph® R650-SC, which are included with this package. There is a total of 15 profiles, representing various tree species. Some are drilled bark-to-bark, while others were drilled bark-to-pith because of their dimensions.

Loading a single file is simple:
```{r}
dp <- dpload(dp.file = system.file("extdata", "00010001.dpa", package = "densitr"))
dp
```

The function returns an S3 object with the class of **dp**, which is essentially a list with two items. In the first, `$data`, you will find a data frame with all of the actual profile data. Column *position* of that data frame holds the horizontal position as saved by the device when drilling. Column *amplitude* holds actual density values as recorded by the device. In the second list of an dpa object, `$footer`, you will find additional information about the individual density profile, as recorded by the device.

View dp measurement data:
```{r}
head(dp$data)
```

View dp additional data:
```{r}
head(dp$footer)
```

We can also plot an individual density profile:
```{r, fig.width=15, fig.height=5}
plot(dp)
```

When plotting density profiles, units for both axis are extracted from `$footer`, as is the measurement ID.

Loading several files is also easy, just specify the argument `dpa.directory` with a folder path. This returns a list of density profiles. By default this function works recursively and also looks into all subfolder, set the `recursive = FALSE` avoid that.
```{r, results = 'asis'}
dp.list <- dpload(dp.directory = system.file("extdata", package = "densitr"))
```

Inspect the list by displaying the first two items:
```{r}
head(dp.list, 2)
```

# Trimming
A typical resistance drilling measurement starts with an increase in resistance values in between the measurement start and the immersion of the needle in the wood. Similarly, if the needle exits of the opposite of the wood, there will be a decrease in values due to lack of resistance. Bark-to-bark measurements should be trimmed on both sides, while bark-to-pith only at the beginning.

## Looking under the hood
**densitr** provides two functions that try to automate start and end detection: `dpdetect_s` and `dpdetect_e` by using checkpoint binary segmentation. They are usually not used manually, see section on automatic trimming. Let's find where the first profile in our list actually starts:
```{r}
dpdetect_s(dp.list[[1]])
```

The function return the horizontal position (equivalent to row number), where the start was detected. See function documentation for more information on how it works. The same function can also return a diagnostic plot.
```{r, fig.width=15, fig.height=10}
dpdetect_s(dp.list[[1]], return.plot = TRUE)
```

End detection is also possible on bark-to-bark profiles:
```{r}
dpdetect_e(dp.list[[1]])
```

Diagnostic plot can also be displayed when detecting profile end. If either start or end is not detected, it will also print a warning message.

## Automatic trimming
In order to trim an individual profile after obtaining start and end profiles, you would have to subset the profile using start and end positions. Calling `dptrim` on a density profile will automate the whole process. It will try do detect start and end positions and trim those portions away, returning a profile without the starting or ending portion of the profile.
```{r}
dptrim(dp.list[[1]])
```

`dptrim` is also capable of displaying a diagnostic plot:
```{r, fig.width=15, fig.height=5}
dptrim(dp.list[[1]], return.plot = TRUE)
```

`dptrim` can only be called on individual profiles. In order to run automatic trimming on a list of density profiles use `dptriml`. While this is just a convenience wrapper for `pbapply::pblapply` or `lapply`, it also provides a trimming report at the end:

```{r}
dp.trimmed  <- dptriml(dp.list)
```

If you have `pbapply` installed, you will get a nice progress bar displaying current progress. This is useful when trimming a large number of profiles at once. Note, you can also supply `dptriml` with an additional argument `cl`, which specifies number of cores to run trimming in parallel. This will significantly speed up the whole process. `pbapply` package is also required for this.

The equivalent functions to `dptrim` and `dptriml` to trim the profiles only at the beginning are `dptrim_s` and `dptriml_s`.

## What to do when automatic trimming fails?
Automatic trimming sometimes fails to detect the starting or ending point within the density profile. Failing to detect ending point is usually a consequence of the profile being bark-to-pith instead of bark-to-bark. In those cases (an others) you can trim the measurement by hand. See vignette on Manual profile trimming for more information.