---
title: "Auditing One Collection: A Quick-Start Guide"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Auditing One Collection: A Quick-Start Guide}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 6, fig.height = 4)

online <- tryCatch({
  con <- url("https://cmr.earthdata.nasa.gov/search/collections.json?page_size=1",
             open = "rb")
  close(con)
  TRUE
}, error = function(e) FALSE)

knitr::opts_chunk$set(eval = online)
```

```{r offline-note, echo = FALSE, eval = !online, results = "asis"}
cat(
  "CMR (`cmr.earthdata.nasa.gov`) wasn't reachable when this vignette was ",
  "built, so the live walkthrough below was skipped. Re-build with a ",
  "network connection to see it run.", sep = ""
)
```

This is the narrow path: you have one collection (or a rough idea of
one) and you want to know when its files actually change. If you're
curious about the broader exploration that shaped this package's other
functions -- collection discovery, S3 region gotchas, provider-specific
metadata quirks -- see `vignette("cmr-landscape")` instead. This
article sticks to the five steps that matter for a single collection.

```{r libs, message = FALSE}
library(cmrlifecycle)
```

## Step 1: find your collection

If you already have a `short_name`, skip to Step 2. Otherwise,
`cmr_find_collections()` does a free-text search across CMR's
collection metadata (title, abstract, science keywords, platforms,
instruments, ...):

```{r find}
cmr_find_collections(keyword = "sea surface temperature", provider = "POCLOUD",
                      page_size = 5, granule_count = FALSE) |>
  dplyr::select(short_name, entry_title, provider, version)
```

This example uses this vignette's running example throughout:
`MUR-JPL-L4-GLOB-v4.1`, GHRSST's daily global sea-surface-temperature
analysis.

## Step 2: is it actually alive, and what does it cover?

Before auditing anything, two quick sanity checks. First, the
collection's own declared metadata:

```{r declared}
cmr_find_collections(short_name = "MUR-JPL-L4-GLOB-v4.1") |>
  dplyr::select(temporal_start, temporal_end, ends_at_present, granule_count)
```

`granule_count > 0` confirms this collection actually has UMM-G
granules to audit at all -- some CMR "collections" don't (see
`vignette("cmr-landscape")`'s AADC example). `temporal_start` is
usually reliable; `ends_at_present`/`temporal_end` are declared
metadata that vary a lot in reliability by provider, so don't lean on
them alone --

```{r liveness}
cmr_collection_liveness("MUR-JPL-L4-GLOB-v4.1")
```

`cmr_collection_liveness()` checks real granule activity in a recent
window directly, which is the more trustworthy "is this still going"
signal. `recent_granule_count` capped at your `page_size` (default
2000) just means "at least that many" -- raise `lookback_days` for
low-cadence collections (annual composites, etc.) that might show
`is_live = FALSE` over a short window despite being fine.

## Step 3: run the audit

`cmr_audit_revisions()` sweeps a date range in 3-month chunks and
returns one row per granule, with the full revision/production
lifecycle. This example covers a little over a year to keep the
vignette quick to build; drop `from` further back for a fuller
picture (MUR itself goes back to 2002):

```{r audit}
rev <- cmr_audit_revisions("MUR-JPL-L4-GLOB-v4.1", from = Sys.Date() - 400)
rev |>
  dplyr::select(gran_date, n_rev, insert_date, update_date, production,
                prod_lag) |>
  head()
```

The column dictionary, briefly (full detail in the
[README](https://github.com/hypertidy/cmrlifecycle#what-cmr_audit_revisions-gives-you)):

| column | what it tells you |
|---|---|
| `n_rev` | how many times the CMR *metadata record* was touched -- often bulk provider events, **not** evidence the file changed |
| `insert_date`/`update_date` | UMM `ProviderDates` -- first publish / latest metadata touch |
| `production` | `DataGranule.ProductionDateTime` -- when the science file itself was generated. **This is the churn-immune signal** |
| `*_lag` | each date above minus `gran_date`, in days |

```{r nrev-range}
range(rev$n_rev)
```

If that range is wide (as it typically is for actively-maintained
GHRSST products), don't read it as "this file was regenerated that
many times" -- cross-check against `prod_lag` instead, which is immune
to metadata churn.

## Step 4: look at it

```{r plot, fig.alt = "Two diagnostic plots: CMR revision count vs granule date, and production lag in days vs granule date"}
plot_cmr_lifecycle(rev)
```

Two things to look for:

- **Revision count vs. granule date** (top plot): a staircase pattern
  (roughly constant counts jumping at specific dates across many
  granules) usually means `n_rev` is tracking a bulk metadata event,
  not per-granule data changes.
- **Production lag vs. granule date** (bottom plot): expect a flat
  band at a stable finalization lag for settled/older dates, with
  smaller lags only at the frontier (recent dates still on a
  near-real-time product, not yet replaced by the finalized file).

## When your collection doesn't play by MUR's rules

Two things assumed above don't hold universally:

**Opaque `GranuleUR`, no embedded date.** `gran_date` defaults to
parsing a leading `YYYYMMDD` out of `granule_ur` -- true for
GHRSST-style products, not universal. Some collections (e.g. MODIS
swath products via LAADS) use an opaque `GranuleUR` like
`"LAADS:9975745835"`. `cmr_audit_revisions()` warns (not errors) if
`gran_date` comes back entirely `NA`; the fix is to derive it from
`temporal_start` instead:

```{r opaque, eval = FALSE}
rev_swath <- cmr_audit_revisions("MOD021KM",
  from = as.Date("2026-08-01"), to = as.Date("2026-08-02"),
  gran_date_fun = function(df) as.Date(df$temporal_start))
```

**Too many granules worldwide.** Swath products can produce hundreds
of granules a day globally. `cmr_find_granules()`/`cmr_audit_revisions()`
take an optional `bounding_box` (`c(west, south, east, north)`) or
`polygon` filter to subset to a region of interest instead:

```{r spatial, eval = FALSE}
cmr_find_granules("MOD021KM", as.Date("2026-08-01"), as.Date("2026-08-01"),
                   bounding_box = c(140, -45, 155, -35))
```

## If your collection is cloud-hosted

Check `region`/`s3_credentials_endpoint` before relying on `s3_url` for
compute placement -- `s3://` links are only cheap from matching-region
compute, and require an Earthdata Login credential exchange regardless
of region:

```{r s3check}
cmr_find_collections(short_name = "MUR-JPL-L4-GLOB-v4.1") |>
  dplyr::select(region, s3_credentials_endpoint)
```

`vignette("cmr-landscape")` covers why this matters in more detail.

## Cheat sheet

```r
library(cmrlifecycle)

# 1. find it (skip if you know short_name)
cmr_find_collections(keyword = "...")

# 2. sanity check
cmr_collection_liveness("your-short-name")
cmr_find_collections(short_name = "your-short-name")

# 3. audit
rev <- cmr_audit_revisions("your-short-name", from = as.Date("2024-01-01"))

# 4. look
plot_cmr_lifecycle(rev)

# only if needed:
# - opaque GranuleUR:  gran_date_fun = function(df) as.Date(df$temporal_start)
# - swath volume:      bounding_box = c(west, south, east, north)
# - cloud compute:     check region / s3_credentials_endpoint first
```