Command Line Interface

Installing Datacube-explorer installs the cubedash-gen script, a Click command line interface, in your virtualenv. Executed from the terminal, this script gives access to built-in, extension, and application-defined commands. The --help option will give more information about any commands and options.

Summary generation

Initialise and create product summaries:

cubedash-gen --init --all

cubedash-gen

A tool to generate and update Explorer’s caches of datacube data.

Explorer’s view of ODC data is too expensive to calculate on every page load (or API call), so Explorer uses its own tables to calculate this summary information ahead-of-time.

The cubedash-gen command creates these summaries in a schema called cubedash, separate from datacube’s own schema.

This can be re-run each time changes are made to an ODC product to show the updates in Explorer.

The first run will be slow as it will scan all data, but later runs will be faster as it scans only modified data. (unless –force-refresh is specified)

—

Datacube config

To choose which datacube to point to, it takes identical datacube config (-C) and environment (-E) options as the datacube command, and reads identical datacube config files and environment variables.

ie. It will use the datacube that is shown by running the command datacube system check

See datacube’s own docs for this configuration handling.

—

Examples

Create Explorer’s schemas, then update (or create) all product summaries:

cubedash-gen –init –all

Update (or create) all product summaries:

cubedash-gen –all

Recreate all information for two products:

cubedash-gen –force-refresh ls8_nbart_scene ls8_level1_scene

Drop all of Explorer’s additions to the database:

cubedash-gen –drop

cubedash-gen [OPTIONS] [PRODUCT_NAMES]...

Options

-E, --env <env>

The ODC environment to use. Defaults to ‘default’.

--all

Refresh all products in the datacube, rather than the specified list.

--epsg <epsg_code>

The equal-area epsg code to use internally for grouping spatial data. (default: 6933)

-v, --verbose

Enable all log messages, instead of just errors.

Logging goes to stdout unless –event-log-file is specified.

Logging is coloured plain-text if going to a tty, and jsonl format otherwise.

Use twice to enable debug logging too.

-j, --jobs <jobs>

Number of concurrent worker subprocesses to use (default: 3)

This should match how many io-and-cpu-heavy queries your DB would like to handle concurrently.

-tz, --timezone <timezone>

Timezone for the geolocation

-l, --event-log-file <event_log_file>

Output log messages to file, in jsonl format

--refresh-stats, --no-refresh-stats

Refresh general statistics tables that cover all products (default: true)

This can be slow, and only needs to be done once (at the end) if calling cubedash-gen repeatedly

--minimum-scan-window <minimum_scan_window>

Always rescan this window of time for dataset changes, even if the refresh tool has run more recently.

This is useful if you have something that doesn’t make rows visible immediately, such as a sync service from another location.

Example values: ‘24h’ or ‘1h30m’

--force-refresh, --no-force-refresh

Force all time periods to be regenerated, rather than just applying updates to existing ones.

(default: false)

--reset-incremental-position, --no-reset-incremental-position

Reset the incremental-update position to the most recent known dataset in Explorer’s tables.

This is useful in development if copying an existing Explorer schema to another database, as the incremental positions according to that new database will be wrong.

(default: false)

--recreate-dataset-extents, --append-dataset-extents

Rebuild Explorer’s existing dataset extents even if they don’t seem to be updated. (default: false)

This is useful if you’ve patched datasets or products in-place with new geometry or regions.

--force-concurrently

Refresh materialised views concurrently in Postgres. (default: false)

This will avoid taking any locks when updating statistics, but has some caveats, see https://www.postgresql.org/docs/10/sql-refreshmaterializedview.html

--init-database, --init, --no-init-database

Check the Explorer schema and create it (or apply updates to it) if needed, before doing anything else.

--drop-database, --drop

Drop all of Explorer’s database schema+caches and exit.

Arguments

PRODUCT_NAMES

Optional argument(s)

Run application

A simple cubedash-run command is available to run Explorer locally:

$ cubedash-run
    * Running on http://localhost:8080/ (Press CTRL+C to quit)

cubedash-run

A simple way to run Explorer locally.

It will use your default datacube settings

(overridable with datacube environment variables, such as DATACUBE_ENVIRONMENT)

cubedash-run [OPTIONS]

Options

--debug

Enable debug mode

--version

-v, --verbose

Enable all log messages, instead of just errors.

Logging goes to stdout unless –event-log-file is specified.

Logging is coloured plain-text if going to a tty, and jsonl format otherwise.

Use twice to enable debug logging too.

-l, --event-log-file <event_log_file>

Output jsonl logs to file

-h, --hostname <hostname>

-p, --port <port>

-w, --workers <workers>

(see cubedash-run –help for list of options)

But Explorer can be run using any typical python wsgi server, for example gunicorn:

pip install gunicorn
gunicorn -b '127.0.0.1:8080' -w 4 cubedash:create_app()

Products will begin appearing one-by-one as the summaries are generated in the background. If impatient, you can manually navigate to a product using /<product_name. (Eg /ls5_nbar_albers)