pds

pds

PDS Tools, to work with Planetary Data System datasets.

Functions

Name	Description
clear_index_cache	Drop cached index frames held in memory by :func:`get_index`.
get_index	Retrieve a specific index file .
missing_pids	Return the subset of `pids` that are NOT present in `df`.
read_pids_file	Read PIDs from a file or stdin, with CSV column resolution.
resolve_pids	Map each requested PID to the full PRODUCT_IDs it resolves to.

clear_index_cache

pds.clear_index_cache(dotted_index_key=None)

Drop cached index frames held in memory by :func:get_index.

get_index keeps the last-loaded frame for each index so repeated lookups in one process (e.g. a batch plp fetch or a notebook session) don’t re-read the parquet each time. Call this to force the next get_index to reload from disk — useful after an out-of-band parquet change, or in tests.

Parameters

Name	Type	Description	Default
dotted_index_key	str	Clear only this index. `None` (default) clears every cached index.	`None`

[source]

get_index

pds.get_index(
    dotted_index_key,
    allow_refresh=False,
    force_refresh=False,
    rebuild_parquet=False,
    force_config_update=False,
    *,
    pids=None,
    columns=None,
    prefix=False,
)

Retrieve a specific index file .

A check is made for possible updates to the index file once per day.

Parameters

Name	Type	Description	Default
dotted_index_key	str	Main identifier for the index to retrieve. Example: ‘mro.ctx.edr’	required
allow_refresh	bool	If True, download the latest version if an update is available.	`False`
force_refresh	bool	Download the latest version unconditionally.	`False`
rebuild_parquet	bool	If True, rebuild the parquet file from existing downloaded files only.	`False`
force_config_update	bool	If True, force update of local URL config file from remote source.	`False`
pids	(Iterable[str], keyword - only)	Product IDs to filter the returned DataFrame to. When provided, rows are matched by exact string comparison against the index’s product-id column (resolved via :func:`pid_column`). PIDs that don’t match any row are silently dropped from the result; use :func:`missing_pids` to find them.	`None`
prefix	(bool, keyword - only)	When True, any `pids` entry that has no exact match but is a leading prefix of one or more PRODUCT_IDs expands to all of those rows (see :func:`resolve_pids`). Exact matches are unaffected.	`False`
columns	(Iterable[str], keyword - only)	Column names to project the returned DataFrame to, in the order given. Exact (case-sensitive) match against the parquet’s column set; unknown names raise :class:`KeyError` listing the available columns. `None` (default) keeps every column.	`None`

Returns

Name	Type	Description
	pandas.DataFrame	DataFrame containing the index data, read from the local parquet file. When `pids` is given, only rows matching those PIDs are returned; when `columns` is given, only those columns are returned.

[source]

missing_pids

pds.missing_pids(df, dotted_index_key, pids)

Return the subset of pids that are NOT present in df.

Useful for batch-PID workflows to surface “which IDs did the index not know about?” alongside the filtered DataFrame returned by get_index(key, pids=...).

Parameters

Name	Type	Description	Default
df	pandas.DataFrame	Index DataFrame (filtered or full — only the configured product-id column needs to be present).	required
dotted_index_key	str	Index key to resolve the product-id column name via :func:`pid_column`.	required
pids	Iterable[str]	PIDs to check.	required

Returns

Name	Type	Description
	list[str]	PIDs in input order that are absent from `df`’s product-id column. Order is preserved; duplicates in the input are NOT deduplicated (the caller decides whether that matters).

[source]

read_pids_file

pds.read_pids_file(source, *, index_key=None, pid_key=None, suffix=None)

Read PIDs from a file or stdin, with CSV column resolution.

Dispatch rule (first match wins):

pid_key is given → CSV mode regardless of source.
File with .csv, .tsv or .tab extension → CSV mode.
Stdin ("-") whose first non-blank line contains a comma or a tab → CSV mode (small heuristic so head file.csv | plp fetch ... Just Works without an explicit flag).
Anything else → plain text; one PID per line, blanks and #-prefixed comments stripped.

In CSV mode the delimiter is auto-detected from the header line: a tab in the first line selects TSV, otherwise comma. This makes tab-separated exports (a common spreadsheet “download as TSV” output) parse into real columns instead of collapsing into one. The product-id column is then determined by pid_key (explicit override) or :func:pid_column (auto-detect via the catalog’s IndexConfig registry, using index_key). Raises :class:ValueError listing the columns when neither resolves — so the caller can pass the right pid_key.

The comma/tab sniff is intentionally tiny. PDS product IDs don’t contain commas or tabs, so plain-text input is reliably distinguishable from tabular input at the first-line level. The cost of being wrong is a clear ValueError pointing the user at --pid-key.

Designed to back the --pids-from CLI option in plp fetch / plp indexes select so users can feed a saved CSV (e.g. the output of plp indexes select --format csv) without pre-extracting a one-PID-per-line file.

Parameters

Name	Type	Description	Default
source	Path or str	File path, or `"-"` for stdin.	required
index_key	str	Dotted index key used to auto-detect the PID column in CSVs via the catalog’s IndexConfig registry. Ignored when `pid_key` is given. Accepts either an index key (`cassini.iss.index`) or a catalog product key (`cassini.iss.edr_sat`) — `pid_column` looks up both forms.	`None`
pid_key	str	Explicit column name for CSV input; wins over auto-detection. Must be an existing column in the CSV. When set, also forces CSV parsing on stdin / non-csv-extension paths — the flag is the user’s declaration that the input is tabular.	`None`
suffix	str	Appended to each PID after reading. Convenient for files that carry observation-level identifiers when the downstream caller actually needs a more specific product (e.g. HiRISE obsids `PSP_003092_0985` + `"_RED"` → `PSP_003092_0985_RED`). Applied after CSV column extraction; the underlying values are not mutated.	`None`

Returns

Name	Type	Description
	list[str]	PIDs in file order. No deduplication. Empty values from the CSV column are NOT filtered — that’s the caller’s responsibility.

Raises

Name	Type	Description
	ValueError	For `.csv` input when neither `pid_key` nor `index_key` resolves to an existing column.
	KeyError	When `pid_key` is given but not present in the CSV columns.

[source]

resolve_pids

pds.resolve_pids(dotted_index_key, pids, df, *, prefix=False)

Map each requested PID to the full PRODUCT_IDs it resolves to.

For each input PID, in order:

if it exactly matches a value in the index’s product-id column (resolved via :func:pid_column), it maps to [pid];
else if prefix is True and it is a leading prefix (str.startswith) of one or more PRODUCT_IDs, it maps to all of those, sorted;
else it maps to [] (the PID is missing).

An exact match always wins over prefix expansion. This is the generic “short product ID” mechanism: a HiRISE obsid handed to a per-CCD index expands to every CCD product, a CTX orbit prefix expands to that orbit’s products, etc. — with no instrument-specific logic.

Returns

Name	Type	Description
	dict[str, list[str]]	Mapping from input PID to its resolved PRODUCT_IDs. Duplicate input PIDs collapse to a single key.

[source]