Tutorial

from planetarypy.pds.apps import find_indexes, get_index, find_instruments
find_instruments?
Signature: find_instruments(mission: str) -> list
Docstring: Find existing instruments for a mission.
File:      ~/Dropbox/src/nbplanetary/planetarypy/pds/apps.py
Type:      function
find_instruments("mro")
['hirise', 'ctx']

Now, seeing that the CTX instrument is supported, one could ask what index files (= catalogs) are available, using the dotted code for mission.instrument:

find_indexes("mro.ctx")
['edr']

We only have one index file for EDR data. Let’s get it, using the now 3-segments dot-code (it returns a pandas Dataframe):

index = get_index('mro.ctx.edr', refresh=False)
index.head()
VOLUME_ID FILE_SPECIFICATION_NAME ORIGINAL_PRODUCT_ID PRODUCT_ID IMAGE_TIME INSTRUMENT_ID INSTRUMENT_MODE_ID LINE_SAMPLES LINES SPATIAL_SUMMING ... SUB_SOLAR_LATITUDE SUB_SPACECRAFT_LONGITUDE SUB_SPACECRAFT_LATITUDE SOLAR_DISTANCE SOLAR_LONGITUDE LOCAL_TIME IMAGE_SKEW_ANGLE RATIONALE_DESC DATA_QUALITY_DESC ORBIT_NUMBER
0 MROX_0001 DATA/CRU_000001_9999_XN_99N999W.IMG 4A_04_0001000400 CRU_000001_9999_XN_99N999W 2005-08-30 15:40:21.549 CTX NIFL 5056 1024 1 ... 0.0 0.0 0.0 0.0 278.89 10.16 0.0 Instrument checkout image of space OK -4242
1 MROX_0001 DATA/CRU_000002_9999_XN_99N999W.IMG 4A_04_0001000500 CRU_000002_9999_XN_99N999W 2005-09-08 15:59:45.313 CTX NIFL 5056 15360 1 ... 0.0 0.0 0.0 0.0 284.48 4.6 0.0 Calibration image of the Moon OK -4126
2 MROX_0001 DATA/CRU_000003_9999_XN_99N999W.IMG 4A_04_0001000600 CRU_000003_9999_XN_99N999W 2005-09-08 16:03:37.927 CTX NIFL 5056 2048 1 ... 0.0 0.0 0.0 0.0 284.48 4.66 0.0 Calibration image of Omega Centauri (globular ... OK -4126
3 MROX_0001 DATA/CRU_000004_9999_XN_99N999W.IMG 4A_04_0001000700 CRU_000004_9999_XN_99N999W 2005-09-08 16:08:23.841 CTX NIFL 5056 2048 1 ... 0.0 0.0 0.0 0.0 284.48 4.74 0.0 Calibration image of Omega Centauri (globular ... OK -4126
4 MROX_0001 DATA/CRU_000005_9999_XN_99N999W.IMG 4A_04_0001000800 CRU_000005_9999_XN_99N999W 2005-09-08 16:11:18.649 CTX NIFL 5056 21504 1 ... 0.0 0.0 0.0 0.0 284.48 4.79 0.0 Calibration image of the Moon OK -4126

5 rows × 51 columns

You might have ssen, that planetarypy realized that the existing index on my disk was outdated and proceeded to get an updated one.

The warning is about badly formatted data on the PDS for which planetarypy had to force path names to the correct letter case. This is not something we can influence, but we reported the issue at the PDS, and here, we just let the user know, that we fudged around that issue.

Note that the catalog ascii times have been converted to proper datetime objects, using the utility functions in planetarypy.utils. This means that you can use these times to create proper plots over time in case you are interested in catalog statistics.

index.IMAGE_TIME
0        2005-08-30 15:40:21.549
1        2005-09-08 15:59:45.313
2        2005-09-08 16:03:37.927
3        2005-09-08 16:08:23.841
4        2005-09-08 16:11:18.649
                   ...          
142363   2023-03-01 00:01:37.834
142364   2023-03-01 00:15:17.076
142365   2023-03-01 00:19:04.486
142366   2023-03-01 00:29:24.818
142367   2023-03-01 00:43:08.662
Name: IMAGE_TIME, Length: 142368, dtype: datetime64[ns]

Using standard pandas operations, we can now filte the data down, and identify a data product we are interested in:

index.columns
Index(['VOLUME_ID', 'FILE_SPECIFICATION_NAME', 'ORIGINAL_PRODUCT_ID',
       'PRODUCT_ID', 'IMAGE_TIME', 'INSTRUMENT_ID', 'INSTRUMENT_MODE_ID',
       'LINE_SAMPLES', 'LINES', 'SPATIAL_SUMMING', 'SCALED_PIXEL_WIDTH',
       'PIXEL_ASPECT_RATIO', 'EMISSION_ANGLE', 'INCIDENCE_ANGLE',
       'PHASE_ANGLE', 'CENTER_LONGITUDE', 'CENTER_LATITUDE',
       'UPPER_LEFT_LONGITUDE', 'UPPER_LEFT_LATITUDE', 'UPPER_RIGHT_LONGITUDE',
       'UPPER_RIGHT_LATITUDE', 'LOWER_LEFT_LONGITUDE', 'LOWER_LEFT_LATITUDE',
       'LOWER_RIGHT_LONGITUDE', 'LOWER_RIGHT_LATITUDE', 'MISSION_PHASE_NAME',
       'TARGET_NAME', 'SPACECRAFT_CLOCK_START_COUNT',
       'FOCAL_PLANE_TEMPERATURE', 'LINE_EXPOSURE_DURATION', 'OFFSET_MODE_ID',
       'SAMPLE_FIRST_PIXEL', 'SCALED_IMAGE_WIDTH', 'SCALED_IMAGE_HEIGHT',
       'SPACECRAFT_ALTITUDE', 'TARGET_CENTER_DISTANCE', 'SLANT_DISTANCE',
       'USAGE_NOTE', 'NORTH_AZIMUTH', 'SUB_SOLAR_AZIMUTH',
       'SUB_SOLAR_LONGITUDE', 'SUB_SOLAR_LATITUDE', 'SUB_SPACECRAFT_LONGITUDE',
       'SUB_SPACECRAFT_LATITUDE', 'SOLAR_DISTANCE', 'SOLAR_LONGITUDE',
       'LOCAL_TIME', 'IMAGE_SKEW_ANGLE', 'RATIONALE_DESC', 'DATA_QUALITY_DESC',
       'ORBIT_NUMBER'],
      dtype='object')
index.EMISSION_ANGLE.describe()
count     142368.0
mean      4.396005
std      18.999974
min            0.0
25%            0.1
50%           1.56
75%           5.94
max          999.9
Name: EMISSION_ANGLE, dtype: Float64
index.LOCAL_TIME.describe()
count     142368.0
mean     15.055919
std       1.427996
min            0.0
25%          14.65
50%          15.12
75%          15.53
max          23.99
Name: LOCAL_TIME, dtype: Float64
angle_filter = index.EMISSION_ANGLE < 1.0

Let’s also calculate a mean latitude and longitude instead of the image corners, for rough location filtering. As a habit, we recommend to assign lower case letters to column names that we add on top of the PDS ones:

index = index.assign(
    lat_mean=(
        index.UPPER_LEFT_LATITUDE
        + index.UPPER_RIGHT_LATITUDE
        + index.LOWER_LEFT_LATITUDE
        + index.LOWER_RIGHT_LATITUDE) /
    4)
index = index.assign(
    lon_mean=(
        index.UPPER_LEFT_LONGITUDE
        + index.UPPER_RIGHT_LONGITUDE
        + index.LOWER_LEFT_LONGITUDE
        + index.LOWER_RIGHT_LONGITUDE) /
    4)
index.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 142368 entries, 0 to 142367
Data columns (total 53 columns):
 #   Column                        Non-Null Count   Dtype         
---  ------                        --------------   -----         
 0   VOLUME_ID                     142368 non-null  string        
 1   FILE_SPECIFICATION_NAME       142368 non-null  string        
 2   ORIGINAL_PRODUCT_ID           142368 non-null  string        
 3   PRODUCT_ID                    142368 non-null  string        
 4   IMAGE_TIME                    142368 non-null  datetime64[ns]
 5   INSTRUMENT_ID                 142368 non-null  string        
 6   INSTRUMENT_MODE_ID            142368 non-null  string        
 7   LINE_SAMPLES                  142368 non-null  Int64         
 8   LINES                         142368 non-null  Int64         
 9   SPATIAL_SUMMING               142368 non-null  Int64         
 10  SCALED_PIXEL_WIDTH            142368 non-null  Float64       
 11  PIXEL_ASPECT_RATIO            142368 non-null  Float64       
 12  EMISSION_ANGLE                142368 non-null  Float64       
 13  INCIDENCE_ANGLE               142368 non-null  Float64       
 14  PHASE_ANGLE                   142368 non-null  Float64       
 15  CENTER_LONGITUDE              142368 non-null  Float64       
 16  CENTER_LATITUDE               142368 non-null  Float64       
 17  UPPER_LEFT_LONGITUDE          142368 non-null  Float64       
 18  UPPER_LEFT_LATITUDE           142368 non-null  Float64       
 19  UPPER_RIGHT_LONGITUDE         142368 non-null  Float64       
 20  UPPER_RIGHT_LATITUDE          142368 non-null  Float64       
 21  LOWER_LEFT_LONGITUDE          142368 non-null  Float64       
 22  LOWER_LEFT_LATITUDE           142368 non-null  Float64       
 23  LOWER_RIGHT_LONGITUDE         142368 non-null  Float64       
 24  LOWER_RIGHT_LATITUDE          142368 non-null  Float64       
 25  MISSION_PHASE_NAME            142368 non-null  string        
 26  TARGET_NAME                   142368 non-null  string        
 27  SPACECRAFT_CLOCK_START_COUNT  142368 non-null  string        
 28  FOCAL_PLANE_TEMPERATURE       142368 non-null  Float64       
 29  LINE_EXPOSURE_DURATION        142368 non-null  Float64       
 30  OFFSET_MODE_ID                142368 non-null  string        
 31  SAMPLE_FIRST_PIXEL            142368 non-null  Int64         
 32  SCALED_IMAGE_WIDTH            142368 non-null  Float64       
 33  SCALED_IMAGE_HEIGHT           142368 non-null  Float64       
 34  SPACECRAFT_ALTITUDE           142368 non-null  Float64       
 35  TARGET_CENTER_DISTANCE        142368 non-null  Float64       
 36  SLANT_DISTANCE                142368 non-null  Float64       
 37  USAGE_NOTE                    142368 non-null  string        
 38  NORTH_AZIMUTH                 142368 non-null  Float64       
 39  SUB_SOLAR_AZIMUTH             142368 non-null  Float64       
 40  SUB_SOLAR_LONGITUDE           142368 non-null  Float64       
 41  SUB_SOLAR_LATITUDE            142368 non-null  Float64       
 42  SUB_SPACECRAFT_LONGITUDE      142368 non-null  Float64       
 43  SUB_SPACECRAFT_LATITUDE       142368 non-null  Float64       
 44  SOLAR_DISTANCE                142368 non-null  Float64       
 45  SOLAR_LONGITUDE               142368 non-null  Float64       
 46  LOCAL_TIME                    142368 non-null  Float64       
 47  IMAGE_SKEW_ANGLE              142368 non-null  Float64       
 48  RATIONALE_DESC                142368 non-null  string        
 49  DATA_QUALITY_DESC             142368 non-null  string        
 50  ORBIT_NUMBER                  142368 non-null  Int64         
 51  lat_mean                      142368 non-null  Float64       
 52  lon_mean                      142368 non-null  Float64       
dtypes: Float64(34), Int64(5), datetime64[ns](1), string(13)
memory usage: 62.9 MB
lat_filter = index.lat_mean.between(-80,-60)
lon_filter = index.lon_mean.between(0, 10)
index[angle_filter & lat_filter & lon_filter]
VOLUME_ID FILE_SPECIFICATION_NAME ORIGINAL_PRODUCT_ID PRODUCT_ID IMAGE_TIME INSTRUMENT_ID INSTRUMENT_MODE_ID LINE_SAMPLES LINES SPATIAL_SUMMING ... SUB_SPACECRAFT_LATITUDE SOLAR_DISTANCE SOLAR_LONGITUDE LOCAL_TIME IMAGE_SKEW_ANGLE RATIONALE_DESC DATA_QUALITY_DESC ORBIT_NUMBER lat_mean lon_mean
1267 MROX_0035 DATA/P03_002074_1155_XN_64S004W.IMG 4A_04_1008005C00 P03_002074_1155_XN_64S004W 2007-01-05 01:02:50.418 CTX NIFL 5056 12288 1 ... -64.59 225779573.9 161.53 16.02 90.1 Dunes in Wegener Crater in MOC R15-01815 OK 2074 -64.59 4.61
4551 MROX_0165 DATA/P09_004738_1020_XN_78S008W.IMG 4A_04_1014016300 P09_004738_1020_XN_78S008W 2007-07-31 14:55:12.379 CTX NIFL 5056 52224 1 ... -77.97 210058626.6 286.93 15.36 90.4 South polar landforms OK 4738 -77.965 5.9325
4677 MROX_0173 DATA/P10_004883_1020_XN_78S005W.IMG 4A_04_1016007800 P10_004883_1020_XN_78S005W 2007-08-11 22:05:40.494 CTX NIFL 5056 52224 1 ... -78.09 211455781.6 293.82 15.27 90.4 South polar seasonal cap edge OK 4883 -78.08 4.57
5184 MROX_0197 DATA/P11_005252_1199_XI_60S008W.IMG 4A_04_101900A800 P11_005252_1199_XI_60S008W 2007-09-09 16:17:03.275 CTX ITL 5056 14336 1 ... -60.18 215826377.3 310.92 14.58 90.1 South mid- to high-latitude crater OK 5252 -60.1875 8.6425
5395 MROX_0209 DATA/P11_005384_1066_XN_73S007W.IMG 4A_04_1018017C00 P11_005384_1066_XN_73S007W 2007-09-19 23:03:52.412 CTX NIFL 5056 35840 1 ... -73.48 217613174.8 316.85 14.82 90.3 Sisyphi Cavi OK 5384 -73.48 7.05
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
133106 MROX_4264 DATA/U09_074575_1037_XN_76S005W.IMG 4A_04_117A02AC00 U09_074575_1037_XN_76S005W 2022-06-24 05:06:01.401 CTX NIFL 5056 52224 1 ... -76.26 206646214.2 252.89 15.96 90.4 Sisyphi Cavi region OK 74575 -76.2575 4.98
133926 MROX_4298 DATA/U10_074878_1161_XN_63S005W.IMG 4A_04_117C022300 U10_074878_1161_XN_63S005W 2022-07-17 19:55:57.184 CTX NIFL 2528 24576 1 ... -63.93 207381603.7 267.84 15.35 90.2 Southern highlands OK 74878 -63.94 5.53
134438 MROX_4315 DATA/U10_075023_1075_XN_72S001W.IMG 4A_04_117C042B00 U10_075023_1075_XN_72S001W 2022-07-29 03:04:41.315 CTX NIFL 5056 7168 1 ... -72.61 208145471.6 274.94 15.41 90.1 Ride-along with HiRISE OK 75023 -72.6275 1.005
135612 MROX_4352 DATA/U12_075498_1055_XN_74S006W.IMG 4A_04_1180007200 U12_075498_1055_XN_74S006W 2022-09-04 03:17:19.126 CTX NIFL 5056 7168 1 ... -74.52 212296356.7 297.69 15.13 90.1 Ride-along with HiRISE OK 75498 -74.535 6.62
136976 MROX_4402 DATA/U13_076012_1155_XN_64S004W.IMG 4A_04_118201BB00 U13_076012_1155_XN_64S004W 2022-10-14 04:35:24.111 CTX NIFL 5056 7168 1 ... -64.62 218924928.8 321.08 14.71 90.1 Ride-along with HiRISE OK 76012 -64.6275 3.995

119 rows × 53 columns

Copying a PRODUCT_ID, we can now automatically download it via the CTX utilities:

pid = "P03_002074_1155_XN_64S004W"
from planetarypy import ctx

Above could issue a warning that is about not finding an ISIS installation, which is required for SOME of this library’s features. Make sure your current environment has the environment variable ISISROOT and ISISDATA defined, so that planetarypy can find the ISIS tools, in case you want to use functions related to them.

Now let’s use the CTX class to access data:

data = ctx.CTX(pid)
data.download()
File exists. Use `overwrite=True` to download fresh.

The data was downloaded to a mission- and instrument-based storage folder that was defined the first time you imported planetarypy. You can verify it’s location by looking at the config object:

from planetarypy.config import config
config.storage_root
Path('/home/ayek72/mnt/slowdata/planetarypy')

We can now read the data into a multi-dimensional xarray, which supports elaborate indexing via its coordinates.

data.edr_da
<xarray.DataArray 'P03_002074_1155 EDR' (y: 12288, x: 5056)>
[62128128 values with dtype=uint8]
Coordinates:
  * x        (x) float64 0.5 1.5 2.5 3.5 ... 5.054e+03 5.054e+03 5.056e+03
  * y        (y) float64 0.5 1.5 2.5 3.5 ... 1.229e+04 1.229e+04 1.229e+04
Attributes: (12/19)
    BANDWIDTH:                  
    CENTER_FILTER_WAVELENGTH:   
    DATA_SET_ID:                "MRO-M-CTX-2-EDR-L0-V1.0"
    FILTER_NAME:                
    INSTRUMENT_ID:              CTX
    INSTRUMENT_NAME:            "CONTEXT CAMERA"
    ...                         ...
    START_TIME:                 2007-01-05T01:02:50.418
    STOP_TIME:                  2007-01-05T01:03:13.479
    TARGET_NAME:                MARS
    _FillValue:                 0
    scale_factor:               1.0
    add_offset:                 0.0

But one also get the image data simply as a numpy array:

data.edr_da.values
array([[ 9,  9, 10, ...,  8, 10,  8],
       [ 9,  8, 10, ...,  9, 10,  9],
       [ 9,  8, 10, ...,  8, 11, 10],
       ...,
       [11, 10, 11, ..., 10, 10, 10],
       [10,  9, 10, ...,  9, 10, 10],
       [11, 10, 10, ..., 11, 11, 10]], dtype=uint8)

Or plot the image using holoviews and datashader, so that even large images won’t bog down your browser:

data.plot_edr()

Note, that the above data is the uncalibrated EDR, so the dynamics of the image seem a little rough. If you have configured the ISIS properly, so that the current Python environment is aware of it (using ISISROOT and ISISDATA env variables), you can calibrate this data yourself now: