from planetarypy.config import config
from planetarypy.instruments.mro.ctx import EDR, ctxconfig
import tomlkitTutorial: CTX EDR data
Goals of this tutorial are to demonstrate
- retrieving EDR data using full or shortened PRODUCT_ID
- control how it is stored locally by a config file
- work with metadata and other attributes of EDR class
Introduction
Minimal goals of an instrument planetarypy module are to
- retrieve the EDR (PDS3 lingo) or Raw (PDS4 lingo) data easily, and
- to store the retrieved data transparent but configurable to the user in a similar way the data is archived on the PDS/PSA.
The methods offered here know where the data are stored, hence the user does not need to deal with that part, but it can be configured via a dedicated instrument configuration file.
A substantial part of the functionality stems from using so called index files that provide relevant metadata. These index files are being downloaded by planetarypy from the data archives automatically, in case they are not on your local system yet.
Config files
There are two config files of concern here, the main config file that decides where all managed data from planetarypy will be stored, and the instrument config file:
config["storage_root"] # main config file'/Users/maye/planetarypy_data'The tomlkit.dumps is only used here for pretty printing. The config object works like a standard Python dictionary.
print(tomlkit.dumps(ctxconfig)) # just for pretty printing# configuration for MRO CTX data
[edr]
# where to search locally
# if empty-> config.storage_root/mro/pds
local_mirror = "/Volumes/planet/Mars/CTX/pds"
# add the volume folder to path?
with_volume = true
# does each product_id have it's own subfolder?
with_pid = false
# where to get new data that is not in storage (should be taken from index maybe?)
#url = "https://pds-imaging.jpl.nasa.gov/data/mro/ctx"
url = "https://pdsimage2.wr.usgs.gov/Mars_Reconnaissance_Orbiter/CTX"
# where to store data if local_mirror is not writable and new data comes in
# defaults again to config.storage_root/mro/pds
local_storage = ""
[calib]
# where to search calibrated data
# if not absolute path, added to config.storage_root
storage = ""
# add volume folder to path?
with_volume = true
# add subfolder for each product_id?
with_pid = true
# what file extensions to use
# these will be extended by the file type, e.g. "lev2.tif" or "lev1.cub"
calibrated_ext = ".lev1"
mapped_ext = ".lev2"
As you can see the instrument config file has two sections, one for the EDR data and one for calibrated data, that the ctx module also supports (covered in another tutorial).
The config relevant to EDR data canm be accessed via standard dictionary key access:
edr_config = ctxconfig["edr"]
print(tomlkit.dumps(edr_config))# where to search locally
# if empty-> config.storage_root/mro/pds
local_mirror = "/Volumes/planet/Mars/CTX/pds"
# add the volume folder to path?
with_volume = true
# does each product_id have it's own subfolder?
with_pid = false
# where to get new data that is not in storage (should be taken from index maybe?)
#url = "https://pds-imaging.jpl.nasa.gov/data/mro/ctx"
url = "https://pdsimage2.wr.usgs.gov/Mars_Reconnaissance_Orbiter/CTX"
# where to store data if local_mirror is not writable and new data comes in
# defaults again to config.storage_root/mro/pds
local_storage = ""
The key/value pairs of the configuration here provide both standard and advanced possibilities possibly not yet implemented for other instrument modules of planetarypy.
local_mirror: provide a different source of EDR filesUseful if your office already has a local server of all EDR files
url: data archive source urllocal_storageplanetarypyusually puts this into the above mentionedconfig.storage_root/mro/ctxbut this keyword allows to override that.fine control of how data is stored:
with_volume: with a volume folder?Like it is done on PDS often.
with_pid: with each product_id having its own folder?This becomes useful if one starts to create a lot of derived products.
Simple data access
Let’s practice getting data. Both the full and the shortened first 15 chars of the product_id are ok to identify an image:
pid = "B03_010694_1868_XI_06N285W" # standard CTX product_id
pid = "J05_046771_1950_XN_15N254W"pid_short = pid[:15]
pid_short'J05_046771_1950'edr = EDR(pid)
edrEDR(pid='J05_046771_1950_XN_15N254W') # Volume: mrox_2682edr = EDR(pid_short)
edrEDR(pid='J05_046771_1950_XN_15N254W') # Volume: mrox_2682edr.pid'J05_046771_1950_XN_15N254W'edr.volume'mrox_2682'edr.short_pid'J05_046771_1950'edr.image_time # this comes from the metadata in the index fileTimestamp('2016-07-18 21:30:48.920000')edr.meta # this is the whole metadata entry for this product_idvolume_id MROX_2682
file_specification_name DATA/J05_046771_1950_XN_15N254W.IMG
original_product_id 4A_04_10EC031E00
product_id J05_046771_1950_XN_15N254W
image_time 2016-07-18 21:30:48.920000
instrument_id CTX
instrument_mode_id NIFL
line_samples 1120
lines 27648
spatial_summing 1
scaled_pixel_width 5.43
pixel_aspect_ratio 1.11
emission_angle 2.3
incidence_angle 53.78
phase_angle 51.53
center_longitude 254.74
center_latitude 15.01
upper_left_longitude 254.62
upper_left_latitude 13.61
upper_right_longitude 254.51
upper_right_latitude 13.63
lower_left_longitude 254.97
lower_left_latitude 16.4
lower_right_longitude 254.86
lower_right_latitude 16.41
mission_phase_name ESP
target_name MARS
spacecraft_clock_start_count 1153344692:142
focal_plane_temperature 295.2
line_exposure_duration 1.877
offset_mode_id 197/230/219
sample_first_pixel 3840
scaled_image_width 6.07
scaled_image_height 166.44
spacecraft_altitude 275.5
target_center_distance 3670.37
slant_distance 275.71
usage_note N
north_azimuth 277.03
sub_solar_azimuth 171.1
sub_solar_longitude 305.76
sub_solar_latitude -3.49
sub_spacecraft_longitude 254.92
sub_spacecraft_latitude 15.0
solar_distance 216757673.6
solar_longitude 188.12
local_time 15.39
image_skew_angle 90.1
rationale_desc Landforms north of the western Nepenthes Mensae
data_quality_desc OK
orbit_number 46771
short_pid J05_046771_1950
month_col J05
Name: Metadata for J05_046771_1950_XN_15N254W, dtype: objectedr.data_okTrueedr.pathPath('/Users/maye/planetarypy_data/missions/mro/ctx/mrox_2682/J05_046771_1950_XN_15N254W.IMG')edr.path.exists()Trueedr.download()--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[19], line 1 ----> 1 edr.download() AttributeError: 'EDR' object has no attribute 'download'