utilities

Download and management utilities

Source code

General Methods

IS2view.utilities.get_data_path(relpath: list | str | Path)[source]

Get the absolute path within a package from a relative path

Parameters:

relpath: list, str or pathlib.Path: relative path

IS2view.utilities.import_dependency(name: str, extra: str = '', raise_exception: bool = False)[source]

Import an optional dependency

Adapted from pandas.compat._optional::import_optional_dependency

Parameters:

name: str: Module name
extra: str, default “”: Additional text to include in the ImportError message
raise_exception: bool, default False: Raise an ImportError if the module is not found

Returns:

module: obj: Imported module

IS2view.utilities.dependency_available(name: str, minversion: str | None = None)[source]

Checks whether a module is installed without importing it

Adapted from xarray.namedarray.utils.module_available

Parameters:

name: str: Module name
minversionstr, optional: Minimum version of the module

Returns:

availablebool: Whether the module is installed

IS2view.utilities.get_hash(local: str | IOBase | Path, algorithm: str = 'md5')[source]

Get the hash value from a local file or BytesIO object

Parameters:

local: obj, str or pathlib.Path: BytesIO object or path to file
algorithm: str, default ‘md5’: hashing algorithm for checksum validation

IS2view.utilities.url_split(s: str)[source]

Recursively split a url path into a list

Parameters:

s: str: url string

IS2view.utilities.get_unix_time(time_string: str, format: str = '%Y-%m-%d %H:%M:%S')[source]

Get the Unix timestamp value for a formatted date string

Parameters:

time_string: str: formatted time string to parse
format: str, default ‘%Y-%m-%d %H:%M:%S’: format for input time string

IS2view.utilities.s3_client(HOST: str = 'https://data.nsidc.earthdatacloud.nasa.gov/s3credentials', timeout: int | None = None, region_name: str = 'us-west-2')[source]

Get AWS s3 client for NSIDC data in the cloud https://data.nsidc.earthdatacloud.nasa.gov/s3credentials

Parameters:

HOST: str: NSIDC AWS S3 credential host
timeout: int or NoneType, default None: timeout in seconds for blocking operations
region_name: str, default ‘us-west-2’: AWS region name

Returns:

client: obj: AWS s3 client for NSIDC Cumulus

IS2view.utilities.s3_filesystem(HOST: str = 'https://data.nsidc.earthdatacloud.nasa.gov/s3credentials', timeout: int | None = None, region_name: str = 'us-west-2')[source]

Get AWS s3 file system object for NSIDC data in the cloud https://data.nsidc.earthdatacloud.nasa.gov/s3credentials

Parameters:

HOST: str: NSIDC AWS S3 credential host
timeout: int or NoneType, default None: timeout in seconds for blocking operations
region_name: str, default ‘us-west-2’: AWS region name

Returns:

session: obj: AWS s3 file system session for NSIDC Cumulus

IS2view.utilities.s3_bucket(presigned_url: str)[source]

Get a s3 bucket name from a presigned url

Parameters:

presigned_url: str: s3 presigned url

Returns:

bucket: str: s3 bucket name

IS2view.utilities.s3_key(presigned_url: str)[source]

Get a s3 bucket key from a presigned url

Parameters:

presigned_url: str: s3 presigned url or https url

Returns:

key: str: s3 bucket key for object

IS2view.utilities.s3_presigned_url(bucket: str, key: str)[source]

Get a s3 presigned url from a bucket and object key

Parameters:

bucket: str: s3 bucket name
key: str: s3 bucket key for object

Returns:

presigned_url: str: s3 presigned url

IS2view.utilities.generate_presigned_url(bucket: str, key: str, expiration: int = 3600)[source]

Generate a presigned https URL to share an S3 object

Parameters:

bucket: str: s3 bucket name
key: str: s3 bucket key for object
expiration: int: Time in seconds for the presigned URL to remain valid

Returns:

presigned_url: str: s3 presigned https url

IS2view.utilities._create_default_ssl_context() → SSLContext[source]: Creates the default SSL context

IS2view.utilities._create_ssl_context_no_verify() → SSLContext[source]: Creates an SSL context for unverified connections

IS2view.utilities._set_ssl_context_options(context: SSLContext) → None[source]: Sets the default options for the SSL context

IS2view.utilities.attempt_login(urs: str = 'urs.earthdata.nasa.gov', context=<ssl.SSLContext object>, password_manager: bool = False, get_ca_certs: bool = False, redirect: bool = False, authorization_header: bool = True, **kwargs)[source]

attempt to build a urllib opener for NASA Earthdata

Parameters:

urs: str, default urs.earthdata.nasa.gov: Earthdata login URS 3 host
context: obj, default IS2view.utilities._default_ssl_context: SSL context for urllib opener object
password_manager: bool, default True: Create password manager context using default realm
get_ca_certs: bool, default False: Get list of loaded “certification authority” certificates
redirect: bool, default False: Create redirect handler object
authorization_header: bool, default False: Add base64 encoded authorization header to opener
username: str, default from environmental variable: NASA Earthdata username
password: str, default from environmental variable: NASA Earthdata password
retries: int, default 5: number of retry attempts
netrc: str, default ~/.netrc: path to .netrc file for authentication

Returns:

opener: obj: OpenerDirector instance

IS2view.utilities.build_opener(username: str, password: str, context=<ssl.SSLContext object>, password_manager: bool = True, get_ca_certs: bool = False, redirect: bool = False, authorization_header: bool = False, urs: str = 'https://urs.earthdata.nasa.gov')[source]

Build urllib opener for NASA Earthdata with supplied credentials

Parameters:

username: str or NoneType, default None: NASA Earthdata username
password: str or NoneType, default None: NASA Earthdata password
context: obj, default IS2view.utilities._default_ssl_context: SSL context for urllib opener object
password_manager: bool, default True: Create password manager context using default realm
get_ca_certs: bool, default False: Get list of loaded “certification authority” certificates
redirect: bool, default False: Create redirect handler object
authorization_header: bool, default False: Add base64 encoded authorization header to opener
urs: str, default ‘https://urs.earthdata.nasa.gov’: Earthdata login URS 3 host

Returns:

opener: obj: OpenerDirector instance

IS2view.utilities.get_token(HOST: str = 'https://urs.earthdata.nasa.gov/api/users/token', username: str | None = None, password: str | None = None, build: bool = True, urs: str = 'urs.earthdata.nasa.gov')[source]

Generate a NASA Earthdata User Token

Parameters:

HOST: str or list: NASA Earthdata token API host
username: str or NoneType, default None: NASA Earthdata username
password: str or NoneType, default None: NASA Earthdata password
build: bool, default True: Build opener and check credentials
timeout: int or NoneType, default None: timeout in seconds for blocking operations
urs: str, default ‘urs.earthdata.nasa.gov’: NASA Earthdata URS 3 host

Returns:

token: dict: JSON response with NASA Earthdata User Token

IS2view.utilities.list_tokens(HOST: str = 'https://urs.earthdata.nasa.gov/api/users/tokens', username: str | None = None, password: str | None = None, build: bool = True, urs: str = 'urs.earthdata.nasa.gov')[source]

List the current associated NASA Earthdata User Tokens

Parameters:

HOST: str: NASA Earthdata list token API host
username: str or NoneType, default None: NASA Earthdata username
password: str or NoneType, default None: NASA Earthdata password
build: bool, default True: Build opener and check credentials
timeout: int or NoneType, default None: timeout in seconds for blocking operations
urs: str, default ‘urs.earthdata.nasa.gov’: NASA Earthdata URS 3 host

Returns:

tokens: list: JSON response with NASA Earthdata User Tokens

IS2view.utilities.revoke_token(token: str, HOST: str = 'https://urs.earthdata.nasa.gov/api/users/revoke_token', username: str | None = None, password: str | None = None, build: bool = True, urs: str = 'urs.earthdata.nasa.gov')[source]

Generate a NASA Earthdata User Token

Parameters:

token: str: NASA Earthdata token to be revoked
HOST: str: NASA Earthdata revoke token API host
username: str or NoneType, default None: NASA Earthdata username
password: str or NoneType, default None: NASA Earthdata password
build: bool, default True: Build opener and check credentials
timeout: int or NoneType, default None: timeout in seconds for blocking operations
urs: str, default ‘urs.earthdata.nasa.gov’: NASA Earthdata URS 3 host

IS2view.utilities.check_credentials()[source]: Check that entered NASA Earthdata credentials are valid

IS2view.utilities.from_nsidc(HOST: str | list, username: str | None = None, password: str | None = None, build: bool = True, timeout: int | None = None, urs: str = 'urs.earthdata.nasa.gov', local: str | ~pathlib.Path | None = None, hash: str = '', chunk: int = 16384, verbose: bool = False, fid=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, mode: oct = 509)[source]

Download a file from a NSIDC https server

Parameters:

HOST: str or list: remote https host
username: str or NoneType, default None: NASA Earthdata username
password: str or NoneType, default None: NASA Earthdata password
build: bool, default True: Build opener and check credentials
timeout: int or NoneType, default None: timeout in seconds for blocking operations
urs: str, default ‘urs.earthdata.nasa.gov’: NASA Earthdata URS 3 host
local: str or NoneType, default None: path to local file
hash: str, default ‘’: MD5 hash of local file
chunk: int, default 16384: chunk size for transfer encoding
verbose: bool, default False: print file transfer information
fid: obj, default sys.stdout: open file object to print if verbose
mode: oct, default 0o775: permissions mode of output local file

Returns:

remote_buffer: obj: BytesIO representation of file
response_error: str or None: notification for response error

IS2view.utilities.cmr_query_release(release: str | int | None)[source]

Build formatted query string for ICESat-2 release

Parameters:

release: str: ICESat-2 data release

Returns:

query_params: str: formatted string for CMR queries

IS2view.utilities.cmr_regions(region: str | list | None)[source]

Check if the submitted ATL14/15 regions are valid

Parameters:

region: str, list or NoneType, default None

Returns:

region_list: list: formatted available ATL14/15 regions

IS2view.utilities.cmr_resolutions(resolution: str | list | None)[source]

Check if the submitted ATL14/15 resolutions are valid

Parameters:

resolution: str, list or NoneType, default None: ICESat-2 ATL14/15 spatial resolution

Returns:

resolution_list: list: formatted available ATL14/15 resolutions

IS2view.utilities.cmr_readable_granules(product: str, **kwargs)[source]

Create list of readable granule names for CMR queries

Parameters:

product: str: ICESat-2 data product
regions: str, list or NoneType, default None: ICESat-2 ATL14/15 region name
resolutions: str, list or NoneType, default None: ICESat-2 ATL14/15 spatial resolution

Returns:

readable_granule_list: list: readable granule names for CMR queries

IS2view.utilities.cmr_filter_json(search_results: dict, endpoint: str = 'data', request_type: str = 'application/x-hdfeos')[source]

Filter the CMR json response for desired data files

Parameters:

search_results: dict

json response from CMR query

endpoint: str, default ‘data’

url endpoint type

'data': NASA Earthdata https archive

'opendap': NASA Earthdata OPeNDAP archive

's3': NASA Earthdata Cumulus AWS S3 bucket

request_type: str, default ‘application/x-hdfeos’

data type for reducing CMR query

Returns:

producer_granule_ids: list: ICESat-2 granules
granule_urls: list: ICESat-2 granule urls from NSIDC

IS2view.utilities.cmr(product: str = None, release: str = None, regions: str | list | None = None, resolutions: str | list | None = None, provider: str = 'NSIDC_CPRD', endpoint: str = 'data', request_type: str = 'application/(x-)?netcdf', opener=None, context: ~ssl.SSLContext = <ssl.SSLContext object>, verbose: bool = False, fid=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)[source]

Query the NASA Common Metadata Repository (CMR) for ICESat-2 data

Parameters:

product: str or NoneType, default None

ICESat-2 data product

release: str or NoneType, default None

ICESat-2 data release

regions: str, list or NoneType, default None

ICESat-2 ATL14/15 region name

resolutions: str, list or NoneType, default None

ICESat-2 ATL14/15 spatial resolution

provider: str, default ‘NSIDC_CPRD’

CMR data provider

endpoint: str, default ‘data’

url endpoint type

'data': NASA Earthdata https archive

'opendap': NASA Earthdata OPeNDAP archive

's3': NASA Earthdata Cumulus AWS S3 bucket

request_type: str, default ‘application/(x-)?netcdf’

data type for reducing CMR query

opener: obj or NoneType, default None

OpenerDirector instance

context: obj, default IS2view.utilities._default_ssl_context

SSL context for urllib opener object

verbose: bool, default False

print file transfer information

fid: obj, default sys.stdout

open file object to print if verbose

Returns:

producer_granule_ids: list: ICESat-2 granules
granule_urls: list: ICESat-2 granule urls from NSIDC

IS2view.utilities.query_resources(**kwargs)[source]

Queries CMR or s3 for available granules

Parameters:

asset: str, default ‘nsidc-https’

Location to get the ICESat-2 gridded land ice data

nsidc-https : NSIDC on-prem DAAC
nsidc-s3 : NSIDC AWS protected s3 bucket
atlas-s3 : s3 bucket in us-west-2
atlas-local : local directory

bucket: str, default ‘is2view’

Project AWS s3 bucket name

directory: str or NoneType, default None

Working data directory if local

product: str, default ‘ATL15’

ICESat-2 gridded land ice product

ATL14 : land ice height
ATL15 : land ice height change

release: str, default ‘004’

ICESat-2 data release

version: str, default ‘01’

ICESat-2 data version

region: str, default ‘AA’

ICESat-2 ATL14/15 region name

AA : Antarctic
CN : Northern Canadian Archipelagobb
CS : Southern Canadian Archipelago
GL : Greenland
IS : Iceland
SV : Svalbard
RA : Russian High Arctic

resolution: str, default ‘01km’

ICESat-2 ATL14/15 spatial resolution

100m : 100 meters horizontal
01km : 1 kilometer horizontal
10km : 10 kilometers horizontal
20km : 20 kilometers horizontal
40km : 40 kilometers horizontal

format: str, default ‘nc’

Data format for ICESat-2 granules

nc : Native netCDF4
zarr : Cloud-optimized zarr

Returns:

granule: str: presigned url or path for granule