Open a SAFE product#

import xarray as xr

from xarray_eop.api import open_datatree
from xarray_eop.path import EOPath

Open a specific part of the product as a dataset#

The specific part may be determined by a given netcdf file from the SAFE or a given group corresponding to the EOPF zarr-like format.

Product stored in the local filesystem:#

import importlib.resources
SAMPLE_PATH = importlib.resources.files("tests.ut.data")
product = SAMPLE_PATH / "S3B_OL_1_ERR_sample.SEN3"

One can access the data by specifying a group corresponding to the EOPF zarr-like format:

group="/conditions/instrument"
ds = xr.open_dataset(product,file_or_group=group,engine="sentinel-3")
ds
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/xarray/namedarray/core.py:514: UserWarning: Duplicate dimension names present: dimensions {'bands'} appear more than once in dims=('bands', 'bands'). We do not yet support duplicate dimension names, but we do allow initial construction of the object. We recommend you rename the dims immediately to become distinct, as most xarray functionality is likely to fail silently if you do not. To rename the dimensions you will need to set the ``.dims`` attribute of each variable, ``e.g. var.dims=('x0', 'x1')``.
  warnings.warn(
<xarray.Dataset> Size: 2kB
Dimensions:                       (bands: 5, detectors: 37, bands1: 5, bands2: 5)
Dimensions without coordinates: bands, detectors, bands1, bands2
Data variables:
    fwhm                          (bands, detectors) float32 740B ...
    lambda0                       (bands, detectors) float32 740B ...
    relative_spectral_covariance  (bands1, bands2) float32 100B ...
    solar_flux                    (bands, detectors) float32 740B ...

or by opening a whole netcdf file

file="instrument_data.nc"
ds = xr.open_dataset(product,file_or_group=file,engine="sentinel-3")
ds
<xarray.Dataset> Size: 103kB
Dimensions:                       (rows: 103, columns: 122, bands: 5,
                                   detectors: 37, bands1: 5, bands2: 5)
Dimensions without coordinates: rows, columns, bands, detectors, bands1, bands2
Data variables:
    detector_index                (rows, columns) float32 50kB ...
    frame_offset                  (rows, columns) float32 50kB ...
    lambda0                       (bands, detectors) float32 740B ...
    FWHM                          (bands, detectors) float32 740B ...
    solar_flux                    (bands, detectors) float32 740B ...
    relative_spectral_covariance  (bands1, bands2) float32 100B ...
Attributes: (12/17)
    netCDF_version:         4.2 of Jul  8 2020 09:28:41 $
    product_name:           S3B_OL_1_ERR____20230506T015316_20230506T015616_2...
    title:                  OLCI Level 1b Product, Instrument Data Set
    institution:            LR1
    source:                 IPF-OL-1-EO 06.17
    history:                 
    ...                     ...
    start_time:             2023-05-06T01:53:16.029027Z
    stop_time:              2023-05-06T01:56:16.040795Z
    processing_baseline:    OL__L1_.003.03.00
    comment:                 
    ac_subsampling_factor:  16
    al_subsampling_factor:  1

Product stored in S3 bucket#

It is also possible to access a product in a cloud storage. S3 cloud storage is currently the only handled object-storage.

xarray-eop provide an EOPath classe derived from pathlib.Path which supports S3 paths and well as fsspec-like URL chain.

buc_name = EOPath("s3://buc-acaw-dpr")
product = buc_name / "Samples/SAFE/S3B_OL_1_ERR____20230506T015316_20230506T015616_20230711T065804_0179_079_117______LR1_D_NR_003.SEN3"
ds = xr.open_dataset(
    product,
    file_or_group="instrument_data.nc",
    engine="sentinel-3",
)
ds
<xarray.Dataset> Size: 11MB
Dimensions:                       (rows: 1023, columns: 1217, bands: 21,
                                   detectors: 3700, bands1: 21, bands2: 21)
Dimensions without coordinates: rows, columns, bands, detectors, bands1, bands2
Data variables:
    detector_index                (rows, columns) float32 5MB ...
    frame_offset                  (rows, columns) float32 5MB ...
    lambda0                       (bands, detectors) float32 311kB ...
    FWHM                          (bands, detectors) float32 311kB ...
    solar_flux                    (bands, detectors) float32 311kB ...
    relative_spectral_covariance  (bands1, bands2) float32 2kB ...
Attributes: (12/17)
    netCDF_version:         4.2 of Jul  8 2020 09:28:41 $
    product_name:           S3B_OL_1_ERR____20230506T015316_20230506T015616_2...
    title:                  OLCI Level 1b Product, Instrument Data Set
    institution:            LR1
    source:                 IPF-OL-1-EO 06.17
    history:                 
    ...                     ...
    start_time:             2023-05-06T01:53:16.029027Z
    stop_time:              2023-05-06T01:56:16.040795Z
    processing_baseline:    OL__L1_.003.03.00
    comment:                 
    ac_subsampling_factor:  16
    al_subsampling_factor:  1

Open the whole SAFE product as a datatree structure#

dt = open_datatree(product)
dt
<xarray.DatasetView> Size: 0B
Dimensions:  ()
Data variables:
    *empty*

In the case of the Sentinel-3 product, data is a collection of netcdf files. To allow lazy loading, each variable is first computed and then rechunk. To speed up the process, it is possible to make a temporary copy of the product in the local filesystem, by setting the optional argument fs_copy to True

dt = open_datatree(product,fs_copy=True)
dt
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/xarray/namedarray/core.py:514: UserWarning: Duplicate dimension names present: dimensions {'bands'} appear more than once in dims=('bands', 'bands'). We do not yet support duplicate dimension names, but we do allow initial construction of the object. We recommend you rename the dims immediately to become distinct, as most xarray functionality is likely to fail silently if you do not. To rename the dimensions you will need to set the ``.dims`` attribute of each variable, ``e.g. var.dims=('x0', 'x1')``.
  warnings.warn(
<xarray.DatasetView> Size: 0B
Dimensions:  ()
Data variables:
    *empty*
dt.measurements.oa17_radiance.plot()
<matplotlib.collections.QuadMesh at 0x7f99acdb4bd0>
../_images/ae424e11c92f3a188209522d8e07320bdf73b4887e6abdcfa9a22526076d6625.png
import matplotlib.pyplot as plt
plt.figure(figsize=(14, 6))
ax = plt.axes()
dt.measurements.oa17_radiance.plot.pcolormesh(
    ax=ax,
    x="longitude", y="latitude", add_colorbar=False
)
<matplotlib.collections.QuadMesh at 0x7f99acc4d3d0>
../_images/18fa274c2ae077eab73854a6467e04a657ebc6368322d3dc8b194f4dd95fa7ce.png