Tutorial¶

# call this cell first until we get this project on pip
import sys
sys.path.append('..')

from pysat_kamodo import Pysat_Kamodo
import pysat

Instrument Initialization¶

To initialize a pysat instrument, we require that a date be specified along with the usual pysat keyword arguments:

import pandas as pd

pki = Pysat_Kamodo('2009, 1, 1', # Pysat_Kamodo allows string dates
                  platform = 'cnofs', # pysat keyword
                  name='vefi', # pysat keyword
                  tag='dc_b',# pysat keyword
                  )
pki

loaded .cnofs_vefi
Returning cnofs vefi dc_b data for 01 January 2009

$$\begin{equation}\operatorname{B_{north}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{up}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{west}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{IGRF north}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{IGRF up}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{IGRF west}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{latitude}{\left(t \right)} [degrees] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{longitude}{\left(t \right)} [degrees] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{altitude}{\left(t \right)} [km] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{dB_{zon}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{dB_{mer}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{dB_{par}}{\left(t \right)} [nT] = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{year}{\left(t \right)} = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{dayofyear}{\left(t \right)} = \lambda{\left(t \right)}\end{equation}$$ $$\begin{equation}\operatorname{B_{flag}}{\left(t \right)} = \lambda{\left(t \right)}\end{equation}$$

The above Kamodo instance shows all available variables from the pysat cnofs vefi instrument together with units. The left-hand side of these equations represents the variables loaded by pysat. The right-hand side $\lambda(t)$ expressions mean that there is some time interpolator defined for these variables that is not in closed form. More information for each variable, including the instrument's citation info, can be read from the variable docs:

help(pki.B_flag)

Help on function B_flag:

B_flag(t=DatetimeIndex(['2009-01-01 00:00:00', '2009-01-01 00:00:01',
               '2009-01-01 00:00:02', '2009-01-01 00:00:03',
               '2009-01-01 00:00:04', '2009-01-01 00:00:05',
               '2009-01-01 00:00:06', '2009-01-01 00:00:07',
               '2009-01-01 00:00:08', '2009-01-01 00:00:09',
               ...
               '2009-01-01 23:59:49', '2009-01-01 23:59:50',
               '2009-01-01 23:59:51', '2009-01-01 23:59:52',
               '2009-01-01 23:59:53', '2009-01-01 23:59:54',
               '2009-01-01 23:59:55', '2009-01-01 23:59:56',
               '2009-01-01 23:59:57', '2009-01-01 23:59:58'],
              dtype='datetime64[ns]', length=80594, freq=None))
    B_flag time interpolator

    parameters:
    t: datetime series or list

    returns: B_flag [] pandas series


    citation: Supports the Vector Electric Field Instrument (VEFI)
    onboard the Communication and Navigation Outage Forecasting
    System (C/NOFS) satellite. Downloads data from the
    NASA Coordinated Data Analysis Web (CDAWeb).

    Description from CDAWeb:

    The DC vector magnetometer on the CNOFS spacecraft is a three axis, fluxgate
    sensor with active thermal control situated on a 0.6m boom.  This magnetometer
    measures the Earth's magnetic field using 16 bit A/D converters at 1 sample per
    sec with a range of .. 45,000 nT.  Its primary objective on the CNOFS
    spacecraft is to enable an accurate V x B measurement along the spacecraft
    trajectory.  In order to provide an in-flight calibration of the magnetic field
    data, we compare the most recent POMME model (the POtsdam Magnetic Model of the
    Earth, http://geomag.org/models/pomme5.html) with the actual magnetometer
    measurements to help determine a set of calibration parameters for the gains,
    offsets, and non-orthogonality matrix of the sensor axes.  The calibrated
    magnetic field measurements are provided in the data file here. The VEFI
    magnetic field data file currently contains the following variables:
    B_north   Magnetic field in the north direction
    B_up      Magnetic field in the up direction
    B_west    Magnetic field in the west direction

    The data is PRELIMINARY, and as such, is intended for BROWSE PURPOSES ONLY.

    A brief discussion of the C/NOFS mission and instruments can be found at
    de La Beaujardière, O., et al. (2004), C/NOFS: A mission to forecast
    scintillations, J. Atmos. Sol. Terr. Phys., 66, 1573–1591,
    doi:10.1016/j.jastp.2004.07.030.

    Parameters
    ----------
    platform : string
        'cnofs'
    name : string
        'vefi'
    tag : string
        Select measurement type, one of {'dc_b'}
    sat_id : string
        None supported

    Note
    ----
    - tag = 'dc_b': 1 second DC magnetometer data

    Warnings
    --------
    - The data is PRELIMINARY, and as such, is intended for BROWSE PURPOSES ONLY.
    - Limited cleaning routine.
    - Module not written by VEFI team.

Interpolation¶

These "Kamodofied" variables support automatic time interpolation. First let's specify a few arbitrary moments in time.

t1 = pd.datetime(2009, 1, 1, 3, 39, 45)
t2 = pd.datetime(2009, 1, 1, 3, 42, 33)
t3 = pd.datetime(2009, 1, 1, 3, 51, 10)
times = [t1, t2, t3]

To interpolate a given variable at these times, simply call the variable like you would a function:

pki.B_up(times)

2009-01-01 03:39:45    12086.362305
2009-01-01 03:42:33    14458.384766
2009-01-01 03:51:10    14879.630859
Name: B_up, dtype: float32

Our Kamodo object uses the native instrument time as a default parameter. This means we can easily retrieve the original raw data by not specifying a time:

# head pandas method just shows the first 5 results
pki.B_up().head()

2009-01-01 00:00:00   -3984.774658
2009-01-01 00:00:01   -3966.702637
2009-01-01 00:00:02   -3951.631592
2009-01-01 00:00:03   -3936.454102
2009-01-01 00:00:04   -3921.183594
Name: B_up, dtype: float32

Visualization¶

Kamodo uses plotly for visualization and easy web deployment. The default time allows Kamodo to determine an appropriate range for these quick-look graphics.

import plotly.io as pio
fig = pki.plot('B_up')
pio.write_image(fig, 'figure1.svg')

Interactive plotting¶

To plot interacvtively in the notebook use plotly's iplot and plot tools.

from plotly.offline import init_notebook_mode, plot, iplot
init_notebook_mode(connected = True)

# renders in notebook cell
iplot(fig)

# saves to html and opens in new browser tab
plot(fig)

PyHC standards¶

Pysat_Kamodo objects have data and meta attributes, in keeping with the Python-in-Heliophysics Community standards. These are not to be confused with pysat's data and meta attributes, which are stored in Pysat_Kamodo._instrument.meta and Pysat_Kamodo._instrument.data, respectively.

Pysat_Kamodo meta attributes¶

The meta attribute illustrates Kamodo's minimalistic view of the world: under the hood, Kamodo uses the symbolic programing library Sympy, so it only needs to keep track of symbols and units.

pki.meta

Kamodo also has some book-keeping metadata tied to indvidual variables:

pki.B_flag.meta

{'units': '',
 'citation': "Supports the Vector Electric Field Instrument (VEFI)\nonboard the Communication and Navigation Outage Forecasting\nSystem (C/NOFS) satellite. Downloads data from the\nNASA Coordinated Data Analysis Web (CDAWeb).\n\nDescription from CDAWeb:\n\nThe DC vector magnetometer on the CNOFS spacecraft is a three axis, fluxgate\nsensor with active thermal control situated on a 0.6m boom.  This magnetometer\nmeasures the Earth's magnetic field using 16 bit A/D converters at 1 sample per\nsec with a range of .. 45,000 nT.  Its primary objective on the CNOFS\nspacecraft is to enable an accurate V x B measurement along the spacecraft\ntrajectory.  In order to provide an in-flight calibration of the magnetic field\ndata, we compare the most recent POMME model (the POtsdam Magnetic Model of the\nEarth, http://geomag.org/models/pomme5.html) with the actual magnetometer\nmeasurements to help determine a set of calibration parameters for the gains,\noffsets, and non-orthogonality matrix of the sensor axes.  The calibrated\nmagnetic field measurements are provided in the data file here. The VEFI\nmagnetic field data file currently contains the following variables:\nB_north   Magnetic field in the north direction\nB_up      Magnetic field in the up direction\nB_west    Magnetic field in the west direction\n\nThe data is PRELIMINARY, and as such, is intended for BROWSE PURPOSES ONLY.\n\nA brief discussion of the C/NOFS mission and instruments can be found at\nde La Beaujardière, O., et al. (2004), C/NOFS: A mission to forecast\nscintillations, J. Atmos. Sol. Terr. Phys., 66, 1573–1591,\ndoi:10.1016/j.jastp.2004.07.030.\n\nParameters\n----------\nplatform : string\n    'cnofs'\nname : string\n    'vefi'\ntag : string\n    Select measurement type, one of {'dc_b'}\nsat_id : string\n    None supported\n\nNote\n----\n- tag = 'dc_b': 1 second DC magnetometer data\n\nWarnings\n--------\n- The data is PRELIMINARY, and as such, is intended for BROWSE PURPOSES ONLY.\n- Limited cleaning routine.\n- Module not written by VEFI team.\n\n",
 'equation': None,
 'hidden_args': []}

Pysat_Kamodo data attributes¶

The high-level data attribute provides a dictionary-access to the underlying instrument data. Custom function data is also available here.

pki.data['B_north'].head()

2009-01-01 00:00:00    18440.412109
2009-01-01 00:00:01    18437.386719
2009-01-01 00:00:02    18433.964844
2009-01-01 00:00:03    18430.533203
2009-01-01 00:00:04    18427.074219
Name: B_north, dtype: float32

The instrument data is also available at the variable level:

pki.B_north.data.head()

2009-01-01 00:00:00    18440.412109
2009-01-01 00:00:01    18437.386719
2009-01-01 00:00:02    18433.964844
2009-01-01 00:00:03    18430.533203
2009-01-01 00:00:04    18427.074219
Name: B_north, dtype: float32

In this case, these are the same as the instrument data. However, Kamodo variables can come from any Kamodo-compatible function or data source, not just pysat data. See Kamodo's documentation on Kamodofication and the turorial on Kamodofying Models for examples.