API Reference

Module: CLEDB_PREPINV

CLEDB_PREPINV.CLEDB_PREPINV.sobs_preprocess(sobs_in, headkeys, params)

Main function to process an input observation and ingest the relevant header keywords. Generally, this function iterates over the observation maps and sends each pixel to the internal functions. It returns a processed observation array (input dependent) that is ready for analysis. Additional products are calculated via its subfunctions. e.g. a height map (used to match databases), signal statistics, etc.

Parameters:

sobs_infltarr

Input data array

HeadkeysInput List of header keys, or record array, or structure

Input list of header keys that describes the sobs_in data

paramsclass

Passes a set of inversion controling parameters

Returns:

sobs_totfltarr

Contains the background subtracted, integrated, and normalized Stokes IQUV spectra for 1-line ([xs,ys,4]) or 2-line ([xs,ys,8]) observations.

yobsfltarr

Used to construct a height projection for each observed voxel in units or R.

sobs_totrotfltarr

Derived from sobs_tot. The Stokes Q and U components are rotated along the center of the Sun to match the reference direction for linear polarization (the reference in which the database is created by CLEDB_BUILD). The variable is initialized as a “zero” array that is returned in the case of 1-line observations to keep a standardized function input/output needed for Numba vectorization.

aobsfltarr

Stores the linear polarization angle transformation performed by the OBS_QUROTATE function. This information will be used to derotate the matched database profiles found by the CLEDB_INVPROC 2-line inversion function for comparison with the input Stokes profiles.

dobsfltarr

Line ratio density array. This is used to match corresponding databases.

snrfltarr

Returns the signal to noise ratio(SNR) of the root mean square (RMS) of the total counts in each Stokes profile. The SNR calculation is correspondent to the ratio between RMS intensity in the line core and background counts (the variance). This measurement shows the quality in the signal for a particular observed voxel.

backgroundfltarr

Returns averaged background counts for each observed voxel and each Stokes component.

wlarrlist of fltarr

List containing a set of wavelength dispersion axie for each observed line.

keyvalslist

list of parameters needed. These are read and processed from the observation metadata or processed depending on the input data type. Generated by the sobs_preprocess*<–*obs_headstructproc functions.

issuemaskfltarr

An array that encodes potential issues appearing during processing ONLY in the case of succesful processing. These are warnings, with respect to each pixel. Errors are handed by specific exceptions. This array will be updated across all modules. The tentative issuemask implementation is described separately.

CLEDB_PREPINV.CLEDB_PREPINV.sdb_preprocess(yobs, dobs, keyvals, wlarr, params)

Main script to find and preload all necessary databases compatible with the sobs_preprocessed data.

Returns databases as a list along with an encoding corresponding to each voxel of the observation. This is compatible to NUMBA object mode.

Parameters:

yobsfltarr

Array of observation heights of the shape of the input data array.

dobsfltarr

Array of computed observation plasma densities for matching with the database.

keyvalslist

Processed observation header keys.

wlarrlist of fltarr

List containing a set of wavelength dispersion axis for each observed line.

paramsclass

Passes a set of inversion controling parameters.

Returns:

dbnamesstrarr

Array of entire database set file names to match to the observation.

databasefltarr

Set of unique databases loaded into memory.

dbhdrlist

header storing the configuration of the entire database set.

db_enc_flnmstrarr (output DEBUG/TEST only for verbose ==4)

Array of filenames keeping track of which database applies to which pixel as resulting from sdb_findelongationanddens (same database can apply to multiple pixels).

db_uniqlist or strings (output DEBUG/TEST only for verbose ==4)

list that reduces the number of databases recorded in db_enc_flnm to help loading only the unique entries.

db_encfltarr (output DEBUG/TEST only for verbose ==4)

Array for each pixel that stores the index of the entry in db_uniq of respective db_enc_flnm entries to be uniquely loaded in the database variable.

CLEDB_PREPINV.CLEDB_PREPINV.sdb_fileingest(dbdir, nline, tline, verbose)

Small function that returns filename and index of elongation y available in the full database set of calculations.

CLEDB_PREPINV.CLEDB_PREPINV.sdb_findelongationanddens(xx, yy, y, d, dbynumbers)

Returns index of elongation y and density d in database set coresponding to the observed pixel.

The calculation is faster ingested as a function rather than an inline calculation!

CLEDB_PREPINV.CLEDB_PREPINV.sdb_parseheader(dbheaderfile)

Reads and parses the database header (db.hdr file) and returns the parameters contained in the set of calculations.

db.hdr text file needs to have the specific CLE or PyCELP version format described in the CLEDB readme.

CLEDB_PREPINV.CLEDB_PREPINV.sdb_read(fildb, dbhdr, verbose)

Reading database files in memory and store in convenient variables.

Returns:

dbfltarr [ncalc,nline,4]

Output database file -1.590196830801

CLEDB_PREPINV.CLEDB_PREPINV.sdb_lcgrid(mmin, mmax, n)

Unpacks the grid for ned densities from mmin to mmax logarithms of density and n sampled densities.

CLEDB_PREPINV.CLEDB_PREPINV.obs_headstructproc(sobs_in, headkeys, params)

Function for ingesting a minimal number of needed keywords from the observation headerfile.

Most keywords are defined statically based on specs at time of last updating this function. Dummy headkeys input can be used for other instruments with excercising care as exampled in the MURAM and PSI cases.

Implemented instruments are: a. CoMP/uCoMP/COSMO b. DKIST Cryo-NIRSP c. MURAM example simulation d. PSImas/pycelp example simulation e. CLE simulation

CLEDB_PREPINV.CLEDB_PREPINV.obs_headstructproc_work_1pix(xx, wcs_pix, ys)

Helper function defining for parallelizing obs_headstructproc.

CLEDB_PREPINV.CLEDB_PREPINV.obs_calcheight(keyvals, hpxygrid)

Helper function for observation preprocessing in obs_headstructproc — Calculates the off-limb heights corresponding to each voxel.

Assumes that both observations have the same pointing.

CLEDB_PREPINV.CLEDB_PREPINV.obs_integrate(sobs_in, wlarr, keyvals, verbose, ncpu, atmred)

Reads the background and peak emission of lines, subtracts the background and integrates the signal.

Function was slow for big arrays. It has been rewritten for efficiency, but lost a lot in terms of clearness. Extended comments are provided for each step/component. Versions prior to “update-iqud” commit contain the original implementation.

Sobs_in is an array of spatial size, wavelength size and 4 components, or a list of two such arrays for two-line obs. The lines are subscribed via ZZ. The last dimension is always 4 corresponding to Stokes IQUV/ the output arrays are of last dimension 4 or 8 based on input. These are subscribed via iterating ZZ; 0:4 or 4:8 via (4*zz):(4*(zz+1))

CLEDB_PREPINV.CLEDB_PREPINV.objFuncStokesI(para, ii)

Objective function for obs_integrate_atmred_1pix – Reduced Chi-Squared Statistic.

CLEDB_PREPINV.CLEDB_PREPINV.obs_integrate_atmred_1pix(para)

Worker function of obs_integrate — used to reduce telluric and photospheric lines from spectra.

Calculates a forward model of spectral line observation assuming single Gaussian profile for coronal line. This function and its helpers are derivatives of the Cryo-NIRSP community data redcution and analysis tutorial: https://bitbucket.org/dkist-community-code/cryonirsp-notebooks/src/main/first_release_specFitting/

Parameters:

LSF_RPOW : Resolving power of line spread function opac : An opacity factor to modify total telluric line absorpance velS : Velocity shift of solar atlas relative to observed spectrum velT : Velocity shift of telluic atlas relative to observed spectrum strayfrac : Additional scalar fraction of straylight within spectrograph (i.e. the “veil” component) icont : Amplitude of the background continuum intensity (dominated by scattered light)

Returns:

ifit : A model fit of the observation

CLEDB_PREPINV.CLEDB_PREPINV.callBFunc(intermediate_result)

Callback function for obs_integrate_atmred_1pix — stop iterations once objective function is <0.3.

CLEDB_PREPINV.CLEDB_PREPINV.vac2air(wave_vac)

Helper function for obs_integrate — Converts wavelengths from vacuum to air-equivalent when loading fitting atlases.

CLEDB_PREPINV.CLEDB_PREPINV.gaussian_filter1d_numba(input_array, sigma)

Helper function for obs_integrate — Applies a Gaussian 1D convolution to 1D array

CLEDB_PREPINV.CLEDB_PREPINV.obs_integrate_work_1pix(xx, yy, zz, nw, wlarr, wcont, dwv, sobs_in_1pix)

Worker function for parallelizing obs_integrate tasks.

CLEDB_PREPINV.CLEDB_PREPINV.obs_qurotate(sobs_tot, keyvals, hpxygrid)

Function for rotating the linear polarization components.

The corresponding angle are using eq. 9 & 10 from Paraschiv & Judge 2022. This implicitly assumes that yobs is kept in the same units as the header keys, either R_sun or arcsec.

CLEDB_PREPINV.CLEDB_PREPINV.obs_cdf(spectr)

Helper function that computes the cdf distribution for a spectra.

CLEDB_PREPINV.CLEDB_PREPINV.obs_dens(sobs_totrot, yobs, keyvals, params)

Compute the density using two stokes I observations (serialized parallel runs for 1 pixel at a time) of Fe XIII.

This function is a stripdown of a standalone density calculation implementation found at https://github.com/arparaschiv/FeXIII-coronal-density/ The function requires a CHIANTI calculation lookup table, usually placed in CLEDB’s main directory. The lookup table can be created using a script from the above github repo.

CLEDB_PREPINV.CLEDB_PREPINV.obs_dens_work_1pix(xx, yy, a_obs, b_obs, chianti_table, y_obs)

Work helper function for parallelizing obs_dens tasks.

Compute the ratio of the observation using the 1074/1079 fraction, then interpolate to the rat component of the chianti_table. Finally recover the encoded density leading up to the ratio at a specific height.

CLEDB_PREPINV.CLEDB_PREPINV.iss_print(issuemask, tline=['fe-xiii_1074', 'fe-xiii_1079'], xx=0, yy=0, iss=0, map1=False)

Function that can be applied to the issuemask array to show punctual or map issues that were flagged during processing with two modes of running.

1. Plot all issues encountered in a specific pixel (default output).

Parameters:

issuemaskintarr

The encoded issue mask array.

tlinestrarr

The line string information from the keywords (keyvals). Default is Fe XIII 1074 and 1079.

xx and yyints

Pixel coordinates to be checked.

Returns:

iss_textstring

All codes that got flagged for location xx and yy.

2. Plot where a specific issue occurs in the entire observation map for either line.

Parameters:

issuemaskintarr

The encoded issue mask array.

issint

Which issue to check for. Consult sobs_preprocess, sdb_preprocess, or documentation at https://cledb.readthedocs.io/en/latest/outputs.html#tentative-issuemask-implementation for codes.

map1boolean

Flag to switch to map output. This needs to be set to false for one-pixel version of this function.

Returns:

iss_instintarr

A map of the physical shape of issuemask that shows where the requested iss issues manifests.

CLEDB_PREPINV.CLEDB_PREPINV.iss_block(x)

Helper function used to decode the information in the issuemask into text for *iss_print” iss_block is duplicated in both prepinv and procinv modules to avoid cross-loading the modules.

CLEDB_PREPINV.CLEDB_PREPINV.iss_text(x)

Helper function that matches one input warning code to a human readable explanation text.

Module: CLEDB_PROC

CLEDB_PROC.CLEDB_PROC.cledb_invproc(sobs_totrot, sobs_dopp, database, db_enc, yobs, aobs, dobs, snr, issuemask, dbhdr, keyvals, nsearch, maxchisq, bcalc, iqud, ncpu, reduced, verbose)

Main function for 2-line inversion the preprocessed data.

Generally, this function iterates over the observation maps, does a few sanity checks and exists if problems are encountered, then sends each pixel to the internal functions. It returns a processed observation array (IQUV or IQUD input dependent) and a matching set of input Stokes pprofiles for double checking the matching being done.

Parameters:

sobs_totrotfltarr

Observation array rotated to the CLEDB polarization frame reference.

sobs_doppfltarr

Doppler analysis resulting array. This is set to zero for IQUV configurations. It is always an input due to numba requirements.

databaselist of fltarrs

List of matched and selected databases for inversion.

db_encintarr

Map the pixel by pixel encoding of the database lists for optimized memory usage. Each pixel is assigned a unique databse, but only one instance of a required database is loaded into memory.

yobsfltarr

Array of observation heights of the shape of the input data array.

aobsfltarr

Array of linear polarization derived azimuths.

dobsfltarr

Array of computed observation plasma densities to match with the database.

snrfltarr

Observation snr statistics of the observed profile.

issuemaskfltarr

Input issuemask resulting from the upstream processing.

dbhdrlist

Metadata information of the loaded database.

keyvalslist

list of header parameters needed. These are read and processed from the observation metadata or processed depending on the input data type.

nsearch,maxchisq,bcalc,iqud,reduced,verbose: boolean, ints, floats

Passes a subset of inversion controling parameters. The entire class can not be passed to the function due to numba requirements.

Returns:

invoutfltarr

Main 2-line inversion output product. Contains the matched database index, the fitting residuals, and 9 inverted physical parameters, for all nsearch closest matching solutions with respect to the input observation. The 11 parameters are: [0] The index of the database entry that was matched at the nsearch rank. [1] The residual of the matched database entry. [2] Plasma density computed via the database. This output is applicable for the Fe XIII 1074.68/1079.79 line ratio (same ion). [3] The apparent height of the observation. [4] Position of the dominant emitting plasma along the LOS. [5] Magnetic field strength recovered via the ratio of obs over db Stokes V. [6] Magnetic field LOS angle in CLE frame. [7] Magnetic field POS Azimuth angle in CLE frame. [8] Bx cartesian projected magnetic field depth/LOS component. [9] By cartesian projected magnetic field horizontal component. [10] Bz cartesian projected magnetic field vertical component.

sfoundfltarr

Returns the set of nsearch derotated and matched Stokes IQUV sets from the database. These can be compared to the input Stokes observation.

CLEDB_PROC.CLEDB_PROC.cledb_invproc_time(sobs_totrot, sobs_dopp, database, db_enc, yobs, aobs, snr, dbhdr, keyvals, nsearch, maxchisq, bcalc, iqud, ncpu, reduced, verbose)

A minimal wrapper that enables timing for CLEDB_INVERT. Timing can no be directly included in function due to full non-python numba compilation.

CLEDB_PROC.CLEDB_PROC.blos_proc(sobs_tot, snr, issuemask, keyvals, consts, params)

This function calculates the line-of-sight magnetic components from STOKES IQUV observations in one line.

This uses the “improved” magnetograph formulation discussed in eq 40 Casini &Judge 99, eq 14 of Plowman 2014, and eq 17 and 18 of Dima & Schad 2020 Follows the discussion in the three papers and adopts different analythical implementations based on the line combination used. As shown in the papers the magnetograph formulation is not precise in terms of recovering the LOS magnetic field. Differences of the order of plus-minus 2 times actual values based on the atomic alignment, F factor, and LOS angle theta can manifest.

The function will produce a set of 2 times degenerate magnetograph along with a classic magnetograph and a field azimuth for each ingested line/observation.

Parameters:

sobs_totfltarr

Observation array. For this purpose this does not need a rotation to the CLEDB polarization frame reference.

snrfltarr

Observation snr statistics of the observed profile

issuemaskfltarr

Input issuemask resulting from the upstream processing.

keyvalslist

List of parameters needed. These are read and processed from the observation metadata or processed depending on the input data type. Generated by the sobs_preprocess*<–*obs_headstructproc functions.

consts

Class containing useful constants that is imported.

params

Class containing inversion control parameters that is imported.

Returns:

blosoutfltarr

Array containint the LOS magnetic field calculation for each pixel. 4 output products for erach line are generated: [0] First degenerate constrained magnetograph solution for each respective line. [1] Second degenerate constrained magnetograph solution for each respective line. [2] “Classic” magnetograph solution for each respective line. Values lie in between the two above degenerate solutions. [3] Magnetic field azimuth angle derived from the Q and U linear polarization components of the respective line.

CLEDB_PROC.CLEDB_PROC.spectro_proc(sobs_in, sobs_tot, snr, issuemask, background, wlarr, keyvals, consts, params)

Ingests the fully prepped data from and calculates spectroscopic outputs for each input line.

This module requires data in the formats as resulting from the CLEDB_PREPINV module. This is a computationally demanding and time consuming function. Optimized input data type sub-modules are envisioned to be integrated into this processing in the future. Currently only a basic/principial approach is implemented. Part of the outputs are used downstream in BLOS_PROC or CLEDB_INVPROC.

Spectro_proc will not run for line integrated data inputs.

Parameters:

sobs_infltarr

Input stokes array that contains the spectral data to be fitted.

sobs_totfltarr

Observation array. For this purpose this does not need a rotation to the CLEDB polarization frame reference.

snrfltarr

Observation snr statistics of the observed profile

backgroundfltarr

Background threshold values. These are used both as an output product and as a preprocess parameter. More details in the cdf_statistics comments.

issuemaskfltarr

Input issuemask resulting from the upstream processing.

wlarrlist of fltarr

List containing a set of wavelength dispersion axie for each observed line.

keyvalslist

List of data header parameters needed. These are read and processed from the observation metadata or processed depending on the input data type.

consts

Class containing useful constants that is imported.

params

Class containing inversion control parameters that is imported.

Returns:

specoutfltarr

Array containint the spectroscopic calculations for each pixel. The dimension outputs are: [0] Wavelength position of the line core. [1] Doppler shift with respect to the theoretical line core defined in the constants class line_ref key. [2] Doppler shift with respect to the theoretical line core defined in the constants class line_ref key. [3:6] Intensity at computed line center wavelength for Stokes I, Stokes Q and U. [6] Intensity at lobe maximum for Stokes V. The signed “core” counts are measured in the core of the absolute strongest lobe. [7] Averaged background intensity outside the line profile for the Stokes I component. [8] Total line FWHM. [9] Non-thermal component of the FWHM line width assuming peak line formation temperature. [10] Fraction of linear polarization Pl with respect to the total Stokes I counts. [11] Fraction of total polarization (linear + circular) Pv with respect to the total Stokes I counts.

CLEDB_PROC.CLEDB_PROC.cdf_statistics(zz, xx, yy, sobs_cal, sobs_tot_1pix, background, wlarr_1pix, keyvals, const, gaussfit, verbose)

Helper function to parallelize computing the statistics of the Stokes I profiles for each pixel.

This is run for each ion/line. Outside loop controls which ion is fed.

Returns:

Line core wavelength. Line shift from reference position. Shift converted to velocities. Stokes I,Q,U, and V linecore intensity. Background intensity of stokes I. Total line width. Non-thermal component of the line width. Fraction of linear polarization and total polarization.

CLEDB_PROC.CLEDB_PROC.blos_proc_1pix(zz, xx, yy, sobs_tot_1pix, const, tline)

Helper compute function for parallelizing the blos_proc function.

CLEDB_PROC.CLEDB_PROC.cledb_matchiquv(xx, yy, sobs_1pix, yobs_1pix, aobs_1pix, dobs_1pix, database_in, dbhdr, snr, nsearch, maxchisq, bcalc, reduced, verbose)

The main solver for the geometry and magnetic field strength in each pixel — version for full Stokes vector.

Returns:

matched database index chi^2 fitting residual The matched observation physics:

Y(radial) position and X(LOS) position; varphi and vartheta angles in CLE geometry; Bx, By, Bx in LOS geometry (transform to cartesian); B field strength.

double line IQUV vector

CLEDB_PROC.CLEDB_PROC.cledb_matchiqud(xx, yy, sobs_1pix, sobsd_1pix, yobs_1pix, aobs_1pix, dobs_1pix, database_in, dbhdr, snr, nsearch, maxchisq, bcalc, reduced, verbose)

The main solver for the geometry and magnetic field strength in each pixel — version for partial Stokes vector.

Returns:

matched database index chi^2 fitting residual The matched observation physics:

Y(radial) position and X(LOS) position; varphi and vartheta angles in CLE geometry; Bx, By, Bx in LOS geometry (transform to cartesian); B field strength.

double line IQUV vector

CLEDB_PROC.CLEDB_PROC.cledb_getsubsetiquv(sobs_1pix, dbhdr, database_in, nsearch)

Helper function to return a subset of the database compatible with the yobs observed height.

CLEDB_PROC.CLEDB_PROC.cledb_getsubsetiqud(sobs_1pix, sobsd_1pix, dbhdr, database_in, nsearch)

Helper function to return a subset of the database compatible with the yobs observed height.

CLEDB_PROC.CLEDB_PROC.cledb_quderotate(dbarr, ang, nf)

Derotates the matched database entry to make it compatible with the observation.

Uses same Mueller transform as obs_qurotate, but applied with the oposite angle to compare the matched database stokes profiles with the observed ones.

CLEDB_PROC.CLEDB_PROC.cledb_partsort(arr, nsearch)

Helper function that produces a fast output similar to np.argpartition with sort.

Performs the simplest possible substitution partial sort. It just returns the first nsearch sorted indexes. The rest of the array remains unsorted.

CLEDB_PROC.CLEDB_PROC.cledb_params(index, dbcgrid, dbtype)

This function is used with the database header to help compute the physics associated to the i, j, k, l entry

For input index, get the i,j,k,l indices in a database.

CLEDB_PROC.CLEDB_PROC.cledb_invparams(i, j, k, l, dbcgrid)

Reverse function of cledb_params. This function is used with the database header to help compute the physics associated with a index entry.

For input i,j,k,l indices in database, get index

CLEDB_PROC.CLEDB_PROC.cledb_elecdens(r)

Computes an electron radial density estimation using the Baumbach formulation, using the Allen Astrophysical quantities.

CLEDB_PROC.CLEDB_PROC.cledb_phys(index, yobs_1pix, dobs_1pix, dbhdr, bcalc, b)

Helper function that returns the CLE and Obs. geometry and magnetic field physics.

This is kept separate from cledb_physcle because it computes projections transformations of the database variables recovered via cledb_physcle.

CLEDB_PROC.CLEDB_PROC.cledb_physcle(index, yobs_1pix, dobs_1pix, dbhdr)

Helper function that returns the CLE frame physics and compatible observation geometry.

This is the primary function that retrieves physics parameters from the database index.

CLEDB_PROC.CLEDB_PROC.obs_cdf(spectr)

Helper that computes the cdf distribution for a spectra.

CLEDB_PROC.CLEDB_PROC.obs_gaussfit(x, ymax, mean, sigma, offset)

Gaussian Function for spectral fitting.