NRL Evaluated Plasma Chemistry (NEPC)

nepc.nepc

This package provides the following functionality for NRL Evaluated Plasma Chemistry (NEPC) style databases:

  • create data files for building a NEPC database from common sources (e.g. LXCat)

  • build a NEPC style database on a MySQL server

  • establishing a connection to a local or remote database

  • access cross section data via the CS class

  • access pre-defined plasma chemistry models via the Model class

  • curate, visualize, and use cross section data

  • perform exploratory data analysis (EDA) of cross section data

  • print statistics about a database (e.g. number of rows in various tables)

Examples

Establish a connection to the database named nepc running on a production server:

>>> cnx, cursor = nepc.connect()

Establish a connection to the database named nepc running on the local machine:

>>> cnx, cursor = nepc.connect(local=True, test=True)

Access the pre-defined plasma chemistry model, fict, in the nepc_test database:

>>> fict = nepc.Model(cursor, "fict")

Print a summary of the fict model, including a stylized Pandas dataframe:

>>> fict.summary()

Additional examples of EDA using nepc are in tests/data/eda. Examples of methods for building data files for the nepc_test database, including parsing LXCat formatted data, are in tests/data/methods.

class nepc.nepc.CS(cursor, cs_id)

A cross section data set, including metadata and cross section data, from a NEPC MySQL database. :param cursor: A MySQLCursor object. See return value cursor of connect(). :type cursor: cursor.MySQLCursor :param cs_id: i.d. of the cross section in cs and csdata tables :type cs_id: int

metadata
cs_idint

id of the cross section in cs and csdata tables

processstr

name of process from processes table

units_efloat

units of electron energy list e in eV

units_sigmafloat

units of cross section list sigma in m^2

refstr

ref from cs table corresponding to entry in ‘[nepc]/models/ref.bib’

lhsAstr

name of lhsA state from states table

lhsBstr

name of lhsB state from states table

rhsAstr

name of rhsA state from states table

rhsBstr

name of rhsB state from states table

wavelengthfloat

wavelength of photon involved in process in nanometers (nm)

lhs_vint

vibrational energy level of lhsA

rhs_vint

vibrational energy level of rhsA

lhs_jint

rotational energy level of lhsA

rhs_jint

rotational energy level of rhsA

backgroundstr

background text describing origin of data and other important info

lpufloat

lower percent uncertainty

upufloat

upper percent uncertainty

lhsA_longstr

long_name of lhsA state from states table

lhsB_longstr

long_name of lhsB state from states table

rhsA_longstr

long_name of rhsA state from states table

rhsB_longstr

long_name of rhsB state from states table

e_on_lhsint

number of electrons on lhs

e_on_rhsint

number of electrons on rhs

hv_on_lhsint

photon on lhs? (0 or 1)

hv_on_rhsint

photon on rhs? (0 or 1)

v_on_lhsint

vibrational energy level on lhs? (0 or 1)

v_on_rhsint

vibrational energy level on rhs? (0 or 1)

j_on_lhsint

rotational energy level on lhs? (0 or 1)

j_on_rhsint

rotational energy level on rhs? (0 or 1)

Type

dict

data
elist of float

Electron energies in units of units_e eV (see CS.metadata).

sigmalist of float

Cross sections in units of units_sigma \(m^2\) (see CS.metadata).

Type

dict

__len__()

The number of data points in the cross section data set

__str__()

Provides a list of certain metadata: specie, process, threshold, ref (if set), text-formatted reaction, background.

plot(units_sigma=1e-20, plot_param_dict={'linewidth': 1}, xlim_param_dict={'auto': True}, ylim_param_dict={'auto': True}, ylog=False, xlog=False, show_legend=True, filename=None, width=10, height=10)

Plot a single cross section data set.

Parameters
  • units_sigma (float, optional) – desired units of the y-axis in \(m^2\)

  • plot_param_dict (dict, optional) – kwargs to pass to matplotlib.axes.Axes.plot()

  • xlim_param_dict (dict, optional) – kwargs to pass to matplotlib.axes.Axes.set_xlim()

  • ylim_param_dict (dict, optional) – kwargs to pass to matplotlib.axes.Axes.set_ylim()

  • ylog (bool, optional) – whether y-axis is log scale (default is False)

  • xlog (bool, optional) – whether x-axis is log scale (default is False)

  • show_legend (bool, optional) – whether to display the legend or not (default is True)

  • filename (str, optional) – filename for output, if provided (default is to not output a file)

  • width (float, optional) – width of plot

  • height (float, optional) – height of plot

Returns

Plot of the cross section data, CS.data, with formatting using information in the metadata, CS.metadata.

Return type

matplotlib.axes.Axes

reaction_text_side(side, latex=False)

Return the LaTeX for the LHS of the process involved in a nepc cross section. :param side: ‘LHS’ or ‘RHS’ to indicate which side of the reaction to return. :type side: str :param latex: Determine whether or not to output the LaTeX formatted text. :type latex: bool, default False

Returns

The plain text for either the LHS or RHS of the process involved in a NEPC cross section (LaTeX) formatted if indicated.

Return type

str

class nepc.nepc.CustomCS(cursor=None, cs_id=None, metadata=None, data=None)

Extends CS to provide a custom cross section data set.

If building upon an existing CS, must provide cursor and cs_id as well as one or both of metadata and data.

If building a CustomCS from scratch, must provide metadata and data.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • cs_id (int) – i.d. of the cross section in cs and csdata tables

  • metadata (dict) – one or more of the attributes of CS

  • data (dict) – same as attributes of CS

metadata

see Attributes of CS

Type

dict

data
elist[float]

electron energy

sigmalist[float]

cross section

Type

dict

class nepc.nepc.CustomModel(cursor=None, model_name=None, cs_id_list=[], cs_list=[], metadata=None)

Extends Model to provide a customized collection of cross sections.

The cross sections can be of class CS (from the NEPC database) or CustomCS (user created/modified). Options:

  1. If building upon an existing NEPC model, must provide cursor and model_name. May also provide a filter to select cross sections that meet criteria.

  2. If building from existing cross sections, must provide cursor and cs_id_list.

  3. If building from a list of custom cross sections, must provide cs_list.

Must do at least one of a, b, or c, and can do any combination thereof.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • model_name (str) – The name of a NEPC model (see [nepc.wiki]/models

  • cs_id_list (list of int) – List of cs_id’s to pull from NEPC database.

  • cs_list (list of CustomCS) – List of user-defined cross sections of CustomCS type.

  • metadata (dict) – Dictionary of filter criteria to select specific CS’s from a Model.

cs

A list of cross section data and metadata of CS or CustomCS type.

Type

list of CS or CustomCS

class nepc.nepc.Model(cursor, model_name)

A pre-defined collection of cross sections from a NEPC database

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • model_name (str) – Name of a NEPC model (pre-defined collection of cross sections)

cs

cross section data in the NEPC format (CS)

Type

list[CS]

unique

set with Model.set_unique, all unique electron energies in all CS.data of the Model

Type

list[float]

__len__()

number of cross sections in the model

plot(units_sigma=1e-20, process='', plot_param_dict={'linewidth': 1}, xlim_param_dict={'auto': True}, ylim_param_dict={'auto': True}, ylog=False, xlog=False, show_legend=True, filename=None, max_plots=10, width=10, height=10)

Plot cross section data in the Model.

Parameters
  • process (str) – If provided, the process that should be plotted.

  • units_sigma (float) – Desired units of the y-axis in m^2.

  • plot_param_dict (dict) – kwargs to pass to ax.plot

  • xlim(ylim)_param_dict (dict) – dictionary of kwargs to pass to ax.set_x(y)lim

  • ylog (bool) – whether y-, x-axis is log scale

  • xlog (bool) – whether y-, x-axis is log scale

  • show_legend (bool) – whether to display the legend or not

  • filename (str) – filename for output, if provided (default is to not output a file)

  • max_cs (int) – maximum number of CS to put on graph

Returns

Plot of the cross section data, CS.data, with formatting using information in the metadata, CS.metadata within the model.

Return type

matplotlib.axes.Axes

set_unique()

sets Model.unique

subset(metadata=None)

Select the cross sections in the model matching the provided metadata

Parameters

metadata (dict) – see CS.metadata

Returns

cs_subset – cross section data in the NEPC format (CS)

Return type

list[CS]

summary(metadata=None, lower=None, upper=None, sort=[])

Summarize the NEPC model.

Prints the following information:
  • Number of cross sections in the model

  • Number of cross sections matching metadata, if provided

Returns a stylized Pandas dataframe with headers given by:

headers = [“cs_id”, “lhsA”, “rhsA”, “process”,

“reaction”, “threshold”, “E_peak”, “E_upper”, “sigma_max”, “lpu”, “upu”]

Parameters
  • metadata (dict) – see CS.metadata

  • lower (int) – lower bound of model index to include in summary

  • upper (int) – upper bound of model index to include in summary

  • sort (list[str]) – headers by which the stylized Pandas table is sorted

Returns

cs_df – A stylized Pandas DataFrame containing the cs_id, process, range of electron energies (E_lower, E_upper), maximum sigma (sigma_max), and lpu/upu’s for each cross section in the model (or subset of the model if metadata is provided)

Return type

pandas.io.formats.style.Styler

nepc.nepc.connect(local=False, DBUG=False, test=False, github=False)

Establish a connection to a NEPC MySQL database

Parameters
  • local (bool, optional) – Access a database on localhost; otherwise use the production server (default False).

  • DBUG (bool, optional) – Print debug info (default False).

  • test (bool, optional) – If true, access the nepc_test database; otherwise, connect to the nepc database.

  • github (bool, optional) – If true, connect to database on GitHub

Returns

  • cnx (connection.MySQLConnection) – A connection to a NEPC MySQL database.

  • cursor (cursor.MySQLCursor) – A MySQLCursor object that can execute operations such as SQL statements. cursor interacts with the NEPC server using the cnx connection.

nepc.nepc.count_table_rows(cursor, table: str)

Return the number of rows in a MySQL table.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • table (str) – Name of a table in the NEPC database at cursor.

Returns

Number of rows in table.

Return type

int

nepc.nepc.cs_e(cursor, cs_id)

Get the electron energies for a cross section dataset in a NEPC database corresponding to a given cs_id.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • cs_id (int) – The cs_id for a cross section dataset in the NEPC database at cursor.

Returns

Electron energies for the cross section dataset corresponding to cs_id.

Return type

list of float

nepc.nepc.cs_e_sigma(cursor, cs_id)

Get electron energy and cross section data for a given cs_id in a NEPC database.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • cs_id (int) – The cs_id for a cross section dataset in the NEPC database at cursor.

Returns

  • list of float – Electron energies for the cross section dataset corresponding to cs_id.

  • list of float – Cross sections for the cross section dataset corresponding to cs_id.

nepc.nepc.cs_metadata(cursor, cs_id)

Get metadata for a given cs_id in a NEPC database.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • cs_id (int) – The cs_id for a cross section dataset in the NEPC database at cursor.

Returns

See CS.metadata. List items are in same order as CS.metadata.

Return type

list

nepc.nepc.cs_sigma(cursor, cs_id)

Get the cross sections for a cross section dataset in a NEPC database corresponding to a given cs_id.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • cs_id (int) – The cs_id for a cross section dataset in the NEPC database at cursor.

Returns

Cross sections for the cross section dataset corresponding to cs_id.

Return type

list of float

nepc.nepc.model_cs_id_list(cursor, model_name)

Get a list of cs_id’s for a model in a NEPC database.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • model_name (str) – Name of a model in the NEPC MySQL database

Returns

cs_id_list – cs_id’s corresponding to cross sections in the model

Return type

list of int

nepc.nepc.table_as_df(cursor, table, columns='*')

Return a table in a MySQL database as a pandas DataFrame.

Parameters
  • cursor (cursor.MySQLCursor) – A MySQLCursor object. See return value cursor of connect().

  • table (str) – Name of a table in the NEPC database at cursor.

  • columns (list of str, optional) – Which columns to get. (Default is to get all columns.)

Returns

Table in the form of a pandas DataFrame

Return type

DataFrame

nepc.util.util

Utility methods for the nepc module.

nepc.util.util.get_size(obj, seen=None)

Recursively find the size of an object.

nepc.util.util.mkdir(outdir)

Try to make a directory and raise an exception if the directory already exists

Parameters

outdir (str) – The directory to be made.

nepc.util.util.rmdir(outdir)

Try to remove a tree and raise an exception if this isn’t possible.

Parameters

outdir (str) – The directory to be removed.

nepc.util.util.wc_fxn(file_to_count)

Return the number of lines in a file using the commandline utility wc.

nepc.util.config

Get environment variables used by nepc scripts.

nepc.util.config.nepc_cs_home()

Returns the path to the nepc_cs directory stored in NEPC_CS_HOME environment variable.

nepc.util.config.nepc_home()

Returns the path to the nepc directory stored in the NEPC_HOME environment variable.

nepc.util.config.production()

Returns the I.P. address or URL of the production server stored in the NEPC_PRODUCTION environment variable.

nepc.util.config.user_home()

Returns the user’s home directory.

nepc.util.parser

Adapted from BOLOS (https://github.com/aluque/bolos). Adaptations:
  • replaced ‘next()’ with ‘__next__()’ for Python 3

  • broke out large comment to add separate dictionary entries for process, param, species, etc.

Notes

This module contains the code required to parse BOLSIG+-compatible files. To make the code re-usabe in other projects it is independent from the rest of the BOLOS code.

Most users would only use the method parse() in this module, which is documented below.

nepc.util.parser.get_next_ids(test=False)

Get the next ids from a file

Parameters

test (bool, optional) – If true, the command is run for the nepc_test database. Otherwise, it is run for the nepc database. (Default is the nepc database.)

Returns

  • int – Next cs_id to use

  • int – Next csdata_id to use

nepc.util.parser.get_states(test=False)

Get lists of name’s and long_name’s from states.tsv file.

nepc.util.parser.parse(filename, has_arg=True, debug=False)

Parses a BOLSIG+ cross-sections file.

Parameters
  • filename (str) – Name of a Bolsig+-compatible cross-sections file.

  • has_arg (bool, optional) – Whether threshold values are present in each block

Returns

processes – A list with all processes, in dictionary form, included in the file.

Return type

list of dict

Notes

This function does not return process.Process instances so that the parser is independent of the rest of the code and can be re-used in other projects. If you want to convert a process in dictionary form d to a process.Process instance, use

>>> process = process.Process(**d)
nepc.util.parser.remove_crs(mystring)

Removes new lines

nepc.util.parser.text_array_to_float_array(text_array, omit_regexp='')

Convert an array of strings to an array of floats

Parameters
  • text_array (str array) –

  • converted (An array of strings to be) –

Returns

  • data (float array)

  • An array of floats containing float-converted data

  • from text_array

nepc.util.parser.write_cs_to_file(filename, data_array, cs_id, start_csdata_id, process, units_e, units_sigma, ref='\\N', lhs_a='\\N', lhs_b='\\N', rhs_a='\\N', rhs_b='\\N', threshold='-1', wavelength='-1', lhs_v=- 1, rhs_v=- 1, lhs_j=- 1, rhs_j=- 1, background='\\N', lpu='-1', upu='-1')

Write both the cross-section data and metadata to a file :param Contains a combination of parameters from write_data_to_file and: :param write_metadata_to_file:

nepc.util.parser.write_csdata_to_file(data_dict, filename: str, start_csdata_id)

Given a dictionary of electron energies and corresponding cross sections, write the data to a file in the correct format for a NEPC database.

Parameters
  • data_dict (dict) – A dictionary where ‘e’ is a numpy array of electron energies, and ‘sigma’ is a numpy array of cross sections.

  • filename (str) – The filename for the .dat file.

  • start_csdata_id (int) – The first csdata_id for the supplied data.

Returns

The next csdata_id to use.

Return type

int

nepc.util.parser.write_data_to_file(data_array, filename, start_csdata_id)

Given an array of electron energies and corresponding cross sections, write the data to a file in the correct format for a NEPC database.

Parameters
  • data_array (numpy.ndarray) – A numpy array of values to be entered into the filenumpy array. Each row is of length 2 (e, sigma).

  • filename (str) – Name of the file where values of data_array should be written.

  • start_csdata_id (int) – The first csdata_id for the supplied data.

Returns

The next csdata_id to use.

Return type

int

nepc.util.parser.write_metadata_to_file(filename, cs_id, process, units_e, units_sigma, ref='\\N', lhs_a='\\N', lhs_b='\\N', rhs_a='\\N', rhs_b='\\N', threshold='-1', wavelength='-1', lhs_v=- 1, rhs_v=- 1, lhs_j=- 1, rhs_j=- 1, background='\\N', lpu='-1', upu='-1')

Given metadata for a cross section data set, write the metadata to a file in the correct format for a NEPC database.

Parameters
  • filename (str) – Name of the file where values of data_array should be written.

  • cs_id (int) – cs_id corresponding to the cross section in the database.

  • process (str) – The short name corresponding to the electron scattering process in the database.

  • units_e (float) – The units, in eV, of the electron energies.

  • units_sigma (float) – The units, in \(m^2\), of the cross sections.

  • ref (str) – The short name for the reference corresponding to the dataset.

  • lhs_a (str) – The short name for the lhs_a state for the process.

  • lhs_b (str) – The short name for the lhs_b state.

  • rhs_a (str) – The short name for the rhs_a state.

  • rhs_b (str) – The short name for the rhs_b state.

  • threshold (float) – The threshold electron energy for the process.

  • wavelength (float) – The wavelength of the photon associated with the process, if applicable.

  • lhs_v (int) – The vibrational energy associated with the LHS state, if applicable.

  • rhs_v (int) – The vibrational energy associated with the RHS state, if applicable.

  • lhs_j (int) – The rotational energy associated with the LHS state, if applicable.

  • rhs_j (int) – The rotational energy associated with the RHS state, if applicable.

  • background (str) – Background information for the cross section data.

  • lpu (float) – Lower percent uncertainty of the cross section data.

  • upu (float) – Upper percent uncertainty of the cross section data.

Returns

The next cs_id to use.

Return type

int

nepc.util.parser.write_models_to_file(filename, models_array)

Write model metadata to a file.

Parameters
  • filename (str) – The name of the file where the model data should be stored.

  • models_array (list of str) – A list of models that should be added to filename.

nepc.util.parser.write_next_id_to_file(next_cs_id, next_csdata_id, test=False)

Write out the next id’s for the database to a file.

Parameters
  • next_cs_id (int) – The next cs_id to use

  • next_csdata_id (int) – The next csdata_id to use

  • test (bool, optional) – If true, the command is run for the nepc_test database. Otherwise, it is run for the nepc database. (Default is the nepc database.)

nepc.util.parser.write_pd_data_to_file(data_frame, filename, start_csdata_id)

Given a dataframe of csdata, write this to a file in the correct format :param data_frame: :type data_frame: pandas DataFrame :param A DataFrame containing cross section data to be entered into the file: :param filename: :type filename: file :param Name of the file where values of data_frame should be entered: :param start_cs_data_id: :type start_cs_data_id: int :param The id where the data should be placed:

Returns

  • csdata_id

  • The next csdata_id to use

Other Topics

Follow along with nepc development at https://github.com/USNavalResearchLaboratory/nepc.

Example exploratory data analysis with nepc can be found here: https://github.com/USNavalResearchLaboratory/nepc/blob/master/tests/data/eda/01_lxcat_n2_fict.ipynb

Indices and tables