nems.recording module

class nems.recording.Recording(signals, meta=None, name=None, signal_views=None)[source]

Bases: object

__init__(signals, meta=None, name=None, signal_views=None)[source]

Signals argument should be a dictionary of signal objects.

copy()[source]

Returns a copy of this recording.

property epochs

The epochs of a recording is the superset of all signal epochs.

__getitem__(key)[source]
__setitem__(key, val)[source]
set_view(view_idx=0)[source]

choose a different view, typically a different masking for jackknifing. returns a shallow copy of the recording, signals preserved in place

view_subset(view_range)[source]

shallow copy recording, preserving a subset of views view_range - list of view ids to keep

views(view_range=None)[source]
property view_count

return how many views exist in this recording

tile_views(view_count=1)[source]

repeat current signals dict view_count times in self.signal views returns a shallow copy of the recording, signals preserved in place

static load(uri)[source]

DEPRECATED??? REPLACED by regular functions?

Loads from a local .tgz file, a local directory, from s3, or from an HTTP URL containing a .tgz file. Examples:

# Load all signals in the gus016c-a2 directory rec = Recording.load(‘/home/myuser/gus016c-a2’) rec = Recording.load(’file:///home/myuser/gus016c-a2’)

# Load the local tar gz directory. rec = Recording.load(’file:///home/myuser/gus016c-a2.tgz’)

# Load a tgz file served from a flat filesystem rec = Recording.load(’http://potoroo/recordings/gus016c-a2.tgz’)

# Load a tgz file created by the nems-baphy interafce rec = Recording.load(’http://potoroo/baphy/271/gus016c-a2’)

# Load from S3: rec = Recording.load(‘s3://nems.lbhb… TODO’)

static load_dir(directory_or_targz)[source]

Loads all the signals (CSV/JSON pairs) found in DIRECTORY or .tgz file, and returns a Recording object containing all of them. DEPRECATED???

static load_targz(targz)[source]

Loads the recording object from a tgz file. DEPRECATED???

static load_url(url)[source]

Loads the recording object from a URL. File must be tgz format. DEPRECATED???

static load_from_arrays(arrays, rec_name, fs, sig_names=None, signal_kwargs={})[source]

DEPRECATED??? Generates a recording object, and the signal objects it contains, from a list of array-like structures of the form channels x time (see signal.py for more details about how arrays are represented

by signals).

If any of the arrays are more than 2-dimensional, an error will be thrown. Also pay close attention to any RuntimeWarnings from the signal class regarding improperly-shaped arrays, which may indicate that an array was passed as time x channels instead of the reverse.

arrayslist of array-like

The data to be converted to a recording of signal objects. Each item should be 2-dimensional and convertible to a numpy ndarray via np.array(x). No constraints are enforced on the dtype of the arrays, but in general float values are expected by most native NEMS functions.

rec_namestr

The name to be given to the new recording object. This will also be assigned as the recording attribute of each new signal.

fsint or list of ints

The frequency of sampling of the data arrays - used to interconvert between real time and time bins (see signal.py). If int, the same fs will be assigned to each signal. If list, the length must match the length of arrays.

sig_nameslist of strings (optional)

Name to attach to the signal created from each array. The length of this list should match that of arrays. If not specified, the signals will be given the generic names: [‘signal1’, ‘signal2’, …].

signal_kwargslist of dicts

Keyword arguments to be passed through to each signal object. The length of this list should match the length of arrays, and may be padded with empty dictionaries to ensure this constraint. For example:

[{‘chans’: [‘1 kHz’, ‘3 kHz’]}, {‘chans’: [‘one’, ‘two’]}, {}]

Would assign channel names ‘1 kHz’ and ‘3 kHz’ to the signal for the first array, ‘one’ and ‘two’ for the second array, and no channel names (or any other arguments) for the third array.

Valid keyword arguments are: chans, epochs, meta,

and safety_checks

recrecording object

New recording containing a signal for each array.

classmethod from_nwb(nwb_file, nwb_format)[source]

The NWB (Neurodata Without Borders) format is a unified data format developed by the Allen Brain Institute. Data is stored as an HDF5 file, with the format varying depending how the data was saved.

References

Parameters
  • nwb_file – path to the nwb file

  • nwb_format – specifier for how the data is saved in the container

Returns

a recording object

save(uri='', uncompressed=False)[source]

Saves this recording to a URI as a compressed .tgz file. Returns the URI of what was saved, or None if there was a problem.

Optional argument ‘uncompressed’ may be used to force the save to occur as a directory full of uncompressed files, but this only works for URIs that point to the local filesystem.

For example:

# Save to a local directory, use automatic filename rec.save(‘/home/username/recordings/’)

# Save it to a local file, with a specific name rec.save(‘/home/username/recordings/my_recording.tgz’)

# Same, but with an explicit file:// prefix rec.save(’file:///home/username/recordings/my_recording.tgz’)

# Save it to the nems_db running on potoroo, use automatic filename rec.save(’http://potoroo/recordings/’)

# Save it to the nems_db running on potoroo, specific filename rec.save(’http://potoroo/recordings/my_recording.tgz’)

# Save it to AWS (TODO, Not Implemented, Needs credentials) rec.save(‘s3://nems.amazonaws.com/somebucket/’)

save_dir(directory)[source]

Saves all the signals (CSV/JSON pairs) in this recording into DIRECTORY in a new directory named the same as this recording.

save_targz(uri)[source]

Saves all the signals (CSV/JSON pairs) in this recording as a .tgz file at a local URI.

as_targz()[source]

Returns a BytesIO containing all the rec’s signals as a .tgz stream. You may either send this over HTTP or save it to a file. No temporary files are created in the creation of this .tgz stream.

Example of saving an in-memory recording to disk:

rec = Recording(…) with open(‘/some/path/test.tgz’, ‘wb’) as fh:

tgz = rec.as_targz() fh.write(tgz.read()) tgz.close() # Don’t forget to close it!

save_url(uri, compressed=False)[source]

Saves this recording to a URL. Returns the URI if it succeeded, else None. Check the return code to see if the URL save failed and you need to save locally instead. e.g.

# Example: Try to save remotely, or save locally if it fails if not rec.save_url(url):

rec.save(‘/tmp/’) # Save to /tmp as a fallback

_save_metadata(md_fh, fmt='%.18e')[source]

Save this signal to a CSV file + JSON sidecar. If desired, you may use optional parameter fmt (for example, fmt=’%1.3e’) to alter the precision of the floating point matrices.

get_signal(signal_name)[source]

Returns the signal object with the given signal_name, or None if it was was found.

signal_name should be a string

add_signal(signal)[source]

Adds the signal equal to this recording. Any existing signal with the same name will be overwritten. No return value.

_split_helper(fn)[source]

For internal use only by the split_* functions.

split_at_time(fraction)[source]

Calls .split_at_time() on all signal objects in this recording. For example, fraction = 0.8 will result in two recordings, with 80% of the data in the left, and 20% of the data in the right signal. Useful for making est/val data splits, or truncating the beginning or end of a data set.

FOR silly reasons having to do with the ordering of val stimuli,

“r” is actually the beginning of the signal – used for val “l” is the end, used for est

split_by_epochs(epochs_for_est, epochs_for_val)[source]

Returns a tuple of estimation and validation data splits: (est, val). Arguments should be lists of epochs that define the estimation and validation sets. Both est and val will have non-matching data NaN’d out.

split_using_epoch_occurrence_counts(epoch_regex=None, keepfrac=1, filemask=None, verbose=False, **context)[source]

Returns (est, val) given a recording rec, a signal name ‘stim_name’, and an epoch_regex that matches ‘various’ epochs. This function will throw an exception when there are not exactly two values for the number of epoch occurrences; i.e. low-rep epochs and high-rep epochs.

keepfrac: if <1: save only keepfrac fraction of the trials

NOTE: This is a fairly specialized function that we use in the LBHB lab. We have found that, given a limited recording time, it is advantageous to have a variety of sounds presented to the neuron (i.e. many low-repetition stimuli) for accurate estimation of its parameters. However, during the validation process, it helps to have many repetitions of the same stimuli so that we can more accurately estimate the peri- stimulus time histogram (PSTH). This function tries to split the data into those two data sets based on the epoch occurrence counts.

get_epoch_indices(epoch_name, allow_partial_epochs=False)[source]
jackknife_mask_by_epoch(njacks, jack_idx, epoch_name, tiled=True, invert=False, allow_partial_epochs=False)[source]
Creates mask or updates existing mask, with subset of epochs

matching epoch_name set to False

Optional argument ‘invert’ causes everything BUT the matched epochs to be NaN’d. njacks determines the number of jackknifes to divide the epochs into, and jack_idx determines which one to return.

‘Tiled’ makes each jackknife use every njacks’th occurrence, and is probably best explained by the following example…

If there are 18 occurrences of an epoch, njacks=5, invert=False, and tiled=True, then the five jackknifes will have these epochs NaN’d out:

jacknife[0]: 0, 5, 10, 15 jacknife[1]: 1, 6, 11, 16 jacknife[2]: 2, 7, 12, 17 jacknife[3]: 3, 8, 13 jacknife[4]: 4, 9, 14

Note that the last two jackknifes have one fewer occurrences.

If tiled=False, then the pattern of NaN’d epochs becomes sequential:

jacknife[0]: 0, 1, 2, 3 jacknife[1]: 4, 5, 6, 7, jacknife[2]: 8, 9, 10, 11, jacknife[3]: 12, 13, 14, 15, jacknife[4]: 16, 17

Here we can see the last jackknife has 2 fewer occurrences.

jackknife_masks_by_epoch(njacks, epoch_name, tiled=True, invert=False, allow_partial_epochs=False)[source]
jackknife_mask_by_time(njacks, jack_idx, tiled=True, invert=False)[source]

To function in place of jackknife_mask_by_epoch for cases where you wish to fit all data evenly, including that which is not contained in an epoch mask.

jackknife_masks_by_time(njacks, tiled=True, invert=False)[source]
jackknife_by_epoch(njacks, jack_idx, epoch_name, tiled=True, invert=False, only_signals=None, excise=False)[source]

By default, calls jackknifed_by_epochs on all signals and returns a new set of data. If you would only like to jackknife certain signals, while copying all other signals intact, provide their names in a list to optional argument ‘only_signals’.

DEPRECATED???– use masks

jackknife_by_time(nsplits, split_idx, only_signals=None, invert=False, excise=False)[source]

By default, calls jackknifed_by_time on all signals and returns a new set of data. If you would only like to jackknife certain signals, while copying all other signals intact, provide their names in a list to optional argument ‘only_signals’.

DEPRECATED??? – use masks

jackknifes_by_epoch(nsplits, epoch_name, only_signals=None)[source]
jackknifes_by_time(nsplits, only_signals=None)[source]
jackknife_inverse_merge()[source]

merges views from jackknife validation data into a single view

currently two different approaches, depending on whether mask signal is present.

concatenate_recordings(recordings)[source]

Concatenate more recordings on to the end of this Recording, and return the result. Recordings must have identical signal names, channels, and fs, or an exception will be thrown. meta of the new recording will be inherited from recordings[0]

select_epoch()[source]
select_times(times, padding=0, reset_epochs=False)[source]
nan_times(times, padding=0)[source]
create_mask(epoch=None, base_signal=None)[source]
inputs:
epoch: {None, boolean, ndarray, string, list}

if None, defaults to False if False, initialize mask signal to False for all times if True, initialize mask signal to False for all times if Tx1 ndarray, True where ndarray is true, False elsewhere if Nx2 ndarray, True in N epoch times if string (eoch name), mask is True for epochs with .name==string if list of strings (epoch names), mask is OR combo of all strings if list of tuples (epoch times), mask is OR combo of all epoch times

TODO: add epochs, base signal parameters

or_mask(epoch, invert=False)[source]

Make rec[‘mask’] == True for all {epoch} or where current mask true. Mask is created if it doesn’t exist See create_mask for input formats for ‘epoch’

ex:

rec.or_mask([‘HIT_TRIAL’, ‘PASSIVE_EXPERIMENT’]) will return a new recording with rec[‘mask’] == True for all PASSIVE EXPERIMENT and all HIT TRIAL epochs

and_mask(epoch, invert=False)[source]

Make rec[‘mask’] == True for all epochs where current mask is also true. Mask is created if it doesn’t exist See create_mask for input formats for ‘epoch’

example use:

newrec = rec.or_mask([‘ACTIVE_EXPERIMENT’]) newrec = rec.and_mask([‘REFERENCE’, ‘TARGET’])

newrec[‘mask’] == True only during REFERENCE and TARGET epochs contained within ACTIVE_EXPERIMENT epochs

apply_mask(reset_epochs=False, mask_name='mask')[source]

Used to excise data based on boolean called mask. Returns new recording with only data specified mask. To make mask, see “create_mask”

nan_mask(remove_epochs=True)[source]

Nan-out data based on boolean signal called mask. Returns new recording with only data specified mask. To make mask, see “create_mask” :param remove_epochs: (True) if true, delete epochs that are all nan :return: rec : copy of self with masked periods set to nan

remove_masked_epochs()[source]

Delete epochs that fall outside of the mask periods :return: rec : copy of self with masked epochs removed

nems.recording.load_recording_from_targz(targz)[source]
nems.recording.load_recording_from_targz_stream(tgz_stream)[source]

Loads the recording object from the given .tgz stream, which is expected to be a io.BytesIO object. For hdf5 files, copy to temporary directory and load with hdf5 utility

nems.recording.load_recording(uri)[source]

Loads from a local .tgz file, a local directory, from s3, or from an HTTP URL containing a .tgz file. Examples:

# Load all signals in the gus016c-a2 directory rec = Recording.load(‘/home/myuser/gus016c-a2’) rec = Recording.load(’file:///home/myuser/gus016c-a2’)

# Load the local tar gz directory. rec = Recording.load(’file:///home/myuser/gus016c-a2.tgz’)

# Load a tgz file served from a flat filesystem rec = Recording.load(’http://potoroo/recordings/gus016c-a2.tgz’)

# Load a tgz file created by the nems-baphy interafce rec = Recording.load(’http://potoroo/baphy/271/gus016c-a2’)

# Load from S3: rec = Recording.load(‘s3://nems.lbhb… TODO’)

nems.recording.load_recording_from_dir(directory_or_targz)[source]

Loads all the signals (CSV/JSON pairs) found in DIRECTORY or .tgz file, and returns a Recording object containing all of them.

nems.recording.load_recording_from_url(url)[source]

Loads the recording object from a URL. File must be tgz format.

nems.recording.load_recording_from_arrays(arrays, rec_name, fs, sig_names=None, signal_kwargs={})[source]

Generates a recording object, and the signal objects it contains, from a list of array-like structures of the form channels x time (see signal.py for more details about how arrays are represented

by signals).

If any of the arrays are more than 2-dimensional, an error will be thrown. Also pay close attention to any RuntimeWarnings from the signal class regarding improperly-shaped arrays, which may indicate that an array was passed as time x channels instead of the reverse.

arrayslist of array-like

The data to be converted to a recording of signal objects. Each item should be 2-dimensional and convertible to a numpy ndarray via np.array(x). No constraints are enforced on the dtype of the arrays, but in general float values are expected by most native NEMS functions.

rec_namestr

The name to be given to the new recording object. This will also be assigned as the recording attribute of each new signal.

fsint or list of ints

The frequency of sampling of the data arrays - used to interconvert between real time and time bins (see signal.py). If int, the same fs will be assigned to each signal. If list, the length must match the length of arrays.

sig_nameslist of strings (optional)

Name to attach to the signal created from each array. The length of this list should match that of arrays. If not specified, the signals will be given the generic names: [‘signal1’, ‘signal2’, …].

signal_kwargslist of dicts

Keyword arguments to be passed through to each signal object. The length of this list should match the length of arrays, and may be padded with empty dictionaries to ensure this constraint. For example:

[{‘chans’: [‘1 kHz’, ‘3 kHz’]}, {‘chans’: [‘one’, ‘two’]}, {}]

Would assign channel names ‘1 kHz’ and ‘3 kHz’ to the signal for the first array, ‘one’ and ‘two’ for the second array, and no channel names (or any other arguments) for the third array.

Valid keyword arguments are: chans, epochs, meta,

and safety_checks

recrecording object

New recording containing a signal for each array.

nems.recording.jackknife_inverse_merge(rec_list)[source]

merges list of jackknife validation data into a signal recording

currently two different approaches, depending on whether mask signal is present.

nems.recording.get_demo_recordings(directory=None, name=None, unpack=False)[source]

Saves all sample recordings in the LBHB public s3 bucket to nems/recordings/, or to the specified directory. By default, the recordings will be kept in a compressed format; however, specifying unpack=True will instead save them uncompressed in a subdirectory.