qcodes.data.data_set

DataSet class and factory functions.

class qcodes.data.data_set.DataSet(location=None, arrays=None, formatter=None, io=None, write_period=5)[source]

Bases: qcodes.utils.helpers.DelegateAttributes

A container for one complete measurement loop.

May contain many individual arrays with potentially different sizes and dimensionalities.

Normally a DataSet should not be instantiated directly, but through new_data or load_data.

Parameters
  • location (Union[str,bool]) – A location in the io manager, or False for an only-in-memory temporary DataSet. Note that the full path to or physical location of the data is a combination of io + location. the default DiskIO sets the base directory, which this location is a relative path inside.

  • io (Optional[io_manager]) – base physical location of the DataSet. Default DataSet.default_io is initially DiskIO('.') which says the root data directory is the current working directory, ie where you started the python session.

  • arrays (Optional[List[qcodes.data.data_array.DataArray]) – arrays to add to the DataSet. Can be added later with self.add_array(array).

  • formatter (Optional[Formatter]) – sets the file format/structure to write (and read) with. Default DataSet.default_formatter which is initially GNUPlotFormat().

  • write_period (Optional[float]) – Only if mode=LOCAL, seconds between saves to disk. If not LOCAL, the DataServer handles this and generally writes more often. Use None to disable writing from calls to self.store. Default 5.

delegate_attr_dicts: List[str] = ['arrays']

A list of names (strings) of dictionaries which are (or will be) attributes of self, whose keys should be treated as attributes of self.

default_io = <DiskIO, base_location='/home/runner/work/Qcodes/Qcodes/docs'>
default_formatter = <qcodes.data.gnuplot_format.GNUPlotFormat object>
location_provider = <qcodes.data.location.FormatLocation object>
background_functions: Dict[str, Callable[[...], Any]] = {}

The value fn is a callable accepting no arguments, and key is a name to identify the function and help you attach and remove it.

In DataSet.complete we call each of these periodically, in the order that they were attached.

Note that because this is a class attribute, the functions will apply to every DataSet. If you want specific functions for one DataSet you can override this with an instance attribute.

sync()[source]

Synchronize this DataSet with the DataServer or storage.

If this DataSet is on the server, asks the server for changes. If not, reads the entire DataSet from disk.

Returns

True if this DataSet is live on the server

Return type

bool

fraction_complete()[source]

Get the fraction of this DataSet which has data in it.

Returns

the average of all measured (not setpoint) arrays’

fraction_complete() values, independent of the individual array sizes. If there are no measured arrays, returns zero.

Return type

float

complete(delay=1.5)[source]

Periodically sync the DataSet and display percent complete status.

Also, each period, execute functions stored in (class attribute) self.background_functions. If a function fails, we log its traceback and continue on. If any one function fails twice in a row, it gets removed.

Parameters

delay (float) – seconds between iterations. Default 1.5

get_changes(synced_indices)[source]

Find changes since the last sync of this DataSet.

Parameters

synced_indices (dict) – {array_id: synced_index} where synced_index is the last flat index which has already been synced, for any (usually all) arrays in the DataSet.

Returns

keys are array_id for each array with changes,

values are dicts as returned by DataArray.get_changes and required as kwargs to DataArray.apply_changes. Note that not all arrays in synced_indices need be present in the return, only those with changes.

Return type

Dict[dict]

add_array(data_array)[source]

Add one DataArray to this DataSet, and mark it as part of this DataSet.

Note: DO NOT just set data_set.arrays[id] = data_array, because this will not check if we are overwriting another array, nor set the reference back to this DataSet, nor that the array_id in the array matches how you’re storing it here.

Parameters

data_array (DataArray) – the new array to add

Raises

ValueError – if there is already an array with this id here.

remove_array(array_id)[source]

Remove an array from a dataset

Throws an exception when the array specified is refereced by other arrays in the dataset.

Parameters

array_id (str) – array_id of array to be removed

store(loop_indices, ids_values)[source]

Insert data into one or more of our DataArrays.

Parameters
  • loop_indices (tuple) – the indices within whatever loops we are inside. May have fewer dimensions than some of the arrays we are inserting into, if the corresponding value makes up the remaining dimensionality.

  • values (Dict[Union[float, Sequence]]) – a dict whose keys are array_ids, and values are single numbers or entire slices to insert into that array.

default_parameter_name(paramname: Optional[str] = None) Optional[str][source]

Return name of default parameter for plotting

The default parameter is determined by looking into metdata[‘default_parameter_name’]. If this variable is not present, then the closest match to the argument paramname is tried.

Parameters

paramname – Name to match to parameter name

Returns

Name of the default parameter, or None if no parameter is found

default_parameter_array(paramname='amplitude')[source]

Return default parameter array

Parameters

paramname (str) – Name to match to parameter name. Defaults to ‘amplitude’

Returns

array corresponding to the default parameter

Return type

DataArray

See also

default_parameter_name

read()[source]

Read the whole DataSet from storage, overwriting the local data.

read_metadata()[source]

Read the metadata from storage, overwriting the local data.

write(write_metadata=False, only_complete=True, filename=None)[source]

Writes updates to the DataSet to storage. N.B. it is recommended to call data_set.finalize() when a DataSet is no longer expected to change to ensure files get closed

Parameters
  • write_metadata (bool) – write the metadata to disk

  • only_complete (bool) – passed on to the match_save_range inside self.formatter.write. Used to ensure that all new data gets saved even when some columns are strange.

  • filename (Optional[str]) – The filename (minus extension) to use. The file gets saved in the usual location.

write_copy(path=None, io_manager=None, location=None)[source]

Write a new complete copy of this DataSet to storage.

Parameters
  • path (Optional[str]) – An absolute path on this system to write to. If you specify this, you may not include either io_manager or location.

  • io_manager (Optional[io_manager]) – A new io_manager to use with either the DataSet’s same or a new location.

  • location (Optional[str]) – A new location to write to, using either this DataSet’s same or a new io_manager.

add_metadata(new_metadata)[source]

Update DataSet.metadata with additional data.

Parameters

new_metadata (dict) – new data to be deep updated into the existing metadata

save_metadata()[source]

Evaluate and save the DataSet’s metadata.

finalize(filename=None, write_metadata=True)[source]

Mark the DataSet complete and write any remaining modifications.

Also closes the data file(s), if the Formatter we’re using supports that.

Parameters
  • filename (Optional[str]) – The file name (minus extension) to write to. The location of the file is the usual one.

  • write_metadata (bool) – Whether to save a snapshot. For e.g. dumping raw data inside a loop, a snapshot is not wanted.

snapshot(update=False)[source]

JSON state of the DataSet.

get_array_metadata(array_id)[source]

Get the metadata for a single contained DataArray.

Parameters

array_id (str) – the array to get metadata for.

Returns

metadata for this array.

Return type

dict

__repr__()[source]

Rich information about the DataSet and contained arrays.

to_xarray() xarray.core.dataset.Dataset[source]

Convert the dataset to an xarray Dataset

classmethod from_xarray(xarray_dataset: xarray.core.dataset.Dataset) qcodes.data.data_set.DataSet[source]

Convert the dataset to an xarray DataSet

delegate_attr_objects: List[str] = []

A list of names (strings) of objects which are (or will be) attributes of self, whose attributes should be passed through to self.

omit_delegate_attrs: List[str] = []

A list of attribute names (strings) to not delegate to any other dictionary or object.

qcodes.data.data_set.dataset_to_xarray_dictionary(data_set: qcodes.data.data_set.DataSet, include_metadata: bool = True) Dict[str, Any][source]

Convert QcodesDataSet to dictionary.

Parameters
  • data_set – The data to convert.

  • include_data – If True then include the ndarray field.

  • include_metadata – If True then include the metadata.

Returns

Dictionary containing the serialized data.

qcodes.data.data_set.load_data(location=None, formatter=None, io=None)[source]

Load an existing DataSet.

Parameters
  • location (Optional[str]) – the location to load from. Default is the current live DataSet. Note that the full path to or physical location of the data is a combination of io + location. the default DiskIO sets the base directory, which this location is a relative path inside.

  • formatter (Optional[Formatter]) – sets the file format/structure to read with. Default DataSet.default_formatter which is initially GNUPlotFormat().

  • io (Optional[io_manager]) – base physical location of the DataSet. Default DataSet.default_io is initially DiskIO('.') which says the root data directory is the current working directory, ie where you started the python session.

Returns

A new DataSet object loaded with pre-existing data.

qcodes.data.data_set.new_data(location=None, loc_record=None, name=None, overwrite=False, io=None, **kwargs)[source]

Create a new DataSet.

Parameters
  • location (Optional[Union[str,Callable, Bool]]) –

    If you provide a string, it must be an unused location in the io manager. Can also be:

    • a Callable location provider with one required parameter (the io manager), and one optional (record dict), which returns a location string when called

    • False - denotes an only-in-memory temporary DataSet.

    Note that the full path to or physical location of the data is a combination of io + location. the default DiskIO sets the base directory, which this location is a relative path inside. Default DataSet.location_provider which is initially FormatLocation()

  • loc_record (Optional[dict]) – If location is a callable, this will be passed to it as record

  • name (Optional[str]) – overrides the name key in the loc_record.

  • overwrite (bool) – Are we allowed to overwrite an existing location? Default False.

  • io (Optional[io_manager]) – base physical location of the DataSet. Default DataSet.default_io is initially DiskIO('.') which says the root data directory is the current working directory, ie where you started the python session.

  • arrays (Optional[List[qcodes.data.data_array.DataArray]) – arrays to add to the DataSet. Can be added later with self.add_array(array).

  • formatter (Optional[Formatter]) – sets the file format/structure to write (and read) with. Default DataSet.default_formatter which is initially GNUPlotFormat().

  • write_period (Optional[float]) – seconds between saves to disk.

Returns

A new DataSet object ready for storing new data in.

qcodes.data.data_set.qcodes_dataset_to_xarray_dataset(data_set: qcodes.data.data_set.DataSet) xarray.core.dataset.Dataset[source]

Convert QCoDeS gridded dataset to xarray dataset

qcodes.data.data_set.xarray_dataset_to_qcodes_dataset(xarray_data_set: xarray.core.dataset.Dataset) qcodes.data.data_set.DataSet[source]

Convert QCoDeS gridded dataset to xarray dataset

qcodes.data.data_set.xarray_dictionary_to_dataset(xarray_dictionary: Dict[str, Any]) qcodes.data.data_set.DataSet[source]

Convert xarray dictionary to Qcodes DataSet.

Parameters

xarray_dictionary – data to convert

Returns

QCoDeS dataSet with converted data.