qcodes.dataset.data_set

exception qcodes.dataset.data_set.CompletedError[source]

Bases: RuntimeError

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class qcodes.dataset.data_set.DataSet(path_to_db: Optional[str] = None, run_id: Optional[int] = None, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None, exp_id: Optional[int] = None, name: Optional[str] = None, specs: Optional[Union[List[qcodes.dataset.descriptions.param_spec.ParamSpec], qcodes.dataset.descriptions.dependencies.InterDependencies_]] = None, values: Optional[List[Optional[Union[str, complex, List, numpy.ndarray, bool]]]] = None, metadata: Optional[Mapping[str, Any]] = None, shapes: Optional[Dict[str, Tuple[int, ...]]] = None, in_memory_cache: bool = True)[source]

Bases: Sized

Create a new DataSet object. The object can either hold a new run or an already existing run. If a run_id is provided, then an old run is looked up, else a new run is created.

Parameters
  • path_to_db – path to the sqlite file on disk. If not provided, the path will be read from the config.

  • run_id – provide this when loading an existing run, leave it as None when creating a new run

  • conn – connection to the DB; if provided and path_to_db is provided as well, then a ValueError is raised (this is to prevent the possibility of providing a connection to a DB file that is different from path_to_db)

  • exp_id – the id of the experiment in which to create a new run. Ignored if run_id is provided.

  • name – the name of the dataset. Ignored if run_id is provided.

  • specs – paramspecs belonging to the dataset or an InterDependencies_ object that describes the dataset. Ignored if run_id is provided.

  • values – values to insert into the dataset. Ignored if run_id is provided.

  • metadata – metadata to insert into the dataset. Ignored if run_id is provided.

  • shapes – An optional dict from names of dependent parameters to the shape of the data captured as a list of integers. The list is in the same order as the interdependencies or paramspecs provided. Ignored if run_id is provided.

  • in_memory_cache – Should measured data be keep in memory and available as part of the dataset.cache object.

persistent_traits = ('name', 'guid', 'number_of_results', 'parameters', 'paramspecs', 'exp_name', 'sample_name', 'completed', 'snapshot', 'run_timestamp_raw', 'description', 'completed_timestamp_raw', 'metadata', 'dependent_parameters', 'parent_dataset_links', 'captured_run_id', 'captured_counter')
background_sleep_time = 0.001
prepare(*, snapshot: Mapping[Any, Any], interdeps: qcodes.dataset.descriptions.dependencies.InterDependencies_, shapes: Optional[Dict[str, Tuple[int, ...]]] = None, parent_datasets: Sequence[Mapping[Any, Any]] = (), write_in_background: bool = False) None[source]
property cache: qcodes.dataset.data_set_cache.DataSetCacheWithDBBackend
property run_id: int
property captured_run_id: int
property path_to_db: str
property name: str
property table_name: str
property guid: str
property snapshot: Optional[Dict[str, Any]]

Snapshot of the run as dictionary (or None)

property snapshot_raw: Optional[str]

Snapshot of the run as a JSON-formatted string (or None)

property number_of_results: int
property counter: int
property captured_counter: int
property parameters: str
property paramspecs: Dict[str, qcodes.dataset.descriptions.param_spec.ParamSpec]
property dependent_parameters: Tuple[qcodes.dataset.descriptions.param_spec.ParamSpecBase, ...]

Return all the parameters that explicitly depend on other parameters

property exp_id: int
property exp_name: str
property sample_name: str
property run_timestamp_raw: Optional[float]

Returns run timestamp as number of seconds since the Epoch

The run timestamp is the moment when the measurement for this run started.

property description: qcodes.dataset.descriptions.rundescriber.RunDescriber
property metadata: Dict[str, Any]

Return a list of Link objects. Each Link object describes a link from this dataset to one of its parent datasets

the_same_dataset_as(other: qcodes.dataset.data_set.DataSet) bool[source]

Check if two datasets correspond to the same run by comparing all their persistent traits. Note that this method does not compare the data itself.

This function raises if the GUIDs match but anything else doesn’t

Parameters

other – the dataset to compare self to

run_timestamp(fmt: str = '%Y-%m-%d %H:%M:%S') Optional[str][source]

Returns run timestamp in a human-readable format

The run timestamp is the moment when the measurement for this run started. If the run has not yet been started, this function returns None.

Consult with time.strftime() for information about the format.

property completed_timestamp_raw: Optional[float]

Returns timestamp when measurement run was completed as number of seconds since the Epoch

If the run (or the dataset) is not completed, then returns None.

completed_timestamp(fmt: str = '%Y-%m-%d %H:%M:%S') Optional[str][source]

Returns timestamp when measurement run was completed in a human-readable format

If the run (or the dataset) is not completed, then returns None.

Consult with time.strftime for information about the format.

toggle_debug() None[source]

Toggle debug mode, if debug mode is on all the queries made are echoed back.

set_interdependencies(interdeps: qcodes.dataset.descriptions.dependencies.InterDependencies_, shapes: Optional[Dict[str, Tuple[int, ...]]] = None) None[source]

Set the interdependencies object (which holds all added parameters and their relationships) of this dataset and optionally the shapes object that holds information about the shape of the data to be measured.

get_parameters() List[qcodes.dataset.descriptions.param_spec.ParamSpec][source]
add_metadata(tag: str, metadata: Any) None[source]

Adds metadata to the DataSet. The metadata is stored under the provided tag. Note that None is not allowed as a metadata value, and the tag has to be a valid python identified (e.g. containing alphanumeric characters and underscores).

Parameters
  • tag – represents the key in the metadata dictionary

  • metadata – actual metadata

add_snapshot(snapshot: str, overwrite: bool = False) None[source]

Adds a snapshot to this run

Parameters
  • snapshot – the raw JSON dump of the snapshot

  • overwrite – force overwrite an existing snapshot

property pristine: bool

Is this DataSet pristine? A pristine DataSet has not yet been started, meaning that parameters can still be added and removed, but results can not be added.

property running: bool

Is this DataSet currently running? A running DataSet has been started, but not yet completed.

property started: bool

Has this DataSet been started? A DataSet not started can not have any results added to it.

property completed: bool

Is this DataSet completed? A completed DataSet may not be modified in any way.

mark_started(start_bg_writer: bool = False) None[source]

Mark this DataSet as started. A DataSet that has been started can not have its parameters modified.

Calling this on an already started DataSet is a NOOP.

Parameters

start_bg_writer – If True, the add_results method will write to the database in a separate thread.

mark_completed() None[source]

Mark DataSet as complete and thus read only and notify the subscribers

add_results(results: Sequence[Mapping[str, Optional[Union[str, complex, List, numpy.ndarray, bool]]]]) None[source]

Adds a sequence of results to the DataSet.

Parameters

results – list of name-value dictionaries where each dictionary provides the values for the parameters in that result. If some parameters are missing the corresponding values are assumed to be None

It is an error to provide a value for a key or keyword that is not the name of a parameter in this DataSet.

It is an error to add results to a completed DataSet.

get_parameter_data(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) Dict[str, Dict[str, numpy.ndarray]][source]

Returns the values stored in the DataSet for the specified parameters and their dependencies. If no parameters are supplied the values will be returned for all parameters that are not them self dependencies.

The values are returned as a dictionary with names of the requested parameters as keys and values consisting of dictionaries with the names of the parameters and its dependencies as keys and numpy arrays of the data as values. If the dataset has a shape recorded in its metadata and the number of datapoints recorded matches the expected number of points the data will be returned as numpy arrays in this shape. If there are less datapoints recorded than expected from the metadata the dataset will be returned as is. This could happen if you call get_parameter_data on an incomplete dataset. See dataset.cache.data for an implementation that returns the data with the expected shape using NaN or zeros as placeholders.

If there are more datapoints than expected the dataset will be returned as is and a warning raised.

If some of the parameters are stored as arrays the remaining parameters are expanded to the same shape as these.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a list of empty arrays is returned.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

Dictionary from requested parameters to Dict of parameter names to numpy arrays containing the data points of type numeric, array or string.

to_pandas_dataframe_dict(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) Dict[str, pandas.core.frame.DataFrame][source]

Returns the values stored in the DataSet for the specified parameters and their dependencies as a dict of pandas.DataFrame s Each element in the dict is indexed by the names of the requested parameters.

Each DataFrame contains a column for the data and is indexed by a pandas.MultiIndex formed from all the setpoints of the parameter.

If no parameters are supplied data will be be returned for all parameters in the DataSet that are not them self dependencies of other parameters.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a dict of empty pandas.DataFrame s is returned.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

Dictionary from requested parameter names to pandas.DataFrame s with the requested parameter as a column and a indexed by a pandas.MultiIndex formed by the dependencies.

get_data_as_pandas_dataframe(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) Dict[str, pandas.core.frame.DataFrame][source]

Returns the values stored in the DataSet for the specified parameters and their dependencies as a dict of pandas.DataFrame s Each element in the dict is indexed by the names of the requested parameters.

Each DataFrame contains a column for the data and is indexed by a pandas.MultiIndex formed from all the setpoints of the parameter.

If no parameters are supplied data will be be returned for all parameters in the DataSet that are not them self dependencies of other parameters.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a dict of empty pandas.DataFrame s is returned.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

Dictionary from requested parameter names to pandas.DataFrame s with the requested parameter as a column and a indexed by a pandas.MultiIndex formed by the dependencies.

to_pandas_dataframe(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) pandas.core.frame.DataFrame[source]

Returns the values stored in the DataSet for the specified parameters and their dependencies as a concatenated pandas.DataFrame s

The DataFrame contains a column for the data and is indexed by a pandas.MultiIndex formed from all the setpoints of the parameter.

If no parameters are supplied data will be be returned for all parameters in the DataSet that are not them self dependencies of other parameters.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a dict of empty pandas.DataFrame s is returned.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

pandas.DataFrame s with the requested parameter as a column and a indexed by a pandas.MultiIndex formed by the dependencies.

Example

Return a pandas DataFrame with

df = ds.to_pandas_dataframe()

to_xarray_dataarray_dict(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) Dict[str, xr.DataArray][source]

Returns the values stored in the DataSet for the specified parameters and their dependencies as a dict of xr.DataArray s Each element in the dict is indexed by the names of the requested parameters.

If no parameters are supplied data will be be returned for all parameters in the DataSet that are not them self dependencies of other parameters.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a dict of empty xr.DataArray s is returned.

The dependent parameters of the Dataset are normally used as coordinates of the XArray dataframe. However if non unique values are found for the dependent parameter values we will fall back to using an index as coordinates.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

Dictionary from requested parameter names to xr.DataArray s with the requested parameter(s) as a column(s) and coordinates formed by the dependencies.

Example

Return a dict of xr.DataArray with

dataarray_dict = ds.to_xarray_dataarray_dict()

to_xarray_dataset(*params: Union[str, qcodes.dataset.descriptions.param_spec.ParamSpec, qcodes.instrument.parameter._BaseParameter], start: Optional[int] = None, end: Optional[int] = None) xr.Dataset[source]

Returns the values stored in the DataSet for the specified parameters and their dependencies as a xr.Dataset object.

If no parameters are supplied data will be be returned for all parameters in the DataSet that are not then self dependencies of other parameters.

If provided, the start and end arguments select a range of results by result count (index). If the range is empty - that is, if the end is less than or equal to the start, or if start is after the current end of the DataSet – then a empty xr.Dataset s is returned.

The dependent parameters of the Dataset are normally used as coordinates of the XArray dataframe. However if non unique values are found for the dependent parameter values we will fall back to using an index as coordinates.

Parameters
  • *params – string parameter names, QCoDeS Parameter objects, and ParamSpec objects. If no parameters are supplied data for all parameters that are not a dependency of another parameter will be returned.

  • start – start value of selection range (by result count); ignored if None

  • end – end value of selection range (by results count); ignored if None

Returns

xr.Dataset with the requested parameter(s) data as xr.DataArray s and coordinates formed by the dependencies.

Example

Return a concatenated xr.Dataset with

xds = ds.to_xarray_dataset()

write_data_to_text_file(path: str, single_file: bool = False, single_file_name: Optional[str] = None) None[source]

An auxiliary function to export data to a text file. When the data with more than one dependent variables, say “y(x)” and “z(x)”, is concatenated to a single file it reads:

x1 y1(x1) z1(x1) x2 y2(x2) z2(x2) .. .. .. xN yN(xN) zN(xN)

For each new independent variable, say “k”, the expansion is in the y-axis:

x1 y1(x1) z1(x1) x2 y2(x2) z2(x2) .. .. .. xN yN(xN) zN(xN) k1 y1(k1) z1(k1) k2 y2(k2) z2(k2) .. .. .. kN yN(kN) zN(kN)

Parameters
  • path – User defined path where the data to be exported

  • single_file – If true, merges the data of same length of multiple dependent parameters to a single file.

  • single_file_name – User defined name for the data to be concatenated. If no extension is passed (.dat, .csv or .txt), .dat is automatically appended.

Raises
  • DataLengthException – If the data of multiple parameters have not same length and wanted to be merged in a single file.

  • DataPathException – If the data of multiple parameters are wanted to be merged in a single file but no filename provided.

subscribe(callback: Callable[[Any, int, Optional[Any]], None], min_wait: int = 0, min_count: int = 1, state: Optional[Any] = None, callback_kwargs: Optional[Mapping[str, Any]] = None) str[source]
subscribe_from_config(name: str) str[source]

Subscribe a subscriber defined in the qcodesrc.json config file to the data of this DataSet. The definition can be found at subscription.subscribers in the qcodesrc.json config file.

Parameters

name – identifier of the subscriber. Equal to the key of the entry in qcodesrc.json::subscription.subscribers.

unsubscribe(uuid: str) None[source]

Remove subscriber with the provided uuid

unsubscribe_all() None[source]

Remove all subscribers

get_metadata(tag: str) str[source]
export(export_type: Optional[Union[qcodes.dataset.export_config.DataExportType, str]] = None, path: Optional[str] = None, prefix: Optional[str] = None) None[source]

Export data to disk with file name {prefix}{run_id}.{ext}. Values for the export type, path and prefix can also be set in the “dataset” section of qcodes config.

Parameters
  • export_type – Data export type, e.g. “netcdf” or DataExportType.NETCDF, defaults to a value set in qcodes config

  • path – Export path, defaults to value set in config

  • prefix – File prefix, e.g. qcodes_, defaults to value set in config.

Raises

ValueError – If the export data type is not specified, raise an error

property export_path: Optional[str]
property export_info: qcodes.dataset.exporters.export_info.ExportInfo
qcodes.dataset.data_set.generate_dataset_table(guids: Sequence[str], conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None) str[source]

Generate an ASCII art table of information about the runs attached to the supplied guids.

Parameters
  • guids – Sequence of one or more guids

  • conn – A ConnectionPlus object with a connection to the database.

Returns: ASCII art table of information about the supplied guids.

qcodes.dataset.data_set.load_by_counter(counter: int, exp_id: int, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None) qcodes.dataset.data_set.DataSet[source]

Load a dataset given its counter in a given experiment

Lookup is performed in the database file that is specified in the config.

Note that the counter used in this function in not preserved when copying data to another db file. We recommend using load_by_run_spec() which does not have this issue and is significantly more flexible.

Parameters
  • counter – counter of the dataset within the given experiment

  • exp_id – id of the experiment where to look for the dataset

  • conn – connection to the database to load from. If not provided, a connection to the DB file specified in the config is made

Returns

DataSet of the given counter in the given experiment

qcodes.dataset.data_set.load_by_guid(guid: str, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None) qcodes.dataset.data_set.DataSet[source]

Load a dataset by its GUID

If no connection is provided, lookup is performed in the database file that is specified in the config.

Parameters
  • guid – guid of the dataset

  • conn – connection to the database to load from

Returns

DataSet with the given guid

Raises
  • NameError – if no run with the given GUID exists in the database

  • RuntimeError – if several runs with the given GUID are found

qcodes.dataset.data_set.load_by_id(run_id: int, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None) qcodes.dataset.data_set.DataSet[source]

Load a dataset by run id

If no connection is provided, lookup is performed in the database file that is specified in the config.

Note that the run_id used in this function in not preserved when copying data to another db file. We recommend using load_by_run_spec() which does not have this issue and is significantly more flexible.

Parameters
  • run_id – run id of the dataset

  • conn – connection to the database to load from

Returns

DataSet with the given run id

qcodes.dataset.data_set.load_by_run_spec(*, captured_run_id: Optional[int] = None, captured_counter: Optional[int] = None, experiment_name: Optional[str] = None, sample_name: Optional[str] = None, sample_id: Optional[int] = None, location: Optional[int] = None, work_station: Optional[int] = None, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None) qcodes.dataset.data_set.DataSet[source]

Load a run from one or more pieces of runs specification. All fields are optional but the function will raise an error if more than one run matching the supplied specification is found. Along with the error specs of the runs found will be printed.

Parameters
  • captured_run_id – The run_id that was originally assigned to this at the time of capture.

  • captured_counter – The counter that was originally assigned to this at the time of capture.

  • experiment_name – name of the experiment that the run was captured

  • sample_name – The name of the sample given when creating the experiment.

  • sample_id – The sample_id assigned as part of the GUID.

  • location – The location code assigned as part of GUID.

  • work_station – The workstation assigned as part of the GUID.

  • conn – An optional connection to the database. If no connection is supplied a connection to the default database will be opened.

Raises

NameError – if no run or more than one run with the given specification exists in the database

Returns

DataSet matching the provided specification.

qcodes.dataset.data_set.new_data_set(name: str, exp_id: Optional[int] = None, specs: Optional[List[qcodes.dataset.descriptions.param_spec.ParamSpec]] = None, values: Optional[List[Optional[Union[str, complex, List, numpy.ndarray, bool]]]] = None, metadata: Optional[Any] = None, conn: Optional[qcodes.dataset.sqlite.connection.ConnectionPlus] = None, in_memory_cache: bool = True) qcodes.dataset.data_set.DataSet[source]

Create a new dataset in the currently active/selected database.

If exp_id is not specified, the last experiment will be loaded by default.

Parameters
  • name – the name of the new dataset

  • exp_id – the id of the experiments this dataset belongs to, defaults to the last experiment

  • specs – list of parameters to create this dataset with

  • values – the values to associate with the parameters

  • metadata – the metadata to associate with the dataset

  • in_memory_cache – Should measured data be keep in memory and available as part of the dataset.cache object.

Returns

the newly created DataSet

class qcodes.dataset.data_set_cache.DataSetCache(dataset: qcodes.dataset.data_set_cache.DatasetType)[source]

Bases: Generic[qcodes.dataset.data_set_cache.DatasetType]

The DataSetCache contains a in memory representation of the data in this dataset as well a a method to progressively read data from the db as it is written and methods to append data as it is received without writing it to disk. The cache can either be loaded from the db or produced as an in memory cache. It is not possible to combine these two ways of producing a dataset cache. The cache is available in the same formats as DataSet.get_parameter_data and DataSet.to_pandas_dataframe_dict

property rundescriber: qcodes.dataset.descriptions.rundescriber.RunDescriber
property live: Optional[bool]

If true this cache has been produced by appending data as measured. If false the data has been read from disk. If None, then the cache does not yet have any data.

data() ParameterData[source]

Loads data from the database on disk if needed and returns the cached data. The cached data is in almost the same format as DataSet.get_parameter_data. However if a shape is provided as part of the dataset metadata and fewer datapoints than expected are returned the missing values will be replaced by NaN or zeroes depending on the datatype.

Returns

The cached dataset.

load_data_from_db() None[source]

Load the data from an on-disk format in case the cache is not live

Should be implemented in a specific subclass that knows how to read data from disk

add_data(new_data: Mapping[str, Mapping[str, numpy.ndarray]]) None[source]
to_pandas_dataframe_dict() Dict[str, pd.DataFrame][source]

Convert the cached dataset to Pandas dataframes. The returned dataframes are in the same format DataSet.to_pandas_dataframe_dict.

Returns

A dict from parameter name to Pandas Dataframes. Each dataframe represents one parameter tree.

to_pandas_dataframe() pd.DataFrame[source]

Convert the cached dataset to Pandas dataframes. The returned dataframes are in the same format DataSet.to_pandas_dataframe_dict.

Returns

A dict from parameter name to Pandas Dataframes. Each dataframe represents one parameter tree.

to_pandas() Dict[str, pd.DataFrame][source]

Returns the values stored in the dataset.data_set.DataSet as a concatenated pandas.DataFrame s

The DataFrame contains a column for the data and is indexed by a pandas.MultiIndex formed from all the setpoints of the parameter.

Note that if the dataset contains data for multiple parameters that do not share the same setpoints it is recommended to use to_pandas_dataframe_dict

Returns

pandas.DataFrame s with the requested parameter as a column and a indexed by a pandas.MultiIndex formed by the dependencies.

to_xarray_dataarray_dict() Dict[str, xr.DataArray][source]

Returns the values stored in the dataset.data_set.DataSet as a dict of xr.DataArray s Each element in the dict is indexed by the names of the dependent parameters.

Returns

Dictionary from requested parameter names to xr.DataArray s with the requested parameter(s) as a column(s) and coordinates formed by the dependencies.

to_xarray_dataset() xr.Dataset[source]

Returns the values stored in the dataset.data_set.DataSet as a xr.Dataset object.

Note that if the dataset contains data for multiple parameters that do not share the same setpoints it is recommended to use to_xarray_dataarray_dict

Returns

xr.Dataset with the requested parameter(s) data as xr.DataArray s and coordinates formed by the dependencies.

class qcodes.dataset.data_set_cache.DataSetCacheWithDBBackend(dataset: qcodes.dataset.data_set_cache.DatasetType)[source]

Bases: qcodes.dataset.data_set_cache.DataSetCache[DataSet]

load_data_from_db() None[source]

Loads data from the dataset into the cache. If new data has been added to the dataset since the last time this method was called, calling this method again would load that new portion of the data and append to the already loaded data. If the dataset is marked completed and data has already been loaded no load will be performed.

add_data(new_data: Mapping[str, Mapping[str, numpy.ndarray]]) None
data() ParameterData

Loads data from the database on disk if needed and returns the cached data. The cached data is in almost the same format as DataSet.get_parameter_data. However if a shape is provided as part of the dataset metadata and fewer datapoints than expected are returned the missing values will be replaced by NaN or zeroes depending on the datatype.

Returns

The cached dataset.

property live: Optional[bool]

If true this cache has been produced by appending data as measured. If false the data has been read from disk. If None, then the cache does not yet have any data.

property rundescriber: qcodes.dataset.descriptions.rundescriber.RunDescriber
to_pandas() Dict[str, pd.DataFrame]

Returns the values stored in the dataset.data_set.DataSet as a concatenated pandas.DataFrame s

The DataFrame contains a column for the data and is indexed by a pandas.MultiIndex formed from all the setpoints of the parameter.

Note that if the dataset contains data for multiple parameters that do not share the same setpoints it is recommended to use to_pandas_dataframe_dict

Returns

pandas.DataFrame s with the requested parameter as a column and a indexed by a pandas.MultiIndex formed by the dependencies.

to_pandas_dataframe() pd.DataFrame

Convert the cached dataset to Pandas dataframes. The returned dataframes are in the same format DataSet.to_pandas_dataframe_dict.

Returns

A dict from parameter name to Pandas Dataframes. Each dataframe represents one parameter tree.

to_pandas_dataframe_dict() Dict[str, pd.DataFrame]

Convert the cached dataset to Pandas dataframes. The returned dataframes are in the same format DataSet.to_pandas_dataframe_dict.

Returns

A dict from parameter name to Pandas Dataframes. Each dataframe represents one parameter tree.

to_xarray_dataarray_dict() Dict[str, xr.DataArray]

Returns the values stored in the dataset.data_set.DataSet as a dict of xr.DataArray s Each element in the dict is indexed by the names of the dependent parameters.

Returns

Dictionary from requested parameter names to xr.DataArray s with the requested parameter(s) as a column(s) and coordinates formed by the dependencies.

to_xarray_dataset() xr.Dataset

Returns the values stored in the dataset.data_set.DataSet as a xr.Dataset object.

Note that if the dataset contains data for multiple parameters that do not share the same setpoints it is recommended to use to_xarray_dataarray_dict

Returns

xr.Dataset with the requested parameter(s) data as xr.DataArray s and coordinates formed by the dependencies.

qcodes.dataset.data_set_cache.append_shaped_parameter_data_to_existing_arrays(rundescriber: qcodes.dataset.descriptions.rundescriber.RunDescriber, write_status: Mapping[str, Optional[int]], existing_data: Mapping[str, Mapping[str, numpy.ndarray]], new_data: Mapping[str, Mapping[str, numpy.ndarray]]) Tuple[Dict[str, Optional[int]], Dict[str, Dict[str, numpy.ndarray]]][source]

Append datadict to an already existing datadict and return the merged data.

Parameters
  • rundescriber – The rundescriber that describes the run

  • write_status – Mapping from dependent parameter name to number of rows written to the cache previously.

  • new_data – Mapping from dependent parameter name to mapping from parameter name to numpy arrays that the data should be appended to.

  • existing_data – Mapping from dependent parameter name to mapping from parameter name to numpy arrays of new data.

Returns

Updated write and read status, and the updated data

qcodes.dataset.data_set_cache.load_new_data_from_db_and_append(conn: qcodes.dataset.sqlite.connection.ConnectionPlus, table_name: str, rundescriber: qcodes.dataset.descriptions.rundescriber.RunDescriber, write_status: Mapping[str, Optional[int]], read_status: Mapping[str, int], existing_data: Mapping[str, Mapping[str, numpy.ndarray]]) Tuple[Dict[str, Optional[int]], Dict[str, int], Dict[str, Dict[str, numpy.ndarray]]][source]

Append any new data in the db to an already existing datadict and return the merged data.

Parameters
  • conn – The connection to the sqlite database

  • table_name – The name of the table the data is stored in

  • rundescriber – The rundescriber that describes the run

  • write_status – Mapping from dependent parameter name to number of rows written to the cache previously.

  • read_status – Mapping from dependent parameter name to number of rows read from the db previously.

  • existing_data – Mapping from dependent parameter name to mapping from parameter name to numpy arrays that the data should be inserted into. appended to.

Returns

Updated write and read status, and the updated data