BF-DATA - Data Management

Ver. 1.3.2 (2023-02-09)

This module provides various elementary data management classes.

class mlpro.bf.data.DataStoring(p_variables)

Bases: object

This class provides a functionality to store values of variables during training/simulation.

C_VAR0 = 'Frame ID'

add_frame(p_frame_id)

To store unique sections in a variable (e.g episodes in RL, etc.)

memorize(p_variable, p_frame_id, p_value)

To store a particular variable into a memory

get_values(p_variable, p_frame_id=None)

To obtain value from the memory

list_to_chunks(p_data, p_chunksize)

compress(p_chunksize)

save_data(p_path, p_filename=None, p_delimiter='\t') → bool

To save stored data in memory_dict as a readable file format

load_data(p_path, p_filename, p_delimiter='\t') → bool

To load data from a readable file format and store them into the DataStoring class format

class mlpro.bf.data.BufferElement(p_element: dict)

Bases: object

Base class implementation for buffer element

add_value_element(p_val: dict)

Adding new value to the element container

Parameters:

p_val (dict) – Elements in dictionary

get_data()

Get the buffer element.

Returns:

Returns the buffer element.

class mlpro.bf.data.Buffer(p_size=1)

Bases: object

Base class implementation for buffer management.

add_element(p_elem: BufferElement)

Add element to the buffer.

Parameters:

p_elem (BufferElement) – Element of Buffer

clear()

Resets buffer.

get_latest()

Returns latest buffered element.

get_all()

Return all buffered elements.

get_sample(p_num: int)

Sample some element from the buffer.

Parameters:

p_num (int) – Number of sample

Returns:

Samples in dictionary

_gen_sample_ind(p_num: int) → list

Generate random indices from the buffer.

Parameters:

p_num (int) – Number of sample

Returns:

List of incides

_extract_rows(p_list_idx: list)

Extract the element in the buffer based on a list of indices.

Parameters:

p_list_idx (list) – List of indices

Returns:

Samples in dictionary

is_full() → bool

Check if the buffer is full.

Returns:

True, if the buffer is full

class mlpro.bf.data.BufferRnd(p_size=1)

Bases: Buffer

Buffer implmentation with random sampling

_gen_sample_ind(p_num: int) → list

Generate random indicies

Parameters:

p_num (int) – Number of sample

Returns:

List of indicies