BF-DATA - Data Management

Ver. 1.4.0 (2024-04-28)

This module provides various elementary buffer management classes.

class mlpro.bf.data.buffers.BufferElement(p_element: dict)

Bases: object

Base class implementation for buffer element

add_value_element(p_val: dict)

Adding new value to the element container

Parameters:

p_val (dict) – Elements in dictionary

get_data()

Get the buffer element.

Returns:

Returns the buffer element.

class mlpro.bf.data.buffers.Buffer(p_size=1)

Bases: object

Base class implementation for buffer management.

add_element(p_elem: BufferElement)

Add element to the buffer.

Parameters:

p_elem (BufferElement) – Element of Buffer

clear()

Resets buffer.

get_latest()

Returns latest buffered element.

get_all()

Return all buffered elements.

get_sample(p_num: int)

Sample some element from the buffer.

Parameters:

p_num (int) – Number of sample

Returns:

Samples in dictionary

_gen_sample_ind(p_num: int) → list

Generate random indices from the buffer.

Parameters:

p_num (int) – Number of sample

Returns:

List of incides

_extract_rows(p_list_idx: list)

Extract the element in the buffer based on a list of indices.

Parameters:

p_list_idx (list) – List of indices

Returns:

Samples in dictionary

is_full() → bool

Check if the buffer is full.

Returns:

True, if the buffer is full

class mlpro.bf.data.buffers.BufferRnd(p_size=1)

Bases: Buffer

Buffer implmentation with random sampling

_gen_sample_ind(p_num: int) → list

Generate random indicies

Parameters:

p_num (int) – Number of sample

Returns:

List of indicies

Ver. 1.4.0 (2024-04-28)

This module provides various elementary data management classes.

class mlpro.bf.data.datastoring.DataStoring(p_variables)

Bases: object

This class provides a functionality to store values of variables during training/simulation.

C_VAR0 = 'Frame ID'

add_frame(p_frame_id)

To store unique sections in a variable (e.g episodes in RL, etc.)

memorize(p_variable, p_frame_id, p_value)

To store a particular variable into a memory

get_values(p_variable, p_frame_id=None)

To obtain value from the memory

list_to_chunks(p_data, p_chunksize)

compress(p_chunksize)

save_data(p_path, p_filename=None, p_delimiter='\t') → bool

To save stored data in memory_dict as a readable file format

load_data(p_path, p_filename, p_delimiter='\t', p_frame=True, p_header=True) → bool

To load data from a readable file format and store them into the DataStoring class format

Ver. 0.1.0 (2024-12-11)

This module provides classes to deal with persistent configuration data.

class mlpro.bf.data.cfg_file.ConfigFile(p_fname: str)

Bases: object

Stores configuration data in a local JSON file.

Parameters:

p_fname (str) – Name of the local JSON file.

get(p_key)

Returns the values stored for the specified key. If no values were found an exception is raised.

Parameters:

p_key – Key.

Returns:

Values stored for the specified key.

Return type:

values

set(p_key, p_values) → bool

Stores the values of the specified key.

Parameters:

• p_key – Key.

• p_values – Values to be stored.

Returns:

True, if storing was successfull. False otherwise.

Return type:

bool