Bulk Goods Plant

Ver. 2.3.2 (2023-08-22)

This module provides an RL environment of Bulk Good Laboratory Plant (BGLP).

class mlpro.rl.pool.envs.bglp.Actuator(minpower, maxpower, minaction, maxaction, masscoeff)

Bases: object

This class serves as a parent class of different types of actuators, which provides the main attributes of an actuator in the BGLP environment.

Parameters:

minpower (float) – minimum power of an actuator.
maxpower (float) – maximum power of an actuator.
minaction (float) – minimum action of an actuator.
maxaction (float) – maximum action of an actuator.
masscoeff (float) – mass transport coefficient of an actuator.

reg_a

list of existing actuators in the environment.

Type:: list of objects

idx_a

length of reg_a.

Type:: int

power_max

maximum power of an actuator.

Type:: float

power_min

minimum power of an actuator.

Type:: float

power_coeff

power coefficient of an actuator, if necessary.

Type:: float

action_max

maximum action of an actuator.

Type:: float

action_min

minimum action of an actuator.

Type:: float

mass_coeff

mass transport coefficient of an actuator.

Type:: float

t_activated

a time indicator about an actuator is activated.

Type:: float

t_end

a time indicator about the end of an activation sequence of the actuator.

Type:: float

status

status of an actuator, false means inactive and true means active.

Type:: bool

cur_mass_transport

current transported mass of an actuator.

Type:: float

cur_power

current power consumption of an actuator.

Type:: float

cur_action

current taken action of an actuator in RL context.

Type:: float

cur_speed

current speed of an actuator.

Type:: float

type_

a short name for an actuator, usually 3 capital letters (e.g VAC, BLT).

Type:: str

reg_a = []

power_coeff = 0

t_activated = 0

t_end = 0

status = False

type_ = ''

idx_a = 0

power_max = 0

power_min = 0

action_min = 0

action_max = 0

mass_coeff = 0

cur_mass_transport = 0

cur_power = 0

cur_action = 0

cur_speed = 0

class mlpro.rl.pool.envs.bglp.VacuumPump(name, minpower, maxpower, minaction, maxaction, masscoeff)

Bases: Actuator

This class inherits Actuator class and serves as a child class of Actuator. This class represents a type of actuators in the BGLP environment, namely Vacuum Pump. Vacuum Pumps are mostly used to transport material from mini hoppers to silos. However, the parameter of each vacuum pump can be dissimilar to each other based on their settings.

Parameters:

name (str) – specific name or id of a vacuum pump (e.g. Vac_A, etc.).
minpower (float) – minimum power of a vacuum pump.
maxpower (float) – maximum power of a vacuum pump.
minaction (float) – minimum action of a vacuum pump.
maxaction (float) – maximum action of a vacuum pump.
masscoeff (float) – mass transport coefficient of a vacuum pump.

reg_v

list of existing vacuum pumps.

Type:: list of objects

idx_v

length of reg_v.

Type:: int

name

specific name or id of a vacuum pump.

Type:: str

t_end_max

maximum end of activation time of a vacuum pump with respect to current time.

Type:: float

reg_v = []

t_end_max = 0

idx_v = 0

name = ''

start_t(now, duration, overwrite=False)

This method calculates the activation time and the end of activation time of the vacuum pump. This method is called, if the vacuum pump would like to be activated or updated.

Parameters:

now (float) – current time of the system.
duration (float) – duration of the vacuum pump being activated or the action by an agent in RL context.
overwrite (bool, optional) – To indicate whether the current operation can be overwritten or not.

calc_mass(now)

This method calculates the transported mass flow by the vacuum pump for a time step.

Parameters:: now (float) – current time of the system.
Returns:: cur_mass_transport – current transported mass.
Return type:: float

calc_power()

This method calculates the power consumption of a vacuum pump.

Returns:: cur_power – current power consumption.
Return type:: float

update(now)

This method calculates whether a vacuum pump must be deactived or not.

Parameters:: now (float) – current time of the system.

deactivate(): This method is used to deactivate a vacuum pump.

class mlpro.rl.pool.envs.bglp.Belt(name, actiontype, minpower, maxpower, minaction, maxaction, masscoeff)

Bases: Actuator

This class inherits Actuator class and serves as a child class of Actuator. This class represents a type of actuators in the BGLP environment, namely Belt. This class can be used for Conveyor Belt, Rotary Feeder, Vibratory Conveyor, or similar type of actuators. Belts are mostly used to transport material from silos to hoppers. However, the parameter of each actuator can be dissimilar to each other based on their settings.

Parameters:

name (str) – specific name or id of an actuator (e.g. Belt_A, etc.).
actiontype (str) – “C” for continuous action, “B” for binary action.
minpower (float) – minimum power of an actuator.
maxpower (float) – maximum power of an actuator.
minaction (float) – minimum action of an actuator.
maxaction (float) – maximum action of an actuator.
masscoeff (float) – mass transport coefficient of an actuator.

reg_b

list of existing actuators.

Type:: list of objects

idx_b

length of reg_b.

Type:: int

name

specific name or id of an actuator.

Type:: str

actiontype

“C” for continuous action, “B” for binary action.

Type:: str

speed

speed of an actuator.

Type:: float

reg_b = []

idx_b = 0

name = ''

actiontype = ''

speed = 0

start_t(now, duration, speed, overwrite=False)

This method calculates the activation time and the end of activation time of the belt. This method is called, if the belt would like to be activated or updated.

Parameters:

now (float) – current time of the system.
duration (float) – duration of the belt being activated, which being defined by the time set.
speed (float) – speed of the belt or the action by an agent in RL context.
overwrite (bool, optional) – To indicate whether the current operation can be overwritten or not.

calc_mass(now)

This method calculates the transported mass flow by the belt for a time step.

Parameters:: now (float) – current time of the system.
Returns:: cur_mass_transport – current transported mass.
Return type:: float

calc_power()

This method calculates the power consumption of a belt.

Returns:: cur_power – current power consumption.
Return type:: float

update(now)

This method calculates whether a belt must be deactived or not.

Parameters:: now (float) – current time of the system.

deactivate(): This method is used to deactivate a belt.

class mlpro.rl.pool.envs.bglp.Reservoir(vol_max, vol_init_abs=0)

Bases: object

This class serves as a parent class of different types of reservoirs, which provides the main attributes of a buffer in the BGLP environment.

Parameters:

vol_max (float) – maximum volume of a reservoir.
vol_init_abs (float, optional) – initial volume of a reservoir. The default is 0.

reg_r

list of existing reservoirs in the environment.

Type:: list of objects

idx_r

length of reg_r.

Type:: int

vol_max

maximum volume of a reservoir.

Type:: float

vol_init_abs

initial volume of a reservoir.

Type:: float

vol_cur_abs

current volume of a reservoir.

Type:: float

vol_cur_rel

current volume of a reservoir in percentage.

Type:: float

change

volume change of a reservoir in a time step.

Type:: float

reg_r = []

idx_r = []

vol_max = 0

vol_init_abs = 0

vol_cur_abs = 0

vol_cur_rel = 0

change = 0

set_change(vol_change)

This method sets up a volume change of a reservoir.

Parameters:: vol_change (float) – volume change of a reservoir in a time step.

update(): This method calculates the current volume of a reservoir after volume changes are made.

class mlpro.rl.pool.envs.bglp.Silo(name, vol_max, vol_cur=0, mode='abs')

Bases: Reservoir

This class inherits Reservoir class and serves as a child class of Reservoir. This class represents a type of buffers in the BGLP environment, namely Silo. Silos are used to temporary stored the transported materials. However, the parameter of each silo can be dissimilar to each other based on their settings.

Parameters:

name (str) – specific name or id of a silo (e.g. Silo_A, etc.).
vol_max (float) – maximum capacity of a silo.
vol_cur (float) – current volume of a silo.
mode (str, optional) – mode of measuring the current volume. “abs” means absolute value, “rel” means percentage. The default is “abs”.

reg_s

list of existing silos.

Type:: list of objects

idx_s

length of reg_s.

Type:: int

name

specific name or id of a silo.

Type:: str

type_

a short name for a silo, usually 3 capital letters (e.g SIL).

Type:: str

reg_s = []

name = ''

idx_s = []

type_ = ''

class mlpro.rl.pool.envs.bglp.Hopper(name, vol_max, vol_cur=0, mode='abs')

Bases: Reservoir

This class inherits Reservoir class and serves as a child class of Reservoir. This class represents a type of buffers in the BGLP environment, namely Hopper. Hoppers are used to temporary stored the transported materials. However, the parameter of each hopper can be dissimilar to each other based on their settings.

Parameters:

name (str) – specific name or id of a hopper (e.g. Hop_A, etc.).
vol_max (float) – maximum capacity of a hopper.
vol_cur (float) – current volume of a hopper.
mode (str, optional) – mode of measuring the current volume. “abs” means absolute value, “rel” means percentage. The default is “abs”.

reg_h

list of existing silos.

Type:: list of objects

idx_h

length of reg_h.

Type:: int

name

specific name or id of a hopper.

Type:: str

type_

a short name for a hopper, usually 3 capital letters (e.g HOP).

Type:: str

reg_h = []

name = ''

idx_h = []

type_ = ''

class mlpro.rl.pool.envs.bglp.BGLP(p_reward_type=1, p_visualize: bool = False, p_logging=True, t_step=0.5, t_set=10.0, demand=0.1, lr_margin=1.0, lr_demand=4.0, lr_power=0.001, margin_p=[0.2, 0.8, 4], prod_target=10000, prod_scenario='continuous', cycle_limit=0)

Bases: Environment

This is the main class of BGLP environment that inherits Environment class from MLPro.

Parameters:

p_reward_type (Reward, optional) – rewarding type. The default is Reward.C_TYPE_EVERY_AGENT.
p_visualize (bool) – Boolean switch for visualisation. Default = False.
p_logging (Log, optional) – logging functionalities. The default is Log.C_LOG_ALL.
t_step (float, optional) – time for each time step (in seconds). The default is 0.5.
t_set (float, optional) – time set for one horizon (in seconds). The default is 10.0.
demand (float, optional) – the constant output flow from the system (in L/s). The default is 0.1.
lr_margin (float, optional) – the learning rate for margin parameter, related to implemented reward function. The default is 1.0.
lr_demand (float, optional) – the learning rate for production demand, related to implemented reward function. The default is 4.0.
lr_power (float, optional) – the learning rate for power consumption, related to implemented reward function. The default is 0.0010.
margin_p (list of floats, optional) – the margin parameter of reservoirs [low, high, multplicator]. The default is [0.2,0.8,4].
prod_target (float, optional) – the production target for batch operation (in L). The default is 10000.
prod_scenario (str, optional) – ‘batch’ means batch production scenario and ‘continuous’ means continuous production scenario. The default is ‘continuous’.

C_NAME

name of the environment.

Type:: str

C_LATENCY

latency.

Type:: timedelta()

C_INFINITY

infinity.

Type:: np.finfo()

C_REWARD_TYPE

rewarding type.

Type:: Reward

sils

list of existing silos.

Type:: list of objects

hops

list of existing hoppers.

Type:: list of objects

ress

list of existing reservoirs.

Type:: list of objects

blts

list of existing belts.

Type:: list of objects

acts

list of existing actuators.

Type:: list of objects

vacs

list of existing vacuum pumps.

Type:: list of objects

con_act_to_res

connection between actuators and reservoirs and setup the sequence.

Type:: list of lists of int

m_param

the margin parameter of reservoirs [low, high, multplicator].

Type:: list of floats

_demand

the constant output flow from the system (in L/s).

Type:: float

t

current time of the system (in seconds).

Type:: float

t_step

time for each time step (in seconds).

Type:: float

lr_margin

the learning rate for margin parameter.

Type:: float

lr_demand

the learning rate for production demand.

Type:: float

lr_power

the learning rate for power consumption.

Type:: float

overflow

current overflow.

Type:: list of floats

power

current power consumption.

Type:: list of floats

transport

current transported mass flow.

Type:: list of floats

reward

current rewards.

Type:: list of floats

levels_init

initial level of reservoirs.

Type:: float

overflow_t

current overflow for a specific buffer in a specific time.

Type:: float

demand_t

current demand for a specific buffer in a specific time.

Type:: float

power_t

current power consumption for a specific actuator in a specific time.

Type:: float

transport_t

current transported material by a specific actuaor in a specific time.

Type:: float

margin_t

current margin for a specific buffer in a specific time.

Type:: float

prod_reached

current production reached in L for batch operation.

Type:: float

prod_target

the production target for batch operation (in L).

Type:: float

prod_scenario

‘batch’ means batch production scenario and ‘continuous’ means continuous production scenario.

Type:: str

cycle_limit

the number of cycle limit.

Type:: int

C_NAME = 'BGLP'

C_LATENCY = datetime.timedelta(seconds=1)

C_INFINITY = 3.4028235e+38

C_REWARD_TYPE = 1

con_act_to_res = []

m_param = []

prod_reached = 0

C_CYCLE_LIMIT = 0

t = 0

t_step = 0

_demand = 0

lr_margin = 0

lr_demand = 0

lr_power = 0

prod_target = 0

prod_scenario = 0

levels_init = 0

sils = []

hops = []

ress = []

blts = []

vacs = []

acts = []

overflow = []

demand = []

power = []

transport = []

overflow_t = 0

demand_t = 0

power_t = 0

transport_t = 0

margin_t = 0

reward = []

static setup_spaces()

This method is used to setup action and state spaces of the system.

The actions and states are normalized between 0 to 1. For the actions, 0 means minimum action and 1 means maximum action. Meanwhile, for the states, 0 means minimum capacity (empty) and 1 means maximum capacity (full)

Returns:

state_space (ESpace()) – state space of the system.
action_space (ESpace()) – action space of the system.

collect_substates() → State

This method is called during resetting the environment.

Returns:: state – current states.
Return type:: State

_reset(p_seed=None) → None

This method is used to reset the environment.

Parameters:: p_seed (int) – Seed parameter for an internal random generator.

_simulate_reaction(p_state: State, p_action: Action) → State

This method is used to calculate the next states of the system after a set of actions.

Parameters:

p_state (State) – State.
p_action (Action) – ACtion.

Returns:

_state – current states.

Return type:

State

_compute_success(p_state: State) → bool

This method computes the success flag. This method can be redefined.

Parameters:: p_state (State) – state.
Returns:: success or not success.
Return type:: bool

_compute_broken(p_state: State) → bool

This method computes the broken flag. This method can be redefined.

Parameters:: p_state (State) – state.
Returns:: broken or not.
Return type:: bool

_compute_reward(p_state_old: State, p_state_new: State) → Reward

This method calculates the reward for different reward types.

Parameters:

p_state_old (State) – previous state.
p_state_new (State) – new state.

Returns:

reward – reward values.

Return type:

Reward

calc_mass_flows(): This method calculates the mass flow transported by actuators.

calc_power(): This method calculates the power consumptions per actuator.

calc_margin(): This method calculates margin level for every reservoir.

set_volume_changes(demandval): This method sets up volume changes for every reservoir.

update_levels(): This method updates the current level of reservoirs.

reset_levels(): This method resets reservoirs.

reset_actuators(): This method resets actuators.

update_actuators(): This method updates actuators.

update(demandval)

This method sets up volume changes, updates reservoirs’ level, and updates actuators.

Parameters:: demandval (float) – production outflow target in L/s.

get_status(t, demandval)

This method calculates overflow, demand, power, transport, and margin. This function will be called every time step.

Parameters:

t (float) – current time in sec.
demandval (float) – production outflow target in L/s.

Returns:

overflow (list of floats) – overflow levels.
demand (list of floats) – demand fulfilled.
power (list of floats) – power consumptions.
transport (list of floats) – transported materials.
margin (list of floats) – margin levels.

set_actions(actions): This method sets up actions for actuators. This function will be called every time set.

calc_state()

This method obtains current levels of reservoirs.

Returns:: levels – level of each reservoir.
Return type:: list of floats

calc_reward()

This method calculates the reward. This method can be redifined! The current reward function is suitable for continuous operation and scalar reward for individual agents.

Returns:: reward – reward for each agent.
Return type:: list of floats

init_plot(p_figure=None)

Initializes the plot functionalities of the class.

Parameters:

p_figure (Matplotlib.figure.Figure, optional) – Optional MatPlotLib host figure, where the plot shall be embedded. The default is None.
p_plot_settings (PlotSettings) – Optional plot settings. If None, the default view is plotted (see attribute C_PLOT_DEFAULT_VIEW).

update_plot()

Updates the plot.

Parameters:: **p_kwargs – Implementation-specific plot data and/or parameters.