PyTorch-based MLP
Ver. 1.2.4 (2023-07-14)
This module provides a template ready-to-use MLP model using PyTorch.
- class mlpro.sl.pool.afct.fnn.pytorch.mlp.PyTorchMLP(p_input_space: ~mlpro.bf.math.basics.MSpace, p_output_space: ~mlpro.bf.math.basics.MSpace, p_output_elem_cls=<class 'mlpro.bf.math.basics.Element'>, p_threshold=0, p_buffer_size=0, p_ada: bool = True, p_visualize: bool = False, p_logging=True, **p_kwargs)
Bases:
MLP,PyTorchHelperFunctionsTemplate class for an adaptive bi-multivariate mathematical function that adapts by supervised learning using PyTorch-based MLP.
- Parameters:
p_input_space (MSpace) – Input space of function
p_output_space (MSpace) – Output space of function
p_output_elem_cls – Output element class (compatible to/inherited from class Element)
p_threshold (float) – Threshold for the difference between a setpoint and a computed output. Computed outputs with a difference less than this threshold will be assessed as ‘good’ outputs. Default = 0.
p_buffer_size (int) – Initial size of internal data buffer. Default = 0 (no buffering).
p_ada (bool) – Boolean switch for adaptivity. Default = True.
p_visualize (bool) – Boolean switch for visualisation. Default = False.
p_logging – Log level (see constants of class Log). Default: Log.C_LOG_ALL
p_kwargs (Dict) – Further model specific parameters (to be specified in child class).
- C_TYPE = 'PyTorch-based Adaptive Function using MLP'
- C_BUFFER_CLS
alias of
PyTorchBuffer
- _setup_model() Sequential
A method to set up a supervised learning network.
- Return type:
A set up supervised learning model
- _init_hyperparam(**p_par)
A method to deal with the hyperparameters related to the MLP model.
Hyperparameters
- p_update_rate :
update rate.
- p_num_hidden_layers :
number of hidden layers.
- p_output_activation_fct :
extra activation function for the output layer.
- p_optimizer :
optimizer.
- p_loss_fct :
loss function.
- p_test_datafloat
the proportion of test data during the sampling process. Default = 0.3.
- p_batch_sizeint
batch size of the buffer. Default = 100.
- p_seed_bufferint
seeding of the buffer. Default = 1.
- p_learning_ratefloat
learning rate of the optimizer. Default = 3e-4.
- p_hidden_sizeint or list
number of hidden neurons. There are two possibilities to set up the hidden size: 1) if hidden_size is an integer, then the number of neurons in all hidden layers are exactly the same. 2) if hidden_size is in a list, then the user can define the number of neurons in each hidden layer, but make sure that the length of the list must be equal to the number of hidden layers.
- p_activation_fcttorch.nn or list
activation function. There are two possibilities to set up the activation function: 1) if activation_fct is a single activation function, then the activation function after all hidden layers are exactly the same. 2) if activation_fct is in a list, then the user can define the activation function after each hidden layer, but make sure that the length of the list must be equal to the number of hidden layers.
- p_weight_bias_initbool, optional
weight and bias initialization. Default : True
- p_weight_inittorch.nn, optional
weight initialization function. Default : torch.nn.init.orthogonal_
- p_bias_inittorch.nn, optional
bias initilization function. Default : lambda x: torch.nn.init.constant_(x, 0)
- p_gain_initint, optional
gain parameter of the weight and bias initialization. Default : np.sqrt(2)
- _add_init(p_model)
This method is optional and is intended for additional initialization process of the _setup_model.
- Parameters:
p_model – model network
- Return type:
updated model network
- _map(p_input: Element, p_output: Element)
Maps a multivariate abscissa/input element to a multivariate ordinate/output element.
- _add_buffer(p_buffer_element: PyTorchIOElement)
This method has a functionality to add data into the buffer.
- Parameters:
p_buffer_element (PyTorchIOElement) – An element of PyTorchBuffer.
- _calc_loss(p_act_output: Tensor, p_pred_output: Tensor) float
This method has a functionality to evaluate the adapted SL model.
- Parameters:
p_act_output (torch.Tensor) – Actual output from the buffer.
p_pred_output (torch.Tensor) – Predicted output by the SL model.
- _optimize(p_loss)
This method provides provide a funtionality to call the optimizer of the feedforward network.
- _adapt_online(p_input: Element, p_output: Element) bool
Adaptation mechanism for PyTorch based model for online learning.
- _adapt_offline(p_dataset: dict) bool
Adaptation mechanism for PyTorch based model for offline learning.
- Parameters:
p_dataset (dict) – a dictionary that consists of a set of data, which are splitted to 2 keys such as input and output. The value of each key is a torch.Tensor of the sampled data.
- Return type:
bool
- forward(p_input: Tensor) Tensor
Forward propagation in neural networks to generate some output using PyTorch.
- _reduce_state(p_state: dict, p_path: str, p_os_sep: str, p_filename_stub: str)
Custom method to reduce the given object state by components that can not be pickled. Further data files can be created in the given path and should use the given filename stub.
- Parameters:
p_state (dict) – Object state dictionary to be reduced by components that can not be pickled.
p_path (str) – Path to store further optional custom data files
p_os_sep (str) – OS-specific path separator.
p_filename_stub (str) – Filename stub to be used for further optional custom data files
- _complete_state(p_path: str, p_os_sep: str, p_filename_stub: str)
Custom method to complete the object state (=self) from external data sources. This method is called by standard method __setstate__() during unpickling the object from an external file.
- Parameters:
p_path (str) – Path of the object pickle file (and further optional related files)
p_os_sep (str) – OS-specific path separator.
p_filename_stub (str) – Filename stub to be used for further optional custom data files