6.2. Getting Started

Here is a concise series to introduce all users to the MLPro-RL in a practical way, whether you are a first-timer or an experienced MLPro user.

If you are a first-timer, then you can begin with Section (1) What is MLPro?.

If you have understood MLPro but not reinforcement learning, then you can jump to Section (2) What is Reinforcement Learning?.

If you have experience in both MLPro and reinforcement learning, then you can directly start with Section (3) What is MLPro-RL?.

After following the below step-by-step guideline, we expect the user understands the MLPro-RL in practice and starts using MLPro-RL.

1. What is MLPro?

If you are a first-time user of MLPro, you might wonder what is MLPro. Therefore, we recommend initially start with understanding MLPro by checking out the following steps:

1. MLPro: An Introduction

2. introduction video of MLPro

3. installing and getting started with MLPro

4. MLPro paper in Software Impact journal

2. What is Reinforcement Learning?

If you have not dealt with reinforcement learning, we recommend starting to understand at least the basic concept of reinforcement learning. There are plenty of references, articles, papers, books, or videos on the internet that explains reinforcement learning. But, for deep understanding, we recommend you to read the book from Sutton and Barto, which is Reinforcement Learning: An Introduction.

3. What is MLPro-RL?

We expect that you have a basic knowledge of MLPro and reinforcement learning. Therefore, you need to understand the overview of MLPro-RL by following the steps below:

1. MLPro-RL introduction page

2. Section 4 of MLPro 1.0 paper

4. Understanding Environment in MLPro-RL

First of all, it is important to understand the structure of an environment in MLPro, which can be found on this page.

Then, you can start following some of our howto files related to the environment in MLPro-RL, as follows:

1. Howto RL-001: Reward

2. Howto RL-AGENT-001: Run an Agent with Own Policy

5. Understanding Agent in MLPro-RL

In reinforcement learning, we have two types of agents, such as a single-agent RL or a multi-agent RL. Both of the types are covered by MLPro-RL. To understand the different possibilities of an agent in MLPro, you can visit this page.

Then, you need to understand how to set up a single-agent and a multi-agent RL in MLPro-RL by following these examples:

1. Howto RL-AGENT-001: Run an Agent with Own Policy

2. Howto RL-AGENT-003: Run Multi-Agent with Own Policy

6. Selecting between Model-Free and Model-Based RL

In this section, you need to select your direction of the RL training, whether it is a model-free RL or a model-based RL. However, firstly, you can pay attention to these two pages, which are RL scenario and training, before selecting either of the paths below.

• Model-Free Reinforcement Learning

  To practice model-free RL in the MLPro-RL package, here are a video and some ready-to-use howto files that can be followed:

  1. A sample application video of MLPro-RL on a UR5 robot

  2. Howto RL-AGENT-002: Train an Agent with Own Policy

  3. Howto RL-AGENT-004: Train Multi-Agent with Own Policy

• Model-Based Reinforcement Learning

  Model-based RL contains two learning paradigms, such as learning the environment (model-based learning) and utilizing the model (e.g. as an action planner). To practice model-based RL in the MLPro-RL package, here are a howto file that can be followed:

  1. Howto RL-MB-001: Train and Reload Model Based Agent (Gym)

  2. Howto RL-MB-002: MBRL with MPC on Grid World Environment

  For more advanced MBRL technique, e.g. applying a native MBRL network, here is an example that can be used as a reference:

  3. Howto RL-MB-003: MBRL on RobotHTM Environment

7. Additional Guidance

After following the previous steps, we hope that you could practice MLPro-RL and start using this subpackage for your RL-related activities. For more advanced features, we highly recommend you to check out the following howto files:

1. Howto RL-AGENT-011: Train and Reload Single Agent (Gym)

2. Howto RL-AGENT-021: Train and Reload Single Agent (MuJoCo)

3. Howto RL-HT-001: Hyperopt

4. Howto RL-HT-002: Optuna

5. Howto RL-ATT-001: Stagnation Detection

6. Howto RL-ATT-002: SB3 Policy with Stagnation Detection