Elements †

In the SkyAI's architecture, the following elements are important:

Modular architecture
The most important architecture of SkyAI is the modularization of the RL or the other machine-learning algorithms. The modular architecture enables the high reusability and the high extensibility.
Writing in C++
To achieve the high execution-speed, SkyAI is implemented in C++. In general, a C++ source code is compiled to an executable whose execution speed is almost the highest level. Each module is implemented as a class of C++. The classes communicate with each other through member functions. We basically use call-by-reference in these functions for the high-speed communication.
Modular implementation
In order to achieve the high flexibility, the modular structure should be changed during execution after compiling the source code. SkyAI wraps the C++ class system, since the member functions for the inter-class communication are needed to be connected and disconnected. Thus, each member function is encapsulated as a port class. Each module can have any number of ports. Ports can be connected and disconnected at any time in execution, which enables to reconfigure the modular structure.
Script interface
A script language is defined to provide an interface of modular manipulations during execution. Specifically, instantiating modules, connecting ports, and setting parameters of the modules (e.g. a learning rate) can be specified in the script language. We refer to the script of SkyAI as an agent script.
Developer's toolkit
To make a module program compatible with the modular architecture, the program should follow some rules and regulations of SkyAI. SkyAI provides some macros and templates*1 to support the developers to easily write system-compatible source code. The macros and the templates can be processed by a standard preprocessor and compiler; i.e. code generators are not required. Thus, it is easy for the developers to modify source code with the system-compatibility.
↑

System using SkyAI †

The center of a software using SkyAI is an agent class (TAgent). The agent class manages whole module instances, and has a parser of the script language. The agent class is provided as generic one, that is, available in any applications. A user instantiates the agent class and calls the parser from the C++ source code (basically, in the main function).

This figure illustrates an example modular structure around an RL module. The Action Space module emits a signal control command to a low-level controller. In an on-line learning system, there are several kinds of cycles, such as episode, action, and time step of low-level controller. The SkyAI modular architecture can handle any kinds of cycles as shown in the figure.

When a module-instantiation directive is written in a script, the agent class instantiate the module, i.e. allocates memory. The agent class also handles directives from a script to connect modules or to set parameters.

The SkyAI architecture enables to modularize RL algorithms as generic ones. On the other hand, the user of SkyAI should make task or robot specific modules, such as a low-level robot controller. Thus, in order to apply SkyAI,

  1. the user implements some task or robot specific modules, then
  2. the user builds them with the provided modules and the agent class, finally
  3. the user writes a script for a specific task.
*1 Strictly, template functions and template classes of C++.
Front page   New List of pages Search Recent changes   Help   RSS of recent changes