Introduction
Start:
#br
>>''SkyAI is a highly modularized '''Reinforcement Learni...
#br
''Table of Contents''
#contents
* Introduction to SkyAI [#b37b9f65]
Designing a behavior by only its objective is essential f...
RL algorithms play an important role in the SkyAI. Thus,...
This introduction is described from a robotics viewpoint....
** What is RL? [#h5fd157e]
Reinforcement learning is a very general approach to enab...
*** Usual Robot Development [#u9f2c1c0]
In general, a developer of a robot have to specify the fo...
- state (sensors) definition,
- action (actuators) definition,
- relation between the sensors, or relation between the s...
- programs with which the robot behaves.
These may be difficult to the end-users of the robot.
*** Using RL [#o1c38ff2]
However, RL algorithms relax them; instead, the robot tri...
- state (sensors) definition,
- action (actuators) definition,
- ''reward'' function which indicates the objective of th...
The state and action definitions can be pre-defined by th...
*** Reward-based Behavior Design [#f24f68b2]
The reward function is the key. It is a scalar function ...
For instance, if we want the robot to walk forward, we ca...
After the reward function is specified, the robot begins ...
*** Adaptable to Unknown Environment [#q8aaeb34]
A remarkable thing is that we do not need to specify the ...
*** Many Applications [#od6df377]
Thus, a lot of researches of RL applications are performe...
- [[MPI Robot Learning Lab:http://robot-learning.de/]] (s...
- [[Video Highlights:http://groups.csail.mit.edu/locomoti...
- [[researches:http://robotics.naist.jp/~akihiko-y/resear...
** Issue of RL and Solutions [#v3f38c1b]
However, there is a big issue of RL methods. Namely, the...
Many researchers are tackling to the learning cost issue....
- dimension reduction,
- model utilization,
- hierarchical structure,
- imitation of the others,
- reusing already learned knowledge.
** Concepts of SkyAI [#u6b7b872]
*** Solving the RL Issues, Creating a Robot's Intelligenc...
The aim of the SkyAI is to integrate these ideas so that ...
*** Highly Modularized Architecture [#q5e19e0d]
The approach of the SkyAI is ''modularization'' of the me...
:High scalability|Modular architecture makes it easy to i...
:High reusability of implementation|Modular architecture ...
:High reusability of learned knowledge|Modular architectu...
*** Once Compiled, Reconfigurable Infinitely [#kce62973]
The SkyAI must be executed in high speed, and should be h...
These are very important features to apply the SkyAI to a...
Thus, the SkyAI is developed so that after a software usi...
*** Developer Friendly [#a170efc9]
Highly-modularized architecture has many benefits as ment...
** Implementation [#zc6cbd26]
Based on the concepts, the SkyAI is implemented as follows.
*** Written in C++ [#l5e417fd]
A program written in C++ is compiled to an executable cod...
*** Core Modular Architecture [#t801758c]
A module is implemented as a class in C++.
In general, communication between classes is performed by...
To provide some basic functions, such as a script interfa...
*** Script Interface [#h39ce6f2]
A script language is defined for the SkyAI. In the scrip...
*** Bridge between C++ Code and Script [#p94a7fe2]
Some macros are provided to make ports easily. The ports...
#codeh(cpp){{
class MSomeModule : public TModuleInterface
{
...
protected:
MAKE_SIGNAL_PORT(signal_execute_action, void(const TAct...
...
};
}}
Thus, making ports does not require a kind of code genera...
The SkyAI also provides a way to handle the parameters of...
#codeh(cpp){{
struct TSomeConfigurations
{
int X;
double Y;
std::vector<double> VecData;
std::vector<std::list<double> > CmpData;
}
}}
just define a "Register" member function and a constructo...
#codeh(cpp){{
TSomeConfigurations (TVariableMap &mmap)
{
Register(mmap);
}
void Register (TVariableMap &mmap)
{
AddToVarMap(mmap, "X" , X);
AddToVarMap(mmap, "Y" , Y);
AddToVarMap(mmap, "VecData", VecData);
AddToVarMap(mmap, "CmpData", CmpData);
}
}}
Then, you can assign the values in the script language as...
#codeh(cpp){{
X = 12
Y = -2.1e+4
VecData = (1, 2.236, 3)
CmpData = {
[] = (0.1, 0.2, 0.3) // push back a list
[] = (0.4, 0.5, 0.6) // ditto
}
}}
Thus, if a developer is familiar with C++, it is easy for...
** Applying SkyAI [#r497cdc9]
In order to apply the SkyAI to a specific problem (a robo...
- a robot module (or an environment module), and,
- a task module.
These modules should be compatible with the modules provi...
Maybe, a developer needs to make some additional modules,...
** Benchmarks [#f0a7ba12]
Some benchmark programs are also provided with the SkyAI....
// Please see the ScreenshotsAndMovies Screenshots and Mo...
** Related Works [#n4476f68]
- [[RL-Library:http://library.rl-community.org/]]
- [[RL-Glue:http://glue.rl-community.org/]]
- [[PyBrain:http://pybrain.org/]]
- [[LibPG:http://code.google.com/p/libpgrl/]]
- [[Reinforcement Learning Toolbox:http://www.igi.tugraz....
End:
#br
>>''SkyAI is a highly modularized '''Reinforcement Learni...
#br
''Table of Contents''
#contents
* Introduction to SkyAI [#b37b9f65]
Designing a behavior by only its objective is essential f...
RL algorithms play an important role in the SkyAI. Thus,...
This introduction is described from a robotics viewpoint....
** What is RL? [#h5fd157e]
Reinforcement learning is a very general approach to enab...
*** Usual Robot Development [#u9f2c1c0]
In general, a developer of a robot have to specify the fo...
- state (sensors) definition,
- action (actuators) definition,
- relation between the sensors, or relation between the s...
- programs with which the robot behaves.
These may be difficult to the end-users of the robot.
*** Using RL [#o1c38ff2]
However, RL algorithms relax them; instead, the robot tri...
- state (sensors) definition,
- action (actuators) definition,
- ''reward'' function which indicates the objective of th...
The state and action definitions can be pre-defined by th...
*** Reward-based Behavior Design [#f24f68b2]
The reward function is the key. It is a scalar function ...
For instance, if we want the robot to walk forward, we ca...
After the reward function is specified, the robot begins ...
*** Adaptable to Unknown Environment [#q8aaeb34]
A remarkable thing is that we do not need to specify the ...
*** Many Applications [#od6df377]
Thus, a lot of researches of RL applications are performe...
- [[MPI Robot Learning Lab:http://robot-learning.de/]] (s...
- [[Video Highlights:http://groups.csail.mit.edu/locomoti...
- [[researches:http://robotics.naist.jp/~akihiko-y/resear...
** Issue of RL and Solutions [#v3f38c1b]
However, there is a big issue of RL methods. Namely, the...
Many researchers are tackling to the learning cost issue....
- dimension reduction,
- model utilization,
- hierarchical structure,
- imitation of the others,
- reusing already learned knowledge.
** Concepts of SkyAI [#u6b7b872]
*** Solving the RL Issues, Creating a Robot's Intelligenc...
The aim of the SkyAI is to integrate these ideas so that ...
*** Highly Modularized Architecture [#q5e19e0d]
The approach of the SkyAI is ''modularization'' of the me...
:High scalability|Modular architecture makes it easy to i...
:High reusability of implementation|Modular architecture ...
:High reusability of learned knowledge|Modular architectu...
*** Once Compiled, Reconfigurable Infinitely [#kce62973]
The SkyAI must be executed in high speed, and should be h...
These are very important features to apply the SkyAI to a...
Thus, the SkyAI is developed so that after a software usi...
*** Developer Friendly [#a170efc9]
Highly-modularized architecture has many benefits as ment...
** Implementation [#zc6cbd26]
Based on the concepts, the SkyAI is implemented as follows.
*** Written in C++ [#l5e417fd]
A program written in C++ is compiled to an executable cod...
*** Core Modular Architecture [#t801758c]
A module is implemented as a class in C++.
In general, communication between classes is performed by...
To provide some basic functions, such as a script interfa...
*** Script Interface [#h39ce6f2]
A script language is defined for the SkyAI. In the scrip...
*** Bridge between C++ Code and Script [#p94a7fe2]
Some macros are provided to make ports easily. The ports...
#codeh(cpp){{
class MSomeModule : public TModuleInterface
{
...
protected:
MAKE_SIGNAL_PORT(signal_execute_action, void(const TAct...
...
};
}}
Thus, making ports does not require a kind of code genera...
The SkyAI also provides a way to handle the parameters of...
#codeh(cpp){{
struct TSomeConfigurations
{
int X;
double Y;
std::vector<double> VecData;
std::vector<std::list<double> > CmpData;
}
}}
just define a "Register" member function and a constructo...
#codeh(cpp){{
TSomeConfigurations (TVariableMap &mmap)
{
Register(mmap);
}
void Register (TVariableMap &mmap)
{
AddToVarMap(mmap, "X" , X);
AddToVarMap(mmap, "Y" , Y);
AddToVarMap(mmap, "VecData", VecData);
AddToVarMap(mmap, "CmpData", CmpData);
}
}}
Then, you can assign the values in the script language as...
#codeh(cpp){{
X = 12
Y = -2.1e+4
VecData = (1, 2.236, 3)
CmpData = {
[] = (0.1, 0.2, 0.3) // push back a list
[] = (0.4, 0.5, 0.6) // ditto
}
}}
Thus, if a developer is familiar with C++, it is easy for...
** Applying SkyAI [#r497cdc9]
In order to apply the SkyAI to a specific problem (a robo...
- a robot module (or an environment module), and,
- a task module.
These modules should be compatible with the modules provi...
Maybe, a developer needs to make some additional modules,...
** Benchmarks [#f0a7ba12]
Some benchmark programs are also provided with the SkyAI....
// Please see the ScreenshotsAndMovies Screenshots and Mo...
** Related Works [#n4476f68]
- [[RL-Library:http://library.rl-community.org/]]
- [[RL-Glue:http://glue.rl-community.org/]]
- [[PyBrain:http://pybrain.org/]]
- [[LibPG:http://code.google.com/p/libpgrl/]]
- [[Reinforcement Learning Toolbox:http://www.igi.tugraz....
Page:
![[SkyAI]](image/logoxs.png "[SkyAI]")
