Running Bioloid Demos

These are demonstrations with a real robot.

Overview

The robot employed here is Bioloid, made by ROBOTIS. To execute the following demos, at least, Bioloid Comprehensive Kit is needed. However, you can use the other type of Bioloid Kit by changing the agent scripts of these demos.

Note: we do not guarantee that the demos work with a real robot (but can advise a way to improve :-).

We use the King Spider model of Bioloid.

kingspider2b.png

A crawling task is performed with this robot.

The objective of the crawling task is to move forward as far as possible. So, the reward is designed so that it is proportional to the velocity of the robot. Bioloid has an infrared ray (IR) sensor to observe the distance from the robot to an obstacle. We use the IR sensor and calculate the velocity for the reward.

The following figure shows a setup of the environment.

rl-bioloid-env.png

Each episode begins with an initial pose, and ends if t>50[s], the robot touches the wall (determined by the operator), or some problems arise.

Build the Demo Program

Execute:

 $ cd benchmarks/bioloid
 $ make

See Documentation/Installation Guide for the detail.

Running Command

Please read Common Usage in advance. The running command is:

 $ ./DEMO_PRG -path PATH_LIST -agent AGENT_FILE -outdir OUT_DIR

The demo-specific elements are:

DEMO_DIR
bioloid (benchmarks/bioloid).
DEMO_PRG
bioloid.out
PATH_LIST
../cmn,m,m/cr
AGENT_FILE
Available agent scripts are listed below.
OUT_DIR
Any directory is possible.

For example, execute the following:

 $ mkdir -p result/rl1
 $ ./bioloid.out -path ../cmn,m,m/cr -agent ql_dcob1 -outdir result/rl1

To exit the program, press Ctrl+c on the terminal, then press 'Q' (shift+q).

In OUT_DIR (result/rl1), the result files are stored. For example, use gnuplot to plot the learning curve as:

 $ gnuplot
 gnuplot> plot 'result/rl1/log-eps-ret.dat' w l
rl1-eps-ret.png

Agent Script

The following files can be specified as AGENT_FILE.

ql_dcob1
5-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
ql_dcob2
2-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
ql_dcob3
3-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
ql_grid1
5-DoF, Q(lambda)-learning, Grid action space, linear action value function (NGnet).
z1
For the Dinosaur model of Bioloid (5-DoF).

Miscellaneous

Serial Communication Trouble

(This part was written in Jun. 10, 2011)

We tested our bioloid program in some linux distributions, and found that some of them have a trouble in the serial communication. Concretely, the program failed to connect bioloid.

The following way can fix this issue.

First, remove the pl2303 module from the kernel by:

 sudo modprobe -r pl2303

Then, reload the same module:

 sudo modprobe pl2303

We found that this problem arises in Ubuntu ver. 10.10 and ver. 11.04. This problem is discussed in Ubuntu Forums.

 

 



Attach file: filerl1-eps-ret.png 1302 download [Information] filerl-bioloid-env.png 1254 download [Information] filekingspider2b.png 1401 download [Information]

Front page   Edit Freeze Diff Backup Upload Copy Rename Reload   New List of pages Search Recent changes   Help   RSS of recent changes
Last-modified: 2011-06-10 (Fri) 02:25:19 (3102d)