Edit of Documentation/Running Demos/bioloid

* Running Bioloid Demos [#j441959c]

These are demonstrations with a real robot.

** Overview [#fd89ee33]

The robot employed here is [[Bioloid:http://www.robotis.com/zbxe/bioloid_en]], made by [[ROBOTIS:http://www.robotis.com/]].
To execute the following demos, at least, Bioloid Comprehensive Kit is needed.
However, you can use the other type of Bioloid Kit by changing the agent scripts of these demos.

Note: we do not guarantee that the demos work with a real robot (but can advise a way to improve :-).

We use the King Spider model of Bioloid.

#ref(kingspider2b.png,center,zoom,300x0)

A crawling task is performed with this robot.

The objective of the crawling task is to move forward as far as possible.
So, the reward is designed so that it is proportional to the velocity of the robot.
Bioloid has an infrared ray (IR) sensor to observe the distance from the robot to an obstacle.
We use the IR sensor and calculate the velocity for the reward.

The following figure shows a setup of the environment.

#ref(rl-bioloid-env.png,center,zoom,400x0)

Each episode begins with an initial pose, and ends if t>50[s], the robot touches the wall (determined by the operator), or some problems arise.

** Build the Demo Program [#yc05a0c6]
Execute:

$ cd benchmarks/bioloid
  $ make

See [[Documentation/Installation Guide]] for the detail.

** Running Command [#k4c65b7e]
Please read [[Common Usage>../Common Usage]] in advance.
The running command is:
  $ ./DEMO_PRG -path PATH_LIST -agent AGENT_FILE -outdir OUT_DIR
The demo-specific elements are:
:DEMO_DIR| bioloid (benchmarks/bioloid).
:DEMO_PRG| bioloid.out
:PATH_LIST| ../cmn,m,m/cr
:AGENT_FILE| Available agent scripts are listed below.
:OUT_DIR| Any directory is possible.

For example, execute the following:
  $ mkdir -p result/rl1
  $ ./bioloid.out -path ../cmn,m,m/cr -agent ql_dcob1 -outdir result/rl1

To exit the program, press Ctrl+c on the terminal, then press 'Q' (shift+q).

In OUT_DIR (result/rl1), the result files are stored.
For example, use gnuplot to plot the learning curve as:
  $ gnuplot
  gnuplot> plot 'result/rl1/log-eps-ret.dat' w l

#ref(rl1-eps-ret.png,center,zoom,300x0)

** Agent Script [#daad775b]
The following files can be specified as AGENT_FILE.

:ql_dcob1 | 5-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
:ql_dcob2 | 2-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
:ql_dcob3 | 3-DoF, Q(lambda)-learning, DCOB (action space), linear action value function (NGnet).
:ql_grid1 | 5-DoF, Q(lambda)-learning, Grid action space, linear action value function (NGnet).
:z1 | For the Dinosaur model of Bioloid (5-DoF).

** Miscellaneous [#haf60902]
*** Serial Communication Trouble [#v8ab3833]
(This part was written in Jun. 10, 2011)

We tested our bioloid program in some linux distributions, and found that some of them have a trouble in the serial communication.  Concretely, the program failed to connect bioloid.

The following way can fix this issue.

First, remove the pl2303 module from the kernel by:
  sudo modprobe -r pl2303
Then, reload the same module:
  sudo modprobe pl2303

We found that this problem arises in Ubuntu ''ver. 10.10'' and ''ver. 11.04''.
This problem is discussed in [[Ubuntu Forums:http://ubuntuforums.org/showthread.php?t=1589316]].

Do not change timestamp