Proj-2013-2014-BrasRobot-Handicap-1

Description and Goals

The long-term goal of this project is to develop a control software manipulator arm for support of persons with disabilities.

Team

• Supervisor: Olivier Richard

• El Hadji Malick FALL, Adji Ndèye Ndaté SAMBE

• Department : RICM 4, Polytech Grenoble

Documents

The Software Requirements Specification (SRS) can be found here

Steps

The projet was attributed on Monday 13th and started the following day January 14th, 2014

Week 1 (January 13th - Janurary 19th)

• Project discovery
• Meeting with the different members of the project for assigning roles. Each pair chooses the part of the project that it wants to process.We choose to process the part on the detection of markers.
• Research on marker detection

Week 2 (January 20th - Janurary 26th)

• Test configuration of the robot
• Learning of C++
• Discovery of OpenCV

In details, we looked at the record and the code of the former group and studied the methods they proposed for the markers detection including hough transform and harritraining. Then we make some researchs to know how OpenCv works. One of the question is what will be the criteria of the markers : color, shape of both.

Week 3 (January 27th - February 2nd)

• Discovery of ArUco

ArUco is a minimal library for Augmented Reality applications based on OpenCv. One of its features of ArUco is Detect markers with a single line of C++ code. This will be very useful for our approach. As it is described in the source forge website, each marker has an unique code indicated by the black and white colors in it. The library detect borders, and analyzes into the rectangular regions which of them are likely to be markers. Then, a decoding is performed and if the code is valid, it is considered that the rectangle is a marker. So our next step will be to choose what kind of marker we will use and define the various steps of the recovery of the marker by the robot. We also start to learn Python langage which will be used to develop ArUco solution. Installing PyDev requires much more time as we though. After successfull completion, we are now developing our own solutions in Python inspired by ArUco's.

Week 4 (February 3rd- February 9th)

As we said, our own solutions are on going process. We expect that in this week they will be completed and fortunately we managed to make our first tests with ArUco. We have experienced a solution with black and white markers with ArUco, and a second with a face detection with OpenCV. The next step is now to be able to test it on video to first measure the efficiency and then take back the coordinates of marker's position that will be interesting for the system group.

facemarker

Arucomarker

Week 5 (February 10th- February 16th)

After a long trying, we have finally succeed to detect several markers contained in a jpeg picture and avi video. We are also able to get back the space coordinates of the marker. We could write it in a file that we will be used by the system team. Our next step is now to detect markers in a live video.

detectedmarkers

MarkersOnVideo

But even if Aruco solution is functional now, we can consider encounter some difficulties. For that OpenCV is still an optional solution. We have already made ​​contact with the Canonball group about it. Knowing that we can retrieve the coordinates x, y, z of the marker, we expect, following our discussion with the group system, to transcribe these data into a text file they will use. The advantage of such a solution is that if the solution proposed by the system group is not practical, we can always use this file to prove the proper functioning of our solution and use it to find another alternative. In the meantime we do our first live detection tests. The first results are rather conclusive but now we will have to determine several scenarios that may happen.

MarkersLive

Week 6 (February 17th- February 23th)

For this week, after making our first tests of real-time detection, we must renew them but this time with the webcam at our disposal. In Aruco, more generally in OpenCV, we have a type VideoCapture from the class of the same name that manages the video stream. It provides an open function whose signature is as follows:

0 is the default webcam of the computer and 1 the first outside webcam connected to the USB port. After making these changes, we are now able to use the webcam provided. Moreover, it should be noted that this phase of the project is a milestone. Before going further, we wanted to make an assessment of future solutions before integration. We first started to make a series of measures to evaluate the performance of detection.

We find that the closer the distance, the greater the chances of detecting a marker. The optimal distance is estimated at 14 cm or less , and that under favorable conditions ( well calibrated light, visible marker) . We also conducted tests on the number of markers needed . We generated a matrix of markers and we tested detection :

Detecting a single marker might fail for different reasons such as poor lightning conditions, fast camera movement, occlusions, etc. To overcome that problem, ArUco allows the use of boards. An Augmented Reality board is a marker composed by several markers arranged in a grid. Boards present two main advantages. First, since there have more than one markers, it is less likely to lose them all at the same time. Second, the more markers are detected, the more points are available for computing the camera extrinsics. As a consecuence, the accuracy obtained increases.

We can see that the ideal would be to have a minimum size of 4x5 to ensure optimal detection . ie we are sure that at least one marker will be detected. Based on these parameters, we can describe a first draft screenplay . The robot detects the marker. By applying the Pythagorean theorem , we can determine the coordinates of the central point. Once these coordinates obtained, the robot performs a translation to this point, as in the diagram below:

Week 7 (February 24th- March 2nd)

During this week we have improved the scenario drawn up last week. After identifying the actions that the robot will perform, we will now look at another problem: how the robot will know the coordinates of the point to be reached?

We considere two possible solutions:

- Write the coordinates on a file and parse it

- Transmit the coordinates through a socket

Both solutions are feasible but the second solution seems more interactive. The first was carried out, while the second is still in progress.

Week 8 (March 3rd- March 9th)

Educational truce

Week 9 (March 10th- March 16th)

TO DO : Implement sockets solution

Appended

How to install Aruco

1- http://miloq.blogspot.com.es/2012/12/install-opencv-ubuntu-linux.html

2- http://miloq.blogspot.fr/2012/12/install-aruco-ubuntu-linux.html