DOOMBA ME 4451: Robotics December 14, 2010 Dr. Harvey ... - helix

DOOMBA
ME 4451: Robotics
December 14, 2010
Dr. Harvey Lipkin, and Dr. Nader Sadegh
Chad Norton, Ryan Lober, Zachary Van Schoyck, Ben Coburn

Initial Plan
For the Doomba project the primary objective was to create a mobile robot
capable of tracking a human in a dynamic environment. The only variable to be inputted
by the user would be a specific distance to maintain from the target. The robot, in this
case the iRobot Create with a webcam, would then calculate the angle and distance to the
target from image processing, determine the necessary movements to reorient itself, then
execute those movements to arrive at the specified distance and a zero degree angle
difference (i.e. directly facing the target). Figure 1 below is a graphical representation of
these operations.
d
0ᵒ
θᵒ
d + x
2
0ᵒ
d
1
Figure 1: Physical representation of tracking and subsequent reorientation
To accomplish this overall task, three operations would have to be accomplished.
First the system would have to be manually calibrated to the environment, second the
camera would have to locate the subject of tracking, and third the locations obtained from
the tracking would have to be transformed in to robotic movements.
Initially this sequence of operations was to be completed using live human
tracking algorithms and real time video. Real time tracking was accomplished using the
Simulink block set, unfortunately Simulink would not communicate with the Create and
we were forced to switch from video tracking to image grabbing using Matlab.
Switching to Matlab as the processing platform reduced the speed of image analysis but
the overall theme of the project was maintained.
1

To accomplish human tracking in Matlab, the target to be tracked had to wear two
green circles spaced 9” center to center apart. Using the calibration functions we would
manually choose the tracked colors from an image grab. After calibrating the system
Matlab would grab images from the video and perform a series of image filters to select
the chosen target color, calculate the properties of the image, filter out noise, determine
which objects were circular, and output the coordinates of the two desired circle
centroids. Finally, a function would take these centroid locations and calculate the angle
and distance to the target, based on camera specific parameters. These measurements
would be compared against the desired distance to maintain, and the current orientation,
and the appropriate robot motions would be executed using the Create Toolbox functions.
This process would then repeat after every move allowing the robot to follow a target.
Robot Kinematics
The bulk of the project revolved around image processing and very minor robot
kinematics were incorporated into the final product with the exception of the movements
preformed by the iRobot. The only parameters given to the robot were distance and angle
changes needed to maintain proper orientation to the target. These commands were
executed using turnAngle and travelDist. Supplying the rotation angle and the distance
difference between the current and desired locations to these two functions, allowed us to
effectively relocate the robot without extensive code or calculations. Although inelegant,
it was the simplest solution to the kinematics issue, and removed the need to focus our
energy on this problem.
After achieving reasonable image processing results, we invested some time in
writing the code necessary to solve the velocity kinematics for the robot in order to use
smoother motions to arrive at the final destination. To do this we calculated the change
in x, y, and angle θ, from one frame to the next. Dividing these values by the processing
time between frames gave us their rates of change. Using these values a reverse velocity
kinematics analysis could be preformed to calculate the turning speed of each wheel. The
equation for this is shown below.
2

where,
Using the command SetDriveWheelsCreate, we could send the calculated wheel
velocities to the robot and have it move in a smooth curved path towards its locations.
The fundamental drawback to this was that calculations of these velocities was dependent
on the CPU time to process the image, and this value was not matching the values we
were expecting and was causing our code to miscalculate the necessary velocities.
Another issue was that the function SetDriveWheelsCreate, had to be stopped by
resetting the values for the wheel speeds to zero. It was decided that because of the
unnecessary problems involved with the smooth trajectory motion that we would use the
proven turn and go mechanism described above, for simplicity.
Challenges/Solutions
The original plan for this project was to try to make the iRobot follow arbitrary
“new” objects with the webcam. The idea was that we could use or modify an already
existing Simulink package to do this, allowing relatively simple programming and letting
us move on to figuring out the rest of the design. This turned out to be un-workable due
to the structure of the original Simulink code.
The Simulink package tracks objects by taking a picture of the background with
no objects of interest present, and then assuming any differences from that background
are interesting objects and should be tracked. This works quite well under the right
conditions, but it requires that the camera be stationary. If the camera moves at all, it will
cause every point on the image to change, making the program try to track the entire
image. This obviously doesn’t work, but it seemed like a good way to at least acquire the
image to be tracked, maybe it could be modified somehow to account for a moving
robot
3

There are two basic ways to handle having a moving camera: attempt to update
the background picture as the robot moves, allowing the continued use of the original
algorithm, or switch to some completely different tracking approach. The first appears to
require a full 3d model of the background environment, because the change in position of
a given image depends on both the relative movement and relative distance of the camera
and the object in question.
On the other hand, tracking by simple color selection and threshold operations
merely requires giving up on tracking arbitrary objects. It also helps with another
problem, that of determining the distance to an object with just one camera.
In order to follow an object with the iRobot, the control system needs two basic
parameters: The angle and distance to the target. The angle comes essentially free with
any camera tracking program, but the distance requires more complicated processes. The
solution to this problem was to place a pair of colored dots on the target and then
determine its distance by measuring the angle between them and using that to determine
the distance to the target. This basic approach also gives us a relatively easy tracking
problem for general target tracking.
The initial difficulty with this approach was our choice of target dots. With a
small camera very low to the ground, in an area lit primarily from above such as the
robotics lab in the MRDC, the image saturates and appears washed out. That is, all
relatively bright colors seem to blend together. This caused problems with our original
choice of bright orange dots on a white background as a target to track. The dots
appeared to be identically colored to the background of the shirt, making tracking
impossible. The solution to this problem had two basic parts: The orange dots were
replaced with a mat green color, reducing issues with washout, and the camera was
elevated significantly above the floor, improving the image and saturation, as shown in
Figure 3 and Figure 4, respectively.
4

Figure 3: Object Tracking Dots
Figure 4: Elevated Camera Stand to improve image quality
This also made selecting appropriate threshold values that simultaneously
detected the target circles and didn’t detect excessive noise. This turns out to be difficult,
partially because appropriate values appear to vary based on lighting and the exact details
of how far the camera is from the target and similar and partially because there's
significant overlap between the value necessary to find the target and a value low enough
to not find anything else. The solution to this problem was to select what would normally
be an excessively large threshold value, accepting somewhat more noise, and then
selecting target blobs by their shape rather than simply selecting the largest two.
5

Implementing this required several filters on the blob tracking. First, we selected
only those blobs which had a ratio of Area to Perimeter squared close to that of a circle
and a non-negligible area. They were then selected by choosing the two with the smallest
horizontal offset, matching the vertically aligned target dots. Finally, to reduce the
possibility of the robot moving after picking an incorrect pair of centroids, a check was
inserted to stop motion and repeat the image analysis loop if the centroids were too far
apart horizontally or not far enough part vertically.
The parts of the project that were not directly related to image tracking were
relatively simple. The basic commands for moving the iRobot were already written, and
worked reasonably well. Developing the trailer to carry the laptop used and the stand to
elevate the camera were both fairly easy. The movement code did include a limit to
prevent the target from being moved out of frame, and to limit the potential trouble
caused by picking an incorrect pair of centroids.
Achievements/Results
As described previously, the overall goal of this project was to have the iRobot
track an object, determine its relative location, and then navigate towards that object
while maintaining a desired distance. In essence, the final product met each of the
requirements. However, the degree to which those goals were met varies.
Using the developed code in Appendix 1, the iRobot can reliably distinguish the
appropriate target from the background. The success of this routine is highly dependent
upon the ambient lighting conditions, light saturation levels and the resolution of the
camera used. While the final product is fairly robust, it is still susceptible to erroneous
identification. An advantage to the developed code is the ability for the robot to continue
to look for the object even if it is out of frame. If the object does return to the frame, the
robot will immediately begin tracking it again.
In order to determine the objects relative position, the code developed in
Appendix 1 was used, as well. The overall tolerance of the trailing distance is not of
significant importance; however the system is accurate to within centimeters of the
desired input distance. As seen in Figure #, the trailer used to tow the laptop provides an
excessive amount of mass to the rear end of the iRobot. This added weight can result in
vehicle either over-rotating because of the added torque or under-rotating because
6

excessive weight that the iRobot is required to turn. This not only hinders the linear
distance of the robot, but it also results in the possibility of the robot not facing the target
object. While the deviations were on average very small, they were present.
Learning Experiences
Based upon the achievements that were reached during the extent of the project,
and the short-comings, our group concluded that there were several adjustments that
could lead to improvements in the design. For example, the robot’s wheel motors were
not sufficient to turn robot and trailer system accurately. A possible solution to this
would be to replace the trailer system with a lighter version or negating it all together.
Removing the need for the trailer would require the use of a wireless camera. Based upon
the short-comings of the camera used, this change would be welcomed. The current
camera was found to provide no distinct advantage in capture rate or in pixel resolution.
Similarly, the field of view of the camera limited the capability of the tracking system.
Therefore, by replacing the camera, several issues could be resolved.
The susceptibility of the image processing to slight variations in color and lighting
was a non-trivial issue that was encountered. In the final design of the project, the image
processing attempted to track two large circles. If the lighting was too saturated or the
background was similar in color, the tracking could be fooled. However, if the tracking
system were configured to detect a light emitting diode (LED), the system may be able to
track more effectively. Without a more effective means to track, the system will be
unacceptably unreliable.
Class Suggestion
The final project could be designed to be a more significant portion of the final
grade in the class. For the amount of time that is spent designing, building, and testing the
project the grade could reflect that time with a higher percentage of the overall grade.
Someone may spend on average 15 hours studying for a midterm and spend twice that on
their project, but the midterm is worth nearly twice the percentage that the entire project
is worth.
7

Appendix
Code to initialize the procedure: start.m
Code to monitor camera vision: test.m
8

Code to send commands to Roomba: Drun.m
9

The following functions were used in support of the 3 codes listed above:
vidsearch.m
ColorDetectHSVimage.m
selectpixelsandsetHSV.m
turnAngle.m
travelDist.m
11