Yasir Niaz Khan

Visual Terrain Classification by 

Mobile Outdoor Robots 

Yasir Niaz Khan and Andreas Zell 

University of Tübingen, Germany

Introduction 

• Ground surface estimation is essential for many 

purposes 

• For localization and mapping 

• For guiding vehicles and humans on ground 

• For a safe traversal for mobile outdoor robots 

• Rescue 

• Surveillance 

• Hazardous areas, etc. 

Yasir Niaz Khan 

2

Related Work 

• Geometrical features 

• LADAR sensor [Vandapel04] 

• Stereo camera [Bajracharya08] 

• Problem: Tall grass vs. small wall 

• Visual features 

• Static aerial images [Hudjakov11] 

• Our Approach: Monocular camera on a real flying 

robot 


3

Driving Robot Platform 

• RC model truck 1:8 

• Dual-core PC 

• 32bit microcontroller 

• Different sensors 

• Point Grey Firefly 

camera 

• 6mm lens 

• 640x480 images 

• Pan-tilt unit – 50cm 

above ground 

• Average speed – 1m/s 

Our Outdoor Robot 

Yasir Niaz Khan 4

Flying Robot Platform 

• AscTec X3D-BL 

Hummingbird 

• Diameter 53 cm 

• Weight 0.5 kg 

• PointGrey FireFly USB 

color camera with VGA 

resolution 

• Gumstix Overo Fire 

single-board with a 

600MHz ARM processor 

• Manual flight 

• Frame-rate of 1 Hz 


Terrain Types 

• 6 Terrain Types 

Grass Asphalt Gravel 

Big tiles Small tiles Bushes 


Artifacts 

• Blur 

• Bumpy surfaces 

• Fast robot motion 

• Vibration of robot 

• Over-exposure 

• Sun glares directly into 

the camera 

• Color changes 

• Depends on day time 

• Shadows 

• Water patches after rain 

• Major scale and 

orientation changes 


Blur and over-exposure 

Color changes 

7

Different Heights – Grass 


8

Local Descriptors 

• Calculated on a grid 

drawn on the image 

• Histogram-based 

• Local Binary Pattern 

• Local Ternary Pattern 

• Interest point-based 

• Speeded Up Robust 

Features (SURF) 

Image divided into 8x8 grid 


Local Binary Pattern [Ojala02] 

• 3x3 window is placed on top of each pixel 

• Neighboring pixels are thresholded 

• Greater than center → 1 

• Less than center → 0 

• Start from upper-left corner and move clockwise 

• Binary Pattern = 10110100 

• Histogram has 256 dimensions 


94 38 54 

23 50 78 

47 66 12 

1 0 1 

0 1 

0 1 0 

10

Local Ternary Pattern [Tan07] 

• Center c, threshold k 

• Pattern = 1(-1)01(-1)10(-1) split into two parts: 

• 10010100 & 01001001 

• Concatenated histograms give 512 dimensions 


94 38 54 

23 50 78 

47 66 12 

1 -1 0 

-1 1 

0 1 -1 

k=5 

11

Speeded Up Robust Features [Bay2006] 

• Interest point detection + descriptor 

• Grid intersections are given as interest points for 

feature calculation 

• Upright SURF 

• Orientation not considered 

• +/-15 rotation invariance 

• Scale is determined through grid search 

• 64 dimensional descriptor – TSURF 


12

Classifier 

• Different Classifiers tested 

• Random Forests 

• Support Vector Machine (SVM) using the Sequential Minimal 

Optimization (SMO) training algorithm 

• Multilayer Perceptron (MLP) 

• LIBLINEAR 

• J48 Decision Tree 

• Naive Bayes 

• k-Nearest Neighbor 

• Best performance - Random Forests 

• Weka software for machine learning 

• PC Cluster for parallel testing 


13

Random Forest 

• A collection of individual decision tree predictors 

• Each node of a tree uses a varying feature 

subset 

• Leaf nodes = actual class labels 

• Class determined by majority voting 

• More trees – better generalization, but increased 

run-time 

• 50 trees gave good classification at satisfactory 

speed 

• 10-fold cross validation 


14

Groundtruth Generation 

• Color code all images according to terrain 

legend 

• No need to cut patches from images 

• Place all images in the same folder 

• Grid is drawn on every image 

• Each grid cell is analyzed for terrain types 

• Cell with maximum pixels of one terrain type is 

assigned to that terrain type 

• Must be >40% 


Groundtruth Generation 


16

Results – Flying Robot 

Gridsize 

LBP LTP TSURF 

10 33.6% 38.9% 99.6% 

20 48.4% 55.7% 96.8% 

30 59.4% 67.3% 88.7% 

40 66.7% 73.6% 81.5% 

50 71.0% 78.1% 76.6% 

60 74.8% 80.3% 71.9% 

70 77.3% 81.7% 71.2% 

80 78.2% 83.3% 66.0% 

90 79.6% 84.7% 67.7% 

100 79.5% 84.4% 63.3% 


Results – Driving Robot 

Grid 

size LBP LTP T-SURF 

10 55.0% 67.2% 99.2% 

20 75.7% 83.0% 97.8% 

30 85.2% 90.0% 96.6% 

40 90.2% 93.3% 95.7% 

50 92.6% 94.7% 94.9% 

60 94.4% 95.7% 95.2% 

70 95.5% 96.8% 94.5% 

80 95.8% 96.8% 93.1% 

90 96.9% 97.4% 93.6% 

100 96.9% 98.1% 94.0% 


Conclusion 

• Terrain classification is possible using a single 

camera on a robot 

• Driving robot driven at fast speed 

• Flying robot flown at different heights 

• Random forest performs good and fast 

classification 

• Terrain can be classified at different resolutions 

for different scenarios: 

• For cell sizes 50x50: LTP 


Thank you

Yasir Niaz Khan

Create successful ePaper yourself

Delete template?

Save as template?