Learning Data Mining with Python

Beating CAPTCHAs with
Neural Networks
Interpreting information contained in images has long been a difficult problem in
data mining, but it is one that is really starting to be addressed. The latest research is
providing algorithms to detect and understand images to the point where automated
commercial surveillance systems are now being used—in real-world scenarios—by
major vendors. These systems are capable of understanding and recognizing objects
and people in video footage.
It is difficult to extract information from images. There is lots of raw data in an
image, and the standard method for encoding images—pixels—isn't that informative
by itself. Images—particularly photos—can be blurry, too close to the targets, too
dark, too light, scaled, cropped, skewed, or any other of a variety of problems that
cause havoc for a computer system trying to extract useful information.
In this chapter, we look at extracting text from images by using neural networks
for predicting each letter. The problem we are trying to solve is to automatically
understand CAPTCHA messages. CAPTCHAs are images designed to be easy for
humans to solve and hard for a computer to solve, as per the acronym: Completely
Automated Public Turing test to tell Computers and Humans Apart. Many websites
use them for registration and commenting systems to stop automated programs
flooding their site with fake accounts and spam comments.
The topics covered in this chapter include:
• Neural networks
• Creating our own dataset of CAPTCHAs and letters
• The scikit-image library for working with image data
• The PyBrain library for neural networks
[ 161 ]