Learning Data Mining with Python

Also, we want to set the final column (column index #1558), which is the class,
to a binary feature. In the Adult dataset, we created a new feature for this. In the
dataset, we will convert the feature while we load it.
converters[1558] = lambda x: 1 if x.strip() == "ad." else 0
Chapter 5
Now we can load the dataset using read_csv. We use the converters parameter to
pass our custom conversion into pandas:
ads = pd.read_csv(data_filename, header=None, converters=converters)
The resulting dataset is quite large, with 1,559 features and more than 2,000 rows.
Here are some of the feature values the first five, printed by inserting ads[:5]
into a new cell:
This dataset describes images on websites, with the goal of determining whether a
given image is an advertisement or not.
The features in this dataset are not described well by their headings. There are two
files accompanying the ad.data file that have more information: ad.DOCUMENTATION
and ad.names. The first three features are the height, width, and ratio of the image
size. The final feature is 1 if it is an advertisement and 0 if it is not.
The other features are 1 for the presence of certain words in the URL, alt text, or
caption of the image. These words, such as the word sponsor, are used to determine
if the image is likely to be an advertisement. Many of the features overlap
considerably, as they are combinations of other features. Therefore, this dataset has a
lot of redundant information.
With our dataset loaded in pandas, we will now extract the x and y data for our
classification algorithms. The x matrix will be all of the columns in our Dataframe,
except for the last column. In contrast, the y array will be only that last column,
feature #1558. Let's look at the code:
X = ads.drop(1558, axis=1).values
y = ads[1558]
[ 95 ]