Learning Data Mining with Python

Chapter 5
Other features describe a dataset in terms of its components:
• Frequency of a given subcomponent, such as a word in a book
• Number of subcomponents and/or the number of different subcomponents
• Average size of the subcomponents, such as the average sentence length
Ordinal features allow us to perform ranking, sorting, and grouping of similar
values. As we have seen in previous chapters, features can be numerical or
categorical. Numerical features are often described as being ordinal. For example,
three people, Alice, Bob and Charlie, may have heights of 1.5 m, 1.6 m and 1.7 m. We
would say that Alice and Bob are more similar in height than are Alice and Charlie.
The Adult dataset that we loaded in the last section contains examples of continuous,
ordinal features. For example, the Hours-per-week feature tracks how many hours
per week people work. Certain operations make sense on a feature like this. They
include computing the mean, standard deviation, minimum and maximum. There is
a function in pandas for giving some basic summary stats of this type:
adult["Hours-per-week"].describe()
The result tells us a little about this feature.
count 32561.000000
mean 40.437456
std 12.347429
min 1.000000
25% 40.000000
50% 40.000000
75% 45.000000
max 99.000000
dtype: float64
Some of these operations do not make sense for other features. For example, it
doesn't make sense to compute the sum of the education statuses.
There are also features that are not numerical, but still ordinal. The Education
feature in the Adult dataset is an example of this. For example, a Bachelor's degree
is a higher education status than finishing high school, which is a higher status than
not completing high school. It doesn't quite make sense to compute the mean of these
values, but we can create an approximation by taking the median value. The dataset
gives a helpful feature Education-Num, which assigns a number that is basically
equivalent to the number of years of education completed. This allows us to quickly
compute the median:
adult["Education-Num"].median()
[ 85 ]