Learning Data Mining with Python

Extracting Features with Transformers
The adult file itself contains two blank lines at the end of the file. By default, pandas
will interpret the penultimate new line to be an empty (but valid) row. To remove
this, we remove any line with invalid numbers (the use of inplace just makes sure
the same Dataframe is affected, rather than creating a new one):
adult.dropna(how='all', inplace=True)
Having a look at the dataset, we can see a variety of features from adult.columns:
adult.columns
The results show each of the feature names that are stored inside an Index object
from pandas:
Index(['Age', 'Work-Class', 'fnlwgt', 'Education',
'Education-Num', 'Marital-Status', 'Occupation', 'Relationship',
'Race', 'Sex', 'Capital-gain', 'Capital-loss', 'Hours-per-week',
'Native-Country', 'Earnings-Raw'], dtype='object')
Common feature patterns
While there are millions of ways to create features, there are some common patterns
that are employed across different disciplines. However, choosing appropriate
features is tricky and it is worth considering how a feature might correlate to the end
result. As the adage says, don't judge a book by its cover—it is probably not worth
considering the size of a book if you are interested in the message contained within.
Some commonly used features focus on the physical properties of the real world
objects being studied, for example:
• Spatial properties such as the length, width, and height of an object
• Weight and/or density of the object
• Age of an object or its components
• The type of the object
• The quality of the object
Other features might rely on the usage or history of the object:
• The producer, publisher, or creator of the object
• The year of manufacturing
• The use of the object
[ 84 ]