Advanced Data Analytics Using Python_ With Machine Learning, Deep Learning and NLP Examples ( 2023)

Decision Tree
Chapter 3 Supervised Learning Using Python
A decision tree is a tree of rules. Each level of the tree represents a
parameter, each node of the level validates a constraint for that level
parameter, and each branch indicates a possible value of parent node
parameter. Figure 3-1 shows an example of a decision tree.
Outlook
Sunny Overcast Rain
Humidity
Yes
Wind
High
Normal
Strong
Weak
No Yes No
Yes
Figure 3-1. Example of decision tree for good weather
Which Attribute Comes First?
One important aspect of the decision tree is to decide the order of features.
The entropy-based information gain measure decides it.
Entropy is a measure of randomness of a system.
Entropy S º pilog 2
pi
c
å
( ) -
i=
1
For example, for any obvious event like the sun rises in the east,
entropy is zero, P=1, and log(p)=0. More entropy means more uncertainty
or randomness in the system.
59