ID3 (Iterative Dichotomiser 3) is a supervised learning algorithm that constructs a decision tree using a greedy approach. It selects the best attribute at each level of the tree using a measure called Information Gain.
Characteristics of ID3:
- Works only with categorical/discrete attributes.
- Uses Information Gain to select the best attribute.
- Prone to overfitting on small datasets.
- No pruning is performed after the tree is built.
- Cannot handle missing values or continuous attributes directly (need to discretize them).
Algorithm:
Entropy & Information Gain Formulas:
Predicting Job Offer using ID3
Dataset: 10 student records with attributes
- CGPA
- Interactiveness
- Practical Knowledge
- Communication Skills
- Target class: Job Offer (Yes/No)
Step 1: Compute Entropy of Job Offer (Target Attribute)
- Yes = 7, No = 3
