The Naive Bayes algorithm is a supervised classification algorithm used for binary or multi-class classification. It works on the Bayes theorem principle and assumes that all features in the dataset are independent of each other (hence called “naive”).
Characteristics:
- Based on Bayes’ Theorem.
- Assumes feature independence.
- Assigns equal weightage to all features.
- Works well for large datasets.
- It is simple, fast, and effective.
- Each feature contributes independently to the probability of a given class.
Applications of Naive Bayes:
- Face recognition
- Text classification
- Recommendation systems
Naive Bayes Algorithm Steps
- Compute the prior probability for each target class.
- Compute the frequency matrix and likelihood probability for each feature.
- Apply Bayes’ theorem to calculate the probability of all hypotheses.
- Classify the test instance using the Maximum A Posteriori (MAP) hypothesis — i.e., select the class with the highest posterior probability.
Why It’s Called ‘Naive’?
Because it naively assumes that all features are statistically independent from each other, even if they may be dependent in reality.