Counts to Posteriors

Combining the Evidence

The class score is an exact product of one prior and the displayed feature likelihoods. Because the products are rational, no log or decimal approximation is needed.

highlighted = computed this step

The score product

The unnormalized score is the prior times the feature likelihoods. For A the exact product is 2/3·3/4·1/2 = 1/4.

score⁡(A)=1/4\operatorname{score}(A)=1/4
Combining evidenceScores are exact products of the prior and likelihoods.Naive Bayes from countstotal=6; counts=A:4, B:2feature counts: F1=1: A=3, B=1; F2=1: A=2, B=1query: F1=1, F2=1; smoothing=noneclassP(c)likelihood count/countscoreAP(A)=4/6 (=2/3)P(F1=1|A)=3/4P(F2=1|A)=2/4 (=1/2)1/4BP(B)=2/6 (=1/3)P(F1=1|B)=1/2P(F2=1|B)=1/21/12naive independence assumed; no smoothing; no logs or decimals

Why the B score is smaller

For B the product is 1/3·1/2·1/2 = 1/12.

score⁡(B)=1/12\operatorname{score}(B)=1/12
Combining evidenceScores are exact products of the prior and likelihoods.Naive Bayes from countstotal=6; counts=A:4, B:2feature counts: F1=1: A=3, B=1; F2=1: A=2, B=1query: F1=1, F2=1; smoothing=noneclassP(c)likelihood count/countscoreAP(A)=4/6 (=2/3)P(F1=1|A)=3/4P(F2=1|A)=2/4 (=1/2)1/4BP(B)=2/6 (=1/3)P(F1=1|B)=1/2P(F2=1|B)=1/21/12naive independence assumed; no smoothing; no logs or decimals

Summary

The independence assumption is what permits this product. The products are still exact rational arithmetic from integer counts.

score⁡(A)>score⁡(B)\operatorname{score}(A)>\operatorname{score}(B)
Combining evidenceScores are exact products of the prior and likelihoods.Naive Bayes from countstotal=6; counts=A:4, B:2feature counts: F1=1: A=3, B=1; F2=1: A=2, B=1query: F1=1, F2=1; smoothing=noneclassP(c)likelihood count/countscoreAP(A)=4/6 (=2/3)P(F1=1|A)=3/4P(F2=1|A)=2/4 (=1/2)1/4BP(B)=2/6 (=1/3)P(F1=1|B)=1/2P(F2=1|B)=1/21/12naive independence assumed; no smoothing; no logs or decimals