Counts to Posteriors

Add-One Smoothing Policy

Add-one smoothing is a visible count policy chosen before scoring. This side example shows a zero feature count becoming an exact nonzero fraction.

highlighted = computed this step

A side example with a zero count

Keep the main scoring path separate. In this side example, feature F3 has value 1. Its visible count is 0 in class A and 1 in class B.

F3=1:A count 0,B count 1F3=1:\quad A\text{ count }0,\quad B\text{ count }1
Add-one smoothing policyA side example shows how zero counts become nonzero fractions.Add-one smoothing from visible countsfeature F3=1; binary values=2; policy chosen before scoringadd-one rule: numerator=count+1; denominator=class count+number of valuesclassclass countfeature countnumeratordenominatorlikelihoodA400+1=14+2=61/6B211+1=22+2=41/2zero counts become nonzero fractions under this side-example policythe main scoring lesson can still use smoothing=none

Add one before the ratio

With add-one smoothing, class A uses numerator 0 plus 1 and denominator 4 plus 2. That gives 1/6.

P(F3=1∣A)=(0 + 1)/(4 + 2)=1/6P(F3=1\mid A)=(0\,+\,1)/(4\,+\,2)=1/6
Add-one smoothing policyA side example shows how zero counts become nonzero fractions.Add-one smoothing from visible countsfeature F3=1; binary values=2; policy chosen before scoringadd-one rule: numerator=count+1; denominator=class count+number of valuesclassclass countfeature countnumeratordenominatorlikelihoodA400+1=14+2=61/6B211+1=22+2=41/2zero counts become nonzero fractions under this side-example policythe main scoring lesson can still use smoothing=none

The same policy for B

Class B uses numerator 1 plus 1 and denominator 2 plus 2, giving 1/2. This policy is chosen before scoring; the existing scoring lesson still uses smoothing none.

P(F3=1∣B)=(1 + 1)/(2 + 2)=1/2P(F3=1\mid B)=(1\,+\,1)/(2\,+\,2)=1/2
Add-one smoothing policyA side example shows how zero counts become nonzero fractions.Add-one smoothing from visible countsfeature F3=1; binary values=2; policy chosen before scoringadd-one rule: numerator=count+1; denominator=class count+number of valuesclassclass countfeature countnumeratordenominatorlikelihoodA400+1=14+2=61/6B211+1=22+2=41/2zero counts become nonzero fractions under this side-example policythe main scoring lesson can still use smoothing=none