Neighbors and Regions

Feature Scale Changes Nearest

The same original rows can have a different nearest neighbor when one feature scale changes. This lesson keeps both squared-distance calculations exact for one query.

highlighted = computed this step

Raw distances

Use query Q=(0, 6), Red point R=(1, 0), and Blue point B=(4, 10). The raw squared distances are 37 to Red and 32 to Blue, so the raw nearest neighbor is Blue.

d2(Q,R)=37,d2(Q,B)=32⇒Blued^2(Q,R)=37,\quad d^2(Q,B)=32\Rightarrow \text{Blue}
Feature scale changes nearestExact raw and scaled squared distances for one query.feature scale changes nearestone query, exact squared distances; no square roots or decimalssame original rows, different feature scale, different nearest neighbor; one exact query only; NOT a normalizationrecommendation; NOT model quality; NOT future dataviewqueryshown rowsd^2 to Redd^2 to BluenearestrawQ=(0,6)Red=(1,0); Blue=(4,10)3732Bluescaled y/10Q=(0,3/5)Red=(1,0); Blue=(4,1)34/25404/25Redsame original rows; changing the feature scale changes the smaller exact distance

Scale y by one tenth

Now keep x unchanged and replace each y value with y divided by 10. The query becomes (0, 3/5). Red stays (1, 0), and Blue becomes (4, 1). The squared distances are 34/25 to Red and 404/25 to Blue.

dscaled2(Q,R)=34/25,dscaled2(Q,B)=404/25d^2_{\text{scaled}}(Q,R)=34/25,\quad d^2_{\text{scaled}}(Q,B)=404/25
Feature scale changes nearestExact raw and scaled squared distances for one query.feature scale changes nearestone query, exact squared distances; no square roots or decimalssame original rows, different feature scale, different nearest neighbor; one exact query only; NOT a normalizationrecommendation; NOT model quality; NOT future dataviewqueryshown rowsd^2 to Redd^2 to BluenearestrawQ=(0,6)Red=(1,0); Blue=(4,10)3732Bluescaled y/10Q=(0,3/5)Red=(1,0); Blue=(4,1)34/25404/25Redsame original rows; changing the feature scale changes the smaller exact distance

Compare the nearest row

With raw coordinates, Blue is nearest. With y scaled by one tenth, Red is nearest. These are the same original rows and one exact query, but a different feature scale changes the smaller squared distance. This is not a normalization recommendation, not a model-quality claim, and not a claim about future data.

raw nearest=Blue,scaled nearest=Red\text{raw nearest}=\text{Blue},\quad \text{scaled nearest}=\text{Red}
Feature scale changes nearestExact raw and scaled squared distances for one query.feature scale changes nearestone query, exact squared distances; no square roots or decimalssame original rows, different feature scale, different nearest neighbor; one exact query only; NOT a normalizationrecommendation; NOT model quality; NOT future dataviewqueryshown rowsd^2 to Redd^2 to BluenearestrawQ=(0,6)Red=(1,0); Blue=(4,10)3732Bluescaled y/10Q=(0,3/5)Red=(1,0); Blue=(4,1)34/25404/25Redsame original rows; changing the feature scale changes the smaller exact distance