Nearest Neighbors

kNN Classify by Majority Vote

Classify a query point by the majority class among its k=3 nearest neighbors. One loop computes (squared_distance, index) pairs to avoid sqrt (squaring preserves ranking); .sort() orders them; a second loop over the first 3 tallies votes by looking up labels via index; max(votes, key=votes.get) picks the winner. Library: sklearn.neighbors.KNeighborsClassifier(n_neighbors=3).fit(X, y) .predict([query]). RESULT: predicted class label.

By hand

With scikit-learn

KNeighborsClassifier(n_neighbors=3) uses Euclidean distance by default. kneighbors([query]) exposes the 3 nearest distances and indices for the snapshot. RESULT is the string predicted by .predict.

naive.py 
X_train = [[1,1], [2,1], [4,1], [8,7], [9,8], [7,9]]
y_train = ['A', 'B', 'A', 'B', 'B', 'B']
query = [2, 2]
n_train = len(X_train)
dists = []
for i in range(n_train):
    d0 = X_train[i][0] - query[0]
    d1 = X_train[i][1] - query[1]
    dists.append((d0*d0 + d1*d1, i))
dists.sort()
votes = {}
for i in range(3):
    lbl = y_train[dists[i][1]]
    votes[lbl] = votes.get(lbl, 0) + 1
winner = max(votes, key=votes.get)
print('RESULT:', repr(winner))
library.py 
from sklearn.neighbors import KNeighborsClassifier
from dalib.display import set_display
set_display()

X_train = [[1,1], [2,1], [4,1], [8,7], [9,8], [7,9]]
y_train = ['A', 'B', 'A', 'B', 'B', 'B']
query = [2, 2]
clf = KNeighborsClassifier(n_neighbors=3)
clf.fit(X_train, y_train)
dists, indices = clf.kneighbors([query])
neighbors = [y_train[i] for i in indices[0]]
pred = str(clf.predict([query])[0])
print('k=3 neighbors:', neighbors)
print('distances:', [round(float(d), 4) for d in dists[0]])
print('RESULT:', repr(pred))

k=3 neighbors: ['B', 'A', 'A']
distances: [1.0, 1.4142, 2.2361]
RESULT: 'A'

Implementation notes

  • Squared distances are used for ranking (avoids math.sqrt): √a < √b ↔ a < b for non-negative a, b. The votes are over labels, not distances, so the actual distance value is not needed.
  • The 2-1 vote (A:2, B:1) shows majority overriding the single nearest neighbor (which is class B at sq=1). A pure 3-0 unanimous vote would not demonstrate majority voting.
  • Training points and query are chosen so no two training points are equidistant from the query and the 3rd/4th boundary is unambiguous (sq=5 vs sq=61) — eliminating tie-breaking ambiguity between naive and sklearn.
  • sklearn's KNeighborsClassifier auto-selects an efficient neighbor-search structure (algorithm='auto'); it applies the same Euclidean metric, so distances match euclidean-distance (this chapter) exactly.
  • With k=3 and a unanimous vote (3-0), prediction is unambiguous. Ties (e.g. k=2 with one A and one B) require a tie-breaking rule — sklearn uses the class with the smaller index in classes_.
  • Cross-reference: euclidean-distance (this chapter) for the distance formula used to rank neighbors.