k Nearest Neighbors

Frank Acito
Additional contact information
Frank Acito: Indiana University

Chapter Chapter 10 in Predictive Analytics with KNIME, 2023, pp 209-227 from Springer

Abstract: Abstract K Nearest Neighbors (kNN) is a powerful and intuitive data mining model for classification and regression tasks. As an instance-based or memory-based learning algorithm, kNN classifies new objects based on their similarity to known objects in the training data. Unlike parametric models, kNN is non-parametric and does not rely on assumptions about data distributions. The main advantage of kNN is its simplicity and ability to handle large datasets efficiently. However, one of its drawbacks is that it requires scanning all the training data each time a new observation needs to be classified, which can be time-consuming for large datasets. The kNN algorithm calculates the distances between the new observation and all existing data points. The k nearest neighbors are selected based on the smallest distances, and their majority class or average value is used for classification or regression. For classification tasks, kNN is considered a “lazy” algorithm because it does not create an explicit model during training. Instead, it stores the entire dataset and makes decisions on new observations instantly. In contrast, “eager” algorithms, like logistic regression, build a model during training that is then used for predictions. In addition to classification, kNN can also be used for regression tasks. It can capture non-linear relationships between predictors and continuous target variables without requiring a predefined model. While kNN is flexible and robust to different target variables and distributions, it requires standardizing predictors to avoid bias from variables with large values. It also suffers from the “curse of dimensionality,” where the performance degrades in high-dimensional spaces due to increased sparsity. Despite its limitations, kNN remains a valuable tool in data mining, especially when dealing with non-linear relationships and a lack of strict assumptions about the data. Careful data preprocessing and optimization of the value of k can help improve its performance in various applications.

Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-45630-5_10

Ordering information: This item can be ordered from
http://www.springer.com/9783031456305

DOI: 10.1007/978-3-031-45630-5_10

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().