A dynamic classification unit for online segmentation of big data via small data buffers
Authors: Khalemsky, Anna; Gelbard, Roy
Journal: Decision Support Systems (2020)
DOI: 10.1016/j.dss.2019.113157
In many segmentation processes, we assign new cases according to a model that was built on the basis of past cases. As long as the new cases are “similar enough” to the past cases, segmentation proceeds normally. However, when a new case is substantially diferent from the known cases, a reexamination of the previously created segments is required. The reexamination may result in the creation of new segments or in the updating of the existing ones. In this paper, we assume that in big and dynamic data environments it is not possible to reexamine all past data and, therefore, we suggest using small groups of selected cases, stored in small data bufers, as an alternative to the collection of all past data. We present an incremental dynamic classifier that supports real-time unsupervised segmentation in big and dynamic data environments. In order to reduce the computational efort of unsupervised clustering in such environments, the suggested model performs calculations only on the relevant data bufers that store the relevant representative cases. In addition, the suggested model can serve as a dynamic classification unit (DCU) that can act as an autonomous agent, as well as collaborat…