Distributed event detection using the Storm System

Abstract

Distributed event detection is the process of identifying specific occurrences of interest in incoming data available at a number of distributed nodes. The traditional approach for detecting events implies central collection and processing of data, which is impractical for a number of reasons. Firstly, since the number of nodes might be large, collecting information centrally is not always possible or efficient. This happens because the amount of information to be transmitted may be huge and the available bandwidth insufficient to accommodate the transmission. Secondly, in some networks such as sensor networks, transmitting information draws additional power, which is not desirable because usually the sensors have limited battery power and their power source may be irreplaceable. Subsequently, it is not recommended to send all available information to a central node because this transmission will consume a substantial amount of power. For these reasons, one of the most significant limitations of sensor networks is the need to reduce energy consumption. The geometric method was proposed relatively recently and allows a network to monitor in a distributed way if the value of a complex function, even nonlinear, calculated using incoming data is over or under a specific threshold value. Thus, composite events can be distributely detected if they are expressed as a threshold monitoring function. The geometric method imposes a set of local constraints on each node and manages to reduce the need for communication between the nodes as long as the constraints are satisfied. In this work, the geometric method is implemented, using the real-time distributed computation framework named Storm, for distributed event detection. A topology implementing the geometric method has been constructed using the components provided by the Storm framework, allowing scalable processing of data acquired from nodes and monitoring of certain functions. Finally, the system also allows runtime addition of new functions that can be monitored.