Reliable speculative processing of out-of-order event streams in generic publish/subscribe middlewares

In surveillance, sports, finances, etc., distributed event-based systems are used to detect meaningful events with low latency in high data rate event streams. Both known approaches to deal with the predominant out-of-order event arrival at the distributed detectors have their shortcomings: buffering approaches introduce latencies for event ordering and stream revision approaches may result in system overloads due to unbounded retraction cascades. This paper presents a speculative processing technique for out-of-order event streams that enhances typical buffering approaches. In contrast to other stream revision approaches our novel technique encapsulates the event detector, uses the buffering technique to delay events but also speculatively processes a portion of it, and adapts the degree of speculation at runtime to fit the available system resources so that detection latency becomes minimal. Our technique outperforms known approaches on both synthetical data and real sensor data from a Realtime Locating System (RTLS) with several thousands of out-of-order sensor events per second. Speculative buffering exploits system resources and reduces latency by 40% on average.