MacroBase: Prioritizing Attention in Fast Data

• Firas Abuzaid ^[1] ; Peter Bailis ; Jialin Ding ; Edward Gan ; Samuel Ross Madden ; Deepak Narayanan ; Kexin Rong ; Sahaana Suri
  1. [1] Firas Abuzaid
• Localización: ACM transactions on database systems, ISSN 0362-5915, Vol. 43, Nº 4, 2018
• Idioma: inglés
• Enlaces
  • Texto completo
• Resumen

  • As data volumes continue to rise, manual inspection is becoming increasingly untenable. In response, we present MacroBase, a data analytics engine that prioritizes end-user attention in high-volume fast data streams. MacroBase enables efficient, accurate, and modular analyses that highlight and aggregate important and unusual behavior, acting as a search engine for fast data. MacroBase is able to deliver order-of-magnitude speedups over alternatives by optimizing the combination of explanation (i.e., feature selection) and classification tasks and by leveraging a new reservoir sampler and heavy-hitters sketch specialized for fast data streams. As a result, MacroBase delivers accurate results at speeds of up to 2M events per second per query on a single core. The system has delivered meaningful results in production, including at a telematics company monitoring hundreds of thousands of vehicles.