• A performance regression is a degradation of software performance due to a change in code. A common way to identify such regressions is to manually create tests for them. This is known as regression testing. Although widely used, regression testing can be slow and error-prone due to its historically manual nature. Moreover, such tests' utility is usually constrained by the expertise of the developer creating them. Due to these limitations, researchers have turned their attention to the automatic generation of regression tests. In this paper, we present AutoPerf - a novel approach to automate regression testing that utilizes three core techniques: (i) zero-positive learning, (ii) autoencoders, and (iii) hardware telemetry. We demonstrate AutoPerf's generality and efficacy against 3 types of performance regressions across 10 real performance bugs in 7 benchmark and open-source programs. On average, AutoPerf exhibits 4% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches.
  • In this position paper, we describe our vision of the future of machine programming through a categorical examination of three pillars of research. Those pillars are: (i) intention, (ii) invention, and(iii) adaptation. Intention emphasizes advancements in the human-to-computer and computer-to-machine-learning interfaces. Invention emphasizes the creation or refinement of algorithms or core hardware and software building blocks through machine learning (ML). Adaptation emphasizes advances in the use of ML-based constructs to autonomously evolve software.
  • Classical anomaly detection (AD) is principally concerned with point-based anomalies, anomalies that occur at a single point in time. While point-based anomalies are useful, many real-world anomalies are range-based, meaning they occur over a period of time. Therefore, applying classical point-based accuracy measures to range-based AD systems can be misleading. In this paper, we present a new mathematical model that more accurately gauges the classification correctness of AD systems for range-based anomalies. Unlike prior work, our mathematical definitions are a superset of the classical AD definitions, enabling our system to also subsume point-based anomalies. Moreover, our system is broadly generalizable and provides a number of specialization functions that can control the application's bias along a multi-dimensional axis.
  • This short paper describes our ongoing research on Greenhouse - a zero-positive machine learning system for time-series anomaly detection.
  • Classical anomaly detection is principally concerned with point-based anomalies, anomalies that occur at a single data point. In this paper, we present a new mathematical model to express range-based anomalies, anomalies that occur over a range (or period) of time.
  • Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradigms in a single system called S-Store. In this way, S-Store can simultaneously accommodate OLTP and streaming applications. We present a simple transaction model for streams that integrates seamlessly with a traditional OLTP system. We chose to build S-Store as an extension of H-Store, an open-source, in-memory, distributed OLTP database system. By implementing S-Store in this way, we can make use of the transaction processing facilities that H-Store already supports, and we can concentrate on the additional implementation features that are needed to support streaming. Similar implementations could be done using other main-memory OLTP platforms. We show that we can actually achieve higher throughput for streaming workloads in S-Store than an equivalent deployment in H-Store alone. We also show how this can be achieved within H-Store with the addition of a modest amount of new functionality. Furthermore, we compare S-Store to two state-of-the-art streaming systems, Spark Streaming and Storm, and show how S-Store matches and sometimes exceeds their performance while providing stronger transactional guarantees.