A performance regression is a degradation of software performance due to a
change in code. A common way to identify such regressions is to manually create
tests for them. This is known as regression testing. Although widely used,
regression testing can be slow and error-prone due to its historically manual
nature. Moreover, such tests' utility is usually constrained by the expertise
of the developer creating them. Due to these limitations, researchers have
turned their attention to the automatic generation of regression tests. In this
paper, we present AutoPerf - a novel approach to automate regression testing
that utilizes three core techniques: (i) zero-positive learning, (ii)
autoencoders, and (iii) hardware telemetry. We demonstrate AutoPerf's
generality and efficacy against 3 types of performance regressions across 10
real performance bugs in 7 benchmark and open-source programs. On average,
AutoPerf exhibits 4% profiling overhead and accurately diagnoses more
performance bugs than prior state-of-the-art approaches.
In this position paper, we describe our vision of the future of machine
programming through a categorical examination of three pillars of research.
Those pillars are: (i) intention, (ii) invention, and(iii) adaptation.
Intention emphasizes advancements in the human-to-computer and
computer-to-machine-learning interfaces. Invention emphasizes the creation or
refinement of algorithms or core hardware and software building blocks through
machine learning (ML). Adaptation emphasizes advances in the use of ML-based
constructs to autonomously evolve software.
Classical anomaly detection (AD) is principally concerned with point-based
anomalies, anomalies that occur at a single point in time. While point-based
anomalies are useful, many real-world anomalies are range-based, meaning they
occur over a period of time. Therefore, applying classical point-based accuracy
measures to range-based AD systems can be misleading. In this paper, we present
a new mathematical model that more accurately gauges the classification
correctness of AD systems for range-based anomalies. Unlike prior work, our
mathematical definitions are a superset of the classical AD definitions,
enabling our system to also subsume point-based anomalies. Moreover, our system
is broadly generalizable and provides a number of specialization functions that
can control the application's bias along a multi-dimensional axis.
This short paper describes our ongoing research on Greenhouse - a
zero-positive machine learning system for time-series anomaly detection.
Classical anomaly detection is principally concerned with point-based
anomalies, anomalies that occur at a single data point. In this paper, we
present a new mathematical model to express range-based anomalies, anomalies
that occur over a range (or period) of time.
Stream processing addresses the needs of real-time applications. Transaction
processing addresses the coordination and safety of short atomic computations.
Heretofore, these two modes of operation existed in separate, stove-piped
systems. In this work, we attempt to fuse the two computational paradigms in a
single system called S-Store. In this way, S-Store can simultaneously
accommodate OLTP and streaming applications. We present a simple transaction
model for streams that integrates seamlessly with a traditional OLTP system. We
chose to build S-Store as an extension of H-Store, an open-source, in-memory,
distributed OLTP database system. By implementing S-Store in this way, we can
make use of the transaction processing facilities that H-Store already
supports, and we can concentrate on the additional implementation features that
are needed to support streaming. Similar implementations could be done using
other main-memory OLTP platforms. We show that we can actually achieve higher
throughput for streaming workloads in S-Store than an equivalent deployment in
H-Store alone. We also show how this can be achieved within H-Store with the
addition of a modest amount of new functionality. Furthermore, we compare
S-Store to two state-of-the-art streaming systems, Spark Streaming and Storm,
and show how S-Store matches and sometimes exceeds their performance while
providing stronger transactional guarantees.