Rethinking the application-database interface

Abstract

From social networking websites to bank transactions, we interact with data-intensive applications every day. Such applications are typically hosted on an application server that interacts with a database server to manipulate persistent data. To make such applications efficient, developers face the daunting task of mastering the intricacies of both the programming system and the database management system. For instance, while many application features can be implemented in either the application or pushed into the database, it is difficult for a programmer to decide where to place a given computation as the decision is typically workload-driven. Unfortunately, making the wrong choice often results in drastic performance hit. In this talk, I will show how examining both the programming system and the database management system at the same time allows us to significantly improve the performance of data-intensive applications. To illustrate such cross-system optimization opportunities, I have designed, built, and evaluated three systems: Query By Synthesis, a tool that converts functionality written as imperative code into relational queries; Sloth, a system that combines queries embedded in applications into batches; and Pyxis, a system that seamlessly moves computation between application and database servers. Using real-world examples, I will show that these systems allow orders of magnitude performance improvement and graceful adaptation to changing server environments while preserving the high-level programming interface to the developer. I will furthermore highlight research opportunities and challenges in applying similar techniques to other system problems.