Soup

Soup

A Flexibility-First Database

• Jon Gjengset
• Malte Schwarzkopf
• Nikhil Benesch
• Eddie Kohler
• M. Frans Kaashoek
• Robert Morris

Web applications ♥ databases

Almost all web applications use relational/noSQL databases.
Existing systems are convenient, and familiar, but not perfect:

• Schema construction and migration is painful.
• Caching and cache invalidation is hard.
• Reactive applications need streaming results.

Soup: a flexibility-first database

• Keep all raw writes.
• Queries are computations over history of writes.
• Database is given queries ahead of time.
• Query results are precomputed (and cached).
• Data-flow enables streaming, incremental processing of writes.

A schemaless database with automatic caching and streaming queries.

Soup: what does it look like?

Example: a common SQL query

			SELECT
			    article.title,
			    COUNT(vote.user)
			FROM article
			LEFT JOIN vote
			    ON (article.id = vote.article)
			GROUP BY article.id
			WHERE article.id = ?
		

Observation: query is a layered computation

			SELECT
			    article.title,
			    COUNT(vote.user)
			FROM article
			LEFT JOIN vote
			    ON (article.id = vote.article)
			GROUP BY article.id
			WHERE article.id = ?
		

Internal system view

Adding more queries

The cost of flexibility

What we're working on

• Query rewriting strategies.
• Efficient and transparent queryset changes.
• Persistence and snapshots of materialized views.
• Multi-machine (including client) sharding and replication.

Thank you. Get in touch at jon@tsp.io.