Email content

View in browser

Materialize

We’ve been busy working with our design partners to turn their operational data into live context layers for agents and apps. The idea in a nutshell: agent tools call your operational databases, those writes flow into Materialize and within about a second Materialize updates an agent’s worldview of your entire business (either directly or by publishing updates to a search index). This creates an interactive experience where humans and agents can collaborate in real time.

We’re seeing more and more of these roll out in production; here’s a case study from the team at Bilt showing these ideas in practice. They have an AI concierge that books reservations and opens disputes, and a RAG pipeline that blends embeddings with live attributes. Their end-to-end freshness went from about an hour to ~1 second, and query latency fell from two seconds to single-digit milliseconds. This also had big implications on token efficiency: "We serve most concierge traffic on fast, efficient models and get frontier-quality answers, because the context does the heavy lifting, not the model."

Of course, we don’t get everything right on the first try; the feedback from our customers makes it into our product weekly. You'll find the full details in the release notes, but here are some of my favorite updates!

Agentic developer and operator experience

New agent skills. Agents can now create ontologies, optimize performance, and diagnose freshness issues. This lets you automate and speed up your iteration cycles with Materialize. In our internal Materialize “dogfood” context graph, our agents were able to reduce our memory consumption by 50%!

Expanded connectors

Sink to Databricks on AWS. The Materialize Iceberg Sink now integrates with the Databricks Unity Catalog. We previously shipped support for AWS S3 Tables and GCP BigLake. Transform your data in Materialize to give your agents fresh context, and then use this sink to continually push updates into your lakehouse for historical analysis and training.

Improved performance

Dictionary compression (public preview). Materialize now stores repeated column values once and references them by row, so columns with many repeated values use much less memory in steady state. Results vary depending on the shape of your data, but we’ve seen memory footprints reduced by as much as 80%. This gives you more headroom for ad hoc queries and to absorb memory spikes without resizing.

Faster MySQL snapshots (private preview). Initial MySQL snapshots (which use CHAR or VARCHAR primary keys) now run in parallel, so large sources are ready to query much sooner. In our tests, a 2-billion-row table went from 220 minutes to 43 minutes, an 80% reduction.