Current Architecture

Data Catalog Lives in GitHub

Why?

• Humans and agents can both see and fix metadata
• Incremental changes with version history
• Issues and review workflow
• Controlled Deployments: branch staging production

Arad’s Wish

“Users should be able to ingest new data while chatting with Axes.”

What follows?

• Private datasets
• GitHub is no longer viable for the data catalog
• Growler lives in Chat Plot

New Data Workflow

• Users can upload their own data or sick Growler on a dataset in the wild.
• Users guide the agent to curate the metadata for their datasets— hopefully it one-shots it but we can never be sure.
• Data can be marked “public”
• Admins can mark data “approved” and it becomes available by default

Data Model

• Version history is a stream of CatalogUpdate events. See spec
• Databases have branches/checkpoints. All our databases (PostgreSQL, Qdrant and ClickHouse) have every version of every database.
• More Sources: Periodic Updates, Connections to External/Live Data (s3, psql, etc) filtered through ClickHouse, API fetch.
• Linear history vs forking/branching. Implications for collaboration and the trade off for more complexity in UI and code.

Agent Architecture

• Agent architecture spec
• Coding tasks start and stop in the same subagent loop
• Agent Loops Spec

A Social Network for Data

• MOAR Data Better
• Axes becomes a way to share things with others: data, artifacts, insights, generated UI, reports
• Network effects: more public data more useful more public data

New UI

• Subagents
• Upload a dataset
• Review metadata changes
• Browse datasets: version management, etc

Eval In The New Paradigm

Again: Build it inside Chat Plot

• Test data and code changes in isolation
• The same data model
• Sources: raw data files, reports, academic publications, blog posts
• Claims: numbers extracted from those sources; these can become “verified” if the agents reproduce them
• Retrospective Eval: read a user’s chat
• Generated Eval: create a new chat with a known answer
• A test suite on metadata and schema updates
• Evaluations of tweaks to our agents and search results

Roadmap

• Coding Agents
• Manual Data Updates
• Curation Agent
• Eval