Speed
-
ClickHouse columnar database designed for real-time analytical queries on massive datasets
with exceptional speed
-
Qdrant vector database designed for storing, searching, and managing high-dimensional vector embeddings
with filtering capabilities
https://console.cloud.google.com/compute/instances?authuser=5&project=chat-plot-staging
Better Metadata: Data Catalog
https://github.com/Roker-Research/data-catalog
-
Current workflow
- Gram starts from scratch every time
- Northstar starts from scratch every time
-
Proposed workflow
- Constant, incremental improvement
- Human-guided surgical updates
- Fewer tokens
- Fast ingestion
Better Context
https://staging.axes.com/admin/system-prompt
https://staging.axes.com/admin/search
- Variables vs Columns
- Seeding the context with dataset list
- Examples in system prompt
- Clean output