Events land in your warehouse, not ours

TL;DR

ProForma now ships a warehouse-first tracking SDK — @proforma/analytics — whose destination is a table in your warehouse. Drop one snippet into a website, and within seconds an events table starts filling up in DuckDB+Iceberg or BigQuery, alongside a Source catalog entry, three starter models (pageviews, sessions, daily_active_users), and a pre-built dashboard page. No third-party analytics service, no reverse ETL.

Why this matters

"Ingest" was always the weakest of ProForma's three pillars. You could connect a warehouse and ask the agent to transform or act, but the canonical startup question — how do I capture what's happening in my product? — usually meant adopting a separate analytics vendor and then paying a second system to copy the data back.

Product analytics companies are, fundamentally, middlemen between your app and your warehouse. If the warehouse is the source of truth the business runs on (and we think it is), the middleman should go away. The SDK you paste into your website should write to your warehouse on the first hop.

That's the shipped feature. The shape of the system is telling:

No new primitives. A tracking product built entirely out of Source, Model, Page, Script, Chat — exactly the set we started with.

What changed

• @proforma/analytics, a new workspace package. ProformaBrowser for websites (IIFE + ESM, ~3 KB gzipped) with cookie anonymous_id, $pageview autocapture, sessionStorage identity, and sendBeacon-backed unload flushes. ProForma for Node with a beforeExit drain. Zero runtime dependencies.
• POST /ingest/v1/events — public, CORS-open, auth'd by a project-scoped key (proforma_pk_... for browsers, proforma_sk_... for servers). Batches go into a new event_ingest_buffer table in Postgres; the existing worker drains them every tick using FOR UPDATE SKIP LOCKED, same pattern the script executions loop uses.
• Warehouse writer — DuckDB/Iceberg gets multi-row INSERT into a VARCHAR-column events table (Iceberg has no JSON type, so properties/context are JSON strings). BigQuery gets tabledata.insertAll streaming, a native JSON type on properties/context, DAY partitioning on received_at, and (event_name, user_id) clustering.
• Opinionated catalog bootstrap — on the first successful flush, ProForma (a) creates the events table, (b) registers it as a Source, (c) inserts three starter Models as views, (d) inserts a ready-made dashboard page into the project. By the time the user returns to the tab after dropping in the snippet, there's already something to click on.
• Chat integration — list_ingest_keys, create_ingest_key, and describe_events_schema let the agent set up tracking and write SQL against events without guessing column names.
• Docs — Tracking events for analysts opening ProForma for the first time.

How it works

browser SDK ──POST──▶ /ingest/v1/events
                        │
                        ▼
            resolve ingest_key, CORS, rate-limit
                        │
                        ▼
            INSERT INTO event_ingest_buffer
                        │
             worker tick (~5s)     FOR UPDATE SKIP LOCKED
                        │
                        ▼
   ┌──────────────────────────────┬─────────────────────────┐
   │ DuckDB + Iceberg:            │ BigQuery:               │
   │ multi-row INSERT via         │ table.insert streaming, │
   │ Lakekeeper REST catalog      │ native JSON columns     │
   └──────────────────────────────┴─────────────────────────┘
                        │
                        ▼
              events table (local MinIO in dev)

Public keys are stored plaintext on purpose — they're meant to be embedded in browser JS, and storing them plaintext keeps the hot path lookup O(1) on a unique index. Server keys (proforma_sk_...) are hash-only after creation.

What's next

At startup-scale volumes, per-batch Iceberg commits accumulate small files over time. The plan's Future section calls out the next two investments explicitly:

• Add PARTITIONED BY (days(received_at)) on the DuckDB/Iceberg path once DuckDB's Iceberg writer exposes partition spec DDL (or by calling Lakekeeper directly after CREATE TABLE).
• Ship a starter compaction Script that rewrites and compacts older snapshots — it fits cleanly into context.step() + context.models.refresh(). Another example of the scripts-as- pipelines thesis doing its job.

We're not building a Python SDK or a native mobile SDK yet. And we're not building destinations / reverse-ETL — if you want to forward events to Slack or Mixpanel, write a Script. The warehouse is the first destination; fan-out is code.