| 1 | # `telemetry-ingest` — Codewhale's first-party telemetry endpoint |
| 2 | |
| 3 | A Cloudflare Worker that accepts the batches described in |
| 4 | [`docs/TELEMETRY.md`](../docs/TELEMETRY.md) and writes them to Workers Analytics |
| 5 | Engine. One POST route. No response body on any path. The handler never reads |
| 6 | client-IP metadata and the closed payload schema accepts no address field. |
| 7 | |
| 8 | It lives here and not in `web/` because the site is a separate deploy with its |
| 9 | own build (Next.js via OpenNext); this is a single 13 KiB script with no assets, |
| 10 | and coupling the two would mean a telemetry change rebuilding the marketing site. |
| 11 | |
| 12 | **Deployed and live** at `https://telemetry.codewhale.net/v1/telemetry`, which |
| 13 | is the shipped default for `telemetry_endpoint`. workers.dev is disabled; that |
| 14 | hostname is the only way in. |
| 15 | |
| 16 | The current source defaults usage analytics on under notice version5, with a |
| 17 | clear opt-out. Prior explicit declines remain off; a missing acceptance record |
| 18 | is not treated as consent. Schema3 carries `notice_version: 5` policy metadata; |
| 19 | legacy schema2 retains its original explicit `consent_version: 4` contract. |
| 20 | Schema1 clients are accepted only into first-party Analytics Engine. Local |
| 21 | schema3 validation is not proof this updated Worker has been deployed. A user who wants |
| 22 | to contact nobody sets `telemetry_endpoint = ""`, which writes batches to |
| 23 | `$CODEWHALE_HOME/telemetry/dryrun.jsonl` instead. |
| 24 | |
| 25 | --- |
| 26 | |
| 27 | ## Optional PostHog processor |
| 28 | |
| 29 | The new sink is **unconfigured by default**. It uses no analytics SDK and does |
| 30 | not create a second collector. `POSTHOG_HOST` must be exactly |
| 31 | `https://us.i.posthog.com` or `https://eu.i.posthog.com`, and |
| 32 | `POSTHOG_PROJECT_TOKEN` must be supplied as a Worker secret. The separate |
| 33 | `POSTHOG_IP_SAFE_EGRESS_VERIFIED="true"` prerequisite requires the staging |
| 34 | receipt below. Never commit a real token. Missing any prerequisite or using |
| 35 | an untrusted host leaves the sink off. |
| 36 | Deployment, token configuration, retention/privacy settings, and activation |
| 37 | require separate operator approval; none is established by the local tests. |
| 38 | |
| 39 | Validated schema3 batches carrying `notice_version: 5` and legacy schema2 |
| 40 | batches carrying explicit `consent_version: 4` can reach the processor. Legacy v1 always remains first-party only. Capture uses |
| 41 | `/batch/`, fixed anonymous/no-geo controls, no incoming headers, no cookies, |
| 42 | no redirects, no response parsing, no retries, and a 1.5-second timeout. |
| 43 | PostHog failure is isolated from the successful Analytics Engine write. |
| 44 | See [the complete disclosure](../docs/TELEMETRY.md) and |
| 45 | [PostHog's capture contract](https://posthog.com/docs/api/capture). |
| 46 | |
| 47 | **Egress activation remains unproved.** Cloudflare can add client-IP headers to |
| 48 | Worker subrequests destined for non-Cloudflare zones, even when application |
| 49 | code builds fresh headers. The guard is an operator prerequisite, not code |
| 50 | that strips platform headers. Before setting it, record a staging receipt for |
| 51 | the actual deployment and regional destination proving original client IPs |
| 52 | are absent from all received headers, including `CF-Connecting-IP`, |
| 53 | `X-Forwarded-For`, and `X-Real-IP`. Requalify if the egress path changes. Without |
| 54 | that receipt, leave the sink disabled; use a detached first-party delivery |
| 55 | context if the deployed network cannot satisfy the invariant. Local fetch |
| 56 | mocks do not prove edge behavior. See [Cloudflare's header behavior](https://developers.cloudflare.com/fundamentals/reference/http-headers/#cf-connecting-ip-in-worker-subrequests). |
| 57 | |
| 58 | The cross-repository CWC contract is generated, not independently authored: |
| 59 | `node scripts/export-product-schema.mjs` (Node 22.18+) exports |
| 60 | `schema/cwc-product-v3.schema.json` from this validator's constants. The v2 |
| 61 | artifact remains available for existing clients. It accepts |
| 62 | only `web-app` / `desktop`, one closed `product_usage` event, and the fixed |
| 63 | no-fingerprint browser envelope. Copies must be compared with this generated |
| 64 | artifact when changing the schema. `test/golden/browser-v3.json` is a complete |
| 65 | website wire example; CWC changes only its surface to `web-app` or `desktop`. |
| 66 | `operations_summary` is separate operator-consented service health: six `u32` |
| 67 | aggregates on `control-plane` with a new random install ID per batch, never |
| 68 | individual request records or end-user identifiers. |
| 69 | |
| 70 | The website/app same-origin proxy, when configured, must validate and forward |
| 71 | only the closed batch body and reconstruct a content-type header. Do not pass |
| 72 | through the incoming request, cookies, user agent, authentication, or metadata. |
| 73 | The ingest's two-header invariant remains unchanged. |
| 74 | |
| 75 | ## The one property that matters |
| 76 | |
| 77 | `docs/TELEMETRY.md` publishes: |
| 78 | |
| 79 | > Batches are **IP-stripped at ingest**. No IP is stored, logged, or joined to |
| 80 | > `install_id`. |
| 81 | |
| 82 | This Worker is the whole of what makes that sentence true. There is no other |
| 83 | component. So: |
| 84 | |
| 85 | - `src/index.ts` reads exactly two request headers — `content-type` and |
| 86 | `content-length` — and nothing else, ever. |
| 87 | - It never touches the `cf` property of the request, so country, colo, city, |
| 88 | region, ASN and coordinates are never in scope. |
| 89 | - `src/datapoint.ts` builds every stored row, and it cannot see the request at |
| 90 | all: its input type is the validated batch body. |
| 91 | - Nothing logs. `invocation_logs` is off in `wrangler.jsonc`, because Cloudflare |
| 92 | describes those as "enriched with information available to Cloudflare in the |
| 93 | context of the invocation" — exactly the class of automatic per-request record |
| 94 | this service promises not to keep. |
| 95 | - `test/no-ip.test.ts` reads the shipped source as text and fails the build if |
| 96 | any of those names appears, if the set of headers read grows past two, if a |
| 97 | `console.*` call is added, or if a `Response` is ever constructed with a body. |
| 98 | That file is the only place in this directory where the forbidden header names |
| 99 | are written down. A later edit cannot add one quietly. |
| 100 | |
| 101 | Debugging without an IP is a solved problem: the schema carries `os`, `arch`, |
| 102 | `libc`, `surface`, `app_version` and `git_sha`, which is what crash triage |
| 103 | actually needs. |
| 104 | |
| 105 | ## What it stores |
| 106 | |
| 107 | Everything in [`docs/TELEMETRY.md`](../docs/TELEMETRY.md) and nothing else. The |
| 108 | validator in `src/schema.ts` is a **closed** field set: an unexpected key |
| 109 | anywhere in the batch — envelope, event, `counters`, `errors`, `turn_wall` — |
| 110 | rejects the whole batch with `400`. That is the point of the design. A future |
| 111 | client bug that starts attaching a path, a prompt, or a customer's provider |
| 112 | table name gets refused by the server rather than quietly stored. |
| 113 | |
| 114 | `test/schema-doc.test.ts` parses the field names and enum spellings back out of |
| 115 | `docs/TELEMETRY.md` and asserts set equality against the validator, and |
| 116 | `test/ingest.test.ts` posts `crates/telemetry/tests/golden/v1.json` — the |
| 117 | client's own pinned v1 wire form — and asserts it is accepted byte for byte and |
| 118 | that deleting *any* key from it is rejected. The doc, the Rust client, and this |
| 119 | endpoint cannot drift apart without a red test. |
| 120 | |
| 121 | ### Column layout |
| 122 | |
| 123 | One Analytics Engine data point per event. A batch carries at most 200 events |
| 124 | (`BATCH_MAX_EVENTS`) and Analytics Engine allows 250 data points per invocation, |
| 125 | so a conforming batch never needs a second pass. |
| 126 | |
| 127 | The layout is **positional and append-only** — Analytics Engine columns are |
| 128 | `blob1..blob20` / `double1..double20`, and the names below exist only in the SQL |
| 129 | you write. Renumbering silently rewrites every historical query. To add a field, |
| 130 | take the next free slot. |
| 131 | |
| 132 | | column | contents | |
| 133 | |---|---| |
| 134 | | `index1` | `install_id` — random v4 UUID, client-rotated every 90 days. The only identifier in the schema. | |
| 135 | | `blob1` | `event` — `install_or_upgrade` \| `session_start` \| `session_end` \| `panic` \| `product_usage` \| `operations_summary` | |
| 136 | | `blob2` | `surface` | |
| 137 | | `blob3` | `os` | |
| 138 | | `blob4` | `arch` | |
| 139 | | `blob5` | `libc` | |
| 140 | | `blob6` | `app_version` | |
| 141 | | `blob7` | `git_sha`, `''` for every locally built binary | |
| 142 | | `blob8` | `tty` — `'true'` \| `'false'` | |
| 143 | | `blob9` | `install_kind` (`install_or_upgrade` only) | |
| 144 | | `blob10` | `previous_version` (`install_or_upgrade` only) | |
| 145 | | `blob11` | `session_source` (`session_start` only) | |
| 146 | | `blob12` | `duration_bucket` (`session_end` only) | |
| 147 | | `blob13` | `exit_class` (`session_end` only) | |
| 148 | | `blob14` | `cold_start_bucket` (`session_end` only; `''` on surfaces that do not measure it) | |
| 149 | | `blob15` | `providers`, comma-joined, already sorted and deduplicated | |
| 150 | | `blob16` | `panic_site` (`panic` only) — a `crates/…` path or the literal `<dep>` | |
| 151 | | `blob17` | `sent_at` — the *batch* timestamp. Events carry none. | |
| 152 | | `blob18` | `aggregate_counters` — closed JSON counts, for `product_usage` or `operations_summary` | |
| 153 | | `blob19` | `schema_version` | |
| 154 | | `blob20` | `privacy_version`: notice5 for v3, consent4 for v2, empty for v1; `blob19` keeps their meanings distinct | |
| 155 | | `double1..10` | `counters`: `turns`, `tool_calls`, `fleet_dispatch`, `workflow_run`, `subagent_spawn`, `mcp_server_connected`, `memory_search`, `approval_modal_shown`, `approval_auto_allowed`, `command_palette_open` | |
| 156 | | `double11..16` | `errors`: `auth_preflight_failed`, `provider_http_4xx`, `provider_http_5xx`, `tool_denied_by_policy`, `tool_timeout`, `network_error` | |
| 157 | | `double17..20` | `turn_wall`: `lt_5s`, `5_30s`, `30_120s`, `gte_120s` | |
| 158 | |
| 159 | Columns not relevant to an event are `''` / `0`. `tty` is a blob because the 20 |
| 160 | doubles are exactly used by the three numeric structs — Analytics Engine's |
| 161 | ceiling is 20. |
| 162 | |
| 163 | ### What it structurally cannot store |
| 164 | |
| 165 | Not "does not"; **cannot**, given the code as written: |
| 166 | |
| 167 | - **The client IP, and anything derived from it** — never read. See above. |
| 168 | - **Any geo** — country, colo, city, region, ASN, coordinates, timezone. |
| 169 | - **Any key the schema does not name.** Unknown key ⇒ `400` for the entire |
| 170 | batch, so there is no path from an unexpected field to storage. |
| 171 | - **Any free-form string.** The published schema has no free-form string type and |
| 172 | no open-keyed map. Every field is an integer, a boolean, or a closed enum, |
| 173 | except `app_version`, `git_sha` and `panic_site` — each of which has a regex |
| 174 | here, so a path, a prompt, a URL, or a branch name fails the shape check. |
| 175 | - **A provider table name.** `providers` entries must be lowercase hyphenated ids |
| 176 | and the array must be sorted and deduplicated; `acme_internal_gateway` is |
| 177 | rejected. (See "known gap" below.) |
| 178 | - **A panic message.** Only `panic_site`, and only inside the `crates/` |
| 179 | allowlist or the literal `<dep>`. |
| 180 | - **Per-event timestamps.** There are none in the schema; only `sent_at`, |
| 181 | per batch. |
| 182 | - **Response content.** Every response is a bare status with a `null` body, so |
| 183 | the endpoint cannot echo back what it received or what it holds. |
| 184 | |
| 185 | **Known gap, stated plainly.** `providers` is the one field whose *value* space |
| 186 | this endpoint cannot close. The authoritative list is |
| 187 | `codewhale_config::provider::all_providers()`, a Rust registry with no generated |
| 188 | artifact to read, and hard-coding a copy here would drift into silently dropping |
| 189 | a real user's route. The client closes it (`Event::is_bounded` → |
| 190 | `is_known_provider_id`) before the POST is made; the server enforces the shape a |
| 191 | closed `&'static str` enum can produce, plus the doc's sorted-and-deduplicated |
| 192 | rule. If a generated provider-id list ever lands in the repo, wire it in here. |
| 193 | |
| 194 | ### Retention |
| 195 | |
| 196 | Cloudflare stores Analytics Engine data for **three months**, and that is not |
| 197 | configurable — a ceiling rather than a policy, since no setting could make it |
| 198 | longer. `docs/TELEMETRY.md` states it. |
| 199 | |
| 200 | --- |
| 201 | |
| 202 | ## Deploy |
| 203 | |
| 204 | Live. The commands below are the ones that produced the current deployment and |
| 205 | the ones that will produce the next one. |
| 206 | |
| 207 | ```sh |
| 208 | cd telemetry-ingest |
| 209 | npm install |
| 210 | npm test # 109 tests, including the doc weld and the IP guard |
| 211 | npx wrangler deploy --dry-run --outdir=.wrangler/dry-run # no account touched |
| 212 | npx wrangler deploy # <- the only command that publishes anything |
| 213 | ``` |
| 214 | |
| 215 | The `routes` block in `wrangler.jsonc` binds it to `telemetry.codewhale.net` as |
| 216 | a custom domain, so the endpoint URL is |
| 217 | `https://telemetry.codewhale.net/v1/telemetry`. The workers.dev subdomain is |
| 218 | disabled: that hostname is the only way in. |
| 219 | |
| 220 | Verified against the live endpoint before the client default was changed: the |
| 221 | client's golden batch returns `204` with a zero-byte body; an unknown key |
| 222 | returns `400`; `GET` returns `405`; a wrong content type returns `415`; a `POST` |
| 223 | to `/` returns `404`. Reading back from Analytics Engine returned exactly two |
| 224 | rows — `session_start` and `session_end`, carrying `install_id`, `surface=tui`, |
| 225 | `os=macos` — the documented shape and nothing else. |
| 226 | |
| 227 | **Re-run the verification below after any deploy.** The client default now |
| 228 | points here, so a regression in this Worker is a regression in a promise |
| 229 | `docs/TELEMETRY.md` makes to users. |
| 230 | |
| 231 | ### Analytics Engine dataset setup |
| 232 | |
| 233 | There is none. The dataset named in `wrangler.jsonc` |
| 234 | (`codewhale_telemetry`) is created implicitly on the first successful |
| 235 | `writeDataPoint`, so there is nothing to provision ahead of the deploy. Confirm |
| 236 | it exists after the first batch: |
| 237 | |
| 238 | ```sh |
| 239 | curl "https://api.cloudflare.com/client/v4/accounts/$CF_ACCOUNT_ID/analytics_engine/sql" \ |
| 240 | --header "Authorization: Bearer $CF_API_TOKEN" \ |
| 241 | --data "SHOW TABLES" |
| 242 | ``` |
| 243 | |
| 244 | The token needs **Account → Account Analytics → Read**. Querying is out of band |
| 245 | through this API; the Worker itself has no read path at all. |
| 246 | |
| 247 | ### Rate limiting |
| 248 | |
| 249 | `ratelimits` binding, 20 POSTs per 60s, **keyed on `install_id`** from the |
| 250 | validated batch body. Never on a network address — an IP-keyed limiter would |
| 251 | mean this Worker handles IPs, which is the whole thing it must not do. That is a |
| 252 | weaker limiter (an `install_id.json` can be rewritten between POSTs) and it is |
| 253 | the right trade: Cloudflare's edge already absorbs volumetric abuse, and the |
| 254 | client only flushes once per session anyway (one three-second attempt at |
| 255 | shutdown). |
| 256 | |
| 257 | ### Size cap |
| 258 | |
| 259 | `MAX_BODY_BYTES` is 72 KiB, computed rather than guessed. A conforming client |
| 260 | sends at most 200 events totalling at most 65536 bytes (`BATCH_MAX_EVENTS` and |
| 261 | `BATCH_MAX_BYTES` in `crates/telemetry/src/actor.rs`, both hard ceilings because |
| 262 | `parse_events` breaks *before* crossing them), plus 199 commas and ~375 bytes of |
| 263 | envelope keys and values — 66110 bytes worst case. 72 KiB is ~11% headroom. |
| 264 | |
| 265 | The 512-record / 256 KiB rings in `crates/telemetry/src/buffer.rs` are the *disk* |
| 266 | cap, not the wire cap: a full ring drains as three batches, never as one POST. |
| 267 | |
| 268 | `content-length` is checked first as a cheap reject, but it is client-supplied, |
| 269 | so the real bound is enforced while reading the body and the stream is cancelled |
| 270 | the moment it goes over. |
| 271 | |
| 272 | --- |
| 273 | |
| 274 | ## Owner reports and queries |
| 275 | |
| 276 | The routine activity report is a checked-in command rather than SQL copied |
| 277 | from a chat: |
| 278 | |
| 279 | ```sh |
| 280 | CF_ACCOUNT_ID=... CF_API_TOKEN=... npm run report:active-installs |
| 281 | CF_ACCOUNT_ID=... CF_API_TOKEN=... npm run report:active-installs -- --days 30 --json |
| 282 | ``` |
| 283 | |
| 284 | The metric is **observed active installs by UTC day**: distinct rotating |
| 285 | anonymous `install_id` values that produced a `session_start` event that day, |
| 286 | plus the weighted number of sessions started. It is never a count of people, |
| 287 | accounts, or total installs, and the report says so next to the numbers. |
| 288 | `npm run report:dau` still works — `scripts/report-dau.mjs` is a pure |
| 289 | re-export of the canonical `scripts/report-active-installs.mjs`, kept only so |
| 290 | the old name keeps working — but the canonical name matches what the number |
| 291 | actually is. |
| 292 | |
| 293 | One command produces all four things the ledger asks for: |
| 294 | |
| 295 | - the **daily series** (default `--days 15` — 14 complete UTC days plus the |
| 296 | partial current day, marked `*`); |
| 297 | - a **7-day trend** over complete UTC days only: the last 7 days' sum of |
| 298 | daily observed active installs against the previous 7, with the percentage |
| 299 | change. When the window cannot cover both sides it says so instead of |
| 300 | printing zeros; |
| 301 | - **event freshness**: the timestamp of the newest ingested event of any kind |
| 302 | and how stale it is, or a plain statement that nothing has been ingested; |
| 303 | - the **coverage caveats, printed with the numbers** in both text and |
| 304 | `--json` modes: clients older than the telemetry feature, opted-out |
| 305 | installs, and non-emitting environments (kill switches, fleet workers, |
| 306 | offline shutdowns, dropped flushes) are invisible, so every count is a |
| 307 | lower bound; and the 90-day id rotation means week-over-week comparisons |
| 308 | are not a retention metric. |
| 309 | |
| 310 | The token needs **Account → Account Analytics → Read**. The report performs |
| 311 | two read-only SQL requests (the series and the freshness probe), does not |
| 312 | print credentials, and rejects windows beyond Analytics Engine's fixed 90-day |
| 313 | retention. Its read path is pinned by `test/report-active-installs.test.ts`: |
| 314 | the install id appears only inside `count(DISTINCT …)`, no payload column |
| 315 | (`blob2`+, `double*`) is ever selected, and the output wording can never label |
| 316 | the result as users, people, or accounts. |
| 317 | |
| 318 | The underlying ad-hoc queries are one query each, which is what the column |
| 319 | layout was chosen for. Run them against the SQL API: |
| 320 | |
| 321 | ```sh |
| 322 | query() { |
| 323 | curl -sS "https://api.cloudflare.com/client/v4/accounts/$CF_ACCOUNT_ID/analytics_engine/sql" \ |
| 324 | --header "Authorization: Bearer $CF_API_TOKEN" --data "$1" |
| 325 | } |
| 326 | ``` |
| 327 | |
| 328 | `_sample_interval` says how many original rows a stored row represents; |
| 329 | Analytics Engine downsamples high-volume indexes, so every count is weighted by |
| 330 | it rather than using bare `count()`. |
| 331 | |
| 332 | ### (a) How many installs and sessions |
| 333 | |
| 334 | ```sql |
| 335 | SELECT |
| 336 | count(DISTINCT index1) AS installs, |
| 337 | sumIf(_sample_interval, blob1 = 'session_start') AS sessions_started, |
| 338 | sumIf(_sample_interval, blob1 = 'session_end') AS sessions_ended, |
| 339 | sumIf(_sample_interval, blob9 = 'install') AS first_installs, |
| 340 | sumIf(_sample_interval, blob9 = 'upgrade') AS upgrades, |
| 341 | sumIf(_sample_interval, blob13 = 'clean') AS clean_exits |
| 342 | FROM codewhale_telemetry |
| 343 | WHERE timestamp > NOW() - INTERVAL '7' DAY |
| 344 | ``` |
| 345 | |
| 346 | Read `installs` as what the published doc says it is and nothing more: the id |
| 347 | rotates every 90 days and is regenerated whenever the telemetry directory is |
| 348 | cleared, so **no count derived from `install_id` is a user count**. It is a lower |
| 349 | bound on distinct machine-installs seen in the window, and it undercounts a |
| 350 | returning user across a rotation. |
| 351 | |
| 352 | Add `, blob6` to `SELECT` and `GROUP BY blob6` to cut by `app_version`; `blob3` |
| 353 | for OS, `blob2` for surface. |
| 354 | |
| 355 | ### (b) Which error classes and panic sites dominate |
| 356 | |
| 357 | ```sql |
| 358 | SELECT |
| 359 | blob16 AS panic_site, |
| 360 | sum(_sample_interval) AS rows, |
| 361 | sum(double11 * _sample_interval) AS auth_preflight_failed, |
| 362 | sum(double12 * _sample_interval) AS provider_http_4xx, |
| 363 | sum(double13 * _sample_interval) AS provider_http_5xx, |
| 364 | sum(double14 * _sample_interval) AS tool_denied_by_policy, |
| 365 | sum(double15 * _sample_interval) AS tool_timeout, |
| 366 | sum(double16 * _sample_interval) AS network_error |
| 367 | FROM codewhale_telemetry |
| 368 | WHERE timestamp > NOW() - INTERVAL '7' DAY |
| 369 | AND blob1 IN ('session_end', 'panic') |
| 370 | GROUP BY panic_site |
| 371 | ORDER BY rows DESC |
| 372 | ``` |
| 373 | |
| 374 | One query, two answers, because `panic_site` is `''` on every non-panic row: |
| 375 | |
| 376 | - the **`panic_site = ''` row** carries the six error-class totals across all |
| 377 | `session_end` events in the window — that is the error ranking; |
| 378 | - every **other row** is one panic site, ranked by how often it fired. |
| 379 | |
| 380 | Sessions that ended in a panic are visible either way: `blob13 = 'panic'` on the |
| 381 | `session_end` row, and the `panic` event carries the site. |
| 382 | |
| 383 | --- |
| 384 | |
| 385 | ## Verifying no IP is stored |
| 386 | |
| 387 | Three checks, in increasing order of how convincing they are. |
| 388 | |
| 389 | **1. The source cannot ask for it.** `npm test` runs `test/no-ip.test.ts`, which |
| 390 | greps the shipped source. Prove the guard is live by adding a |
| 391 | `request.headers.get("CF-Connecting-IP")` line to `src/index.ts` and re-running — |
| 392 | three tests go red — then revert. |
| 393 | |
| 394 | **2. Every stored column is accounted for.** The schema is closed and the layout |
| 395 | above is exhaustive; there is no free slot an address could occupy. Confirm the |
| 396 | deployed dataset has exactly the columns you expect: |
| 397 | |
| 398 | ```sh |
| 399 | query "SELECT * FROM codewhale_telemetry LIMIT 1 FORMAT JSON" |
| 400 | ``` |
| 401 | |
| 402 | The result carries `dataset`, `timestamp`, `_sample_interval`, `index1`, and the |
| 403 | `blob*`/`double*` columns. There is no address column, because Analytics Engine |
| 404 | has no implicit one — every column is written by `writeDataPoint`. |
| 405 | |
| 406 | **3. Search the stored data for an address shape.** After the first real |
| 407 | batches, this returns zero rows: |
| 408 | |
| 409 | ```sql |
| 410 | SELECT count() AS suspicious |
| 411 | FROM codewhale_telemetry |
| 412 | WHERE timestamp > NOW() - INTERVAL '7' DAY |
| 413 | AND ( |
| 414 | match(index1, '\\d+\\.\\d+\\.\\d+\\.\\d+') |
| 415 | OR match(concat(blob1, blob2, blob3, blob4, blob5, blob6, blob7, blob8, |
| 416 | blob9, blob10, blob11, blob12, blob13, blob14, blob15, |
| 417 | blob16, blob17), '\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}|:[0-9a-f]{1,4}:') |
| 418 | ) |
| 419 | ``` |
| 420 | |
| 421 | Also confirm nothing is being logged: with `invocation_logs` off and no |
| 422 | `console.*` call in the source, **Workers Logs → this Worker** should show no |
| 423 | per-request entries at all. |
| 424 | |
| 425 | --- |
| 426 | |
| 427 | ## Local development |
| 428 | |
| 429 | ```sh |
| 430 | npm install |
| 431 | npm test # vitest, 109 tests |
| 432 | npm run typecheck # tsc --noEmit |
| 433 | npm run check # wrangler deploy --dry-run — touches no account |
| 434 | npm run dev # wrangler dev --local |
| 435 | ``` |
| 436 | |
| 437 | `wrangler dev --local` runs the real `workerd` with local Analytics Engine and |
| 438 | rate-limit bindings. Post the client's own golden batch at it: |
| 439 | |
| 440 | ```sh |
| 441 | curl -i -X POST http://127.0.0.1:8787/v1/telemetry \ |
| 442 | -H 'content-type: application/json' \ |
| 443 | --data-binary @../crates/telemetry/tests/golden/v1.json |
| 444 | # HTTP/1.1 204 No Content, zero-byte body |
| 445 | |
| 446 | curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8787/v1/telemetry |
| 447 | # 405, Allow: POST |
| 448 | ``` |
| 449 | |
| 450 | ### Responses |
| 451 | |
| 452 | | status | when | |
| 453 | |---|---| |
| 454 | | `204` | accepted, zero-byte body, no headers | |
| 455 | | `400` | not JSON, or fails the published schema — including any unknown key | |
| 456 | | `404` | POST to a path other than `/v1/telemetry` | |
| 457 | | `405` | any method other than POST, on any path (`Allow: POST`) | |
| 458 | | `413` | body over `MAX_BODY_BYTES` | |
| 459 | | `415` | content type is not `application/json` | |
| 460 | | `429` | this `install_id` is over the rate limit | |
| 461 | | `500` | internal error; nothing was written | |
| 462 | |
| 463 | Every one of them has an empty body. The client |
| 464 | (`crates/telemetry/src/client.rs`) reads only the status class and drops the |
| 465 | batch on anything that is not 2xx — no retry, no backoff, no re-queue — so a |
| 466 | rejection is invisible to the user by construction, and a 5xx here can never |
| 467 | become a client-visible error. That is what lets this endpoint fail closed: |
| 468 | when in doubt, refuse the batch. |
| 469 |