Platform API access tiers (Meta & Google)
Both of V2's platform credentials sit on entry-level API access tiers, while the V1 stack's credentials had standard access on both platforms. This single fact — one tier ceiling per platform — is the root cause of every fleet-wide rate-limit wave observed since cutover. It is not a pacing problem, and no amount of scheduler tuning removes it; both fixes are human tier applications in the platforms' consoles.
Diagnosed by live probes on 2026-08-18 (memory plane 32407 / 32414). Until the upgrades land, the conveyor pacing, per-account throttle isolation, adaptive window splitting, async insights jobs, and pooled poll budgets documented in Ingestion are what keep syncs completing inside these ceilings. After the upgrades they remain as headroom, not survival mechanisms.
Meta: app on Marketing API "development access"
| Credential | Meta app 722299047588298 (system-user token) |
| Tier | ads_api_access_tier: development_access |
| Evidence | X-Business-Use-Case-Usage response header returns the tier literally; chronic code-4 / subcode-1504022 "Application request limit reached" errors at request volumes V1 handled trivially |
| Scope of the ceiling | Per app — a shared point budget across every ad account the token touches, with a 300s decay window (reads 1pt, writes 3pt) |
| Symptom signature | Multiple clients' Meta accounts throttle in the same minutes; sync-health shows error_class: throttled with code 4 (app-level), not code 17 (account-level) |
Probe (1 call): GET act_<id>?fields=id with the vault creds, then read the
X-Business-Use-Case-Usage header — it names the tier outright. (debug_token
resolves which app a token belongs to.)
Upgrade path (human): developers.facebook.com → app 722299047588298 → App Review → request Ads Management Standard Access (the Marketing API "Standard Access" advanced-access item). Apps tied to a verified business are often auto-granted. Standard access raises the app budget by orders of magnitude.
Google: developer token on "Basic Access"
| Credential | The Google Ads developer token (owned by the manager/MCC account; per-client OAuth creds live in the vault) |
| Tier | Basic Access — 15,000 operations/day, shared |
| Evidence | Live probe 2026-08-18 08:51Z: 429 RESOURCE_EXHAUSTED with quotaErrorDetails: { rateScope: "DEVELOPER", rateName: "Number of operations for basic access", retryDelay: "18269s" } |
| Scope of the ceiling | Per developer token — one daily operations pool across every client's Google account. rateScope: DEVELOPER is the tell; an account-scoped limit would read ACCOUNT |
| Symptom signature | Every Google account in the fleet 429s together (google searchStream 429 on TS, VLS, CS, Pup Labs, iPhone Freedom simultaneously) |
Two behaviors that cost us diagnosis time — do not relearn them:
- The window is rolling, not midnight-anchored. Fresh 429s arrive after
midnight Pacific;
retryDelaycounts down as old operations age out of a moving 24-hour window. Never wait for a "daily reset" that isn't coming. - Sync-health truncates the interesting part. The stored
last_sync_errorcuts off beforequotaErrorDetails, so the error column shows a generic 429. The probe script below exists precisely to print the full verdict.
Probe (1 operation, read-only):
# needs production env in the shell (vercel env pull)
npx tsx scripts/probe-google-quota.ts [client-slug] [account-id]
# defaults: top-speed-ts 2006860536
Prints QUOTA-RESET on success, or Google's full quota error including
rateScope, rateName, and the live retryDelay countdown.
Upgrade path (human): Google Ads UI → Tools → API Center on the manager account that owns the developer token → apply for Standard Access (short questionnaire). Standard access removes the daily operations cap under fair-use policy.
How the sync architecture interacts with these ceilings
- Inngest cooldowns are retry scheduling, not stored state. A quota 429
becomes
MarketingRateLimitError→RetryAfterErrorwith the platform's own retry hint; the run is parked, nothing is flagged in the database. "Is the connection back?" is answered by probing or by the next successful sync —last_sync_errorclears on success, so recovery shows aslast_synced_atnewer thanlast_sync_error_at. - Throttling is isolated per account binding (see Ingestion): one exhausted account defers only itself, and the paid-account conveyor keeps rotating the rest. Under a tier ceiling this shows up as several accounts parked at once — expected, not a bug.
- Error classification lives in one place:
src/lib/marketing/platform-fetch.ts(platformHttpError) is the single authority for what counts as a throttle (Meta codes 4/17/32, subcodes 80000–80999,Retry-After; Google per-minute vs per-day exhaustion). Per-day Google exhaustion is deliberately not retried in-process.
Status ledger
| Date | Event |
|---|---|
| 2026-08-18 | Both tiers diagnosed by live probe; upgrade applications are with the operator. Update this row when a tier changes, and re-run both probes to confirm before deleting any throttling mitigation. |