Skip to main content

Architecture & Work Queues

Education scholarship programs and Education Savings Account (ESA) platforms experience extreme traffic seasonality. Reimbursement submissions routinely surge by 10x to 50x during semester purchasing windows and deadline cutoff dates.

To guarantee high availability, zero dropped claims, tenant isolation, and strict protection against downstream AI rate limits (OpenAI TPM/RPM or Google Cloud Vertex AI quotas), HeyPeppy implements an entirely serverless Google Cloud Tasks + Firebase Functions v2 pipeline.

Queue Guarantees & Multi-Tenant Throttling​

1. Tenant-Aware Concurrency Limiting (maxConcurrentDispatches)​

Unlike traditional HTTP servers that can be overwhelmed by sudden bursts, Cloud Tasks holds incoming tasks safely in the queue and dispatches them at a controlled concurrency rate (e.g. 50 tasks/second per tenant pool).

  • Result: Downstream LLM APIs never return 429 Too Many Requests, and one tenant's submission rush never exhausts quota allocated to another tenant.

2. Exponential Retries & Backoff​

If a transient failure occurs (e.g., an upstream provider 503 error or temporary network timeout), Cloud Tasks automatically retries the task:

  • Max Attempts: 5
  • Initial Backoff: 10 seconds
  • Max Backoff: 300 seconds (5 minutes)
  • Doubling Factor: 3

3. Dead-Letter Handling​

If a task fails after 5 attempts (e.g. corrupted PDF file), the job status in Firestore is marked as failed, the specific error message is recorded in the tenant's job record, and an alert is triggered in Google Cloud Monitoring.

Polling vs. Webhooks​

PatternWhen to UseRecommended Latency
Webhooks (Recommended)High-volume batch processing and background pipelines. Your server receives an HTTP POST with HMAC signature when ready.Instant (0s polling overhead)
PollingInteractive user interfaces where the partner portal displays an active progress bar to the parent. Poll GET /v1/jobs/{jobId} every 2–3 seconds.2–5 seconds
Synchronous EndpointLightweight single-item claims where an immediate blocking HTTP response is strictly required. Use /v1/receipts/extract-sync.Direct HTTP connection