# Scale email sync to many mailboxes

Source: https://developer.nylas.com/docs/cookbook/email/scale-email-sync/

Syncing one inbox is a weekend project. Syncing 10,000 of them is a systems problem. The moment you scale past a few hundred mailboxes, polling each account on a timer collapses under provider rate limits, grants start expiring faster than you notice, and a single bad deploy can silently drop messages for thousands of users. The design choices you make early decide whether this stays a background job or becomes a 3 AM page.

This page lays out an architecture for keeping thousands of connected mailboxes in sync with the [Nylas Email API](/docs/reference/api/messages/): event-driven webhooks instead of polling, rate-limit handling that doesn't stampede, automatic reconnect when grants expire, and a one-time backfill for history. Each section answers a question real teams ask when they outgrow the naive approach.


> **Info:** 
> **New to Nylas?** Start with the [quickstart guide](/docs/v3/getting-started/) to set up your app and connect a test account before continuing here.


## How do enterprise SaaS companies handle email sync for thousands of users?

Enterprise teams stop polling and go event-driven. Instead of a worker that wakes every few minutes to ask each mailbox "anything new?", they subscribe once to a webhook stream that pushes `message.created` and `message.updated` notifications as mail arrives. With 10,000 mailboxes, polling every 5 minutes means 2.88 million requests per day before a single message is read. Webhooks turn that into one notification per actual event.

The pattern is a fan-in, not a fan-out. Every connected account routes its changes through a single Nylas webhook destination you register at `POST /v3/webhooks`, so your endpoint receives one ordered stream rather than managing 10,000 polling cursors. Each notification carries the `grant_id` of the affected mailbox plus the object that changed, up to a 1 MB payload. Your job shrinks to receiving, verifying, and queuing. The provider quota math that breaks polling, where every Microsoft 365 and Gmail mailbox carries its own per-user request ceiling, mostly disappears because you only call the API when an event tells you to.

## What is the most scalable architecture for email sync in a SaaS app?

The most scalable design separates three jobs that fail differently: a thin webhook receiver, a durable queue, and worker pools that call the API. The receiver only verifies the `x-nylas-signature` header and enqueues; it never blocks on database writes. Workers pull from the queue and fetch full objects when needed. This split lets you absorb a 100,000-message spike without dropping notifications, because the queue is the buffer.

Three components carry the load. First, a webhook destination registered through `/v3/webhooks` with the trigger types you care about, returning a `200` within a few seconds or Nylas retries. Second, a message queue (SQS, Pub/Sub, or similar) that decouples receipt from processing, so a slow database never causes a dropped event. Third, idempotent workers keyed on message ID, because retries and `.truncated` re-fetches mean the same event can arrive twice. When a payload exceeds 1 MB, the API strips the body and appends `.truncated` to the trigger name, so workers re-query `GET /v3/grants/{grant_id}/messages/{message_id}` for the full object. Design for at-least-once delivery from day one.

The flow below registers a single webhook destination for the message and grant lifecycle triggers that drive sync at scale. You create it once per environment, not per mailbox, and every connected grant routes through it. Note the `notification_email_addresses` field, which alerts your team if the destination starts failing validation.

```bash
curl --request POST \
  --url 'https://api.us.nylas.com/v3/webhooks' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>' \
  --data-raw '{
    "trigger_types": [
      "message.created",
      "message.updated",
      "grant.created",
      "grant.expired",
      "grant.deleted"
    ],
    "description": "production-email-sync",
    "webhook_url": "https://api.yourapp.com/webhooks/nylas",
    "notification_email_addresses": ["oncall@yourapp.com"]
  }'
```

For the full tradeoff between the two models, including when a timed poll still wins, see [webhooks vs polling](/docs/cookbook/use-cases/build/webhooks-vs-polling/).

## How do I handle email sync failures and reconnects in production?

In production, treat grant expiry as a routine event, not an error. OAuth tokens revoke, passwords change, and providers force re-auth, so subscribe to `grant.expired` and `grant.deleted` from the start. When a `grant.expired` notification arrives, that mailbox stops syncing until the user re-authenticates. Catch it, flag the account, and trigger your re-auth flow instead of letting messages silently pile up unsynced.

Grant lifecycle webhooks are the backbone of reliable multi-mailbox sync. The `grant.created` trigger tells you a new mailbox is ready to backfill. The `grant.expired` trigger fires when credentials lapse, which is your cue to prompt the user through `/v3/connect/auth` again. There's one sharp edge documented in the [notifications reference](/docs/reference/notifications/): if a grant is out of service for more than 72 hours, the missed notifications can't be backfilled. Past that 72-hour window you can't rely on catch-up events. Instead, compare the `grant.expired` and `grant.updated` timestamps and re-query `GET /v3/grants/{grant_id}/messages` for anything that changed in the gap. Build that reconciliation path before you need it.

The handler below routes inbound webhooks by trigger type. It re-enables sync on reconnect and marks a grant stale on expiry, the two transitions that decide whether a mailbox stays current.

```python
def handle_webhook(payload):
    trigger = payload["type"]
    grant_id = payload["data"]["object"]["grant_id"]

    if trigger == "grant.expired":
        mark_grant_stale(grant_id)        # stop counting it as healthy
        notify_user_reauth(grant_id)      # send them through /v3/connect/auth
    elif trigger == "grant.created":
        enqueue_backfill(grant_id)        # one-time history import
    elif trigger in ("message.created", "message.updated"):
        enqueue_message(payload["data"]["object"])
    elif trigger == "grant.deleted":
        purge_grant_data(grant_id)        # honor deletion
```

## How do I avoid rate limits when syncing many mailboxes?

Throttle per grant, not just globally, because provider limits are per-mailbox. Microsoft and Google both apply per-user quotas that a fleet-wide limiter can't see, so 500 workers can stay under your global ceiling while still flooding one busy mailbox. Check each provider's current published limit before you tune your throttle. Cap concurrency per `grant_id`, back off on a `429` using the `Retry-After` header, and add jitter so retries don't synchronize into a thundering herd. In practice, even a low per-mailbox cap of 4 to 8 in-flight requests prevents most throttling.

Webhooks already cut most polling traffic, but backfills and re-syncs still send bursts against single accounts. Three habits keep large fleets under quota. Limit concurrent requests per mailbox to a small number, since providers throttle a single account long before they throttle your whole app. Use the `limit` query parameter on `GET /v3/grants/{grant_id}/messages` to pull up to 200 objects per call, which means fewer round trips per page. And run history imports on a separate worker pool from live webhook processing, so a heavy backfill never starves real-time updates. The detailed backoff playbook lives in [handle rate-limit errors](/docs/cookbook/use-cases/build/handle-rate-limit-errors/).

## How do I backfill history for a newly connected mailbox?

Run a one-time backfill when `grant.created` fires, then rely on webhooks for everything after. New mail arriving post-connection generates `message.created` events, but a mailbox holding 50,000 messages from the last decade predates the connection and triggers nothing. Page through `GET /v3/grants/{grant_id}/messages` with a `limit` of 200, follow `next_cursor` until it stops returning a value, then mark the grant backfilled.

Backfill and live sync are two separate jobs with different failure modes. The backfill is a bounded batch task you can pause, checkpoint, and resume by persisting the latest `next_cursor` as `page_token`. The webhook listener is a lightweight always-on process. Keeping them apart means a stalled import of one 80,000-message mailbox never blocks real-time delivery for the other 9,999 accounts. A 20,000-message mailbox finishes in roughly 100 sequential requests at the maximum page size. The full loop, including IMAP cache behavior and resume logic, is covered in [backfill historical email](/docs/cookbook/email/backfill-historical-email/), so this page links it rather than repeating the code.

## Webhooks vs polling for fleet sync

The table compares the two sync models at fleet scale, where the per-mailbox cost compounds. The unified webhook column is what production teams reach for once they pass a few hundred accounts.

| Concern | Polling each mailbox | Nylas webhooks |
| --- | --- | --- |
| **Requests at 10,000 mailboxes** | ~2.88M/day at a 5-minute interval | 1 notification per actual event |
| **Latency to new mail** | Up to your poll interval (minutes) | Near real-time push |
| **Rate-limit exposure** | High; every poll counts against quota | Low; you only call the API on events |
| **Grant expiry detection** | Discovered on the next failed poll | `grant.expired` pushed immediately |
| **Operational cost** | Scales with mailbox count | Scales with event volume |

Polling still wins in one case: a small, fixed fleet of a few dozen mailboxes where a predictable timed sweep is simpler than running a public webhook endpoint. Below a few hundred accounts, the infrastructure for receiving and queuing webhooks may not pay for itself yet. Past that, the math tips hard toward events.

## What's next

- [Get real-time updates with webhooks](/docs/cookbook/use-cases/build/realtime-webhooks/) for the receiver, signature verification, and queue setup
- [Backfill historical email](/docs/cookbook/email/backfill-historical-email/) for the one-time history import each new grant needs
- [Handle rate-limit errors](/docs/cookbook/use-cases/build/handle-rate-limit-errors/) for the backoff and jitter playbook at fleet scale
- [Webhooks vs polling](/docs/cookbook/use-cases/build/webhooks-vs-polling/) for the full decision between the two sync models