Using a Pub/Sub notification channel

In Nylas v3 you can set up a Pub/Sub data connector to receive notifications about activity and changes on the provider. Your project can then consume the notifications, and respond to power your app's logic.

Google Pub/Sub is a event-driven queue that you can divide into several "topics" which you can use to segment event notifications. You can then subscribe to different topics to get notifications as they come in, either individually or in batches, during periods of high volume. Pub/Sub queues also allow you to create a "dead letter" queue to gracefully handle message delivery delays. Pub/Sub is ideal for projects where webhook volume, latency, or deliverability are concerns, or where your project requires deliverability guarantees.

You can use a Pub/Sub connector along with webhooks, and you can use up to five separate Pub/Sub connectors to separate your notifications.

🔍 Good to know: Although you set up the Pub/Sub notification channel on Google Cloud, it can receive notifications about events on any of the providers Nylas supports - not just Google.

Before you begin

Before you can start using Pub/Sub channels, you need the following prerequisites:

A v3 Nylas application.
Your Nylas application's client ID and API key.
A Google Cloud Platform (GCP) project.
- If you're not already using GCP to authenticate Google users, create a GCP project.
- If you already have a Google project for a Google auth app, you can add the Pub/Sub topic for the notifications to that project.
The Google Cloud CLI installed.

Pub/Sub architecture

The Nylas Pub/Sub connector stores information that allows Nylas to connect to and send messages into a Pub/Sub topic on GCP, which your project can consume. Each Nylas application can have up to five Pub/Sub data connectors. (Contact Nylas Support if your project requires more than five.)

You can use Pub/Sub in as a complete replacement for webhook notifications, or alongside existing webhook architecture. For example, you might use a Pub/Sub connector for notification subscriptions that produce a high volume of messages, but use webhooks for lower volume notification subscriptions.

Each Pub/Sub topic can subscribe to any number of notification triggers so you can split your notifications across separate Pub/Sub topics. For example, you can have one topic that handles all email notifications, and separate ones specifically for new events and event changes.

📝 Keep in mind: It might take up to two minutes for you to receive Pub/Sub notifications for newly authenticated grants.

Set up to handle undeliverable notifications

After Nylas delivers the notification to your Pub/Sub queue, it's up to your project to consume the messages.

Nylas recommends that you create a second Pub/Sub topic for each Pub/Sub channel to serve as a "dead letter queue". This topic allows you to collect notifications that your project is unable to consume, so they don't collect in your notification channel and cause latency issues. You can then replay these notifications later, after you resolve any ingestion issues.

See the official Pub/Sub documentation on handling message failures for more information.

Create the Pub/Sub topic

Log in to your Google Cloud console, and go to the Pub/Sub page.
From the Topics page, click Create topic.
In the form that opens, give your topic an ID, and select Add a default subscription.

Make note of the topic name, because you'll use it in the next step.

You can also follow the official Google documentation on creating a Pub/Sub topic using the Google Console, or using the Google Cloud CLI.

Configure the Pub/Sub channel topic

Next, configure the Pub/Sub topic so Nylas can publish notification messages to it. For these steps, use the Google Cloud CLI.

First log in to the CLI tool.
```
gcloud auth login   
```

Select the project you created the Pub/Sub topic in.

gcloud config set project <YOUR_PROJECT_NAME>

Next, add the Nylas service account as a publisher to the topic.

gcloud pubsub topics add-iam-policy-binding <YOUR_TOPIC_NAME> --member=serviceAccount:nylas-datapublisher@nylas-gma-mt.iam.gserviceaccount.com --role=roles/pubsub.publisher

Create a Pub/Sub notification channel for the topic

Next, connect the Pub/Sub queue to your application. You can do this from the Nylas v3 Dashboard by navigating to the Notifications page and clicking Create Pub/Sub Channel, or by making a POST /v3/channels/pubsub request.

This creates a notification channel and sets up the destination where GCP can send the Pub/Sub messages. This request is also where you select the notification triggers you want to this Pub/Sub channel to subscribe to.

curl --request POST \
  --url 'https://api.us.nylas.com/v3/channels/pubsub' \
  --header 'Content-Type: application/json' \
  --header 'Accept: application/json, application/gzip' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>' \
  --data-raw '{
    "description": "Pub Sub Test",
    "trigger_types": [
      "message.send_success"
    ],
    "encryption_key": "",
    "topic": "projects/<YOUR_PROJECT_NAME>/topics/<YOUR_TOPIC_NAME>",
    "notification_email_addresses": [
      "spencer@example.com"
    ]
  }'