Use a PubSub notification channel

In Nylas v3 you can set up a PubSub data connector to receive notifications about activity and changes on the provider. Your project can then consume the notifications, and respond to power your app's logic.

Google PubSub is a event-driven queue that you can divide into several "topics" which you can use to segment event notifications. You can then subscribe to different topics to get notifications as they come in, either individually or in batches, during periods of high volume. PubSub queues also allow you to create a "dead letter" queue to gracefully handle message delivery delays. PubSub is ideal for projects where webhook volume, latency, or deliverability are concerns, or where your project requires deliverability guarantees.

You can use a PubSub connecter along with webhooks, and you can use up to five separate PubSub connectors to separate your notifications.

🔍 Good to know: Although you set up the PubSub notification channel on Google Cloud, it can receive notifications about events on any of the providers Nylas supports - not just Google.

Before you begin

Before you can start using Pub/Sub channels, you need the following prerequisites:

A v3 Nylas application.
Your Nylas application's client ID and API key.
A Google Cloud Platform (GCP) project.
- If you're not already using GCP to authenticate Google users, create a GCP project.
- If you already have a Google project for a Google auth app, you can add the Pub/Sub topic for the notifications to that project.
The Google Cloud CLI installed.

PubSub architecture

The Nylas PubSub connector stores information that allows Nylas to connect to, and send messages into a PubSub topic on GCP, which your project can consume. Each Nylas application can have up to five PubSub data connectors. (Contact Nylas support if your project requires more than five.)

You can use PubSub in as a complete replacement for webhook notifications, or in addition to existing webhook architecture. For example, you might use a PubSub connector for notification subscriptions that produce a high volume of messages, but use webhooks for lower volume notification subscriptions.

Each PubSub topic can subscribe to any number of notification triggers so you can split your notifications across separate PubSub topics, for example to have one that handles all email notifications, and separate ones specifically for new events, and event changes.

Set up to handle undeliverable notifications

After Nylas delivers the notification to your PubSub queue, it is up to your project to consume the messages.

Nylas recommends that you create a second PubSub topic for each PubSub channel to serve as a "dead letter queue". This topic allows you to collect notifications that your project is unable to consume, so they don't collect in your notification channel and cause latency issues. You can then replay these notifications at a later date after you resolve any ingestion issues.

See the official PubSub documentation on handling message failures for more information.

Create the PubSub topic

Log in to your Google Cloud console, and go to the Pub/Sub page.
From the Topics page, click Create topic.
In the form that opens, give your topic an ID, and make sure Add a default subscription is selected.

Make note of the topic name, because you'll use it in the next step.

You can also follow the official Google documentation on creating a PubSub topic using the Google Console, or using the Google Cloud CLI.

Configure the PubSub channel topic

Next, configure the PubSub topic so Nylas can publish notification messages to it. For these steps, use the Google Cloud CLI.

First log in to the CLI tool.
```
gcloud auth login   
```

Select the project you created the PubSub topic in.

gcloud config set project <YOUR_PROJECT_NAME>

Next, add the Nylas service account as a publisher to the topic.

gcloud pubsub topics add-iam-policy-binding <YOUR_TOPIC_NAME> --member=serviceAccount:[email protected]=roles/pubsub.publisher

Create a PubSub notification channel for the topic

Next, connect the PubSub queue to your application by making a POST /v3/channels/pubsub request. This creates a notification channel and sets up the destination where GCP can send the PubSub messages. This request is also where you select the notification triggers you want to this PubSub channel to subscribe to.

curl --request POST \
  --url http://api.us.nylas.com/v3/channels/pubsub \
  --header 'Content-Type: application/json' \
  --header 'Authorization: <NYPAS_API_KEY>' \
  --data '{
  "description": "Pub Sub Dest Test",
  "trigger_types": [
    "message.send_success"
  ],
  "encryption_key": "", /
  "topic": "<YOUR_TOPIC_NAME>",
  "notification_email_addresses": [
    "[email protected]"
  ]
}'

Updating a v3 Google webhook PubSub topic to a general Pubsub channel

If you receive webhook notifications in Nylas v3 about email activity from Google providers, you probably already have a PubSub topic to enable realtime webhook delivery. If you want to use a PubSub channel for email notifications instead, you can completely reconfigure and reregister your existing webhook-relay PubSub topic - however Nylas recommends that you create a new PubSub topic for clarity.

You should not maintain both a realtime-email webhook PubSub topic and a PubSub channel for email triggers because you'll get all Google email notifications twice, doubling your billable message traffic.

Good to know: A PubSub channel in Nylas receives email notifications from all email providers, not just Google. If you are replacing your Google email webhook PubSub with a PubSub channel, you can remove the webhook destinations you used for other email providers, and consolidate your email notifications into the channel.

To switch to using a PubSub channel:

Create a new PubSub topic and configure it as described above.
Create a new Nylas PubSub channel for your application and confirm that it works.
Subscribe your channel to the email notification triggers you want to receive.
Disable and optionally delete the webhook subscription for email triggers, and remove the webhook PubSub topic.