Only show these results:

Data Syncing

When an account is connected to Nylas, the sync engine starts pulling in all mail for the account, prioritizing recent messages first (though mail isn’t strictly added in reverse chronological order). As new email arrives in the user’s mailbox, it is also added in parallel. Click to see an overview of How Nylas Works.

For contacts and calendar syncing, events, and contacts are downloaded from newest to oldest. New events or contacts added after the start of the sync are not downloaded until the initial sync is complete. Since the volume of data for contacts and calendar is typically not large, this isn’t generally a problem.

Nylas continues to keep its data up-to-date with the backend provider through a number of mechanisms: keeping an IMAP IDLE connection open, using Exchange ActiveSync’s “ping” notifications, using webhooks from calendar providers, or, in the case that no better mechanism is available, simply polling the provider.

Syncing Strategies

There are two main ways to pull in email information from an account:

  • The Threads and Messages endpoints allows you retrieve all messages and threads, or filter for a certain subset of them based on various constraints.
  • Deltas allow you to process new data quickly without having to fetch an index of the user’s mailbox or perform a large number of API calls.

You shouldn’t make any assumptions about the sync progress of an account. We recommend using both mechanisms to ensure you’re able to quickly provide data to a user when the account starts syncing (through the threads endpoint), and to quickly update the user when new mail comes through (through deltas). There are other caveats you should keep in mind that are discussed below.

Syncing Transactional Email Providers

When syncing transactional emails, such as Sendgrid or Mailgun, Nylas saves a copy on the IMAP server. This means your customer will also see the emails appear in their email provider account.

Threads and Messages

In some cases, it’s easiest to request data from the Nylas API as it’s needed for display, without ever needing to store it locally. The Threads and Messages endpoints are great for this use case.

For example, if your application provides a custom “Inbox” view, you could fetch data from the /threads endpoint and use it to render HTML or provide data to a single page JavaScript application. Making /threads requests on-demand rather than caching the data in a local database makes it easier to ensure the data is always current, and it’s easy to filter messages too.

When you first connect an account and fetch data from the /threads endpoint you won’t receive all of a user’s email if Nylas is still syncing historical mail. You can’t make any assumptions about whether an account has finished syncing on Nylas’ servers, and it can take anywhere from a few minutes to several days depending on the account size.

Deltas

The Nylas Sync Engine builds a transaction log that records every change as it synchronizes users’ mailboxes. Your application can use this transaction log, exposed through the Delta APIs, to build email applications that process incoming changes quickly, without fetching an index of the user’s mailbox, polling, or making a large number of API requests.

The Delta API documentation describes the endpoint, request parameters, and response objects in detail. It also describes the difference between streaming and long polling and when you would typically use each strategy.

If your application wants to ingest emails as they are processed by the Nylas Sync Engine, the delta stream is an efficient solution.

Deltas and Threads

By themselves, the Threads and Deltas APIs aren’t sufficient to build a full, realtime view of a user’s mailbox, since:

  • Threads only return mail that has already been processed by Nylas. If a user has just connected their account, a request to /threads might only return a few Thread objects that Nylas has processed. A few hours later, /threads might return many thousand objects since more mail has been processed.
  • Deltas only return mail as it’s being processed by Nylas. Listening to deltas tells your application that mail is available which was was not previously synced or has been modified.

To build a robust application that maintains an up-to-date cache of a user’s entire mailbox, you should leverage both Threads and Deltas. Be sure to:

  • Begin listening to deltas (changes to the mailbox) at the same time you start paginating /threads (existing data in the mailbox.)
  • Always “upsert” when you retrieve objects. A delta about a thread should create or update that thread in your cache.

If your application is only interested in caching a subset of mail—for example, mail in the last month, or mail in a particular folder—you should:

  • Limit pagination using threads filter parameters
  • Listen for deltas, but ignore deltas about mail you do not care about.

In general, you should obtain the cursor first before paginating through /threads to ensure you don’t miss any emails.

  1. Authorize an account
  2. Obtain a cursor
  3. Start listening for deltas
  4. Paginate /threads (remember to account for duplicates)

Managing Multiple Account Syncs

The Delta API documentation goes into detail on when to use long polling versus streaming to call the delta endpoint. Depending on what strategy you use and your software stack, the number of parallel processes or open connections when syncing a large number of accounts may be a concern.

In our experience, clients have been successful using the Delta API with large numbers of accounts using the following asynchronous frameworks:

Using Deltas

The delta sync API allows fetching all the changes that occurred after a specific time. Review our deltas documentation for more details about the API.

Ruby SDK

Ruby SDK

    # Get an API cursor. Cursors are API objects identifying an individual change.
# The latest cursor is the id of the latest change which was applied
# to an API object (e.g: a message got read, an event got created, etc.)
cursor = nylas.latest_cursor

last_cursor = nil
nylas.deltas(cursor) do |event, object|
if event == "create" or event == "modify"
if object.is_a?(Nylas::Contact)
puts "#{object.name} - #{object.email}"
elsif object.is_a?(Nylas::Event)
puts "Event!"
end
elsif event == "delete"
# In the case of a deletion, the API only returns the ID of the object.
# In this case, the Ruby SDK returns a dummy object with only the id field
# set.
puts "Deleting from collection #{object.class.name}, id: #{object}"
end
last_cursor = object.cursor
end

# Don't forget to save the last cursor so that we can pick up changes
# from where we left.
save_to_db(last_cursor)

The streaming API will receive deltas in real-time, without needing to repeatedly poll.

MRI

MRI uses EventMachine for async IO.

    cursor = nylas.latest_cursor

EventMachine.run do
nylas.delta_stream(cursor) do |event, object|
if event == "create" or event == "modify"
if object.is_a?(Nylas::Contact)
puts "#{object.name} - #{object.email}"
elsif object.is_a?(Nylas::Event)
puts "Event!"
end
elsif event == "delete"
# In the case of a deletion, the API only returns the ID of the object.
# In this case, the Ruby SDK returns a dummy object with only the id field
# set.
puts "Deleting from collection #{object.class.name}, id: #{object}"
end
end
end

To receive streams from multiple accounts, call delta_stream for each of them inside an EventMachine.run block.

    api_handles = [] # a list of Nylas::API objects

EventMachine.run do
api_handles.each do |a|
cursor = a.latest_cursor()
a.delta_stream(cursor) do |event, object|
puts object
end
end
end

JRuby

The JRuby implementation uses the Simple JSON Streaming gem instead of EventMachine and YAJL. No need for the EventMachine.run block.

    cursor = nylas.latest_cursor

nylas.delta_stream(cursor) do |event, object|
if event == "create" or event == "modify"
if object.is_a?(Nylas::Contact)
puts "#{object.name} - #{object.email}"
elsif object.is_a?(Nylas::Event)
puts "Event!"
end
elsif event == "delete"
# In the case of a deletion, the API only returns the ID of the object.
# In this case, the Ruby SDK returns a dummy object with only the id field
# set.
puts "Deleting from collection #{object.class.name}, id: #{object}"
end
end
```

To receive streams from multiple accounts, call `delta_stream` for each of them inside an `EventMachine`.run` block.

```ruby
api_handles = [] # a list of Nylas::API objects

api_handles.each do |a|
cursor = a.latest_cursor()
a.delta_stream(cursor) do |event, object|
puts object
end
end

Exclude Changes from a Specific Type

Only return messages

    nylas.deltas(cursor, exclude=[Nylas::Contact,
Nylas::Event,
Nylas::File,
Nylas::Tag,
Nylas::Thread]) do |event, object|
if ['create', 'modify'].include? event
puts object.subject
end
end

Expand Messages from the Delta stream

It’s possible to ask the Deltas and delta stream API to return expanded messages directly:

    nylas.deltas(cursor, exclude=[Nylas::Contact,
Nylas::Event,
Nylas::File,
Nylas::Tag,
Nylas::Thread], expanded_view=true) do |event, object|
if ['create', 'modify'].include? event
if obj.is_a?(Nylas::Message)
puts obj.subject
puts obj.message_id
end
end
end

Using Deltas Node

Node SDK Example

    var DELTA_EXCLUDE_TYPES = ['contact', 'calendar', 'event', 'file', 'tag'];
var nylas = Nylas.with(accessToken);

nylas.deltas.latestCursor(function(error, cursor) {

// Save inital cursor.
persistCursor(cursor);

// Start the stream and add event handlers.
var stream = nylas.deltas.startStream(cursor, DELTA_EXCLUDE_TYPES);

stream.on('delta', function(delta) {
// Handle the new delta here.
console.log('Received delta:', delta);
// Save new cursor so this delta doesn't need to be re-fetched for future streams.
persistCursor(delta.cursor);

}).on('error', function(err) {
// Handle errors here, such as by restarting the stream at the last cursor.
console.error('Delta streaming error:', err);

});

// Closing the stream explicitly, if needed
stopButton.addEventListener('click', function() {
stream.close();
});
})

FAQ

When fetching the raw email does Nylas go query to the originating mail server or does it pull from Nylas' database/cache?

Both actually! We store raw MIME messages in an Amazon S3 bucket for 7 days. If you query within those 7 days, we'll return it from there. After the message is removed from S3, then we query the mail provider directly for the raw message to re-fetch it.