Skip to content
Skip to main content

Using Nylas Notetaker

Last updated:

Nylas Notetaker is a meeting bot that you invite to online meetings to record and transcribe the discussion. It’s a standalone Nylas product: send a bot to a meeting with a single API call and no connected account, or pair it with Nylas Connect calendar sync so booked meetings record automatically.

How Notetaker works

Section titled “How Notetaker works”

When you invite Notetaker to a meeting, it joins the session as a participant and records the discussion. After the meeting, it transcribes the recording, optionally generates a summary and action items, and reports each step through webhook notifications. Notetaker supports Google Meet, Microsoft Teams, and Zoom.

Every Notetaker endpoint comes in two forms. Grant-based endpoints under /v3/grants/<NYLAS_GRANT_ID>/notetakers tie the bot to a connected user account, which is what calendar sync and per-user webhooks are built on. Standalone endpoints under /v3/notetakers need no grant at all: you pass a meeting link and Nylas sends the bot. The request and response shapes are otherwise identical, and this page links both forms wherever they differ.

Notetaker can join meetings in four ways:

  • Immediately: Make a POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request with the link to a meeting that’s already started and omit the join_time.
  • On a schedule: Make the same request with a link to a future meeting and its join_time.
  • Automatically from a calendar: Use calendar sync to let Notetaker join meetings on a user’s calendar that match rules you set.
  • From Scheduler bookings: Turn on the Scheduler integration and Notetaker joins every meeting booked through a Scheduler configuration.

Notetaker records until you remove it from the session, the meeting ends, or silence detection triggers. It then processes what it recorded and sends a notetaker.media webhook notification as each file becomes available for download.

Set up Notetaker

Section titled “Set up Notetaker”

To follow along with the samples on this page, you first need to sign up for a Nylas developer account, which gets you a free Nylas application and API key.

For a guided introduction, you can follow the Getting started guide to set up a Nylas account and Sandbox application. When you have those, you can connect an account from a calendar provider (such as Google, Microsoft, or iCloud) and use your API key with the sample API calls on this page to access that account’s data.

Set up Notetaker notifications

Section titled “Set up Notetaker notifications”

Nylas can send webhook notifications about Notetakers, like when they join calls and when recordings are available. You can set this up through the Nylas Dashboard or by making a POST /v3/webhooks request with your webhook_url and the trigger types you want to subscribe to. There are 5 Notetaker triggers:

Invite Notetaker to a meeting

Section titled “Invite Notetaker to a meeting”

Invite Notetaker to a meeting by making a POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request with a link to your session. The join_time is optional. If you leave it out, Notetaker joins the meeting immediately; otherwise it joins at that Unix timestamp.

Everything the bot does is controlled through notetaker_settings. Pass any combination of fields and Nylas applies defaults for the ones you don’t set:

  • name: the display name attendees see.
  • recording_type: audio_video (the default, an MP4) or audio (an MP3 with no video).
  • transcription, summary, and action_items: the outputs to produce. summary and action_items need transcription. See Enable summaries and action items.
  • transcription_settings: language and keyword hints. See Transcription settings.
  • leave_after_silence_seconds: when the bot gives up on a quiet call. See Silence detection.
  • messages: what the bot posts in the meeting chat. See Custom announcements.
  • video_output: the image the bot shows as its camera feed. See Custom video output.
  • retention_time: how long Nylas keeps the media. See Retention policies.

When you invite Notetaker to a meeting, Nylas sends a notetaker.meeting_state webhook notification showing that it’s attempting to join.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "meeting_settings": {
        "video_recording": true,
        "audio_recording": true,
        "transcription": true,
        "transcription_settings": {
          "expected_languages": ["en", "es"],
          "fallback_language": "en"
        },
        "summary": true,
        "summary_settings": {
          "custom_instructions": "Focus on action items related to the product launch."
        },
        "action_items": true,
        "action_items_settings": {
          "custom_instructions": "Group action items by team member."
        },
        "leave_after_silence_seconds": 300,
        "messages": {
          "on_join": [
            {
              "type": "text",
              "content": "👋 Hey team! I'm your AI notetaker — here to capture the key points.",
              "send_after_ms": 5000
            }
          ],
          "on_participant_join": [
            {
              "type": "text",
              "content": "👋 Welcome! Recording and transcription are running.",
              "debounce_ms": 3000
            }
          ]
        }
      },
      "meeting_provider": "Google Meet",
      "meeting_link": "https://meet.google.com/abc-defg-hij",
      "join_time": 1737500936450,
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "connecting",
      "state": "connecting",
      "meeting_state": "waiting_for_entry"
    }
  }
}

Notetaker joins as a non-signed-in user, so meetings restricted to organization members need someone to admit it. If nobody does within 10 minutes of the join time, it times out with a failed_entry status. See Notetaker never joined the meeting.

By default, every invite request schedules a new Notetaker, even if another request already scheduled one for the same meeting. If you’d rather have one bot per meeting, see Notetaker deduplication.

Enable summaries and action items

Section titled “Enable summaries and action items”

When you invite a Notetaker to a meeting or update a scheduled Notetaker, set notetaker_settings.summary and notetaker_settings.action_items to true. Both require transcription to be true. Nylas generates a short summary of the meeting and a list of action items based on the conversation, and delivers each as a JSON file alongside the recording and transcript.

To steer the AI, pass custom instructions of up to 3,000 characters in notetaker_settings.summary_settings.custom_instructions and notetaker_settings.action_items_settings.custom_instructions. Nylas’s AI model takes these instructions into account when generating the output.

Nylas returns URLs for the summary and action items files in the notetaker.media webhook. For the file shapes, see Notetaker media formats; for downloading them, see Handling Notetaker media files.

Silence detection

Section titled “Silence detection”

By default, Notetaker leaves a meeting after 5 minutes (300 seconds) of continuous silence. This ends recordings when a meeting has finished but participants haven’t disconnected the call.

Change the threshold with the leave_after_silence_seconds field in notetaker_settings. The value must be between 10 and 3,600 seconds (1 hour). Set a lower value for short meetings, or raise it for meetings with expected long pauses.

Reusable defaults with Notetaker configurations

Section titled “Reusable defaults with Notetaker configurations”

A Notetaker configuration is a saved set of notetaker_settings that Nylas applies to every Notetaker automatically, so you don’t repeat the same fields on every request. You can save defaults at the application, workspace, calendar, and event level, and the most specific level wins. Configurations are opt-in; inline notetaker_settings keep working as before. See Notetaker configurations for the inheritance model and when to use each level.

Configurations also carry two policy fields that sit alongside notetaker_settings: deduplication_policy, which sends one bot per meeting instead of one per user (see Notetaker deduplication), and dispatch_disabled_until, which stops Notetakers from joining meetings until a timestamp passes (see Pause Notetaker dispatch).

Email people when a recording is ready

Section titled “Email people when a recording is ready”

Notetaker finishes processing minutes after a call ends, and nothing tells the host until they check the API. A workflow on notetaker.media sends the recording and transcript as soon as they exist, and one on notetaker.meeting_state reports a Notetaker that a waiting room kept out.

Get a list of scheduled Notetakers

Section titled “Get a list of scheduled Notetakers”

Make a GET /v3/notetakers or GET /v3/grants/<NYLAS_GRANT_ID>/notetakers request to list Notetakers. Each item includes the bot’s state, so one call tells you which Notetakers are waiting to join, currently recording, or finished.

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Cancel a Notetaker before it joins

Section titled “Cancel a Notetaker before it joins”

If you no longer need a Notetaker before it starts attending a meeting, make a DELETE /v3/notetakers/<NOTETAKER_ID>/cancel or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel request. Cancel works while the Notetaker’s status is one of scheduled, connecting, or waiting_for_entry. Once the bot is in the meeting, use the leave endpoint instead.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Delete a Notetaker

Section titled “Delete a Notetaker”

To permanently delete a Notetaker in any state, make a DELETE /v3/notetakers/<NOTETAKER_ID> or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID> request. Unlike the cancel endpoint, which only works before Notetaker joins a meeting, the delete endpoint works regardless of the Notetaker’s current state. It’s also the way to remove media before the retention window ends, which is 14 days by default.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/notetakers/<NOTETAKER_ID>" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

When a Notetaker is deleted, Nylas sends a notetaker.deleted webhook notification.

Remove Notetaker from a meeting

Section titled “Remove Notetaker from a meeting”

To stop recording before the meeting ends, make a POST /v3/notetakers/<NOTETAKER_ID>/leave or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave request. Use this request only after Notetaker is attending the meeting. If Notetaker hasn’t joined yet, use the cancel request instead. Nylas still processes whatever was recorded up to the moment the bot left, so a 40-minute meeting you stop at minute 20 produces a 20-minute recording and transcript.

curl --request POST \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Nylas sends a notetaker.meeting_state webhook notification with meeting_state set to api_request when Notetaker is removed this way.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "webhook_delivery_attempt": 0,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "calendar_id": "<CALENDAR_ID>",
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "disconnected",
      "state": "disconnected",
      "meeting_state": "api_request"
    }
  }
}

Go deeper

Section titled “Go deeper”