Skip to content
Skip to main content

Using Nylas Notetaker

Nylas Notetaker is a real-time meeting bot that you can invite to online meetings to record and transcribe your discussions.

How Notetaker works

Section titled “How Notetaker works”

When you invite Notetaker to a meeting, it joins the session as a user and records your discussion. It then transcribes your meeting and sends status updates to you using webhook notifications.

Notetaker can join meetings in four ways:

Notetaker records the meeting until you either remove it from the session or the meeting ends. Then, it processes the data it recorded. Nylas sends notetaker.media webhook notifications as each processed file becomes available for download.

Integrate Notetaker with Scheduler

Section titled “Integrate Notetaker with Scheduler”

You can automatically add Notetaker bots to meetings booked through Nylas Scheduler. When enabled, Notetaker automatically joins meetings created through Scheduler bookings, records the session, and generates transcripts and summaries. For detailed information about setting up this integration, see Scheduler & Notetaker Integration.

Set up Notetaker

Section titled “Set up Notetaker”

To follow along with the samples on this page, you first need to sign up for a Nylas developer account, which gets you a free Nylas application and API key.

For a guided introduction, you can follow the Getting started guide to set up a Nylas account and Sandbox application. When you have those, you can connect an account from a calendar provider (such as Google, Microsoft, or iCloud) and use your API key with the sample API calls on this page to access that account’s data.

Set up Notetaker notifications

Section titled “Set up Notetaker notifications”

Nylas can send webhook notifications about Notetakers, like when they join calls and when recordings are available. You can set this up through the Nylas Dashboard or by making a POST /v3/webhooks request with your webhook_url and the trigger types you want to subscribe to. You can subscribe to the following Notetaker webhook triggers:

Invite Notetaker to a meeting

Section titled “Invite Notetaker to a meeting”

When you’re ready, invite Notetaker to a meeting by making a POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request with a link to your session. The join_time is an optional parameter. If you leave it blank, Notetaker joins the meeting immediately.

When you invite Notetaker to a meeting, Nylas sends a notetaker.meeting_state webhook notification showing that it’s attempting to join.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "meeting_settings": {
        "video_recording": true,
        "audio_recording": true,
        "transcription": true,
        "transcription_settings": {
          "expected_languages": ["en", "es"],
          "fallback_language": "en"
        },
        "summary": true,
        "summary_settings": {
          "custom_instructions": "Focus on action items related to the product launch."
        },
        "action_items": true,
        "action_items_settings": {
          "custom_instructions": "Group action items by team member."
        },
        "leave_after_silence_seconds": 300
      },
      "meeting_provider": "Google Meet",
      "meeting_link": "https://meet.google.com/abc-defg-hij",
      "join_time": 1737500936450,
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "connecting",
      "state": "connecting",
      "meeting_state": "waiting_for_entry"
    }
  }
}

Notetaker is always considered a non-signed-in user on the meeting platform. If your meeting is limited to organization members only, you need to approve Notetaker when it tries to join. If you don’t approve its join request within 10 minutes of the scheduled join time, it times out and sends a notetaker.meeting_state webhook notification with the status set to failed_entry.

Nylas doesn’t de-duplicate Notetaker bots. Every POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request you make invites a new Notetaker to the specified meeting.

Enable summaries and action items

Section titled “Enable summaries and action items”

When you invite a Notetaker bot to a meeting or update a scheduled Notetaker, you can also enable summaries and action items for the meeting by setting summary and action_items to true. Nylas automatically generates a short summary of the meeting and a list of action items based on your conversation.

If you want to pass custom instructions for either the summary or list of action items, you can specify summary_settings.custom_instructions and action_items_settings.custom_instructions in your request. Nylas’ AI model takes these instructions into consideration while generating the information.

Nylas returns URLs for files that contain the summary and action items. For information on downloading those files, see Handling Notetaker media files.

Set transcription languages

Section titled “Set transcription languages”

By default, Notetaker uses automatic language detection on every transcription. If your users consistently speak one or more languages and you’ve seen Notetaker mis-identify them — for example, Portuguese coming back as Spanish, or a single-language meeting bouncing between codes — you can pass language hints in meeting_settings.transcription_settings. The hints either force a single language (pass one code in expected_languages) or narrow what auto-detect considers (pass two or more). The recognizer then chooses from the codes you provide instead of guessing across every supported language.

This is a language declaration feature, not translation. Notetaker doesn’t translate transcripts or force code switching — it just constrains the candidate set.

transcription must be set to true for transcription_settings to take effect.

You can set transcription_settings anywhere meeting_settings is accepted:

expected_languages and fallback_language

Section titled “expected_languages and fallback_language”

transcription_settings has two fields:

  • expected_languages — An array of language codes the audio is expected to contain. Required when transcription_settings is non-empty. Must contain at least one supported code, and cannot be null or empty.
  • fallback_language — Optional. The language Notetaker uses if it cannot confidently detect one of the expected_languages. Must be either auto or one of the codes already in expected_languages.

The simplest valid object just declares the expected languages:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "expected_languages": ["en", "es"]
    }
  }
}

When fallback_language is omitted, Notetaker chooses the fallback from expected_languages based on the highest detection confidence score. Setting fallback_language: "auto" produces the same behavior, so it is rarely necessary to pass it explicitly. Nylas does not normalize an omitted fallback_language to auto in stored settings or responses — the field simply stays absent.

If you want a deterministic fallback, set fallback_language to a concrete code that is also in expected_languages:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "expected_languages": ["en", "es"],
      "fallback_language": "en"
    }
  }
}

transcription_settings is replaced as a whole

Section titled “transcription_settings is replaced as a whole”

transcription_settings is treated as one composite setting. Nylas does not field-merge it. To change anything about the language hints — adding a language, swapping the fallback, removing a code — send the full object you want to end up with. You cannot, for example, override only fallback_language while inheriting expected_languages from a parent calendar; you must send both fields together (or omit the object entirely to inherit it unchanged).

Clear language hints

Section titled “Clear language hints”

To clear transcription_settings — either to remove hints already set on a Notetaker, or to override hints inherited from a calendar on a single event — send transcription_settings as either null or {}. The two are equivalent: both remove inherited language hints and return the Notetaker to normal automatic language detection.

{
  "meeting_settings": {
    "transcription_settings": null
  }
}

Supported language codes

Section titled “Supported language codes”

These codes are valid in expected_languages. The same set, plus auto, is valid for fallback_language.

CodeLanguageCodeLanguageCodeLanguageCodeLanguage
afAfrikaansamAmharicarArabicasAssamese
azAzerbaijanibaBashkirbeBelarusianbgBulgarian
bnBengaliboTibetanbrBretonbsBosnian
caCatalancsCzechcyWelshdaDanish
deGermanelGreekenEnglishen_auEnglish (Australian)
en_ukEnglish (British)en_usEnglish (American)esSpanishetEstonian
euBasquefaPersianfiFinnishfoFaroese
frFrenchglGalicianguGujaratihaHausa
hawHawaiianheHebrewhiHindihrCroatian
htHaitian CreolehuHungarianhyArmenianidIndonesian
isIcelandicitItalianjaJapanesejwJavanese
kaGeorgiankkKazakhkmKhmerknKannada
koKoreanlaLatinlbLuxembourgishlnLingala
loLaoltLithuanianlvLatvianmgMalagasy
miMaorimkMacedonianmlMalayalammnMongolian
mrMarathimsMalaymtMaltesemyBurmese
neNepalinlDutchnnNorwegian NynorsknoNorwegian
ocOccitanpaPunjabiplPolishpsPashto
ptPortugueseroRomanianruRussiansaSanskrit
sdSindhisiSinhalaskSlovakslSlovenian
snShonasoSomalisqAlbaniansrSerbian
suSundanesesvSwedishswSwahilitaTamil
teTelugutgTajikthThaitkTurkmen
tlTagalogtrTurkishttTatarukUkrainian
urUrduuzUzbekviVietnameseyiYiddish
yoYorubazhChinese

auto is only valid for fallback_language. It is not a valid entry in expected_languages.

Get a list of scheduled Notetakers

Section titled “Get a list of scheduled Notetakers”

You can make a GET /v3/notetakers or GET /v3/grants/<NYLAS_GRANT_ID>/notetakers request to get a list of scheduled Notetaker bots.

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Cancel a scheduled Notetaker

Section titled “Cancel a scheduled Notetaker”

If you no longer need Notetaker in an upcoming meeting, you can make a DELETE /v3/notetakers/<NOTETAKER_ID>/cancel or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel request to cancel the scheduled Notetaker bot.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Delete a Notetaker

Section titled “Delete a Notetaker”

To permanently delete a Notetaker in any state, make a DELETE /v3/notetakers/<NOTETAKER_ID> or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID> request. Unlike the cancel endpoint, which only works for Notetakers in the scheduled state, the delete endpoint works regardless of the Notetaker’s current state.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/notetakers/<NOTETAKER_ID>" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

When a Notetaker is deleted, Nylas sends a notetaker.deleted webhook notification.

Remove Notetaker from a meeting

Section titled “Remove Notetaker from a meeting”

Notetaker continues recording your meeting until you either remove it from the session, the meeting ends, or it automatically leaves due to silence detection. If you want to stop recording your meeting before it ends, you can make a POST /v3/notetakers/<NOTETAKER_ID>/leave or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave request to remove Notetaker from your session.

curl --request POST \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Nylas sends a notetaker.meeting_state webhook notification when Notetaker is removed from a meeting.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "webhook_delivery_attempt": 0,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "calendar_id": "<CALENDAR_ID>",
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "disconnected",
      "state": "disconnected",
      "meeting_state": "api_request"
    }
  }
}

Silence detection

Section titled “Silence detection”

By default, Notetaker automatically leaves a meeting after 5 minutes (300 seconds) of continuous silence. This helps end recordings when meetings have concluded but participants haven’t disconnected the call.

You can customize the silence detection threshold using the leave_after_silence_seconds field in meeting_settings. The value must be between 10 and 3600 seconds (1 hour). Set a lower value for shorter meetings, or increase it for meetings with expected long pauses.

Troubleshoot Notetaker using history events

Section titled “Troubleshoot Notetaker using history events”

The Notetaker history endpoints give you a complete, ordered timeline of everything that happened to a specific bot so you can see the journey it’s taken.

Call the history endpoint

Section titled “Call the history endpoint”

Use your Nylas API key and the Notetaker ID from an API response or webhook notification.

The data.events array is ordered most recent first. Each item represents a snapshot of the Notetaker bot at a specific point in time, with:

  • created_at: When the event was recorded (Unix timestamp, in seconds).
  • event_type: The kind of change that occurred (notetaker.created, notetaker.updated, notetaker.meeting_state, notetaker.media, or notetaker.deleted).
  • data: The Notetaker payload at that moment, including fields such as state, meeting_state, and (for media events) a media object.

Use history events to debug common issues

Section titled “Use history events to debug common issues”

  • Notetaker never joined the meeting

    • Look for a notetaker.created event to confirm the bot was scheduled.
    • Check later notetaker.meeting_state events:
      • Repeated connecting or a final failed_entry value usually indicates a join or lobby issue on the meeting provider side.
      • If there are no notetaker.meeting_state events at all, verify that the meeting link is valid and the join_time is correct.

  • Notetaker left the meeting earlier than expected

    • Find the last notetaker.meeting_state event where data.meeting_state might be meeting_ended, kicked, or api_request.
    • Compare the created_at timestamp to your expected meeting duration to see whether the call ended early or the bot was removed.

  • Media or transcripts never arrived

    • Confirm that there is at least one notetaker.meeting_state event showing the bot in an attending state with meeting_state set to recording_active.
    • Look for a notetaker.media event:
      • If present, inspect the data.media object for links to recording, transcript, summary, and action_items, and check whether your project downloaded them.
      • If there is no notetaker.media event, the recording or processing likely failed; you can share the full history payload with Nylas Support for further investigation.

  • Configuration or scheduling changed unexpectedly

    • Review notetaker.updated events to see how fields such as join_time, meeting_link, or meeting_settings changed over time.
    • Because events are most recent first, you can step backwards through the array to reconstruct exactly how the Notetaker configuration evolved.