Skip to content
Skip to main content

Using Nylas Notetaker

Last updated:

Nylas Notetaker is a real-time meeting bot that you can invite to online meetings to record and transcribe your discussions.

How Notetaker works

Section titled “How Notetaker works”

When you invite Notetaker to a meeting, it joins the session as a user and records your discussion. It then transcribes your meeting and sends status updates to you using webhook notifications.

Notetaker can join meetings in four ways:

Notetaker records the meeting until you either remove it from the session or the meeting ends. Then, it processes the data it recorded. Nylas sends notetaker.media webhook notifications as each processed file becomes available for download.

Integrate Notetaker with Scheduler

Section titled “Integrate Notetaker with Scheduler”

You can automatically add Notetaker bots to meetings booked through Nylas Scheduler. When enabled, Notetaker automatically joins meetings created through Scheduler bookings, records the session, and generates transcripts and summaries. For detailed information about setting up this integration, see Scheduler & Notetaker Integration.

Set up Notetaker

Section titled “Set up Notetaker”

To follow along with the samples on this page, you first need to sign up for a Nylas developer account, which gets you a free Nylas application and API key.

For a guided introduction, you can follow the Getting started guide to set up a Nylas account and Sandbox application. When you have those, you can connect an account from a calendar provider (such as Google, Microsoft, or iCloud) and use your API key with the sample API calls on this page to access that account’s data.

Set up Notetaker notifications

Section titled “Set up Notetaker notifications”

Nylas can send webhook notifications about Notetakers, like when they join calls and when recordings are available. You can set this up through the Nylas Dashboard or by making a POST /v3/webhooks request with your webhook_url and the trigger types you want to subscribe to. You can subscribe to the following Notetaker webhook triggers:

Invite Notetaker to a meeting

Section titled “Invite Notetaker to a meeting”

When you’re ready, invite Notetaker to a meeting by making a POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request with a link to your session. The join_time is an optional parameter. If you leave it blank, Notetaker joins the meeting immediately.

When you invite Notetaker to a meeting, Nylas sends a notetaker.meeting_state webhook notification showing that it’s attempting to join.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "meeting_settings": {
        "video_recording": true,
        "audio_recording": true,
        "transcription": true,
        "transcription_settings": {
          "expected_languages": ["en", "es"],
          "fallback_language": "en"
        },
        "summary": true,
        "summary_settings": {
          "custom_instructions": "Focus on action items related to the product launch."
        },
        "action_items": true,
        "action_items_settings": {
          "custom_instructions": "Group action items by team member."
        },
        "leave_after_silence_seconds": 300
      },
      "meeting_provider": "Google Meet",
      "meeting_link": "https://meet.google.com/abc-defg-hij",
      "join_time": 1737500936450,
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "connecting",
      "state": "connecting",
      "meeting_state": "waiting_for_entry"
    }
  }
}

Notetaker is always considered a non-signed-in user on the meeting platform. If your meeting is limited to organization members only, you need to approve Notetaker when it tries to join. If you don’t approve its join request within 10 minutes of the scheduled join time, it times out and sends a notetaker.meeting_state webhook notification with the status set to failed_entry.

Nylas doesn’t de-duplicate Notetaker bots. Every POST /v3/notetakers or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers request you make invites a new Notetaker to the specified meeting.

Enable summaries and action items

Section titled “Enable summaries and action items”

When you invite a Notetaker bot to a meeting or update a scheduled Notetaker, you can also enable summaries and action items for the meeting by setting summary and action_items to true. Nylas automatically generates a short summary of the meeting and a list of action items based on your conversation.

If you want to pass custom instructions for either the summary or list of action items, you can specify summary_settings.custom_instructions and action_items_settings.custom_instructions in your request. Nylas’ AI model takes these instructions into consideration while generating the information.

Nylas returns URLs for files that contain the summary and action items. For information on downloading those files, see Handling Notetaker media files.

Set transcription languages

Section titled “Set transcription languages”

meeting_settings.transcription_settings tunes how Notetaker transcribes a meeting. It carries two independent controls: language hints that steer automatic language detection, and keyword hints that bias recognition toward specific terms. Set either, both, or neither. transcription must be true for any of these settings to take effect.

By default, Notetaker uses automatic language detection on every transcription. If your users consistently speak one or more languages and you’ve seen Notetaker mis-identify them (for example, Portuguese coming back as Spanish, or a single-language meeting bouncing between codes), pass language hints in transcription_settings. The hints either force a single language (pass one code in expected_languages) or narrow what auto-detect considers (pass two or more). The recognizer then chooses from the codes you provide instead of guessing across every supported language.

Language hints affect detection only. Notetaker transcribes each meeting in the language spoken and doesn’t translate the transcript into another language.

You can set transcription_settings anywhere meeting_settings is accepted:

expected_languages and fallback_language

Section titled “expected_languages and fallback_language”

The language hints in transcription_settings are two fields:

  • expected_languages — An array of language codes the audio is expected to contain. Optional. When you set it, it must contain at least one supported code and cannot be null or empty. Omit it and transcription considers all supported languages.
  • fallback_language — Optional. When expected_languages is set, the fallback must be one of those codes. When you omit expected_languages, it can be any supported code. Leave it out and the transcriber auto-detects the language. The field isn’t stored, so a GET won’t return it.

The simplest valid object just declares the expected languages:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "expected_languages": ["en", "es"]
    }
  }
}

If you omit fallback_language, the transcriber auto-detects the language. Passing "auto" explicitly does the same thing.

If you want a deterministic fallback, set fallback_language to a concrete code that is also in expected_languages:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "expected_languages": ["en", "es"],
      "fallback_language": "en"
    }
  }
}

When you omit expected_languages, transcription considers all supported languages, and fallback_language isn’t limited to a list. You can set it to any supported code:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "fallback_language": "en"
    }
  }
}

Improve recognition of names and jargon

Section titled “Improve recognition of names and jargon”

Keyword hints bias the transcriber toward domain-specific terms it would otherwise mis-hear, such as people’s names, company names, product names, and acronyms. Like language hints, they live in transcription_settings, and transcription must be true. These two fields are independent of expected_languages, so you can set them on their own.

  • keywords — An array of terms to prioritize during transcription. Up to 200 terms. Each term must be 1 to 200 characters and cannot contain control characters. Cannot be null.
  • use_speaker_names_as_keywords — A boolean. When true, Notetaker adds known speaker names to the keyword set so they’re transcribed accurately. Cannot be null.
{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "keywords": ["Nylas", "AssemblyAI", "OAuth"],
      "use_speaker_names_as_keywords": true
    }
  }
}

You can combine keyword hints with language hints in the same object:

{
  "meeting_settings": {
    "transcription": true,
    "transcription_settings": {
      "expected_languages": ["en", "es"],
      "keywords": ["Nylas", "AssemblyAI"]
    }
  }
}

transcription_settings is replaced as a whole

Section titled “transcription_settings is replaced as a whole”

transcription_settings is treated as one composite setting. Nylas does not field-merge it. To change anything inside it, whether that’s a language code, the fallback, or the keyword list, send the full object you want to end up with. You can’t, for example, add a keyword while inheriting expected_languages from a parent calendar; send every field you want together, or omit the object entirely to inherit it unchanged.

Clear transcription settings

Section titled “Clear transcription settings”

To clear transcription_settings, whether to remove settings already on a Notetaker or to override settings inherited from a calendar on a single event, send transcription_settings as either null or {}. The two are equivalent: both remove inherited language and keyword hints and return the Notetaker to default transcription behavior.

{
  "meeting_settings": {
    "transcription_settings": null
  }
}

Supported language codes

Section titled “Supported language codes”

These codes are valid in both expected_languages and fallback_language.

CodeLanguageCodeLanguageCodeLanguageCodeLanguage
afAfrikaansamAmharicarArabicasAssamese
azAzerbaijanibaBashkirbeBelarusianbgBulgarian
bnBengaliboTibetanbrBretonbsBosnian
caCatalancsCzechcyWelshdaDanish
deGermanelGreekenEnglishen_auEnglish (Australian)
en_ukEnglish (British)en_usEnglish (American)esSpanishetEstonian
euBasquefaPersianfiFinnishfoFaroese
frFrenchglGalicianguGujaratihaHausa
hawHawaiianheHebrewhiHindihrCroatian
htHaitian CreolehuHungarianhyArmenianidIndonesian
isIcelandicitItalianjaJapanesejwJavanese
kaGeorgiankkKazakhkmKhmerknKannada
koKoreanlaLatinlbLuxembourgishlnLingala
loLaoltLithuanianlvLatvianmgMalagasy
miMaorimkMacedonianmlMalayalammnMongolian
mrMarathimsMalaymtMaltesemyBurmese
neNepalinlDutchnnNorwegian NynorsknoNorwegian
ocOccitanpaPunjabiplPolishpsPashto
ptPortugueseroRomanianruRussiansaSanskrit
sdSindhisiSinhalaskSlovakslSlovenian
snShonasoSomalisqAlbaniansrSerbian
suSundanesesvSwedishswSwahilitaTamil
teTelugutgTajikthThaitkTurkmen
tlTagalogtrTurkishttTatarukUkrainian
urUrduuzUzbekviVietnameseyiYiddish
yoYorubazhChinese

auto is valid only for fallback_language, not expected_languages. Omitting fallback_language produces the same auto-detection.

Get a list of scheduled Notetakers

Section titled “Get a list of scheduled Notetakers”

You can make a GET /v3/notetakers or GET /v3/grants/<NYLAS_GRANT_ID>/notetakers request to get a list of scheduled Notetaker bots.

curl --request GET \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Cancel a Notetaker before it joins

Section titled “Cancel a Notetaker before it joins”

If you no longer need Notetaker before it starts attending a meeting, you can make a DELETE /v3/notetakers/<NOTETAKER_ID>/cancel or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel request to cancel the Notetaker bot. Use this request while the Notetaker’s status is scheduled, connecting, or waiting_for_entry.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/cancel" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Delete a Notetaker

Section titled “Delete a Notetaker”

To permanently delete a Notetaker in any state, make a DELETE /v3/notetakers/<NOTETAKER_ID> or DELETE /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID> request. Unlike the cancel endpoint, which only works before Notetaker joins a meeting, the delete endpoint works regardless of the Notetaker’s current state.

curl --request DELETE \
  --url "https://api.us.nylas.com/v3/notetakers/<NOTETAKER_ID>" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

When a Notetaker is deleted, Nylas sends a notetaker.deleted webhook notification.

Remove Notetaker from a meeting

Section titled “Remove Notetaker from a meeting”

Notetaker continues recording your meeting until you either remove it from the session, the meeting ends, or it automatically leaves due to silence detection. If you want to stop recording your meeting before it ends, you can make a POST /v3/notetakers/<NOTETAKER_ID>/leave or POST /v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave request to remove Notetaker from your session. Use this request only after Notetaker is attending the meeting. If Notetaker hasn’t joined yet, use the cancel request instead.

curl --request POST \
  --url "https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/notetakers/<NOTETAKER_ID>/leave" \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>'

Nylas sends a notetaker.meeting_state webhook notification when Notetaker is removed from a meeting.

{
  "specversion": "1.0",
  "type": "notetaker.meeting_state",
  "source": "/nylas/notetaker",
  "id": "<WEBHOOK_ID>",
  "time": 1737500935555,
  "webhook_delivery_attempt": 0,
  "data": {
    "application_id": "<NYLAS_APPLICATION_ID>",
    "object": {
      "id": "<NOTETAKER_ID>",
      "grant_id": "<NYLAS_GRANT_ID>",
      "calendar_id": "<CALENDAR_ID>",
      "event": {
        "ical_uid": "<ICAL_UID>",
        "event_id": "<EVENT_ID>",
        "master_event_id": "<MASTER_EVENT_ID>"
      },
      "object": "notetaker",
      "status": "disconnected",
      "state": "disconnected",
      "meeting_state": "api_request"
    }
  }
}

Silence detection

Section titled “Silence detection”

By default, Notetaker automatically leaves a meeting after 5 minutes (300 seconds) of continuous silence. This helps end recordings when meetings have concluded but participants haven’t disconnected the call.

You can customize the silence detection threshold using the leave_after_silence_seconds field in meeting_settings. The value must be between 10 and 3600 seconds (1 hour). Set a lower value for shorter meetings, or increase it for meetings with expected long pauses.

Troubleshoot Notetaker using history events

Section titled “Troubleshoot Notetaker using history events”

The Notetaker history endpoints give you a complete, ordered timeline of everything that happened to a specific bot so you can see the journey it’s taken.

Call the history endpoint

Section titled “Call the history endpoint”

Use your Nylas API key and the Notetaker ID from an API response or webhook notification.

The data.events array is ordered most recent first. Each item represents a snapshot of the Notetaker bot at a specific point in time, with:

  • created_at: When the event was recorded (Unix timestamp, in seconds).
  • event_type: The kind of change that occurred (notetaker.created, notetaker.updated, notetaker.meeting_state, notetaker.media, or notetaker.deleted).
  • data: The Notetaker payload at that moment, including fields such as state, meeting_state, and (for media events) a media object.

Use history events to debug common issues

Section titled “Use history events to debug common issues”

  • Notetaker never joined the meeting

    • Look for a notetaker.created event to confirm the bot was scheduled.
    • Check later notetaker.meeting_state events:
      • Repeated connecting or a final failed_entry value usually indicates a join or lobby issue on the meeting provider side.
      • If there are no notetaker.meeting_state events at all, verify that the meeting link is valid and the join_time is correct.

  • Notetaker left the meeting earlier than expected

    • Find the last notetaker.meeting_state event where data.meeting_state might be meeting_ended, kicked, or api_request.
    • Compare the created_at timestamp to your expected meeting duration to see whether the call ended early or the bot was removed.

  • Media or transcripts never arrived

    • Confirm that there is at least one notetaker.meeting_state event showing the bot in an attending state with meeting_state set to recording_active.
    • Look for a notetaker.media event:
      • If present, inspect the data.media object for links to recording, transcript, summary, and action_items, and check whether your project downloaded them.
      • If there is no notetaker.media event, the recording or processing likely failed; you can share the full history payload with Nylas Support for further investigation.

  • Configuration or scheduling changed unexpectedly

    • Review notetaker.updated events to see how fields such as join_time, meeting_link, or meeting_settings changed over time.
    • Because events are most recent first, you can step backwards through the array to reconstruct exactly how the Notetaker configuration evolved.