# Using Nylas Notetaker Source: https://developer.nylas.com/docs/v3/notetaker/ Nylas Notetaker is a real-time meeting bot that you can invite to online meetings to record and transcribe your discussions. > **Info:** > **New to Notetaker?** Start with the [Notetaker API quickstart](/docs/v3/getting-started/notetaker/) to send your first bot to a meeting in under 5 minutes. ## How Notetaker works When you invite Notetaker to a meeting, it joins the session as a user and records your discussion. It then transcribes your meeting and sends status updates to you using webhook notifications. > **Info:** > **Notetaker sends a message to attendees using the meeting provider's messaging function a few minutes after joining the call**. This message informs users that Notetaker is recording and transcribing the meeting, and that it's the meeting host's responsibility to collect consent. Notetaker can join meetings in four ways: - **On the fly**: Make a [`POST /v3/notetakers`](/docs/reference/api/standalone-notetaker/invite-standalone-notetaker/) or [`POST /v3/grants//notetakers`](/docs/reference/api/notetaker/invite-notetaker/) request with the link to a meeting that's already started and omit the `join_time`. - **On a schedule**: Make a [`POST /v3/notetakers`](/docs/reference/api/standalone-notetaker/invite-standalone-notetaker/) or [`POST /v3/grants//notetakers`](/docs/reference/api/notetaker/invite-notetaker/) request with a link to a scheduled meeting and its `join_time`. - **Automatically**: Use the [calendar sync feature](/docs/v3/notetaker/calendar-sync/) to let Notetaker join meetings on your calendar automatically. - **With Scheduler**: Automatically add Notetaker bots to meetings booked through Nylas Scheduler. When enabled, Notetaker automatically joins meetings created through Scheduler bookings. For more information, see [Scheduler & Notetaker Integration](/docs/v3/scheduler/scheduler-notetaker-integration/). Notetaker records the meeting until you either [remove it from the session](#remove-notetaker-from-a-meeting) or the meeting ends. Then, it processes the data it recorded. Nylas sends [`notetaker.media` webhook notifications](/docs/reference/notifications/notetaker/notetaker-media/) as each processed file becomes available for download. ## Integrate Notetaker with Scheduler You can automatically add Notetaker bots to meetings booked through Nylas Scheduler. When enabled, Notetaker automatically joins meetings created through Scheduler bookings, records the session, and generates transcripts and summaries. For detailed information about setting up this integration, see [Scheduler & Notetaker Integration](/docs/v3/scheduler/scheduler-notetaker-integration/). ## Set up Notetaker To follow along with the samples on this page, you first need to [sign up for a Nylas developer account](https://dashboard-v3.nylas.com/register?utm_source=docs&utm_medium=devrel-surfaces&utm_campaign=&utm_content=using-apis), which gets you a free Nylas application and API key. For a guided introduction, you can follow the [Getting started guide](/docs/v3/getting-started/) to set up a Nylas account and Sandbox application. When you have those, you can connect an account from a calendar provider (such as Google, Microsoft, or iCloud) and use your API key with the sample API calls on this page to access that account's data. ### Set up Notetaker notifications Nylas can send webhook notifications about Notetakers, like when they join calls and when recordings are available. You can set this up through the Nylas Dashboard or by making a [`POST /v3/webhooks` request](/docs/reference/api/webhook-notifications/post-webhook-destinations/) with your `webhook_url` and the trigger types you want to subscribe to. You can subscribe to the following Notetaker webhook triggers: - [`notetaker.created`](/docs/reference/notifications/notetaker/notetaker-created/) - [`notetaker.updated`](/docs/reference/notifications/notetaker/notetaker-updated/) - [`notetaker.meeting_state`](/docs/reference/notifications/notetaker/notetaker-meeting_state/) - [`notetaker.media`](/docs/reference/notifications/notetaker/notetaker-media/) - [`notetaker.deleted`](/docs/reference/notifications/notetaker/notetaker-deleted/) ## Invite Notetaker to a meeting > **Success:** > **Notetaker currently supports Google Meet, Microsoft Teams, and Zoom sessions**. When you're ready, invite Notetaker to a meeting by making a [`POST /v3/notetakers`](/docs/reference/api/standalone-notetaker/invite-standalone-notetaker/) or [`POST /v3/grants//notetakers`](/docs/reference/api/notetaker/invite-notetaker/) request with a link to your session. The `join_time` is an optional parameter. If you leave it blank, Notetaker joins the meeting immediately. ```bash curl --request POST \ --url "https://api.us.nylas.com/v3/grants//notetakers" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --data '{ "join_time": 1732657774, "meeting_link": "https://meet.google.com/xyz-abcd-ijk", "meeting_settings": { "action_items": true, "action_items_settings": { "custom_instructions": "Only return the 5 most important action items." }, "audio_recording": true, "leave_after_silence_seconds": 360, "summary": true, "summary_settings": { "custom_instructions": "Return this summary in the MEDPIC sales methodology." }, "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"], "fallback_language": "en" }, "video_recording": true }, "name": "Nylas Notetaker" }' ``` ```json [invite-Response (JSON)] { "request_id": "5fa64c92-e840-4357-86b9-2aa364d35b88", "data": { "id": "", "name": "Nyla's Notetaker", "join_time": 1732657774, "meeting_link": "", "meeting_provider": "Google Meet", "state": "scheduled", "meeting_settings": { "action_items": true, "action_items_settings": { "custom_instructions": "Only return the 5 most important action items." }, "audio_recording": true, "summary": true, "summary_settings": { "custom_instructions": "Return this summary in the MEDPIC sales methodology." }, "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"], "fallback_language": "en" }, "video_recording": true } } } ``` When you invite Notetaker to a meeting, Nylas sends a [`notetaker.meeting_state` webhook notification](/docs/reference/notifications/notetaker/notetaker-meeting_state/) showing that it's attempting to join. ```json {19-20} { "specversion": "1.0", "type": "notetaker.meeting_state", "source": "/nylas/notetaker", "id": "", "time": 1737500935555, "data": { "application_id": "", "object": { "id": "", "grant_id": "", "meeting_settings": { "video_recording": true, "audio_recording": true, "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"], "fallback_language": "en" }, "summary": true, "summary_settings": { "custom_instructions": "Focus on action items related to the product launch." }, "action_items": true, "action_items_settings": { "custom_instructions": "Group action items by team member." }, "leave_after_silence_seconds": 300 }, "meeting_provider": "Google Meet", "meeting_link": "https://meet.google.com/abc-defg-hij", "join_time": 1737500936450, "event": { "ical_uid": "", "event_id": "", "master_event_id": "" }, "object": "notetaker", "status": "connecting", "state": "connecting", "meeting_state": "waiting_for_entry" } } } ``` Notetaker is always considered a non-signed-in user on the meeting platform. If your meeting is limited to organization members only, you need to approve Notetaker when it tries to join. If you don't approve its join request within 10 minutes of the scheduled join time, it times out and sends a [`notetaker.meeting_state` webhook notification](/docs/reference/notifications/notetaker/notetaker-meeting_state/) with the `status` set to `failed_entry`. Nylas doesn't de-duplicate Notetaker bots. Every [`POST /v3/notetakers`](/docs/reference/api/standalone-notetaker/invite-standalone-notetaker/) or [`POST /v3/grants//notetakers`](/docs/reference/api/notetaker/invite-notetaker/) request you make invites a new Notetaker to the specified meeting. ## Enable summaries and action items When you [invite a Notetaker bot to a meeting](#invite-notetaker-to-a-meeting) or [update a scheduled Notetaker](/docs/reference/api/notetaker/update-notetaker/), you can also enable summaries and action items for the meeting by setting `summary` and `action_items` to `true`. Nylas automatically generates a short summary of the meeting and a list of action items based on your conversation. If you want to pass custom instructions for either the summary or list of action items, you can specify `summary_settings.custom_instructions` and `action_items_settings.custom_instructions` in your request. Nylas' AI model takes these instructions into consideration while generating the information. Nylas returns URLs for files that contain the summary and action items. For information on downloading those files, see [Handling Notetaker media files](/docs/v3/notetaker/media-handling/). ## Set transcription languages By default, Notetaker uses automatic language detection on every transcription. If your users consistently speak one or more languages and you've seen Notetaker mis-identify them — for example, Portuguese coming back as Spanish, or a single-language meeting bouncing between codes — you can pass language hints in `meeting_settings.transcription_settings`. The hints either **force a single language** (pass one code in `expected_languages`) or **narrow what auto-detect considers** (pass two or more). The recognizer then chooses from the codes you provide instead of guessing across every supported language. This is a language *declaration* feature, not translation. Notetaker doesn't translate transcripts or force code switching — it just constrains the candidate set. `transcription` must be set to `true` for `transcription_settings` to take effect. You can set `transcription_settings` anywhere `meeting_settings` is accepted: - When you [invite](/docs/reference/api/notetaker/invite-notetaker/) or [update](/docs/reference/api/notetaker/update-notetaker/) a Notetaker (grant-based or [standalone](/docs/reference/api/standalone-notetaker/invite-standalone-notetaker/)). - When you [enable Notetaker on a calendar](/docs/v3/notetaker/calendar-sync/) so all matching events inherit the setting. - When you set Notetaker on a specific event, which overrides anything inherited from the calendar. ### `expected_languages` and `fallback_language` `transcription_settings` has two fields: - **`expected_languages`** — An array of language codes the audio is expected to contain. Required when `transcription_settings` is non-empty. Must contain at least one [supported code](#supported-language-codes), and cannot be `null` or empty. - **`fallback_language`** — Optional. The language Notetaker uses if it cannot confidently detect one of the `expected_languages`. Must be either `auto` or one of the codes already in `expected_languages`. The simplest valid object just declares the expected languages: ```json { "meeting_settings": { "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"] } } } ``` When `fallback_language` is omitted, Notetaker chooses the fallback from `expected_languages` based on the highest detection confidence score. Setting `fallback_language: "auto"` produces the same behavior, so it is rarely necessary to pass it explicitly. Nylas does not normalize an omitted `fallback_language` to `auto` in stored settings or responses — the field simply stays absent. If you want a deterministic fallback, set `fallback_language` to a concrete code that is also in `expected_languages`: ```json { "meeting_settings": { "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"], "fallback_language": "en" } } } ``` ### `transcription_settings` is replaced as a whole `transcription_settings` is treated as one composite setting. Nylas does not field-merge it. To change anything about the language hints — adding a language, swapping the fallback, removing a code — send the full object you want to end up with. You cannot, for example, override only `fallback_language` while inheriting `expected_languages` from a parent calendar; you must send both fields together (or omit the object entirely to inherit it unchanged). ### Clear language hints To clear `transcription_settings` — either to remove hints already set on a Notetaker, or to override hints inherited from a calendar on a single event — send `transcription_settings` as either `null` or `{}`. The two are equivalent: both remove inherited language hints and return the Notetaker to normal automatic language detection. ```json { "meeting_settings": { "transcription_settings": null } } ``` ### Supported language codes These codes are valid in `expected_languages`. The same set, plus `auto`, is valid for `fallback_language`. | Code | Language | Code | Language | Code | Language | Code | Language | | ---- | -------- | ---- | -------- | ---- | -------- | ---- | -------- | | `af` | Afrikaans | `am` | Amharic | `ar` | Arabic | `as` | Assamese | | `az` | Azerbaijani | `ba` | Bashkir | `be` | Belarusian | `bg` | Bulgarian | | `bn` | Bengali | `bo` | Tibetan | `br` | Breton | `bs` | Bosnian | | `ca` | Catalan | `cs` | Czech | `cy` | Welsh | `da` | Danish | | `de` | German | `el` | Greek | `en` | English | `en_au` | English (Australian) | | `en_uk` | English (British) | `en_us` | English (American) | `es` | Spanish | `et` | Estonian | | `eu` | Basque | `fa` | Persian | `fi` | Finnish | `fo` | Faroese | | `fr` | French | `gl` | Galician | `gu` | Gujarati | `ha` | Hausa | | `haw` | Hawaiian | `he` | Hebrew | `hi` | Hindi | `hr` | Croatian | | `ht` | Haitian Creole | `hu` | Hungarian | `hy` | Armenian | `id` | Indonesian | | `is` | Icelandic | `it` | Italian | `ja` | Japanese | `jw` | Javanese | | `ka` | Georgian | `kk` | Kazakh | `km` | Khmer | `kn` | Kannada | | `ko` | Korean | `la` | Latin | `lb` | Luxembourgish | `ln` | Lingala | | `lo` | Lao | `lt` | Lithuanian | `lv` | Latvian | `mg` | Malagasy | | `mi` | Maori | `mk` | Macedonian | `ml` | Malayalam | `mn` | Mongolian | | `mr` | Marathi | `ms` | Malay | `mt` | Maltese | `my` | Burmese | | `ne` | Nepali | `nl` | Dutch | `nn` | Norwegian Nynorsk | `no` | Norwegian | | `oc` | Occitan | `pa` | Punjabi | `pl` | Polish | `ps` | Pashto | | `pt` | Portuguese | `ro` | Romanian | `ru` | Russian | `sa` | Sanskrit | | `sd` | Sindhi | `si` | Sinhala | `sk` | Slovak | `sl` | Slovenian | | `sn` | Shona | `so` | Somali | `sq` | Albanian | `sr` | Serbian | | `su` | Sundanese | `sv` | Swedish | `sw` | Swahili | `ta` | Tamil | | `te` | Telugu | `tg` | Tajik | `th` | Thai | `tk` | Turkmen | | `tl` | Tagalog | `tr` | Turkish | `tt` | Tatar | `uk` | Ukrainian | | `ur` | Urdu | `uz` | Uzbek | `vi` | Vietnamese | `yi` | Yiddish | | `yo` | Yoruba | `zh` | Chinese | | | | | `auto` is only valid for `fallback_language`. It is not a valid entry in `expected_languages`. ## Get a list of scheduled Notetakers You can make a [`GET /v3/notetakers`](/docs/reference/api/standalone-notetaker/get-all-standalone-notetakers/) or [`GET /v3/grants//notetakers`](/docs/reference/api/notetaker/get-all-notetakers/) request to get a list of scheduled Notetaker bots. ```bash curl --request GET \ --url "https://api.us.nylas.com/v3/grants//notetakers" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` ## Cancel a scheduled Notetaker If you no longer need Notetaker in an upcoming meeting, you can make a [`DELETE /v3/notetakers//cancel`](/docs/reference/api/standalone-notetaker/cancel-standalone-notetaker/) or [`DELETE /v3/grants//notetakers//cancel`](/docs/reference/api/notetaker/cancel-notetaker/) request to cancel the scheduled Notetaker bot. ```bash curl --request DELETE \ --url "https://api.us.nylas.com/v3/grants//notetakers//cancel" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` ## Delete a Notetaker To permanently delete a Notetaker in any state, make a [`DELETE /v3/notetakers/`](/docs/reference/api/standalone-notetaker/delete-standalone-notetaker/) or [`DELETE /v3/grants//notetakers/`](/docs/reference/api/notetaker/delete-notetaker/) request. Unlike the [cancel endpoint](#cancel-a-scheduled-notetaker), which only works for Notetakers in the `scheduled` state, the delete endpoint works regardless of the Notetaker's current state. > **Warn:** > **Deleting a Notetaker is irreversible.** This is a hard delete that permanently removes the Notetaker and all associated media (recordings, transcripts, thumbnails, summaries, and action items) from Nylas systems. Once deleted, the data cannot be recovered. Make sure you have [downloaded any media files](/docs/v3/notetaker/media-handling/#download-and-store-notetaker-media-files) you need before deleting a Notetaker. ```bash curl --request DELETE \ --url "https://api.us.nylas.com/v3/notetakers/" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` When a Notetaker is deleted, Nylas sends a [`notetaker.deleted` webhook notification](/docs/reference/notifications/notetaker/notetaker-deleted/). ## Remove Notetaker from a meeting Notetaker continues recording your meeting until you either remove it from the session, the meeting ends, or it automatically leaves due to silence detection. If you want to stop recording your meeting before it ends, you can make a [`POST /v3/notetakers//leave`](/docs/reference/api/standalone-notetaker/post-standalone-notetaker-leave/) or [`POST /v3/grants//notetakers//leave`](/docs/reference/api/notetaker/post-notetaker-leave/) request to remove Notetaker from your session. ```bash curl --request POST \ --url "https://api.us.nylas.com/v3/grants//notetakers//leave" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` Nylas sends a [`notetaker.meeting_state` webhook notification](/docs/reference/notifications/notetaker/notetaker-meeting_state/) when Notetaker is removed from a meeting. ```json {20-22} { "specversion": "1.0", "type": "notetaker.meeting_state", "source": "/nylas/notetaker", "id": "", "time": 1737500935555, "webhook_delivery_attempt": 0, "data": { "application_id": "", "object": { "id": "", "grant_id": "", "calendar_id": "", "event": { "ical_uid": "", "event_id": "", "master_event_id": "" }, "object": "notetaker", "status": "disconnected", "state": "disconnected", "meeting_state": "api_request" } } } ``` ## Silence detection By default, Notetaker automatically leaves a meeting after 5 minutes (300 seconds) of continuous silence. This helps end recordings when meetings have concluded but participants haven't disconnected the call. You can customize the silence detection threshold using the `leave_after_silence_seconds` field in `meeting_settings`. The value must be between 10 and 3600 seconds (1 hour). Set a lower value for shorter meetings, or increase it for meetings with expected long pauses. ## Troubleshoot Notetaker using history events The Notetaker history endpoints give you a complete, ordered timeline of everything that happened to a specific bot so you can see the journey it's taken. - **Grant-based Notetakers** - [`GET /v3/grants//notetakers//history`](/docs/reference/api/notetaker/get-notetaker-history/) - **Standalone Notetakers** - [`GET /v3/notetakers//history`](/docs/reference/api/standalone-notetaker/get-standalone-notetaker-history/) ### Call the history endpoint Use your Nylas API key and the Notetaker ID from an API response or webhook notification. ```bash [Grant-based Notetaker history] curl --request GET \ --url "https://api.us.nylas.com/v3/grants//notetakers//history" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` ```bash [Standalone Notetaker history] curl --request GET \ --url "https://api.us.nylas.com/v3/notetakers//history" \ --header 'Accept: application/json' \ --header 'Authorization: Bearer ' ``` ```json [Sample history response] { "request_id": "abc-123-def", "data": { "events": [ { "created_at": 1700000300, "event_type": "notetaker.media", "data": { "grant_id": "d4e78fca-2a90-4b6e-91c3-a7f2bcb0d498", "id": "71c807752c744ad0902f64d43e6cc399", "meeting_link": "https://meet.google.com/abc-def-ghi", "meeting_provider": "Google Meet", "join_time": 1700000050, "object": "notetaker", "state": "available", "status": "available", "meeting_settings": { "audio_recording": true, "video_recording": true, "transcription": true, "transcription_settings": { "expected_languages": ["en", "es"], "fallback_language": "en" }, "summary": true, "action_items": true }, "media": { "recording": "https://storage.googleapis.com/nylas-notetaker-uc1-prod-notetaker/recording.mp4", "recording_duration": "3600", "transcript": "https://storage.googleapis.com/nylas-notetaker-uc1-prod-notetaker/transcript.json", "thumbnail": "https://storage.googleapis.com/nylas-notetaker-uc1-prod-notetaker/thumbnail.jpg", "summary": "https://storage.googleapis.com/nylas-notetaker-uc1-prod-notetaker/summary.txt", "action_items": "https://storage.googleapis.com/nylas-notetaker-uc1-prod-notetaker/action_items.json" } } }, { "created_at": 1700000200, "event_type": "notetaker.meeting_state", "data": { "grant_id": "d4e78fca-2a90-4b6e-91c3-a7f2bcb0d498", "id": "71c807752c744ad0902f64d43e6cc399", "meeting_link": "https://meet.google.com/abc-def-ghi", "meeting_provider": "Google Meet", "join_time": 1700000050, "object": "notetaker", "state": "disconnected", "status": "disconnected", "meeting_state": "meeting_ended" } }, { "created_at": 1700000000, "event_type": "notetaker.created", "data": { "grant_id": "d4e78fca-2a90-4b6e-91c3-a7f2bcb0d498", "id": "71c807752c744ad0902f64d43e6cc399", "meeting_link": "https://meet.google.com/abc-def-ghi", "meeting_provider": "Google Meet", "join_time": 1700000050, "object": "notetaker", "state": "scheduled", "status": "scheduled" } } ] } } ``` The `data.events` array is ordered **most recent first**. Each item represents a snapshot of the Notetaker bot at a specific point in time, with: - **`created_at`**: When the event was recorded (Unix timestamp, in seconds). - **`event_type`**: The kind of change that occurred (`notetaker.created`, `notetaker.updated`, `notetaker.meeting_state`, `notetaker.media`, or `notetaker.deleted`). - **`data`**: The Notetaker payload at that moment, including fields such as `state`, `meeting_state`, and (for media events) a `media` object. ### Use history events to debug common issues - **Notetaker never joined the meeting** - Look for a `notetaker.created` event to confirm the bot was scheduled. - Check later `notetaker.meeting_state` events: - Repeated `connecting` or a final `failed_entry` value usually indicates a join or lobby issue on the meeting provider side. - If there are no `notetaker.meeting_state` events at all, verify that the meeting link is valid and the `join_time` is correct. - **Notetaker left the meeting earlier than expected** - Find the last `notetaker.meeting_state` event where `data.meeting_state` might be `meeting_ended`, `kicked`, or `api_request`. - Compare the `created_at` timestamp to your expected meeting duration to see whether the call ended early or the bot was removed. - **Media or transcripts never arrived** - Confirm that there is at least one `notetaker.meeting_state` event showing the bot in an `attending` state with `meeting_state` set to `recording_active`. - Look for a `notetaker.media` event: - If present, inspect the `data.media` object for links to `recording`, `transcript`, `summary`, and `action_items`, and check whether your project downloaded them. - If there is no `notetaker.media` event, the recording or processing likely failed; you can share the full history payload with Nylas Support for further investigation. - **Configuration or scheduling changed unexpectedly** - Review `notetaker.updated` events to see how fields such as `join_time`, `meeting_link`, or `meeting_settings` changed over time. - Because events are most recent first, you can step backwards through the array to reconstruct exactly how the Notetaker configuration evolved.