# Extract action items with your own LLM

Source: https://developer.nylas.com/docs/cookbook/notetaker/extract-action-items-ai/

Notetaker already gives you an `action_items` list out of the box, but it's a flat array of strings like `["Test the Notetaker implementation."]`. That reads fine in an email recap, yet it falls apart the moment you try to push tasks into Jira, Linear, or a CRM, because there's no assignee, no due date, and nothing typed. When you need structured records instead of a bulleted list, the move is to fetch the raw `transcript` and run your own LLM over it.

This recipe shows how to pull the speaker-labelled transcript, prompt an LLM to return typed action items as JSON, and validate the result before it hits a task tracker. It builds on the media fetch in [Get a meeting transcript and recording](/docs/cookbook/notetaker/get-transcript-and-recording/) and follows the same extraction pattern as [Extract structured data from email](/docs/cookbook/ai/extract-data-from-email/).

## When should you skip the built-in action items?

Use the built-in `action_items` when a plain checklist is enough, like a recap email. Reach for your own LLM when downstream systems need typed fields. A task tracker row usually wants at least 3 fields the built-in list never provides: who owns it, what the task is, and when it's due.

The built-in `summary` and `action_items` are media objects Nylas generates automatically, each delivered as a small JSON file under 1 KB. They're great for human reading and cost you nothing extra. The tradeoff is shape: a string array can't carry an assignee or a date, so a task tracker import either drops those fields or forces a human to fill them in by hand. Running your own model over the `transcript` lets you define the exact schema your tracker expects, including an owner pulled from the `speaker` labels and a due date parsed from phrases like "by next Friday." For the format of the built-in outputs, see [Handling Notetaker media files](/docs/v3/notetaker/media-handling/).

## How do you fetch the raw transcript?

Call the media endpoint, then download the `transcript` object's `url`. The transcript is a JSON file, usually `type: "speaker_labelled"`, with a `transcript` array where each entry carries a `speaker`, `text`, and `start`/`end` times in milliseconds. That speaker attribution is what lets your LLM assign owners.

Make a `GET` request to `/v3/grants/{grant_id}/notetakers/{notetaker_id}/media`, grab `data.transcript.url`, and fetch it. The signed link is valid for 3,600 seconds, so download the content right away rather than storing the URL. The media fetch itself is covered in depth in [Get a meeting transcript and recording](/docs/cookbook/notetaker/get-transcript-and-recording/); here you only need the transcript object. The snippet below pulls the transcript and flattens it into a single labelled string the model can read in one pass.

```python


NYLAS = "https://api.us.nylas.com"
HEADERS = {"Authorization": f"Bearer {os.environ['NYLAS_API_KEY']}"}

def get_transcript_text(grant_id, notetaker_id):
    media = requests.get(
        f"{NYLAS}/v3/grants/{grant_id}/notetakers/{notetaker_id}/media",
        headers=HEADERS).json()["data"]
    doc = requests.get(media["transcript"]["url"]).json()  # link valid 3,600 seconds
    if doc["type"] == "raw":
        return doc["transcript"]                            # plain string, no speakers
    return "\n".join(f'{seg["speaker"]}: {seg["text"]}' for seg in doc["transcript"])
```

## How do you prompt an LLM for typed action items?

Define a strict JSON schema, send the transcript with `temperature` set to 0 for repeatable output, and ask the model to return only items it can ground in the text. The schema below asks for 4 fields per item: an `assignee`, a `task`, a `due_date`, and a `decision` flag.

The prompt does the real work here. Give the model the exact fields you want, tell it to use `null` when a value isn't stated rather than inventing one, and constrain it to JSON so you can parse the response directly. Setting `temperature` to 0 keeps the same transcript producing the same items across runs, which matters when the output feeds an automated pipeline. The example uses an OpenAI-style client, but any chat model with JSON output works the same way. This is the identical pattern used to [extract structured data from email](/docs/cookbook/ai/extract-data-from-email/), just pointed at a transcript instead of a message body.

```python

from openai import OpenAI

client = OpenAI()

SCHEMA_HINT = """Return JSON: {"action_items": [
  {"assignee": string|null, "task": string, "due_date": string|null, "decision": bool}
]}. Use null when a value is not stated. Do not invent owners or dates."""

def extract_items(transcript_text):
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        temperature=0,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": "You extract action items from meeting transcripts."},
            {"role": "user", "content": SCHEMA_HINT + "\n\nTranscript:\n" + transcript_text},
        ],
    )
    return json.loads(resp.choices[0].message.content)["action_items"]
```

## How do you validate before sending to a task tracker?

Never trust raw model output. Check each item has a non-empty `task`, coerce `due_date` to a real date or `null`, and drop anything malformed before it reaches your tracker. One bad row can fail a whole batch import, so filter first.

Validation is the difference between a pipeline that runs unattended and one that pages you at 2 AM. The model returns JSON, but it can still hand back an empty task, a `due_date` like "soon" that isn't a date, or an assignee who wasn't in the meeting. Run every item through a small guard that enforces your tracker's 2 required fields, a task and a due date, normalizes dates to ISO 8601, and discards the rest. The check below keeps only items with a usable task and a parseable date.

```python
from datetime import date

def clean(items):
    valid = []
    for it in items:
        task = (it.get("task") or "").strip()
        if not task:
            continue                                    # skip empty tasks
        due = it.get("due_date")
        try:
            due = date.fromisoformat(due).isoformat() if due else None
        except (ValueError, TypeError):
            due = None                                  # unparseable date becomes null
        valid.append({"assignee": it.get("assignee"), "task": task, "due_date": due})
    return valid

# clean(items) returns the validated list; push it to Jira, Linear, or your CRM, one row per action item
```

## What's next

- [Get a meeting transcript and recording](/docs/cookbook/notetaker/get-transcript-and-recording/) for the full media object and download flow
- [Extract structured data from email](/docs/cookbook/ai/extract-data-from-email/) for the same LLM extraction pattern on email content
- [Handling Notetaker media files](/docs/v3/notetaker/media-handling/) for the transcript shape and built-in summary format
- [Notetaker API](/docs/v3/notetaker/) for creating bots and capturing meetings