Feed email history into an LLM

You want a model to answer “what did we agree on with this vendor?” using a real email thread, not a hallucination. The hard part isn’t the prompt. It’s that raw email is a mess of quoted replies, HTML wrappers, tracking images, and signatures, and a 30-message thread blows past most context windows long before you reach the answer.

This recipe shapes a user’s mailbox into clean, token-budgeted context for an LLM. You fetch a thread, strip each body down to readable text, chunk it to fit a window, and feed only the relevant parts to the model. It’s the retrieval-augmented generation pattern, applied to an inbox instead of a document store.

How do I feed email history into an LLM context?

Feed email into an LLM in four steps: fetch the thread with GET /v3/grants/{grant_id}/threads/{thread_id} to get its message_ids, pull each message body, clean the bodies with PUT /v3/grants/{grant_id}/messages/clean, then chunk the text to fit your model’s context window. One grant covers all 6 providers, so the same code path works for Gmail and Outlook accounts.

The order matters. Cleaning before chunking removes signatures and quoted replies that would otherwise eat 40% of your token budget on noise. A typical business thread of 12 messages is around 8,000 raw tokens but drops to roughly 2,500 after cleaning, which is the difference between one model call and a multi-pass summarization job.

How do I fetch a thread to use as model context?

Start with GET /v3/grants/{grant_id}/threads/{thread_id}, which returns the conversation’s subject, participants, and a message_ids array. The thread object carries only the latest_draft_or_message body, not every message, so you iterate message_ids to fetch each full message. A single thread can hold well over 50 messages, which is why this is a two-step fetch.

The request below reads one thread by ID. Use it when you already know which conversation the model should reason about, for example a vendor thread or a support escalation. URL-encode the thread_id, because Gmail thread IDs can contain characters that return a 404 otherwise.

from nylas import Client

nylas = Client(api_key="<NYLAS_API_KEY>")

thread = nylas.threads.find("<NYLAS_GRANT_ID>", "<THREAD_ID>")
message_ids = thread.data.message_ids
print(f"{len(message_ids)} messages in thread")

Each ID in message_ids maps to one message you fetch through the Messages API. For the full thread field list and how providers map to it, see how email threading works.

How do I clean an email body before sending it to a model?

Clean a body with PUT /v3/grants/{grant_id}/messages/clean, which strips signatures, quoted reply chains, tracking images, and link markup, then returns the readable text in a conversation field. You pass up to 20 message IDs per request in the message_id array. Cleaning a single thread of 12 messages takes one or two calls and removes the boilerplate that pads token counts.

The request below cleans a batch of messages and returns plain text per message. Set ignore_images and ignore_links to true so tracking images and long URLs don’t reach the model. Use remove_conclusion_phrases to drop “Best regards” style closers that add tokens without meaning.

curl --request PUT \
  --url 'https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/messages/clean' \
  --header 'Authorization: Bearer <NYLAS_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "message_id": ["<MESSAGE_ID_1>", "<MESSAGE_ID_2>"],
    "ignore_images": true,
    "ignore_links": true,
    "remove_conclusion_phrases": true
  }'

cleaned = nylas.messages.clean_messages(
    "<NYLAS_GRANT_ID>",
    request_body={
        "message_id": ["<MESSAGE_ID_1>", "<MESSAGE_ID_2>"],
        "ignore_images": True,
        "ignore_links": True,
        "remove_conclusion_phrases": True,
    },
)

context = "\n\n".join(m.conversation for m in cleaned.data)

The endpoint description states the conversation field holds “The cleaned message body. If html_as_markdown is true, the text is Markdown-formatted. Otherwise, Nylas returns plain text.” Plain text is the right default for embeddings, since Markdown syntax adds tokens an embedding model doesn’t need.

How do I chunk an email thread to fit a context window?

Chunk a cleaned thread by counting tokens per message and packing messages into windows that stay under your model’s limit. A practical target is 60% of the window for context, leaving room for the system prompt and the model’s reply. For a 128,000-token model, that’s around 75,000 tokens of email, enough for a 300-message thread after cleaning.

The function below counts tokens with the model’s own token encoder and groups cleaned messages into chunks. Counting before you send prevents the 400 errors and truncation you hit when a thread overflows the window. Each chunk stays a complete unit, so the model never sees half a message.

import tiktoken

encoder = tiktoken.get_encoding("cl100k_base")
MAX_TOKENS = 75000  # ~60% of a 128k window

def chunk_messages(cleaned_messages):
    chunks, current, count = [], [], 0
    for msg in cleaned_messages:
        tokens = len(encoder.encode(msg.conversation))
        if count + tokens > MAX_TOKENS and current:
            chunks.append(current)
            current, count = [], 0
        current.append(msg.conversation)
        count += tokens
    if current:
        chunks.append(current)
    return chunks

When a thread spans multiple chunks, summarize each chunk first, then summarize the summaries. That map-reduce approach is covered end to end in summarize email threads with AI, which handles threads longer than one window.

When should I use retrieval over the whole inbox instead?

Use inbox-wide retrieval when the answer could live in any of thousands of messages, not one known thread. Instead of fetching one conversation, you embed message snippets into a vector store, then retrieve the top matches at query time. The Messages API returns 50 messages per page and up to 200, with a snippet field holding the first 100 characters of each body.

Embed snippets, not full bodies, for the index. A 10,000-message mailbox is roughly 1 million snippet tokens to embed once, versus tens of millions for full bodies. At query time, you fetch then clean only the handful of full messages your vector search returns, so the model reads 5 to 10 cleaned messages rather than the entire mailbox. For the connection and retrieval plumbing, see connect an LLM to a user’s inbox.

Single thread or inbox-wide retrieval: Which fits?

The single-thread fetch and inbox-wide retrieval solve different problems. Pick by whether you know the conversation in advance. A single thread often fits in under 2,500 cleaned tokens, while inbox retrieval sends only the 5 to 10 messages a vector search returns. The table below compares the two on cost and fit.

Concern	Single-thread fetch	Inbox-wide retrieval (Nylas)
When to use	You know the thread ID	Answer could be in any message
Retrieval	`GET /threads/{thread_id}` then per-message fetch	Embed `snippet`, vector search, then fetch top hits
Token cost	Whole thread, often under 2,500 after cleaning	Only the 5-10 retrieved messages
Index needed	None	Vector store of snippet embeddings
Providers	All 6 through one grant	All 6 through one grant

If you only ever answer questions about one open conversation, skip the vector store. The single-thread fetch is simpler, has no index to maintain, and a 12-message thread costs well under a cent to summarize. The inbox-wide pattern earns its complexity only when the relevant message isn’t known ahead of time.

What’s next

Connect an LLM to a user’s inbox for the OAuth grant, fetch, and reply-routing plumbing
Summarize email threads with AI for the map-reduce pattern on threads longer than one window
How email threading works for the thread object and provider mapping
Messages API reference for filters, pagination, and the full message schema