A vendor emails a PDF invoice. A customer replies with a receipt photo. A job applicant sends a resume. The data your app needs sits inside the attached file, not the message body, and every sender formats it differently. Writing a rule-based parser for that variety breaks the moment a new vendor’s layout arrives.
This recipe covers the full inbound path: detect that a message carries attachments, download the raw bytes, and hand a PDF or image to a model that returns typed JSON. It uses two verified endpoints and is honest about where a dedicated OCR vendor still beats a general model.
How does an AI agent parse email attachments and extract structured data?
Section titled “How does an AI agent parse email attachments and extract structured data?”An AI agent parses email attachments in four steps: it detects a new message through a message.created webhook, downloads the file bytes with GET /v3/grants/{grant_id}/attachments/{attachment_id}/download, sends the PDF or image to a vision-capable model with a JSON schema, then validates the typed result before storing it. The schema forces the same fields out of every file format.
The agent never parses raw layouts itself. It reads the attachments array on the message, picks the files that match its content type filter, and pulls each one through a single download call. One fetch path works across all 6 providers, so a Gmail invoice and an Outlook receipt run the same code. Roughly 2% to 5% of messy real-world files still need a human check, so flag low-confidence results instead of trusting them blindly.
How do I download attachments from an email API?
Section titled “How do I download attachments from an email API?”Send a GET to /v3/grants/{grant_id}/attachments/{attachment_id}/download with the message_id as a required query parameter. The response is the raw file as binary, not JSON, so a 2 MB scanned invoice comes back ready to pipe to disk or a model. You get the attachment_id and message_id from the attachments array on the message itself.
Attachments aren’t a separate resource you can list, they live on the message object. Each entry carries an id, filename, content_type, and size in bytes. The download endpoint accepts an optional query_imap=true for IMAP, Yahoo, and iCloud accounts, which pulls the file straight from the IMAP server instead of the Nylas cache. The request below fetches one file. Use --compressed because the API serves downloads gzipped when the client asks, which matters on large files.
curl --compressed --request GET \ --url 'https://api.us.nylas.com/v3/grants/<NYLAS_GRANT_ID>/attachments/<ATTACHMENT_ID>/download?message_id=<MESSAGE_ID>' \ --header 'Authorization: Bearer <NYLAS_API_KEY>'import os, requests
NYLAS = "https://api.us.nylas.com"HEADERS = {"Authorization": f"Bearer {os.environ['NYLAS_API_KEY']}"}
def download_attachment(grant_id, message_id, attachment_id): r = requests.get( f"{NYLAS}/v3/grants/{grant_id}/attachments/{attachment_id}/download", headers=HEADERS, params={"message_id": message_id}, ) r.raise_for_status() return r.content # raw bytes: PDF or imageFor inline-versus-attached mechanics and the SDK download variants, see how to read and download email attachments.
How do I find the attachments on an inbound message?
Section titled “How do I find the attachments on an inbound message?”Fetch the message with GET /v3/grants/{grant_id}/messages/{message_id} and read its attachments array. Each object reports filename, content_type, size, and a stable id you pass straight to the download endpoint. An empty array means there’s nothing to fetch, so a single length check gates all the download work. At the thread level, the thread object exposes a has_attachments boolean you can read first to skip threads with no files.
When a webhook fires, don’t trust the payload’s body directly: Nylas truncates webhook payloads above 1 MB and appends a .truncated suffix to the trigger name. Fetch the full message through the API instead. Filter the array by content_type so you only download the files you can parse, like application/pdf or image/png, and skip the inline signature logos that arrive on most corporate mail. The function below fetches one message and returns the attachments worth processing.
PARSEABLE = {"application/pdf", "image/png", "image/jpeg"}
def parseable_attachments(grant_id, message_id): r = requests.get( f"{NYLAS}/v3/grants/{grant_id}/messages/{message_id}", headers=HEADERS ) r.raise_for_status() msg = r.json()["data"] return [ a for a in msg.get("attachments", []) if not a.get("is_inline") and a["content_type"].split(";")[0].strip() in PARSEABLE ]If you only need the file list, add ?select=attachments to strip the body and headers from the response, which keeps payloads small when you iterate a large mailbox.
How do I extract structured data from invoices and receipts?
Section titled “How do I extract structured data from invoices and receipts?”Route the downloaded file by type, then send it to a model constrained to a JSON schema. A PDF’s extracted text or a receipt image goes to a vision-capable model, which reads a scanned invoice the same way it reads body text. The schema, typically 5 to 10 fields, forces the same typed output whether the source is a tidy invoice or a crumpled receipt photo.
Type the total field as a number, not a string, so the model returns 49.99 instead of "$49.99" and you skip a parse step. Mark fields nullable when they may be absent, because forcing a value is what makes a model invent one. Extraction from a single invoice takes about 2 seconds and costs a fraction of a cent on a small vision model. The OpenAI structured outputs guide covers passing a schema through response_format, which decodes the model against your fields rather than trusting a prompt instruction.
INVOICE_SCHEMA = { "type": "object", "additionalProperties": False, "required": ["invoice_number", "total", "currency"], "properties": { "invoice_number": {"type": "string"}, "total": {"type": "number"}, "currency": {"type": "string"}, "due_date": {"type": ["string", "null"], "format": "date"}, "vendor": {"type": ["string", "null"]}, },}After the model returns, validate every field before it touches your database. A total of 0 or an invoice_number that fails your known pattern signals a missed extraction, and rejecting beats storing a confident wrong value. For the full schema and model-call walkthrough, see extract structured data from email.
Nylas attachments vs parsing provider APIs directly
Section titled “Nylas attachments vs parsing provider APIs directly”A unified download endpoint returns the same attachment shape across Gmail, Outlook, Yahoo, iCloud, IMAP, and Exchange, so one parser handles every inbound file. The Gmail API and Microsoft Graph each expose attachments through their own resource model, with different ID formats and base64 encoding rules, so supporting two providers means two integrations. The table compares the work each path requires.
| Task | Provider APIs directly | Nylas unified API |
|---|---|---|
| Auth setup | Separate OAuth app per provider | 1 connector, OAuth handled for you |
| Find attachments | users.messages.get (Gmail), GET /me/messages/{id}/attachments (Graph) | GET /v3/grants/{grant_id}/messages/{message_id} |
| Download bytes | Base64-decode the part payload per provider | GET /v3/grants/{grant_id}/attachments/{attachment_id}/download |
| Inbound trigger | Gmail Pub/Sub watch, Graph subscription renewals | One message.created webhook |
| Provider coverage | One provider per integration | 6 providers, one code path |
The honest tradeoff: if every mailbox you touch is Gmail and you already run the Gmail API for other features, decoding the attachment part there avoids adding a layer. The unified path wins when you support more than one provider or want to skip per-provider subscription plumbing.
Things to know about attachment size and limits
Section titled “Things to know about attachment size and limits”Attachment size drives where you run the download. The size field is bytes, so check it before fetching: push anything above roughly 1 MB to a background job rather than blocking a request handler, where a large download risks a timeout. Providers cap the full message payload at 25 MB, and on the send side Nylas requires multipart/form-data once a request crosses 3 MB.
Two more facts shape parsing. First, content_type can lie. Some senders stamp application/octet-stream on every file, so check the actual bytes (the magic-number header) before you parse or run anything. Second, cloud-storage links aren’t attachments: Google Drive and OneDrive files shown as “attachments” in the provider UI arrive as <a> tags in the body, not in the attachments array, and need separate Drive or OneDrive access to fetch. Treat attachment IDs as opaque and scoped to their grant and message, since the same physical file attached to two messages has two distinct IDs.
What’s next
Section titled “What’s next”- Read and download email attachments for inline handling and every SDK download variant
- Extract structured data from email for the full model-call and schema walkthrough
- Parse the email body into clean text for the in-body data path
- Messages API reference for every message and attachment field