# Build a web research agent with Workflow SDK **Author:** Anshuman Bhardwaj --- Build a web research agent that searches the web, extracts findings from each source, and returns a cited report without losing progress when a crash, restart, or deployment interrupts a run that can loop many times. You’ll use AI SDK with [Workflow SDK](https://workflow-sdk.dev/) to run a research loop not as one long request but as a sequence of durable, journaled steps, so the search results and findings already recorded survive any interruption. In this guide, you'll build a Next.js app that hands a user's question to an agent running inside [Vercel Workflows](https://vercel.com/workflows), performs web searches across various sources, and extracts findings from each. Along with this, you'll learn to stream the agent's progress to the browser in real time and return a final report with citations. Deploy the template now, or read on for a deeper look at how it all works. ## Quick start with an AI coding agent If you are working with an AI coding agent, hand it the project and this prompt: ### Vercel Plugin Turn your agent into a Vercel expert with this [plugin](https://vercel.com/docs/agent-resources/vercel-plugin). It gives your coding agent current knowledge of the Vercel products this template uses, including Vercel Workflows, AI Gateway, and AI SDK. The plugin is optional; it is not required to use this template or for this guide. `npx plugins add vercel/vercel-plugin` ## Overview The request flow looks like this: `POST /api/research -> start(researchWorkflow, [question]) -> searchWeb step using AI Gateway search -> for each source page, extractFindings step -> synthesizeReport step -> return ResearchReport GET /api/readable/:runId -> read workflow stream -> transform workflow chunks into UI chunks -> send SSE events to the browser GET /api/run/:runId -> read status and returnValue` ## Prerequisites Before you begin, make sure you have: - Node.js 22 or later - [pnpm](https://pnpm.io/) (or npm/yarn) - A [Vercel account](https://vercel.com/signup) ### Configure credentials For local development, link the Vercel project and pull environment variables: `vercel link vercel env pull` AI Gateway authenticates requests using [Vercel OIDC tokens](https://vercel.com/docs/oidc), which Vercel generates and links to your project automatically. ## Create the project Create a new [Next.js](https://nextjs.org/docs) app with `create-next-app` : `pnpm create next-app@latest durable-web-research-agent --yes cd durable-web-research-agent` Then install the Workflow and AI SDK packages: `pnpm add workflow @ai-sdk/workflow ai zod @vercel/oidc ajv` ### Configure Next.js for Workflow Wrap your Next.js config with `withWorkflow`: `import type { NextConfig } from "next"; import { withWorkflow } from "workflow/next"; const nextConfig: NextConfig = { serverExternalPackages: ["@vercel/oidc", "ajv"], }; export default withWorkflow(nextConfig);` ## Define the data types Keep the workflow state compact. Store findings, URLs, snippets, and short page text. Avoid writing the entire crawled websites into the workflow event log. `export type Finding = { claim: string; sourceUrl: string; snippet: string; }; export type ResearchReport = { sections: Array<{ heading: string; body: string; }>; citations: Array<{ claim: string; sourceUrl: string; }>; }; export type SearchWebResult = { query: string; answer: string; sources: Array<{ title?: string; url: string; text?: string; }>; }; export type SourcePage = { title?: string; url: string; text: string; error?: string; }; export type ExtractFindingsResult = { findings: Finding[]; };` ## Create durable research steps The Workflow SDK gives the app durable control flow and the AI SDK gives the app model calls, AI Gateway tools, and structured outputs. Implement the two branches, as follows: - Put `generateText()`, `fetch()`, Gateway search tools, and any other side effects inside `"use step"` functions. - Keep the `"use workflow"` function focused on orchestration, loops over already-returned data, and calls to step functions. - Return compact typed objects from steps so Workflow can journal them and replay the run without calling the model again. The research steps do the work with side effects: - `searchWeb()` calls an AI Gateway search tool. - `extractFindingsFromPages()` calls an AI Gateway model with `Output.object`. - `fetchSourcePage()` helps when your search tool returns URLs without enough page text. The example below uses AI Gateway's `parallelSearch` tool. Search belongs in a step because it is external I/O: it can fail, retry, rate-limit, or return different results over time. Once the step completes, Workflow records the selected sources, and replay can continue from those recorded sources instead of searching again. `parallelSearch` returns source-oriented search results with excerpts that are already useful to an LLM. If your project prefers Perplexity-backed search, swap in `gateway.tools.perplexitySearch({ maxResults: 5 })`. ``import { generateText, gateway, Output } from "ai"; import { z } from "zod"; import type { ExtractFindingsResult, SearchWebResult, SourcePage, } from "./research-types"; const MAX_SOURCE_PAGES = 5; const MAX_PAGE_CHARS = 12000; const extractedFindingsSchema = z.object({ findings: z.array( z.object({ claim: z.string(), sourceUrl: z.string().describe("One of the source URLs provided."), snippet: z.string(), }), ), }); function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null; } function firstString(...values: unknown[]) { return values.find((value): value is string => typeof value === "string"); } function normalizeGatewaySearchOutput(output: unknown): SourcePage[] { if (!isRecord(output)) { return []; } const candidates = Array.isArray(output.results) ? output.results : Array.isArray(output.sources) ? output.sources : []; return candidates .filter(isRecord) .map((result) => { const url = firstString(result.url, result.link, result.sourceUrl); const text = firstString( result.excerpt, result.snippet, result.text, result.content, ); if (!url) { return null; } return { title: firstString(result.title), url, text: (text ?? "").slice(0, MAX_PAGE_CHARS), }; }) .filter((page): page is SourcePage => page !== null) .slice(0, MAX_SOURCE_PAGES); } export async function searchWeb({ query }: { query: string }) { "use step"; const result = await generateText({ model: "openai/gpt-5-nano", prompt: ` Find reliable source pages for this research question: ${query} Return sources that contain concrete technical details, tradeoffs, and implementation facts. Prefer primary docs and high-signal technical articles. `.trim(), tools: { parallel_search: gateway.tools.parallelSearch({ mode: "agentic", maxResults: MAX_SOURCE_PAGES, excerpts: { maxCharsPerResult: MAX_PAGE_CHARS, }, }), }, toolChoice: { type: "tool", toolName: "parallel_search" }, }); const sources = result.toolResults .filter((toolResult) => toolResult.toolName === "parallel_search") .flatMap((toolResult) => normalizeGatewaySearchOutput(toolResult.output)); return { query, answer: result.text, sources, } satisfies SearchWebResult; }`` ### Fetch pages when search only returns URLs Some search tools return URLs and snippets but not enough source text. In that case, add a `fetchSourcePage` step and call it before extraction. Keep it as a workflow step because page fetches are side effects and can fail or need retries. ``function decodeHtmlEntities(value: string) { return value .replaceAll(" ", " ") .replaceAll("&", "&") .replaceAll("<", "<") .replaceAll(">", ">") .replaceAll(""", "\"") .replaceAll("'", "'"); } function htmlToText(html: string) { return decodeHtmlEntities( html .replace(//gi, " ") .replace(//gi, " ") .replace(//gi, " ") .replace(//gi, " ") .replace(/<[^>]+>/g, " ") .replace(/\s+/g, " ") .trim(), ); } export async function fetchSourcePage({ title, url, }: { title?: string; url: string; }): Promise { "use step"; try { const response = await fetch(url, { headers: { accept: "text/html,text/plain;q=0.9,*/*;q=0.8", "user-agent": "Mozilla/5.0 (compatible; DurableResearchAgent/1.0)", }, }); if (!response.ok) { return { title, url, text: "", error: `Fetch failed with status ${response.status}.`, }; } const contentType = response.headers.get("content-type") ?? ""; const body = await response.text(); const text = contentType.includes("html") ? htmlToText(body) : body.replace(/\s+/g, " ").trim(); return { title, url, text: text.slice(0, MAX_PAGE_CHARS), }; } catch (error) { return { title, url, text: "", error: error instanceof Error ? error.message : "Unknown fetch error.", }; } }`` ### Extract findings with structured output Extraction is separate from search on purpose: search finds candidate source material and extraction decides which claims are useful enough to keep and turns messy page text into compact, typed research artifacts. Keeping extraction separate gives you three benefits: - The final report can depend on a stable `Finding[]` instead of raw search prose. - The UI can stream findings as soon as each page is processed. - If synthesis fails, Workflow reuses the already-extracted findings instead of asking the search model to recreate them. Use `Output.object()` so the extraction step returns predictable data. ``export async function extractFindingsFromPages({ pages, question, }: { pages: SourcePage[]; question: string; }): Promise { "use step"; const readablePages = pages.filter((page) => page.text.trim().length > 0); if (readablePages.length === 0) { return { findings: [] }; } const allowedUrls = new Set(readablePages.map((page) => page.url)); const { output } = await generateText({ model: "anthropic/claude-haiku-4.5", output: Output.object({ schema: extractedFindingsSchema, }), prompt: ` Question: ${question} Fetched source pages: ${JSON.stringify( readablePages.map((page) => ({ title: page.title, url: page.url, text: page.text, })), null, 2, )} Extract 8 to 12 concise findings that answer the question. Each finding must use one of the provided page URLs as sourceUrl and include a short snippet from that page text. Prefer concrete technical differences, tradeoffs, use cases, ecosystem facts, deployment details, and limitations. `.trim(), }); return { findings: output.findings.filter((finding) => allowedUrls.has(finding.sourceUrl), ), }; }`` The URL filter after generation prevents the model from inventing citations that were not in the fetched source set. ### Orchestrate the workflow The workflow body should mostly coordinate steps. It can loop over the bounded search results, but it should not perform network requests or model calls directly. This workflow also writes progress events to `getWritable()`. Those events look like AI SDK model-call stream parts, so the UI can render tool starts and finishes. ``import type { ModelCallStreamPart } from "@ai-sdk/workflow"; import { getWritable } from "workflow"; import { extractFindingsFromPages, fetchSourcePage, searchWeb, } from "./research-steps"; import type { Finding } from "./research-types"; import { synthesizeReport } from "./synthesize"; const MAX_SOURCE_PAGES = 5; type WorkflowStreamPart = | ModelCallStreamPart | { type: "finish-step" } | { type: "start-step" }; function findingKey(finding: Finding) { return `${finding.claim}\n${finding.sourceUrl}`; } function dedupeFindings(findings: Finding[]) { return [ ...findings.reduce((deduped, finding) => { deduped.set(findingKey(finding), finding); return deduped; }, new Map()).values(), ]; } async function writeToolCall( writable: WritableStream, toolCallId: string, toolName: string, input: unknown, ) { "use step"; const writer = writable.getWriter(); try { await writer.write({ type: "tool-call", toolCallId, toolName, input, } as WorkflowStreamPart); } finally { writer.releaseLock(); } } async function writeToolResult( writable: WritableStream, toolCallId: string, toolName: string, output: unknown, ) { "use step"; const writer = writable.getWriter(); try { await writer.write({ type: "tool-result", toolCallId, toolName, output, } as WorkflowStreamPart); await writer.write({ type: "finish-step" }); await writer.write({ type: "start-step" }); } finally { writer.releaseLock(); } } export async function researchWorkflow(question: string) { "use workflow"; const writable = getWritable(); const normalizedQuestion = question.trim(); const findings: Finding[] = []; await writeToolCall(writable, "search-0", "searchWeb", { query: normalizedQuestion, maxSources: MAX_SOURCE_PAGES, }); const searchResult = await searchWeb({ query: normalizedQuestion }); await writeToolResult(writable, "search-0", "searchWeb", searchResult); for (const [index, source] of searchResult.sources .slice(0, MAX_SOURCE_PAGES) .entries()) { const fetchCallId = `fetch-${index}`; await writeToolCall(writable, fetchCallId, "fetchPage", source); const page = source.text ? { title: source.title, url: source.url, text: source.text, } : await fetchSourcePage(source); await writeToolResult(writable, fetchCallId, "fetchPage", page); const extractCallId = `extract-${index}`; await writeToolCall(writable, extractCallId, "extractFindings", { sourceUrl: page.url, title: page.title, hasText: page.text.trim().length > 0, }); const extracted = await extractFindingsFromPages({ pages: [page], question: normalizedQuestion, }); findings.push(...extracted.findings); await writeToolResult(writable, extractCallId, "extractFindings", extracted); } return synthesizeReport(normalizedQuestion, dedupeFindings(findings)); }`` To keep the run predictable even if a search provider returns more results than requested, this adds the following guardrails: - the Gateway search tool receives `maxResults: 5` - the workflow slices `searchResult.sources` to `MAX_SOURCE_PAGES` ### Synthesize the final report Synthesis should read only the extracted findings. This keeps the report grounded in durable evidence instead of the model's memory of earlier search text. That design also makes failure recovery cheaper. If the final report step retries, it receives the same `Finding[]` from the event log and does not repeat search or extraction. If there are no findings, return an explicit empty-state report instead of asking the model to improvise. ``import { generateText, Output } from "ai"; import { z } from "zod"; import type { Finding, ResearchReport } from "./research-types"; const reportSchema = z.object({ sections: z.array( z.object({ heading: z.string(), body: z.string(), }), ), citations: z.array( z.object({ claim: z.string(), sourceUrl: z.string().describe("The source URL for the cited claim."), }), ), }); export async function synthesizeReport( question: string, findings: Finding[], ): Promise { "use step"; if (findings.length === 0) { return { sections: [ { heading: "Insufficient sourced findings", body: "The research run completed without any extracted findings, so a cited report cannot be generated from the workflow state.", }, ], citations: [], }; } const { output } = await generateText({ model: "anthropic/claude-sonnet-4.6", output: Output.object({ schema: reportSchema, }), prompt: ` Question: ${question} Findings: ${JSON.stringify(findings, null, 2)} Write a deep, practical research brief that answers the question using only these findings. For comparison questions, include sections for positioning, core capabilities, ecosystem and language/runtime fit, agent/orchestration support, deployment tradeoffs, and a recommendation matrix. Every section must ground its claims in the findings. Include citations for the most important claims using the exact sourceUrl values from the findings. `.trim(), }); return { sections: output.sections, citations: output.citations.length > 0 ? output.citations : findings.slice(0, 10).map((finding) => ({ claim: finding.claim, sourceUrl: finding.sourceUrl, })), }; }`` The deterministic citation fallback matters. Structured output improves shape, but it does not guarantee the model will populate every optional-looking field the way your UI expects. ## Start the workflow from an API route Start the workflow with a `POST` request and return the run ID. This route returns immediately. The browser can then subscribe to the readable stream and poll status by `runId`. `import { NextResponse } from "next/server"; import { start } from "workflow/api"; import { researchWorkflow } from "@/workflows/research-workflow"; export async function POST(req: Request) { const { question } = (await req.json()) as { question?: string }; if (!question?.trim()) { return NextResponse.json( { error: "Question is required." }, { status: 400 }, ); } const run = await start(researchWorkflow, [question.trim()]); return NextResponse.json({ runId: run.runId, }); }` ### Stream workflow progress Workflow streams are read by run ID. The code below converts the workflow stream into AI SDK UI chunks and sends them as Server-Sent Events. ``import { toUIMessageChunk } from "@ai-sdk/workflow"; import type { UIMessageChunk } from "ai"; import { getRun } from "workflow/api"; type RouteContext = { params: Promise<{ runId: string }>; }; type ModelCallStreamPartWithToolName = { type: string; toolName?: string; }; function createResearchUIChunkTransform() { return new TransformStream({ start(controller) { controller.enqueue({ type: "start" }); controller.enqueue({ type: "start-step" }); }, transform(part, controller) { const chunk = toUIMessageChunk(part as never); if (!chunk) { return; } if ( chunk.type === "tool-output-available" && typeof part === "object" && part !== null && "toolName" in part && typeof (part as ModelCallStreamPartWithToolName).toolName === "string" ) { controller.enqueue({ ...chunk, toolName: (part as ModelCallStreamPartWithToolName).toolName, } as UIMessageChunk); return; } controller.enqueue(chunk); }, flush(controller) { controller.enqueue({ type: "finish-step" }); controller.enqueue({ type: "finish" }); }, }); } export async function GET(_request: Request, { params }: RouteContext) { const { runId } = await params; let run; try { run = await getRun(runId); await run.status; } catch { return Response.json( { error: `Run ${runId} was not found.` }, { status: 404 }, ); } const readable = ( run.getReadable() as unknown as ReadableStream ).pipeThrough(createResearchUIChunkTransform()); const encoder = new TextEncoder(); const stream = readable.pipeThrough( new TransformStream({ transform(chunk, controller) { const data = typeof chunk === "string" ? chunk : JSON.stringify(chunk); controller.enqueue(encoder.encode(`data: ${data}\n\n`)); }, }), ); return new Response(stream, { headers: { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache, no-transform", Connection: "keep-alive", "X-Accel-Buffering": "no", }, }); }`` The custom transform preserves `toolName` on output chunks. The UI needs that metadata to decide which tool outputs contain findings. ### Read status and the final report The final report is the workflow return value. ``import { NextResponse } from "next/server"; import { getRun } from "workflow/api"; type RouteContext = { params: Promise<{ runId: string }>; }; function toIso(value: Date | null | undefined) { return value ? value.toISOString() : null; } export async function GET(_request: Request, { params }: RouteContext) { const { runId } = await params; try { const run = await getRun(runId); const [status, workflowName, createdAt, startedAt, completedAt, returnValue] = await Promise.all([ run.status, run.workflowName, run.createdAt, run.startedAt, run.completedAt, run.returnValue.catch(() => null), ]); return NextResponse.json({ runId, status, workflowName, createdAt: toIso(createdAt), startedAt: toIso(startedAt), completedAt: toIso(completedAt), returnValue, }); } catch { return NextResponse.json( { error: `Run ${runId} was not found.` }, { status: 404 }, ); } }`` ### Connect a client The backend is intentionally UI-agnostic. A React console, a CLI, or an internal dashboard can all use the same three contracts: | Endpoint | Client responsibility | | -------------------------- | -------------------------------------------- | | `POST /api/research` | Send `{ question }`, receive `{ runId }` | | `GET /api/readable/:runId` | Subscribe to streamed UI chunks | | `GET /api/run/:runId` | Poll status and read the final `returnValue` | The stream route already converts workflow events into AI SDK UI chunks. A client does not need to know how Workflow stores the run. It only needs to interpret these events: - `tool-input-available` means a workflow phase started, such as `searchWeb`, `fetchPage`, or `extractFindings`. - `tool-output-available` means that phase finished. - `tool-output-available` with `toolName: "extractFindings"` contains `{ findings: Finding[] }`. - `finish` means the readable stream ended. Dedupe streamed findings by `toolCallId` and finding index. A reconnect or retry can replay chunks that the browser has already seen. You now have a web research agent ready to answer your questions. The complete [template](https://github.com/vercel-labs/durable-web-research-agent) includes a polished frontend with real-time streaming for findings and workflow steps. To get started, clone this template’s [GitHub repository](https://github.com/vercel-labs/durable-web-research-agent), [configure credentials](#configure-credentials), and run the local app: `pnpm install pnpm dev` Open `http://localhost:3000` and submit: `When to use Vercel Workflows?` ## Troubleshooting ### Search returns no source pages Inspect `result.toolResults` from the Gateway search call. Gateway tools can differ in output shape. Normalize `results`, `sources`, excerpts, and URLs before passing data to extraction. ### Search provider access is restricted Use a Gateway-executed search tool that is enabled for your team. If `parallelSearch` is not available, try `perplexitySearch`, or configure provider access in AI Gateway. ### The final report has no citations Make sure synthesis receives extracted findings, not raw search text. Add the deterministic citation fallback shown above so the UI always has source URLs when findings exist. ### Live findings stay empty Check that the stream route preserves `toolName` on `tool-output-available` chunks and that the UI reads findings from `extractFindings` outputs. ### Steps rerun on every replay Move side effects out of the workflow body and into `"use step"` functions. Native `fetch`, provider calls, timestamps, random values, and writable stream writes should be inside steps. ## Resources - [Vercel Workflows documentation](https://vercel.com/docs/workflow) - [Workflow SDK documentation](https://workflow-sdk.dev) - [AI SDK documentation](https://ai-sdk.dev/docs) - [AI Gateway provider documentation](https://ai-sdk.dev/providers/ai-sdk-providers/ai-gateway) --- [View full KB sitemap](/kb/sitemap.md)