{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-07-22T18:08:18.068Z"},"content":[{"type":"documentation","id":"d795de05-22fb-4a0c-a666-eae3de9d075b","slug":"research-automation-webhooks","title":"Research Automation: How to Build Real-Time Research Pipelines with Webhooks","url":"https://www.koji.so/docs/research-automation-webhooks","summary":"Architecture guide for automated research pipelines on the Koji Headless API as it exists today: trigger interviews from product events like signup or churn, detect completion via polling or the embed event, pull the AI analysis with the read endpoint, and route insights to Slack or a warehouse. Native webhooks are on the roadmap but not yet available.","content":"# Research Automation: How to Build Real-Time Research Pipelines with Webhooks\n\nThe dream pipeline is simple: a user churns, an interview starts automatically, the AI digs into why, and the insight lands in Slack before your standup. You can build that on the Koji Headless API today. One honest caveat before the architecture: **Koji does not have native outbound webhooks yet**, so the completion signal comes from polling or the embed event rather than a push. Everything else in the pipeline works exactly as you would hope.\n\n---\n\n## The pipeline at a glance\n\n| Stage | What happens | Koji surface |\n|---|---|---|\n| 1. Trigger | A product event starts an interview | `POST /interviews/start` |\n| 2. Collect | The respondent has the conversation | Your UI on the message API, or the embed widget |\n| 3. Detect and analyze | Completion is detected, AI analysis lands | Polling `GET /interviews/{id}`, or the embed completed event |\n| 4. Route | Insights flow to where your team works | Your worker pushes to Slack, CRM, or warehouse |\n\n---\n\n## Stage 1: Trigger interviews from product events\n\nAny event in your system can start an interview: a cancellation, a signup, a support ticket closing, an NPS detractor score. The start endpoint takes no project id (the API key determines the study), and the `respondent` object carries your CRM linkage:\n\n```javascript\n// Example: trigger an exit interview when a user cancels\nasync function onSubscriptionCancelled(user) {\n  const res = await fetch('https://www.koji.so/api/v1/interviews/start', {\n    method: 'POST',\n    headers: {\n      'Authorization': 'Bearer ' + process.env.KOJI_API_KEY,\n      'Content-Type': 'application/json',\n    },\n    body: JSON.stringify({\n      respondent: {\n        external_id: user.id,\n        display_name: user.firstName,\n        metadata: { plan: user.plan, cancelled_at: new Date().toISOString() },\n      },\n      mode: 'text',\n    }),\n  })\n\n  const interview = await res.json()\n\n  // Store the id so your completion worker can poll it later\n  await db.pendingInterviews.insert({\n    interview_id: interview.interview_id,\n    external_id: user.id,\n  })\n\n  // Keep session_token too: your UI drives the conversation with it\n  return interview\n}\n```\n\nThe two fields doing the heavy lifting:\n\n- `external_id` ties the interview to your user record. It comes back in the read response, so the pipeline can join results to the right account.\n- `metadata` is free-form JSON stored on the respondent. Stamp it with the trigger context (plan, cohort, event timestamp) so analysis lands with its context attached.\n\nFull request reference in [Starting Interviews via API](/docs/starting-interviews-via-api).\n\n---\n\n## Stage 2: Collect the conversation\n\nTwo ways to run the interview itself:\n\n- **Your own UI on the message API.** Your client drives the conversation with the `session_token`, streaming AI replies over SSE. See [Sending Messages via API](/docs/sending-messages-via-api) and [Completing Interviews via API](/docs/completing-interviews-via-api).\n- **The embed widget.** Drop the iframe into your app or a follow-up page and let it run the whole conversation. See [Embed Widget Reference](/docs/embed-widget-reference).\n\n---\n\n## Stage 3: Detect completion and pull the analysis\n\nSince there is no webhook push yet, completion detection is pull-based. The workhorse is a scheduled job over your pending interview ids:\n\n```javascript\n// Runs on your server every minute (cron job, worker, or scheduled function).\n// pendingIds holds interview ids you stored when you called the start endpoint.\nconst API_KEY = 'pk_live_wJalrXUtnFEMI4K7MDENGbPxRfiCYz2K'\n\nasync function checkPending(pendingIds) {\n  for (const id of pendingIds) {\n    const res = await fetch(`https://www.koji.so/api/v1/interviews/${id}`, {\n      headers: { 'Authorization': `Bearer ${API_KEY}` },\n    })\n    if (!res.ok) continue\n\n    const interview = await res.json()\n    if (interview.status === 'completed' && interview.analysis) {\n      await handleCompleted(interview) // your \"webhook handler\"\n      // Remove id from your pending store so it is not processed twice\n    }\n  }\n}\n```\n\nKey behaviors to build around:\n\n- `analysis` is `null` until the interview is completed and the background analysis job has finished. Gating on `status === 'completed' && analysis` means one poll pass picks up fully processed interviews only.\n- The read endpoint authenticates with the API key alone (`interview:read` permission, no session token), so the worker needs exactly one server-side key.\n- Respect the per-key rate limit of 60 requests per minute when batching. See [Rate Limits and CORS](/docs/rate-limits-and-cors).\n\nIf your interviews run in the embed widget, the `koji:interview_completed` postMessage event gives you a faster in-browser signal you can forward to your backend. Use it to shave minutes off the loop, with polling as the safety net. Both patterns are compared in [Webhook Setup](/docs/webhook-setup).\n\n---\n\n## Stage 4: Route the insights\n\nOnce the worker has a completed interview with analysis, routing is plain integration work. Slack is the classic first target:\n\n```javascript\n// After your worker fetches a completed interview with analysis\nconst summary = interview.analysis?.summary ?? 'Analysis pending'\n\nawait fetch(process.env.SLACK_WEBHOOK_URL, {\n  method: 'POST',\n  headers: { 'Content-Type': 'application/json' },\n  body: JSON.stringify({\n    text: `Interview ${interview.interview_id} completed ` +\n      `(${interview.stats.message_count} messages)\\n${summary}`,\n  }),\n})\n```\n\nOther destinations follow the same shape: upsert into your warehouse keyed on `respondent.external_id`, attach the summary to the CRM record that triggered the interview, or open a ticket when sentiment crosses a threshold. The read response gives you the full transcript, stats, and analysis in one payload, so a single fetch feeds every destination.\n\n---\n\n## Putting it together\n\nThe whole lifecycle, runnable end to end in Node:\n\n```javascript\n// Full interview lifecycle: start → message → complete → read results.\n// Node 18+ (built-in fetch). Replace the key with your own.\nconst API_KEY = 'pk_live_wJalrXUtnFEMI4K7MDENGbPxRfiCYz2K'\nconst BASE = 'https://www.koji.so/api/v1'\nconst HEADERS = { 'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json' }\n\nasync function main() {\n  // 1. Start\n  const started = await fetch(`${BASE}/interviews/start`, {\n    method: 'POST',\n    headers: HEADERS,\n    body: JSON.stringify({ respondent: { external_id: 'user_8271' }, mode: 'text' }),\n  }).then(r => r.json())\n  console.log('AI:', started.initial_message)\n\n  const session = { 'X-Session-Token': started.session_token }\n\n  // 2. One message exchange (SSE)\n  const res = await fetch(`${BASE}/interviews/${started.interview_id}/message`, {\n    method: 'POST',\n    headers: { ...HEADERS, ...session },\n    body: JSON.stringify({ content: 'I found onboarding confusing.' }),\n  })\n  let reply = ''\n  const reader = res.body.getReader()\n  const decoder = new TextDecoder()\n  let buffer = ''\n  while (true) {\n    const { value, done } = await reader.read()\n    if (done) break\n    buffer += decoder.decode(value, { stream: true })\n    const lines = buffer.split('\\n')\n    buffer = lines.pop()\n    for (const line of lines) {\n      if (!line.startsWith('data: ')) continue\n      const frame = JSON.parse(line.slice(6))\n      if (frame.type === 'chunk') reply += frame.content\n    }\n  }\n  console.log('AI:', reply)\n\n  // 3. Complete\n  await fetch(`${BASE}/interviews/${started.interview_id}/complete`, {\n    method: 'POST',\n    headers: { ...HEADERS, ...session },\n    body: JSON.stringify({ reason: 'natural' }),\n  })\n\n  // 4. Read results (analysis lands asynchronously)\n  const result = await fetch(`${BASE}/interviews/${started.interview_id}`, {\n    headers: HEADERS,\n  }).then(r => r.json())\n  console.log('Transcript messages:', result.stats.message_count)\n}\n\nmain()\n```\n\nIn production the stages split across your event handlers (stage 1), your UI or the embed (stage 2), and one scheduled worker (stages 3 and 4). The worker is the only new infrastructure the pipeline needs.\n\n---\n\n## Native webhooks: coming soon\n\nOutbound webhooks are on the roadmap. When they arrive, stage 3 becomes a push instead of a poll and end-to-end latency drops to near-real-time. Design for the swap now: keep completion handling in one function so the migration is a trigger change, not a rewrite.\n\n---\n\n## FAQ\n\n### Is this pipeline real-time?\n\nNear-real-time. With a one-minute poll cycle, insights land at most a couple of minutes after the respondent finishes. The embed completed event can shave that further for widget-based interviews.\n\n### Do I need one pipeline per study?\n\nOne worker can serve many studies, but API keys are study-scoped, so the worker holds one `interview:read` key per study and picks the right one per pending interview.\n\n### How do I connect an interview back to the user who triggered it?\n\nSet `respondent.external_id` at start time. It comes back as `respondent.external_id` in the read response, ready to join against your user table.\n\n### What permissions should the pipeline's keys have?\n\nThe trigger path needs `interview:start`. The worker needs `interview:read`. If your own UI drives conversations, that client-side key needs `interview:chat` and `interview:complete` too. See [API Key Permissions](/docs/api-permissions).","category":"API Reference","lastModified":"2026-07-14T13:56:43.927899+00:00","metaTitle":"Real-Time Research Pipelines | Koji Headless API","metaDescription":"Build an automated research pipeline with the Koji API: trigger interviews from product events, detect completion, pull analysis, and route insights.","keywords":["research automation webhooks","user research webhook","automated research pipeline","real-time research data","research webhook integration","continuous research automation","qualitative data webhook"],"aiSummary":"Architecture guide for automated research pipelines on the Koji Headless API as it exists today: trigger interviews from product events like signup or churn, detect completion via polling or the embed event, pull the AI analysis with the read endpoint, and route insights to Slack or a warehouse. Native webhooks are on the roadmap but not yet available.","aiPrerequisites":["Familiarity with HTTP and REST APIs","Koji account on Interviews plan or higher for webhooks","A webhook endpoint (can use Zapier or Make for no-code setup)"],"aiLearningOutcomes":["Design a four-stage research pipeline: trigger, collect, analyze, route","Start interviews automatically from product events with external_id linkage","Detect completion with a polling worker or the embed event","Route AI analysis into Slack or a data warehouse"],"aiDifficulty":"advanced","aiEstimatedTime":"12 min"}],"pagination":{"total":1,"returned":1,"offset":0}}