Streaming AI Responses with OpenAI API in Next.js

Sat, 30 Mar 2024 00:00:00 +0000

Streaming improves chat UX by showing tokens as they are generated. Next.js Route Handlers make it straightforward to proxy streams securely.

Route Handler

// app/api/chat/route.ts
import OpenAI from 'openai';

export async function POST(req: Request) {
  const { messages } = await req.json();
  const openai = new OpenAI();

  const stream = await openai.chat.completions.create({
    model: 'gpt-4',
    messages,
    stream: true,
  });

  const encoder = new TextEncoder();
  const readable = new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        const text = chunk.choices[0]?.delta?.content || '';
        if (text) controller.enqueue(encoder.encode(text));
      }
      controller.close();
    },
  });

  return new Response(readable, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

Client Consumption

Use fetch with a reader loop or libraries like Vercel AI SDK’s useChat for React state management.

Streaming on David Lang

Streaming AI Responses with OpenAI API in Next.js

Route Handler

Client Consumption