Streaming

Progressive UI rendering powered by incremental JSON parsing. Users see components building in real time as LLM tokens arrive, instead of staring at a loading spinner until the full response is ready.

Overview

Traditional LLM integrations wait for the entire JSON response before rendering anything. With GenUIKit streaming, every token the model produces is fed into an incremental parser that heals partial JSON on the fly. As soon as enough props are available, the matching component starts rendering and progressively fills in as more data arrives.

The streaming pipeline has three layers that work together:

  • --StreamParser— Low-level incremental JSON parser that heals incomplete tokens into valid objects.
  • --useStreamingUI— React hook that connects a streaming source to your component registry and manages the full render lifecycle.
  • --StreamResolver— Orchestration layer that ties parsing, throttling, abbreviation expansion, and error recovery into one pipeline.
quick-example.tsxtsx
1import { useStreamingUI } from '@genuikit/react';
2import { registry } from './registry';
3 
4function StreamingChat({ responseStream }) {
5 const { element, phase } = useStreamingUI(registry, responseStream);
6 
7 return (
8 <div className="message">
9 {phase === 'pending' && <p>Waiting for response...</p>}
10 {element}
11 {phase === 'complete' && <span className="checkmark">Done</span>}
12 </div>
13 );
14}

StreamParser

The StreamParser from @genuikit/core is the foundation of the streaming system. It accepts raw string chunks (individual tokens from the LLM) and maintains a running parse of the incomplete JSON, healing it into a valid object at every step.

Core API

Method / PropertyTypeDescription
push(chunk)(chunk: string) => voidFeed a new token or chunk of text into the parser.
currentRecord<string, unknown> | nullThe current healed JSON object. Returns null until the first opening brace is seen.
completebooleanTrue when the parser has received a fully closed JSON object.

How JSON Healing Works

When the parser receives a partial token stream, it closes any open strings, arrays, and objects to produce valid JSON at every step. For example, if the LLM has emitted {"city": "Tok, the parser heals it to {"city": "Tok"} so your component can already render with the partial value.

Token-by-Token Example

Here is what happens as tokens arrive one by one:

stream-parser-demo.tstypescript
1import { StreamParser } from '@genuikit/core';
2 
3const parser = new StreamParser();
4 
5// Token 1: opening brace
6parser.push('{"');
7console.log(parser.current); // {}
8console.log(parser.complete); // false
9 
10// Token 2: start of the "type" key
11parser.push('type": "Weather');
12console.log(parser.current); // { type: "Weather" }
13 
14// Token 3: close type value, start props
15parser.push('Card", "props": {"city": "Tok');
16console.log(parser.current);
17// { type: "WeatherCard", props: { city: "Tok" } }
18 
19// Token 4: finish city, add temperature
20parser.push('yo", "temperature": 22');
21console.log(parser.current);
22// { type: "WeatherCard", props: { city: "Tokyo", temperature: 22 } }
23 
24// Token 5: close everything
25parser.push(', "condition": "sunny"}}');
26console.log(parser.current);
27// { type: "WeatherCard", props: { city: "Tokyo", temperature: 22, condition: "sunny" } }
28console.log(parser.complete); // true

Notice that current always returns a valid object, even mid-token. The parser tracks nesting depth, string escape sequences, and number boundaries so it always knows exactly where to insert closing delimiters.

useStreamingUI Hook

The useStreamingUI hook from @genuikit/react is the primary way to connect a streaming LLM response to your component registry. It manages the full lifecycle from pending through skeleton, partial rendering, and completion.

Signature

use-streaming-ui.d.tstypescript
1import { ComponentRegistry } from '@genuikit/core';
2 
3interface StreamingUIOptions {
4 // Shown while waiting for the first meaningful token
5 skeleton?: React.ReactNode;
6 
7 // Rendered if the stream fails or the component cannot be resolved
8 fallback?: React.ReactNode;
9 
10 // Called when the stream completes successfully
11 onComplete?: (result: { type: string; props: Record<string, unknown> }) => void;
12 
13 // Number of automatic retries on stream errors (default: 0)
14 retry?: number;
15}
16 
17interface StreamingUIResult {
18 // The current rendered element (updates progressively)
19 element: React.ReactNode | null;
20 
21 // Current phase of the stream lifecycle
22 phase: 'pending' | 'skeleton' | 'partial' | 'complete' | 'error';
23 
24 // Validation errors if the final props fail schema checks
25 errors: string[] | null;
26 
27 // The raw parsed data at any point in the stream
28 data: Record<string, unknown> | null;
29}
30 
31function useStreamingUI(
32 registry: ComponentRegistry,
33 source: ReadableStream<string> | Response | null,
34 options?: StreamingUIOptions
35): StreamingUIResult;

State Transitions

The hook moves through four phases as the stream progresses. Each phase maps to a distinct rendering state:

pending→skeleton→partial→complete
  • pending — Source is null or has not emitted any tokens yet.
  • skeleton — First bytes arrived, but not enough data to identify the component. Shows the skeleton placeholder.
  • partial — Component type identified and rendering with partial props. Updates on every token.
  • complete — Stream finished. Final props are validated against the schema.

Connecting to fetch()

The hook accepts a Response object directly, so you can pass the result of fetch() without any extra plumbing:

streaming-chat.tsxtsx
1'use client';
2 
3import { useState } from 'react';
4import { useStreamingUI } from '@genuikit/react';
5import { registry } from '@/lib/registry';
6 
7export function StreamingChat() {
8 const [response, setResponse] = useState<Response | null>(null);
9 
10 const { element, phase, errors } = useStreamingUI(registry, response, {
11 skeleton: <div className="animate-pulse h-32 bg-gray-200 rounded-lg" />,
12 fallback: <div className="text-red-500">Failed to render component</div>,
13 onComplete: (result) => {
14 console.log('Stream complete:', result.type);
15 },
16 retry: 2,
17 });
18 
19 async function handleSend(message: string) {
20 const res = await fetch('/api/chat', {
21 method: 'POST',
22 headers: { 'Content-Type': 'application/json' },
23 body: JSON.stringify({ message }),
24 });
25 // Pass the Response directly - the hook reads the stream
26 setResponse(res);
27 }
28 
29 return (
30 <div>
31 <button onClick={() => handleSend('Show me the weather in Tokyo')}>
32 Ask
33 </button>
34 
35 <div className="mt-4">
36 {phase === 'pending' && <p className="text-gray-400">Send a message to start...</p>}
37 {element}
38 {phase === 'complete' && (
39 <p className="text-green-500 text-sm mt-2">Rendering complete</p>
40 )}
41 {errors && (
42 <p className="text-red-400 text-sm mt-2">
43 Validation errors: {errors.join(', ')}
44 </p>
45 )}
46 </div>
47 </div>
48 );
49}

Wire Format

LLM output tokens are expensive. The wire format module in @genuikit/core provides a compact encoding that abbreviates common keys and component names, reducing token count by roughly 30% without any loss of information.

API

FunctionDescription
encode(data, map)Compress a component payload using an abbreviation map.
decode(data, map)Expand an abbreviated payload back to its full form.
generateWireFormatPrompt(registry)Generate the abbreviation map and system prompt instructions for the LLM.
estimateSavings(data, map)Returns the percentage of tokens saved by the compact encoding.

Before & After

Here is a side-by-side comparison showing the token savings:

wire-format-example.tstypescript
1import { encode, decode, estimateSavings } from '@genuikit/core/wire-format';
2 
3// Define your abbreviation map
4const abbreviations = {
5 WeatherCard: 'WC',
6 temperature: 'tmp',
7 condition: 'cnd',
8 humidity: 'hmd',
9 city: 'c',
10 props: 'p',
11 type: 't',
12};
13 
14// Full format (what the LLM would normally produce)
15const full = {
16 type: 'WeatherCard',
17 props: {
18 city: 'Tokyo',
19 temperature: 22,
20 condition: 'sunny',
21 humidity: 65,
22 },
23};
24// JSON: {"type":"WeatherCard","props":{"city":"Tokyo","temperature":22,"condition":"sunny","humidity":65}}
25// ~95 characters
26 
27// Compact wire format
28const compact = encode(full, abbreviations);
29// { t: "WC", p: { c: "Tokyo", tmp: 22, cnd: "sunny", hmd: 65 } }
30// JSON: {"t":"WC","p":{"c":"Tokyo","tmp":22,"cnd":"sunny","hmd":65}}
31// ~63 characters
32 
33console.log(estimateSavings(full, abbreviations));
34// { original: 95, compressed: 63, savedPercent: 33.7 }
35 
36// Decode back to full form on the client
37const restored = decode(compact, abbreviations);
38// { type: "WeatherCard", props: { city: "Tokyo", temperature: 22, condition: "sunny", humidity: 65 } }

Including Abbreviations in the System Prompt

Use generateWireFormatPrompt() to create an instruction block that tells the LLM to use the compact encoding. Include this in your system prompt alongside the tool definitions:

system-prompt.tstypescript
1import { ComponentRegistry } from '@genuikit/core';
2import { generateWireFormatPrompt } from '@genuikit/core/wire-format';
3import { registry } from './registry';
4 
5// Generate the abbreviation map and LLM instructions
6const wireFormatBlock = generateWireFormatPrompt(registry);
7// Returns a string like:
8// "Use these abbreviations when outputting component JSON:
9// WeatherCard -> WC, temperature -> tmp, condition -> cnd, ...
10// Example: {"t":"WC","p":{"c":"Tokyo","tmp":22}} instead of
11// {"type":"WeatherCard","props":{"city":"Tokyo","temperature":22}}"
12 
13const systemPrompt = `
14You are a UI assistant. Respond with component JSON.
15 
16${registry.toToolDefinition()}
17 
18${wireFormatBlock}
19`;

StreamResolver

The StreamResolver orchestrates the full streaming pipeline: reading from a source, parsing tokens, throttling updates, expanding abbreviations, and emitting events. It is the engine that powers useStreamingUI under the hood, and you can use it directly for non-React contexts or advanced customization.

Pipeline

source→parse→throttle→resolve→emit
  • source — Reads chunks from a ReadableStream, SSE connection, or WebSocket.
  • parse — Feeds chunks into StreamParser, heals incomplete JSON.
  • throttle — Batches rapid updates to limit re-renders (configurable interval).
  • resolve — Expands abbreviations and resolves the component from the registry.
  • emit — Fires snapshot, complete, or error events to listeners.

Configuration

stream-resolver-config.tstypescript
1import { StreamResolver } from '@genuikit/core';
2import { registry } from './registry';
3 
4const resolver = new StreamResolver(registry, {
5 // Minimum interval between snapshot events (ms).
6 // Lower = smoother updates, higher = fewer re-renders.
7 throttleMs: 50,
8 
9 // Wire format abbreviation map for decoding compact payloads.
10 abbreviations: {
11 WeatherCard: 'WC',
12 temperature: 'tmp',
13 condition: 'cnd',
14 humidity: 'hmd',
15 city: 'c',
16 props: 'p',
17 type: 't',
18 },
19 
20 // Retry configuration with exponential backoff.
21 retry: {
22 maxAttempts: 3,
23 baseDelayMs: 1000, // First retry after 1s
24 maxDelayMs: 10000, // Cap at 10s
25 backoffFactor: 2, // 1s -> 2s -> 4s
26 },
27});

Event Types

Subscribe to events to react to stream progress:

stream-resolver-events.tstypescript
1import { StreamResolver } from '@genuikit/core';
2import { registry } from './registry';
3 
4const resolver = new StreamResolver(registry, { throttleMs: 50 });
5 
6// Fired on each throttled update with the current partial data
7resolver.on('snapshot', (event) => {
8 console.log('Partial data:', event.data);
9 console.log('Component type:', event.componentType); // null until identified
10 console.log('Progress:', event.bytesReceived);
11});
12 
13// Fired once when the stream ends and the final payload is valid
14resolver.on('complete', (event) => {
15 console.log('Final component:', event.componentType);
16 console.log('Final props:', event.props);
17 console.log('Duration:', event.durationMs, 'ms');
18});
19 
20// Fired if the stream errors or the final payload fails validation
21resolver.on('error', (event) => {
22 console.error('Stream error:', event.error);
23 console.log('Attempt:', event.attempt, 'of', event.maxAttempts);
24 console.log('Will retry:', event.willRetry);
25});
26 
27// Start processing a stream
28const response = await fetch('/api/chat', {
29 method: 'POST',
30 body: JSON.stringify({ message: 'Weather in Tokyo' }),
31});
32 
33resolver.start(response.body!);

Real-World Example

This end-to-end example shows a Next.js API route that streams component JSON from an LLM, paired with a client component that progressively renders the result. Both the server and client code are included.

Server: API Route

The API route calls the LLM with the component tool definitions and streams the response back to the client as a text stream:

app/api/chat/route.tstypescript
1import { NextRequest } from 'next/server';
2import { registry } from '@/lib/registry';
3import { generateWireFormatPrompt } from '@genuikit/core/wire-format';
4 
5export async function POST(req: NextRequest) {
6 const { message } = await req.json();
7 
8 const wireFormatBlock = generateWireFormatPrompt(registry);
9 
10 const llmResponse = await fetch('https://api.openai.com/v1/chat/completions', {
11 method: 'POST',
12 headers: {
13 'Content-Type': 'application/json',
14 Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
15 },
16 body: JSON.stringify({
17 model: 'gpt-4o',
18 stream: true,
19 messages: [
20 {
21 role: 'system',
22 content: `You are a UI assistant. Respond ONLY with component JSON.
23Available components:
24${registry.toToolDefinition()}
25 
26${wireFormatBlock}`,
27 },
28 { role: 'user', content: message },
29 ],
30 }),
31 });
32 
33 // Transform the SSE stream into a plain text stream of JSON tokens
34 const encoder = new TextEncoder();
35 const decoder = new TextDecoder();
36 
37 const stream = new ReadableStream({
38 async start(controller) {
39 const reader = llmResponse.body!.getReader();
40 
41 try {
42 while (true) {
43 const { done, value } = await reader.read();
44 if (done) break;
45 
46 const text = decoder.decode(value, { stream: true });
47 const lines = text.split('\n').filter((line) => line.startsWith('data: '));
48 
49 for (const line of lines) {
50 const data = line.slice(6);
51 if (data === '[DONE]') continue;
52 
53 try {
54 const parsed = JSON.parse(data);
55 const token = parsed.choices?.[0]?.delta?.content;
56 if (token) {
57 controller.enqueue(encoder.encode(token));
58 }
59 } catch {
60 // Skip malformed SSE chunks
61 }
62 }
63 }
64 } finally {
65 controller.close();
66 }
67 },
68 });
69 
70 return new Response(stream, {
71 headers: {
72 'Content-Type': 'text/plain; charset=utf-8',
73 'Transfer-Encoding': 'chunked',
74 'Cache-Control': 'no-cache',
75 },
76 });
77}

Client: Streaming Component

The client component sends a message, passes the fetch response directly to useStreamingUI, and renders progressively:

components/chat-panel.tsxtsx
1'use client';
2 
3import { useState, useCallback } from 'react';
4import { useStreamingUI } from '@genuikit/react';
5import { registry } from '@/lib/registry';
6 
7export function ChatPanel() {
8 const [input, setInput] = useState('');
9 const [response, setResponse] = useState<Response | null>(null);
10 const [history, setHistory] = useState<React.ReactNode[]>([]);
11 
12 const { element, phase, errors } = useStreamingUI(registry, response, {
13 skeleton: (
14 <div className="animate-pulse space-y-3 p-4">
15 <div className="h-4 bg-gray-200 rounded w-3/4" />
16 <div className="h-4 bg-gray-200 rounded w-1/2" />
17 <div className="h-20 bg-gray-200 rounded" />
18 </div>
19 ),
20 fallback: (
21 <div className="border border-red-300 bg-red-50 rounded-lg p-4">
22 <p className="text-red-600 font-medium">Failed to render component</p>
23 {errors && <p className="text-red-400 text-sm mt-1">{errors.join(', ')}</p>}
24 </div>
25 ),
26 onComplete: (result) => {
27 // Archive the completed component into history
28 setHistory((prev) => [...prev, element]);
29 setResponse(null);
30 },
31 retry: 2,
32 });
33 
34 const handleSubmit = useCallback(async () => {
35 if (!input.trim()) return;
36 
37 const res = await fetch('/api/chat', {
38 method: 'POST',
39 headers: { 'Content-Type': 'application/json' },
40 body: JSON.stringify({ message: input }),
41 });
42 
43 setInput('');
44 setResponse(res);
45 }, [input]);
46 
47 return (
48 <div className="flex flex-col h-full">
49 {/* Message history */}
50 <div className="flex-1 overflow-y-auto p-4 space-y-4">
51 {history.map((item, i) => (
52 <div key={i} className="border rounded-lg p-4">
53 {item}
54 </div>
55 ))}
56 
57 {/* Current streaming element */}
58 {phase !== 'pending' && element && (
59 <div className="border rounded-lg p-4 border-blue-200">
60 <div className="flex items-center gap-2 mb-2">
61 <span className="text-xs text-blue-500 font-medium uppercase">
62 {phase}
63 </span>
64 {phase === 'partial' && (
65 <span className="inline-block w-2 h-2 bg-blue-400 rounded-full animate-pulse" />
66 )}
67 </div>
68 {element}
69 </div>
70 )}
71 </div>
72 
73 {/* Input */}
74 <div className="border-t p-4 flex gap-2">
75 <input
76 type="text"
77 value={input}
78 onChange={(e) => setInput(e.target.value)}
79 onKeyDown={(e) => e.key === 'Enter' && handleSubmit()}
80 placeholder="Ask for a UI component..."
81 className="flex-1 border rounded-lg px-4 py-2 focus:outline-none focus:ring-2 focus:ring-blue-500"
82 />
83 <button
84 onClick={handleSubmit}
85 disabled={phase === 'partial' || phase === 'skeleton'}
86 className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50"
87 >
88 Send
89 </button>
90 </div>
91 </div>
92 );
93}