Frontend Architecture for GenAI: Why Your React Patterns Don't Work Anymore

The Traditional Frontend Playbook Is Broken

You've built dozens of React applications. You know how to manage forms, handle API calls, optimize renders, and structure state. Then you integrate an LLM API and suddenly none of your patterns work. Your component re-renders 50 times per second. Your error boundaries don't catch mid-stream failures. Users navigate away and leave zombie connections consuming tokens. Your carefully optimized bundle now includes streaming parsers, token counters, and connection lifecycle managers you've never needed before.

Figure: Broken Traditional Frontend Playbook

The problem isn't your skills—it's that GenAI fundamentally changed what frontends do. Traditional web apps are request-response systems with discrete state transitions. You submit a form, show a spinner, display the result. State is ephemeral. Errors are atomic. Retries are simple. GenAI apps are real-time streaming systems with continuous state evolution. Tokens arrive at 50Hz. Responses are partial and incremental. Errors happen mid-generation. Connection state matters as much as application state.

This isn't about learning a new library or framework. It's about recognizing that the architectural assumptions underlying your existing patterns—synchronous data flow, bounded response times, stateless requests—no longer hold. You're building a different kind of application that requires different architectural thinking.

Mental Model: Frontend as Stream Consumer, Not Request Initiator

The core mental shift is understanding that your frontend is no longer the active party that requests data and waits for a response. It's a passive consumer that subscribes to a data stream, processes events as they arrive, and maintains state across a long-lived connection. This inverts the traditional control flow.

Figure: Frontend as Stream Consumer, Not Request Initiator

In request-response architecture, the frontend controls timing. You send a request when the user clicks submit. You know when the response arrives because the promise resolves. You control when to show loading states, when to update the UI, when to clear state. The lifecycle is deterministic: idle, loading, success, or error.

In streaming architecture, the backend controls timing. It sends tokens whenever the LLM generates them—which might be 10ms apart or 500ms apart depending on model load. Your frontend reacts to events as they arrive. You don't control when tokens come, only how you handle them when they do. The lifecycle is continuous: connecting, streaming, potentially paused by backpressure, eventually completing or erroring.

This mental model affects every architectural decision. State management isn't about "what's the current value" but "what's the accumulated value so far and is the stream still active." Error handling isn't about "did the request fail" but "at what point did the stream fail and what partial data did we successfully receive." Memory management isn't about "clean up after the component unmounts" but "ensure streams are explicitly terminated even when users navigate mid-generation."

The key insight: you're building a real-time system that happens to use HTTP as the transport layer. Treat it like WebSocket communication or video streaming, not like REST API calls. This framing clarifies why your traditional patterns feel inadequate—you're using synchronous tools for asynchronous problems.

Understanding this distinction also reveals why certain patterns are necessary. Connection lifecycle management, explicit stream termination, state accumulation rather than replacement, render batching, and partial response recovery—these aren't optional optimizations. They're fundamental requirements for systems where data arrives continuously and connections persist across time.

Architecture: Eight Layers of Frontend Complexity

GenAI frontends require eight distinct architectural layers that traditional applications either don't need or handle trivially. Each layer has its own state, error modes, and performance characteristics.

Architecture: Eight Layers of Frontend Complexity (Open image in new tab, to see the deep details.)

Layer 1: Connection Manager Layer

Handles stream lifecycle from initialization through termination. Tracks connection state, implements reconnection logic, manages timeouts, and ensures cleanup on navigation. This layer is what distinguishes streaming from REST—you need explicit connection state management.

Layer 2: Stream Parser Layer

Transforms raw byte streams into structured events. Handles SSE parsing, manages partial event buffering, deals with protocol-specific quirks like heartbeat events. Without this, you're processing malformed data.

Layer 3: State Accumulator Layer

Aggregates tokens into complete responses. Maintains conversation history, handles multi-turn context, implements optimistic updates. Traditional state management treats updates as replacements—this treats them as accumulations.

Layer 4: Render Optimizer Layer

Batches state updates to prevent render storms. Implements debouncing, throttling, or frame-based updates. Without this, 50 token updates per second means 50 component renders per second, killing browser performance.

Layer 5: Token Counter and Cost Tracker Layer

Monitors token consumption in real-time, calculates costs, enforces usage limits. This is frontend responsibility now because billing happens per token, not per request. Users need live feedback on consumption.

Layer 6: Context Manager Layer

Tracks conversation windows, manages message threading, handles context pruning when limits are approached. This moves backend concerns to the frontend because users need immediate feedback about context state.

Layer 7: Error Boundary Layer

Handles mid-stream failures, rate limit responses, partial recovery scenarios. Traditional error boundaries catch render errors—these need to catch connection failures, stream interruptions, and protocol violations.

Layer 8: Memory Manager Layer

Ensures connection cleanup on navigation, implements explicit stream termination, prevents memory leaks from abandoned streams. SPAs don't automatically clean up long-lived connections—you must do it explicitly.

Each layer has failure modes that cascade. A missing memory manager causes connection leaks. A naive state accumulator causes render storms. An inadequate error boundary leaves users staring at frozen UIs. You can't skip layers or handle them as afterthoughts—they're all load-bearing.

Implementation: Concrete Patterns for Each Layer

Production GenAI frontends need specific implementation patterns for each architectural layer. Here's what actually works.

Connection Manager with Explicit Lifecycle

// hooks/useStreamConnection.tsimport { useRef, useCallback, useEffect } from 'react';type ConnectionState = 'idle' | 'connecting' | 'connected' | 'streaming' | 'closed' | 'error';export function useStreamConnection() {  const abortControllerRef = useRef<AbortController | null>(null);  const reconnectAttemptsRef = useRef(0);  const maxReconnects = 3;    const connect = useCallback(async (url: string, onEvent: (event: any) => void) => {    // Cancel existing connection    abortControllerRef.current?.abort();        const controller = new AbortController();    abortControllerRef.current = controller;        try {      const response = await fetch(url, {        method: 'POST',        signal: controller.signal,        headers: { 'Accept': 'text/event-stream' }      });            if (!response.ok) {        throw new Error(`HTTP ${response.status}`);      }            const reader = response.body!.getReader();      const decoder = new TextDecoder();            while (!controller.signal.aborted) {        const { done, value } = await reader.read();        if (done) break;                const text = decoder.decode(value);        onEvent({ type: 'data', data: text });      }            reconnectAttemptsRef.current = 0; // Reset on successful completion          } catch (error: any) {      if (error.name === 'AbortError') {        return; // Expected termination      }            // Attempt reconnection with exponential backoff      if (reconnectAttemptsRef.current < maxReconnects) {        reconnectAttemptsRef.current++;        const delay = Math.pow(2, reconnectAttemptsRef.current) * 1000;                await new Promise(resolve => setTimeout(resolve, delay));        return connect(url, onEvent); // Recursive reconnect      }            throw error;    }  }, []);    const disconnect = useCallback(() => {    abortControllerRef.current?.abort();    abortControllerRef.current = null;    reconnectAttemptsRef.current = 0;  }, []);    // Cleanup on unmount - critical for SPAs  useEffect(() => {    return () => {      disconnect();    };  }, [disconnect]);    return { connect, disconnect };}

This pattern handles reconnection with backoff, explicit cleanup, and abort controller management. The key is treating connections as resources that need lifecycle management, not fire-and-forget operations.

State Accumulator with Render Batching

// hooks/useStreamingState.tsimport { useState, useRef, useCallback, useEffect } from 'react';export function useStreamingState() {  const [displayContent, setDisplayContent] = useState('');  const bufferRef = useRef('');  const updateTimerRef = useRef<NodeJS.Timeout | null>(null);  const rafRef = useRef<number | null>(null);    const appendToken = useCallback((token: string) => {    bufferRef.current += token;        // Batch updates using requestAnimationFrame    if (!rafRef.current) {      rafRef.current = requestAnimationFrame(() => {        setDisplayContent(bufferRef.current);        rafRef.current = null;      });    }  }, []);    const flush = useCallback(() => {    if (rafRef.current) {      cancelAnimationFrame(rafRef.current);      rafRef.current = null;    }    setDisplayContent(bufferRef.current);  }, []);    const reset = useCallback(() => {    bufferRef.current = '';    setDisplayContent('');    if (rafRef.current) {      cancelAnimationFrame(rafRef.current);      rafRef.current = null;    }  }, []);    // Cleanup on unmount  useEffect(() => {    return () => {      if (rafRef.current) {        cancelAnimationFrame(rafRef.current);      }    };  }, []);    return {    content: displayContent,    appendToken,    flush,    reset  };}

This batches rapid token updates to match the display refresh rate (typically 60Hz). Without batching, 50 token updates per second causes 50 React renders, 50 virtual DOM diffs, and 50 DOM updates—completely unnecessary when humans can't perceive updates faster than 16ms anyway.

Token Counter and Cost Tracker

// hooks/useTokenTracking.tsimport { useState, useCallback, useRef } from 'react';interface TokenMetrics {  inputTokens: number;  outputTokens: number;  totalTokens: number;  estimatedCost: number;  tokensPerSecond: number;}const PRICING = {  'gpt-4': { input: 0.03, output: 0.06 }, // per 1K tokens  'claude-3-opus': { input: 0.015, output: 0.075 }};export function useTokenTracking(model: keyof typeof PRICING) {  const [metrics, setMetrics] = useState<TokenMetrics>({    inputTokens: 0,    outputTokens: 0,    totalTokens: 0,    estimatedCost: 0,    tokensPerSecond: 0  });    const startTimeRef = useRef<number>(Date.now());  const tokenCountRef = useRef(0);    const trackToken = useCallback(() => {    tokenCountRef.current++;        const elapsed = (Date.now() - startTimeRef.current) / 1000;    const tps = elapsed > 0 ? tokenCountRef.current / elapsed : 0;        const pricing = PRICING[model];    const cost = (tokenCountRef.current / 1000) * pricing.output;        setMetrics({      inputTokens: 0, // Set from initial prompt      outputTokens: tokenCountRef.current,      totalTokens: tokenCountRef.current,      estimatedCost: cost,      tokensPerSecond: tps    });  }, [model]);    const reset = useCallback(() => {    tokenCountRef.current = 0;    startTimeRef.current = Date.now();    setMetrics({      inputTokens: 0,      outputTokens: 0,      totalTokens: 0,      estimatedCost: 0,      tokensPerSecond: 0    });  }, []);    return { metrics, trackToken, reset };}

This gives users real-time cost feedback. Critical for applications where users can rack up hundreds of dollars in a single session if they're not careful. The frontend needs to show cost accumulation live, not just in a billing dashboard later.

Context Window Manager

// hooks/useContextWindow.tsimport { useState, useCallback, useMemo } from 'react';interface Message {  role: 'user' | 'assistant';  content: string;  tokens: number;}export function useContextWindow(maxTokens: number = 8000) {  const [messages, setMessages] = useState<Message[]>([]);    const totalTokens = useMemo(() => {    return messages.reduce((sum, msg) => sum + msg.tokens, 0);  }, [messages]);    const remainingTokens = maxTokens - totalTokens;  const utilizationPercent = (totalTokens / maxTokens) * 100;    const addMessage = useCallback((message: Message) => {    setMessages(prev => {      const newMessages = [...prev, message];      let totalTokenCount = newMessages.reduce((sum, msg) => sum + msg.tokens, 0);            // Prune oldest messages if over limit      while (totalTokenCount > maxTokens && newMessages.length > 2) {        // Keep system message and most recent        const removed = newMessages.splice(1, 1)[0];        totalTokenCount -= removed.tokens;      }            return newMessages;    });  }, [maxTokens]);    const clearHistory = useCallback(() => {    setMessages([]);  }, []);    return {    messages,    totalTokens,    remainingTokens,    utilizationPercent,    addMessage,    clearHistory,    isNearLimit: utilizationPercent > 80  };}

This manages conversation context automatically. When users approach token limits, the UI can show warnings, suggest pruning old messages, or automatically trim history. Without this, users hit context limits unexpectedly and don't understand why the LLM suddenly refuses their request.

Progressive Markdown Renderer

// components/StreamingMarkdown.tsx'use client';import { useMemo } from 'react';import ReactMarkdown from 'react-markdown';interface StreamingMarkdownProps {  content: string;  isStreaming: boolean;}export function StreamingMarkdown({ content, isStreaming }: StreamingMarkdownProps) {  // Only render complete markdown blocks during streaming  const renderableContent = useMemo(() => {    if (!isStreaming) return content;        // Find last complete block (paragraph, code fence, heading)    const lines = content.split('\n');    let lastCompleteIndex = 0;    let inCodeBlock = false;        for (let i = 0; i < lines.length; i++) {      const line = lines[i];            // Track code fences      if (line.trim().startsWith('```')) {        inCodeBlock = !inCodeBlock;        if (!inCodeBlock) {          lastCompleteIndex = i + 1; // Include closing fence        }      } else if (!inCodeBlock && (line.trim() === '' || i === lines.length - 1)) {        lastCompleteIndex = i;      }    }        return lines.slice(0, lastCompleteIndex).join('\n');  }, [content, isStreaming]);    const partialContent = isStreaming     ? content.slice(renderableContent.length)    : '';    return (    <div className="prose max-w-none">      <ReactMarkdown>{renderableContent}</ReactMarkdown>      {partialContent && (        <span className="text-gray-600">          {partialContent}          <span className="inline-block w-2 h-4 bg-gray-800 animate-pulse ml-1" />        </span>      )}    </div>  );}

This prevents markdown rendering from breaking on partial code blocks or incomplete lists. During streaming, only complete markdown structures are rendered. Partial content appears as plain text with a cursor. This avoids flickering as the markdown parser tries to interpret incomplete syntax.

Error Boundary for Stream Failures

// components/StreamErrorBoundary.tsx'use client';import { Component, ReactNode } from 'react';interface Props {  children: ReactNode;  onError?: (error: Error, errorInfo: any) => void;}interface State {  hasError: boolean;  error: Error | null;  errorType: 'connection' | 'stream' | 'rate-limit' | 'unknown';}export class StreamErrorBoundary extends Component<Props, State> {  constructor(props: Props) {    super(props);    this.state = { hasError: false, error: null, errorType: 'unknown' };  }    static getDerivedStateFromError(error: Error): State {    // Classify error type    let errorType: State['errorType'] = 'unknown';        if (error.message.includes('rate limit')) {      errorType = 'rate-limit';    } else if (error.message.includes('connection') || error.name === 'AbortError') {      errorType = 'connection';    } else if (error.message.includes('stream')) {      errorType = 'stream';    }        return { hasError: true, error, errorType };  }    componentDidCatch(error: Error, errorInfo: any) {    this.props.onError?.(error, errorInfo);  }    render() {    if (this.state.hasError) {      return (        <div className="p-4 bg-red-50 border border-red-200 rounded">          <h3 className="font-semibold text-red-800">            {this.getErrorTitle()}          </h3>          <p className="text-red-600 mt-2">            {this.getErrorMessage()}          </p>          <button            onClick={() => this.setState({ hasError: false, error: null })}            className="mt-4 px-4 py-2 bg-red-600 text-white rounded"          >            Retry          </button>        </div>      );    }        return this.props.children;  }    private getErrorTitle(): string {    switch (this.state.errorType) {      case 'rate-limit':        return 'Rate Limit Exceeded';      case 'connection':        return 'Connection Failed';      case 'stream':        return 'Stream Interrupted';      default:        return 'An Error Occurred';    }  }    private getErrorMessage(): string {    switch (this.state.errorType) {      case 'rate-limit':        return 'Too many requests. Please wait a moment before trying again.';      case 'connection':        return 'Unable to establish connection. Check your network and retry.';      case 'stream':        return 'The response stream was interrupted. Partial response may be available.';      default:        return this.state.error?.message || 'Something went wrong.';    }  }}

This provides user-friendly error messages for different failure types. Rate limits need different messaging than network failures. Stream interruptions should preserve partial responses if possible.

Pitfalls and Failure Modes

Render Storms from Naive State Updates

The most common mistake is calling setState on every token arrival. At 50 tokens per second, this means 50 React renders per second. The browser can't keep up. UI becomes laggy. Memory usage spikes. Eventually, the tab freezes.

Detection: open DevTools performance profiler during streaming. If you see continuous render cycles consuming >80% of frame time, you have a render storm. Solution: batch updates with requestAnimationFrame or debounce with fixed intervals (50-100ms).

Memory Leaks from Unclosed Streams

Users navigate away mid-generation. The component unmounts but the fetch request continues. The stream stays open, consuming memory and LLM tokens. In SPAs where users navigate frequently, this accumulates dozens of zombie connections.

Detection: monitor network tab for streams that continue after navigation. Check browser memory profiler for growing heap even when idle. Solution: always implement cleanup in useEffect return functions. Use AbortController to cancel fetch requests on unmount.

Context Window Overflow Without Warning

Users have multi-turn conversations. Each turn adds tokens. Eventually they hit the context limit. The LLM request fails with a cryptic error. Users don't understand why—they just see "request failed."

Detection: track token counts across messages. Calculate remaining context. Solution: show context utilization meter in UI. Warn at 80% capacity. Auto-prune old messages or prompt users to clear history before hitting limits.

Partial Response Loss on Error

Stream fails after receiving 500 tokens. Traditional error handling clears state and shows an error message. The user loses the partial response that might have been useful.

Detection: check if error handlers wipe state unconditionally. Solution: preserve partial content on stream errors. Show error message alongside partial response. Give users option to retry or accept partial result.

Cost Explosion from Background Streaming

User opens multiple tabs or leaves tabs open. Each tab maintains its own stream. Forgotten tabs continue consuming tokens. User racks up hundreds of dollars before noticing.

Detection: implement cost tracking per session. Alert when cost exceeds thresholds. Solution: show persistent cost indicator in UI. Implement page visibility API to pause streams when tab is hidden. Auto-terminate streams after inactivity timeout.

Broken Markdown Rendering During Streaming

Markdown parser tries to render incomplete code blocks. UI flickers as it alternates between code block formatting and plain text. Lists break when only partial items have arrived.

Detection: watch for UI flickering during streaming. Check if markdown elements appear and disappear. Solution: only render complete markdown blocks during streaming. Hold partial blocks as plain text until closing delimiters arrive.

Summary and Next Steps

GenAI frontends require fundamentally different architectural patterns than traditional web applications. The shift from request-response to streaming inverts control flow, makes connection state explicit, and requires eight distinct architectural layers: connection management, stream parsing, state accumulation, render optimization, token tracking, context management, error boundaries, and memory management.

The key insights: treat streaming as a protocol, not an HTTP response variant—implement explicit lifecycle management. Batch renders to prevent storms—50 token updates per second needs batching, not 50 React renders. Track tokens and cost in real-time—users need live feedback on consumption. Manage context windows explicitly—don't wait for backend errors to tell users they're over limits. Preserve partial responses on errors—500 tokens of useful output is better than nothing. Clean up connections on navigation—SPAs don't automatically terminate streams.

Next steps for production: implement comprehensive observability for stream health—track connection duration, token throughput, error rates, and memory growth. Build A/B testing infrastructure to measure streaming UX impact—does it improve engagement or just feel faster. Add intelligent prefetching for multi-turn conversations—predictively load context before users need it. Implement graceful degradation—fall back to polling or long-polling when SSE fails. Build cost prediction models—warn users before expensive operations. Create stream resumption for network interruptions—reconnect and continue from last token rather than restarting.

The patterns described here work for current LLM APIs but will need adaptation as models get faster, responses get longer, and multi-modal streaming becomes common. Stay focused on fundamentals: explicit state management, connection lifecycle control, progressive rendering, and user feedback. These principles survive API changes.

Comments

Following is the valuable comments/questions from Yuvraj Shivaji Dhepe

Question 1:

For the first 4 layers, will using CopilotKit be helpful, it's something like frontend for such agentic systems, they have major integrations with agentic frameworks like Agno, LangGraph etc.

Answer

CopilotKit for Layers 1-4

Answer is Yes and No (It depends)

Yes, CopilotKit abstracts away significant complexity such as connection management, stream parsing, state accumulation, some render optimization and provides React hooks..

When it's worth it:

• You're building standard chat interfaces
• You want rapid prototyping
• Your frontend team isn't deep in streaming internals
• You're okay with opinionated patterns (Meaning that you don't want significant deviations from what is already there in terms of user user experience/journey).

When you'll outgrow it:

• Custom streaming protocols (not standard SSE/WebSocket)
• Fine-grained control over reconnection logic
• Non-chat UIs (streaming dashboards, collaborative editors)
• Performance optimization beyond what CopilotKit provides
• Multi-provider support with custom handling
• Enterprise environment where genAI is just one part of the overall systems

CopilotKit is essentially "batteries included" for the first 4 layers. If it fits your use case, use it. If you need custom behavior, you'll implement those layers yourself anyway.

Question 2:

For layer 5, 6: I was wondering handling these as a common state between frontend and backend. So a websocket connection where backend just updates: context length left/filled, token counter. Will this not be optimal, in cases when there are multiple teams involved: For example in my case our team is involved in agentic part and frontend team would receive all the LLM related information from the backend itself.

Answer

Backend-as-State-Source via WebSocket (Layers 5-6)

Yes, this is not just optimal, it's the correct architecture for multi-team setups.

Your approach as I understood can be described by the following diagram:

Figure: Backend-as-State-Source via WebSocket

Backend computes:

• Token counts (actual, not estimated)
• Context utilization
• Cost accumulation
• Remaining capacity

Middleware maintains:

• Current state snapshot
• Historical metrics
• Per-session tracking

Frontend subscribes:

• Reactive updates when state changes
• No LLM logic required
• Just display what middleware provides

Why this is optimal:

1. Clean separation of concerns

   • Agentic team: owns token logic, context management, LLM interactions
   • Frontend team: displays UI, no domain knowledge needed
   • Middleware: state synchronization layer

2. Single source of truth

   • Backend has actual token counts from provider APIs
   • Frontend can't drift out of sync with reality
   • No "frontend estimates vs backend actual" mismatches

3. Real-time synchronization

   • WebSocket pushes updates immediately
   • Frontend always shows current state
   • No polling, no stale data

4. Scalability

   • Multiple frontend clients can subscribe to same state
   • Middleware can broadcast to all connected clients
   • Backend doesn't care how many frontends exist

Implementation pattern:

// Backend sends via WebSocket{  type: "metrics_update",  session_id: "sess_123",  data: {    tokens: {      input: 245,      output: 1247,      total: 1492,      limit: 8000,      remaining: 6508    },    context: {      messages: 12,      tokens_used: 1492,      utilization_percent: 18.7,      can_add_message: true    },    cost: {      current_session: 0.045,      estimated_next_message: 0.003    }  }}// Frontend just consumesconst [metrics, setMetrics] = useState(null);useEffect(() => {  ws.on('metrics_update', (data) => {    setMetrics(data);  });}, []);// Display<div>  <p>Tokens: {metrics.tokens.total} / {metrics.tokens.limit}</p>  <ProgressBar value={metrics.context.utilization_percent} />  <p>Cost: ${metrics.cost.current_session}</p></div>

This eliminates frontend complexity:

• No token counting logic
• No context window calculations
• No cost estimation formulas
• No model-specific knowledge
• Just subscribe and render

For your multi-team scenario, this is ideal:

• Agentic team controls the state computation
• Frontend team has simple reactive UI
• Clear contract: WebSocket message schema
• Teams can work independently

The middleware/state manager can be:

• Part of your backend (simple approach)
• Separate service (if you need caching, replay, etc.)
• Redis/similar with pub/sub (for multi-instance backends)

Your instinct is correct - backend-managed state via WebSocket is the right pattern when teams are separated and frontend shouldn't have LLM domain logic.

Question 3:

For layer 7, 8: Again I think this heavy lifting is done via CopilotKit, but haven't looked under the hood?

Answer

CopilotKit for Layers 7-8

Layer 7 (Error Boundaries): CopilotKit handles basic error boundaries but not production-grade classification. It catches stream failures but doesn't differentiate:

• Rate limits (needs backoff UI)
• Context overflow (needs pruning suggestion)
• Network interruption (needs retry)
• Provider outage (needs fallback)

You'll still need custom error boundaries for sophisticated error handling.

Layer 8 (Memory Management): CopilotKit does handle cleanup on unmount. This part works well. Their hooks properly abort connections and clean up listeners.

What CopilotKit doesn't handle:

• Cross-tab coordination (multiple tabs streaming)
• Partial response recovery on error
• Custom reconnection strategies
• Stream prioritization under load

For comments/feedback on the article and to connect with me for more insights on Agentic AI systems, security, and practical AI engineering:

Related Articles

• Building ChatGPT-Style Streaming in React: FastAPI + Next.js Production Guide
• Playwright + AI: The Ultimate Testing Power Combo
• Stop Pasting Screenshots: How AI Engineers Document Systems with Mermaid

Follow for more technical deep dives on AI/ML systems, production engineering, and building real-world applications: