LocalRuntime

Overview

With LocalRuntime, the chat history state is managed by assistant-ui. This gives you built-in support for thread management, message editing, reloading and branch switching.

If you need full control over the state of the messages on the frontend, use ExternalStoreRuntime instead.

assistant-ui integrates with any custom REST API. To do so, you define a custom ChatModelAdapter and pass it to the useLocalRuntime hook.

Getting Started

Create a Next.JS project

npx create-next-app@latest my-app
cd my-app

Install `@assistant-ui/react`

npm install @assistant-ui/react

Define a `MyRuntimeProvider` component

Update the MyModelAdapter below to integrate with your own custom API.

@/app/MyRuntimeProvider.tsx

"use client";
 
import type { ReactNode } from "react";
import {
  AssistantRuntimeProvider,
  useLocalRuntime,
  type ChatModelAdapter,
} from "@assistant-ui/react";
 
const MyModelAdapter: ChatModelAdapter = {
  async run({ messages, abortSignal }) {
    // TODO replace with your own API
    const result = await fetch("<YOUR_API_ENDPOINT>", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      // forward the messages in the chat to the API
      body: JSON.stringify({
        messages,
      }),
      // if the user hits the "cancel" button or escape keyboard key, cancel the request
      signal: abortSignal,
    });
 
    const data = await result.json();
    return {
      content: [
        {
          type: "text",
          text: data.text,
        },
      ],
    };
  },
};
 
export function MyRuntimeProvider({
  children,
}: Readonly<{
  children: ReactNode;
}>) {
  const runtime = useLocalRuntime(MyModelAdapter);
 
  return (
    <AssistantRuntimeProvider runtime={runtime}>
      {children}
    </AssistantRuntimeProvider>
  );
}

Wrap your app in `MyRuntimeProvider`

@/app/layout.tsx

import type { ReactNode } from "react";
import { MyRuntimeProvider } from "@/app/MyRuntimeProvider";
 
export default function RootLayout({
  children,
}: Readonly<{
  children: ReactNode;
}>) {
  return (
    <MyRuntimeProvider>
      <html lang="en">
        <body>{children}</body>
      </html>
    </MyRuntimeProvider>
  );
}

Streaming

Declare the run function as an AsyncGenerator (async *run). This allows you to yield the results as they are generated.

@/app/MyRuntimeProvider.tsx

const MyModelAdapter: ChatModelAdapter = {
  async *run({ messages, abortSignal, context }) {
    const stream = await backendApi({ messages, abortSignal, context });
 
    let text = "";
    for await (const part of stream) {
      text += part.choices[0]?.delta?.content || "";
 
      yield {
        content: [{ type: "text", text }],
      };
    }
  },
};

LocalRuntime

Overview

Getting Started

Create a Next.JS project

Install @assistant-ui/react

Define a MyRuntimeProvider component

Wrap your app in MyRuntimeProvider

Streaming

On this page

Install `@assistant-ui/react`

Define a `MyRuntimeProvider` component

Wrap your app in `MyRuntimeProvider`