# How to Stream With OpenAI's Assistant API

## Metadata

- **Published:** 3/27/2024
- **Duration:** 14 minutes
- **YouTube URL:** https://youtube.com/watch?v=JnWlw5jIzFY
- **Channel:** nerding.io

## Description

In this tutorial, we'll walk you through the process of setting up a scalable and efficient backend architecture that leverages the power of serverless computing and artificial intelligence. You'll learn how to integrate OpenAI's API with Vercel's AI package, enabling functions and tools with streaming experiences.

Course: https://forms.gle/PN3RpKD8eHsicL9Z7

📰 News & Resources: https://sendfox.com/nerdingio
📞 Book a Call: https://calendar.app.google/M1iU6X2x18metzDeA

🎥 Chapters
00:00 Introduction
00:20 New Release
01:44 Review Streaming
03:45 Home Automation Example
04:18 OpenAI Assistant Setup
06:53 Vercel AI
08:10 Streaming Example
10:37 Frontend
12:01 OpenAI Functions
14:27 Conclusion

🔗 Links
https://github.com/vercel/ai/releases
https://github.com/vercel/ai/blob/main/examples/next-openai/app/api/assistant/assistant-setup.md
https://platform.openai.com/assistants

⤵️ Let's Connect
https://everefficient.ai
https://nerding.io
https://twitter.com/nerding_io
https://www.linkedin.com/in/jdfiscus/
https://www.linkedin.com/company/ever-efficient-ai/

## Key Highlights

### 1. Streaming OpenAI Assistant API

Vercel AI package now supports streaming with the OpenAI Assistant API, enabling real-time responses and interactions.

### 2. Forward Stream Type & Use Agent Hook

New features include a forward stream response type and the ability to use the use agent hook for set messages, providing more control and type safety.

### 3. Home Automation Assistant Example

The video demonstrates a practical example using a home automation assistant to control room temperature, showcasing function calling.

### 4. Error Handling with Use Assistant

The `useAssistant` hook allows for graceful error handling, enabling the front end to respond appropriately to issues like 429 errors.

### 5. Data Role for Structured Responses

Returning data with a 'data' role allows for specific styling and presentation of function results within the chat interface.


## Summary

## Video Summary: How to Stream With OpenAI's Assistant API

**1. Executive Summary:** This tutorial demonstrates how to leverage Vercel AI's new streaming capabilities for OpenAI's Assistant API, enabling real-time and interactive experiences with custom tools and functions. The video provides a practical example of a home automation assistant, walking through setup, code implementation, and error handling.

**2. Main Topics Covered:**

*   **Introduction to Vercel AI's Streaming for OpenAI Assistants:** Discusses the new features released in the Vercel AI package, focusing on streaming, forward stream response type, and the `useAgent` hook.
*   **OpenAI Assistant Setup:** Guides users through setting up an assistant in the OpenAI platform, including instructions, function definitions (using JSON schemas), and obtaining the assistant ID.
*   **Vercel AI Implementation:** Explains how to clone the Vercel AI repository, install dependencies, and configure environment variables (OpenAI API key and Assistant ID).
*   **Streaming Example (Home Automation Assistant):** Demonstrates a practical application involving a home automation assistant that controls room temperature, showcasing function calling and data handling.
*   **Frontend Implementation and Error Handling:** Discusses the frontend code using the `useAssistant` hook, highlighting its ability to handle errors (e.g., 429 errors) gracefully and providing appropriate responses to the user.
*   **Function Calling and Data Role:** Explains how to define and call functions in the OpenAI assistant, and how to use the 'data' role to style and present function results within the chat interface.

**3. Key Takeaways:**

*   Vercel AI's streaming support for OpenAI Assistants enables real-time responses and interaction with custom tools.
*   The `useAssistant` hook allows for easier integration, error handling, and customization of the frontend experience.
*   Function calling enables the assistant to interact with external APIs and perform specific tasks.
*   The 'data' role is crucial for properly formatting and displaying function results in the chat interface.
*   The video provides a clear step-by-step guide to setting up and implementing a functional streaming OpenAI assistant with Vercel AI.

**4. Notable Quotes or Examples:**

*   **On the new patch:** "Instead of the message context we're actually pulling in this forward stream so our data is kind of staying the same but instead of our run we're actually running a stream."
*   **On Function Definition:** Explanation of the JSON schemas used to define functions like `get_room_temperature` and `set_room_temperature`, including enum-based validation.
*   **On the benefits of `useAssistant`:** "What's really interesting about this is if it fails say you get a 429 or something like that or some other error you could actually capture that and have your front end respond with an appropriate response."
*   **On the 'data' role:** "If the role is data then what we're going to do is a pre- element and actually put the JSON stringified result in here... that's how we're getting our our uh styling around it."

**5. Target Audience:**

*   Developers interested in building AI-powered applications with OpenAI's Assistant API.
*   Frontend and backend engineers looking to implement streaming capabilities in their applications.
*   Individuals familiar with JavaScript/TypeScript, React, Next.js, Vercel, and OpenAI.
*   Developers seeking a practical example of integrating OpenAI Assistants with custom tools and functions.


## Full Transcript

hey everyone welcome to nerding IO I'm JD and today we're going to be going through the open AI assistant API specifically leveraging streaming which was released in the versel AI package we're going to go through a home assistant example and then we're going to dive into some of the more specific specifics around streaming all right so the first thing we're going to do is we're actually going to be looking at the uh releases and so you can kind of see here through the versel releases of the AI package we're looking at like Canary but then the also the uh patches so this is last week I know I've been uh away for a little bit again I've been working through a lot of different AI agents and a lot more audio um and so I found this really interesting that uh it was coming out with the the ability for open AI assistance to be streamed this is super important when you start dealing with a lot of different tooling and being able to re reach out to internal apis the other pieces that have come out are also the fact that um the they specify a forward stream type now and the ability to uh use the use agent hook for set messages they're exposing that so what we're going to do is we're going to focus on the um the first part which is which is the uh streaming portion we're actually going to look at each one of these patches and then just kind of go through the code and then we'll actually build out the example that they have so if we go ahead and we look at the streaming patch there's a a few different things a lot of it's up here is their documentation but if you click on the route underneath the examples and then the app API assistant you'll see a couple of different things so we're still using the same experimental assistant response but what's happening now is instead of the message context we're actually pulling in this forward stream so our data is kind of staying the same but instead of our run we're actually running a stream and it's in here in this thread of the runs you can actually see this create and stream so it's really just one function change but then down below is where all of the uh actions are starting to happen as well as the ability to call out to different tools so not only are we getting the stream coming through but we're actually being able to while uh uh the C like our status is here we're actually able to then look and execute on our different functions so in here we have like the get room temperature so this is incredibly important and like I said we're actually going to run through the code of the example but I just wanted to show that this is the implementation that they started or that they just put in the patch the next piece what for the patch was the uh the response and so now what you have is instead of just a void or an any you actually have the uh response type for forward stream and a run so it actually knows the particular type next there's the so now what we're going to do is we're actually going to go into the example and then we'll start going through the instructions of setting this up so the first thing to pay attention to is we're in the AI uh package of versel we're going to clone this down and we're going to go into setting up the example for the assistant out and so what they're doing in this one is they're creating a uh home automation assistant and what it's going to be doing so we have our prompt and we have our temperature and then we're going to be able to set our room temperature and so what we're going to do now is we're going to go ahead and jump into open Ai and get this set up real quick everyone if you haven't already please remember to like And subscribe it helps more than you know we also have a survey down below where it's actually going to be an upcoming course on building privatized wh labeled business assistants or AI agents all right let's get back to it so now we're in our platform open Ai and we just went to our assistance tab remember you have to have a Plus account for this we're just going to go ahead and click create and what we'll do is we'll just call this uh stream example uh we can go back over to here and what we're going to do is we're going to grab the uh the uh prompt for the instruction we're going to go here and paste that in and then we're going to grab our functions so if you haven't done a function before really it's just a uh a JavaScript or a Json blob but you come down here you click add function you go ahead and paste that in and we'll click save what it's doing is it's basically telling you that these rooms are an enum of the property that uh you're able to grab so and that you have to post that information and then in order to set the room it's going to do the same thing it's requiring uh the room as an enum but then it also Al has the temperature that you want to set that room to that's where you have required room and temperature so we're going to go back to the assistant and add another function and we'll go ahead and save and you don't have to do this but we're just going to put it up to gbt 4 for now just in case um you can test it if you want but really you need your assistant ID in order to set this up inverse else so we'll go ahead and copy that okay so one what we're going to do first you need to clone this repo so you need to clone the AI versel repo and then once you do that you're going to go ahead and put in the uh you can either use pnpm pnpm yeah or npm uh and then you're going to want to go but you're going to want to go in this folder and then we can launch this so what I did is I I pulled this up so I've Clon this down into VSS code and I'm just kind of looking at the AI folder in the examples folder and then next open AI folder and this has all the different examples that we're going to look at once I've cded into that I just ran the uh the Run command for PNP pnpm or npm the other thing you're going to want to do is you're going to want to put in your open AI key as well as your assistant ID remember this is what we copied into our clipboard from the open AI platform next uh so now that we have that and we have this running we're actually just going to go to the page and actually see what's happening and if we come over here just remember that it's Local Host and then the assistant URI so now that we are in this we're going to take a look at we'll go ahead and clear these we're going to take a look at our Network hopefully you can see this um I'll try and make a little little bigger uh and then we'll see what we can do so we're just going to type in here 76 Dees and we'll see what we get and so we can see that our feed is going out we're actually getting an error and that's perfect because remember in order to set the temperature uh we want to we have to pass it the room so it's telling us we need to say which room we're ref referring to and is this in Celsius so let's uh do this again and we'll just say what is the temperature or in the bedroom and I know there's a spelling error though I just want to see what's coming back so first we tried to set a degree and now it's saying it's 76 in the the bedroom so now if we want to actually change it what we can do again this isn't connected to a a database or a API so but it is remembering context in our thread so the way the assistants work is you have um your your assistant and then your thread and then your runs and your messages so what we're going to do is we'll just try and set this in the living room and we'll just say we want it to be uh 65 F because I'm in the states degrees in the living room and now we should actually get a object coming back and so there we go we actually get our object this is actually coming from the uh the tool itself so we're actually setting it but we're seeing that our data is the temp in the living room is I believe that's in Celsius now which is pretty awesome that it's converting and uh it gives us the description as well as the new temperature and the old temperature again that is because in our object here we're setting these principles or properties right so we have our number and then it's making an association between the new temperature and the old temperature so let's take a look at the page so if we go back to VSS code and we look at our page we're doing a couple different interesting things we've got our experimental use assistant remember this was also kind of added and it's referencing use assistant so if we want to see what that does can hover over and it's saying we're defining our API we're checking we can look or def predefine our thread ID and then we can also pass credentials headers and B body and then the way to do an on error so what's really interesting about this is if it fails say you get a 429 or something like that or some other error you could actually capture that and have your front end respond with an appropriate response so not only are we able to like d debug this but we're able to graceful uh do a graceful fail so use this use assistant is super helpful what we're in here we're just assigning our API to assistant and we can look this is the code we were looking at earlier of our route where it's defining our temperature it's posting this information it is getting the thread ID and then or or creating one and then it's creating the message and then we're actually running this through a stream here's our run result our run result of our stream and then we're passing this information so right here when we get to the tools and I probably should have blown this up a little bit more sorry so when we get to our tools what we're doing is we're able to map this information and then check based on the function and the function name which function should we actually call in our open AI assistant function this function could technically be a call to another AP I it could be a call to a database it could be any number of things in here we're just defining it as this is our home temperature array and what are the parameters that we want assigned to it and then we're actually sending that information back and this is the the object that we saw getting printed and we're saying it's a role of data why is this important well if we go back to our page the assistant page we can actually see down here that uh a few different things so again if we have an error we have our error here but then we also have our map so this is all the messages that we're getting and if you notice it's saying based on the role what are we doing if the role is not data then just show the streamed content if the role is data then what we're going to do is a pre- element and actually put the J and stringified result in here so when we're actually returning the data back from the function that is actually getting inserted into our uh display in our message and we're showing it a different way remember though that every single one of these are in our messages of our thread so it's just showcasing that this is a data roll and that's how we're getting our our uh styling around it so if we look back here that's how we can tell that this object needs to go inside of this object needs to go inside of this uh pre element and so again this is super super helpful now that we're doing streaming specifically on assistance but the ability to actually look at this and see a function in the act in action with the uh the stream response which we can see here all right everyone that's it for today what we covered was the open AI assistance API specifically looking at versel AI package that allows us to do streaming this is incredibly beneficial when looking at things for custom functionality or being able to reach out to external apis again if you haven't already please remember to like And subscribe and don't forget the link below for the survey for our upcoming course happy nerding

---

*Generated for LLM consumption from nerding.io video library*