# Use ANY language in LangSmith with REST

## Metadata

- **Published:** 9/6/2023
- **Duration:** 21 minutes
- **YouTube URL:** https://youtube.com/watch?v=7BL0GEdMmgY
- **Channel:** nerding.io

## Description

Hey everyone, welcome to Nerding IO. In this video we will be diving into Lang Chain and exploring the Lang Smith cookbook. We'll be looking at different examples, including Next.js and a REST-based example. I'll guide you through the process of setting up a chain, running a REST API example, and exploring different runs. We'll also cover logging LLM runs and testing for errors. Join me on this exciting journey into Lang Chain!

📰 FREE eBooks & News: https://sendfox.com/nerdingio
👉🏻  Ranked #1 Product of the Day:  https://www.producthunt.com/posts/ever-efficient-ai
📞 Book a Call: https://calendar.app.google/M1iU6X2x18metzDeA

🎥 Chapters
00:00 Introduction
03:21 Setting up LangSmith and running a REST API example
08:50 Logging LLM runs
12:30 Exploring different chains
14:24 Understanding the run logger class
16:15 Running a Fibonacci code example
17:45 Reviewing the logs and traces
19:54 Testing for errors

🔗 Links
https://github.com/langchain-ai/langsmith-cookbook/tree/main
https://github.com/langchain-ai/langsmith-cookbook/blob/main/tracing-examples/rest/rest.ipynb
https://smith.langchain.com/

⤵️ Let's Connect
https://everefficient.ai
https://nerding.io
https://twitter.com/nerding_io
https://www.linkedin.com/in/jdfiscus/
https://www.linkedin.com/company/ever-efficient-ai/

## Key Highlights

### 1. LangSmith REST API Unveiled

LangSmith now offers a REST API, enabling logging runs and leveraging its features from any language or environment, not just Python or JavaScript. This opens doors for wider adoption.

### 2. Flexible Data Logging with API

The REST API allows for comprehensive data logging, including inputs, outputs, errors, metadata (runtime, platform), tags, and events (retry, new token) providing rich context for debugging and analysis.

### 3. Logging LLM Runs via API

The API supports logging LLM runs, capturing details like prompts, model parameters, and function call responses (names and arguments). It mimics OpenAI's API structure for ease of integration.

### 4. Nested Runs & Execution Tracking

Demonstrates creating nested runs using a parent run ID and execution order, allowing for detailed tracing of complex workflows, as shown with the Fibonacci sequence example.

### 5. Error Logging and Tracing

Illustrates logging errors, capturing the error message, and providing a traceback of the execution flow leading to the error, which aids in debugging and issue resolution.


## Summary

## LangSmith REST API: Unleashing Language Model Monitoring Across All Languages

**1. Executive Summary:** This video from Nerding IO explores the LangSmith cookbook, focusing on the recently added REST API. It showcases how to leverage LangSmith's tracing, logging, and error-tracking capabilities from any programming language through REST calls, opening up broader use cases beyond Python and JavaScript.

**2. Main Topics Covered:**

*   **Introduction to LangSmith REST API:** Overview of the new API and its benefits for logging runs and data from diverse environments.
*   **Setting up LangSmith and REST API Example:** Walkthrough of the necessary prerequisites (API Key, Project Name, App ID) and launching the REST API notebook.
*   **Logging LLM Runs via API:** Demonstrating logging inputs, outputs, errors, metadata, tags, and events using POST and PATCH requests to the LangSmith API, including:
    *   Simple data logging
    *   Adding tags and metadata (runtime, platform)
    *   Logging LLM specific data (prompts, model parameters, function calls)
*   **Exploring Nested Runs and Execution Tracking:** Illustrating the creation of nested runs using parent run IDs and execution order, with a Fibonacci sequence example to track intricate workflows.
*   **Error Logging and Tracing:** Demonstrating how to log errors, capture error messages, and provide a traceback of the execution flow for effective debugging.

**3. Key Takeaways:**

*   **Language Agnostic Logging:** The LangSmith REST API allows logging runs and data from any language or environment, expanding its utility beyond Python and JavaScript.
*   **Comprehensive Data Logging:** The API supports detailed data logging, including inputs, outputs, errors, metadata (runtime, platform), tags, and events (retry, new token).
*   **Detailed LLM Tracking:** You can log LLM runs and capture all relevant data, including prompts, parameters, and function call responses, making debugging much easier.
*   **Nested Runs for Complex Workflows:** Parent run IDs and execution order enable detailed tracing of complex workflows with nested runs.
*   **Robust Error Handling:** Error logging includes the error message and a traceback, facilitating efficient debugging.

**4. Notable Quotes or Examples:**

*   *"Although uh you know llms are specifically written in Python mostly and and also JavaScript there's other use cases that you can use to actually log the runs and take advantage of linksmith so if you're using it in different languages or different types of like traces in open telemetry so this is really awesome."* (Illustrates the motivation for the REST API).
*   **Fibonacci Sequence Example:** Used to demonstrate nested runs and logging success/failure paths, showcasing the API's tracing and error handling capabilities.
*   **Logging LLM Calls:** Example showcases logging a function call and the subsequent parameters.

**5. Target Audience:**

*   AI/ML engineers and developers using LangChain.
*   Developers who need to monitor LLM applications built using languages other than Python or JavaScript.
*   Teams looking to implement robust tracing and debugging strategies for their AI workflows.
*   Developers interested in using a centralized tool to trace/debug AI code written in many different programming languages.


## Full Transcript

hey everyone welcome to nerding IO I'm JD today we're going to be looking at link chain and specifically continuing our journey into the langsmith cookbook a lot of different things have been coming out so we're going to kind of tiptoe our way into it but first if you have any AI development needs you can look at our business side ever efficient AI again thanks for the continued support but let's go ahead and dive in because there's a handful of cool things that are coming out so we have been predominantly looking at our tracing examples which we're going to go through today and then we've looked been looking at our testing examples they've also added things on fine tuning and even in the feedback examples they have in xjs this is really exciting because we use nexjs as well we'll start going through some JavaScript examples in Lane chain in another video but what really blew me away is inside the tracing examples they actually have a rest basic sample and that is really awesome so we'll this is what we're going to go through today and the reason this is so awesome is because although uh you know llms are specifically written in Python mostly and and also JavaScript there's other use cases that you can use to actually log the runs and take advantage of linksmith so if you're using it in different languages or different types of like traces in open telemetry so this is really awesome so what we're going to do is we have our python notebook running in the background you want to make sure that you have this cloned if you haven't set up link chain or Ling Smith before you can look at some of our previous videos uh but we're going to go ahead and Dive Right In so what we did is we pulled this down we have our Jupiter oh notebook running and we've gone ahead and launched our rest API example so what this is telling us is that we can actually submit post requests and it will to the API link Smith Lane chain runs and use our X API header as which is our API key you can see down here our environment for our API key if the two of those validate they can authenticate against this which is super awesome so we're going to go through a couple different runs the first one is just going to be a logging run so we're going to go ahead and start this so we are going to go ahead and click our run the cell again this is just showing us our prerequisite really to run Lane Smith you just need your API key the uh app ID and then the project name and so all of these are default in your python system but let's go ahead and run this and actually I should show you the whole screen so basically what this is saying is that we're defining our lane chain key from our environment variable also note I always put this in here it's a dot m file I just like running uh or using files in my folder instead of using the exports so we're going to define a run ID we're going to use request to actually post to the 8th the langsmith API we're going to Define our run name this is going to be and then some just default information so we have our inputs and then this is how we authenticate and then we'll do some work but we can actually patch our request so that we can see we'll see Foo and then we'll see bar so let's go ahead and run this you can see that we're getting a response 200 and now we can actually go to our projects page and actually see this information here so let's so let's take a look if we go to our projects I have been using this tracing cookbook one but we're actually going to see it in the default so we can see right here we have our first run it's called Foo and this is our log we actually have our input is Foo and our output is bar and that comes from the initial part of our posts so we're actually posting to the API and then for our output we're actually patching or updating that request all right so if again this I I just find this really awesome but so what we're looking at here is the additional values that we can put into our post request so we can have things like error we can have some extras we can even have our session name um and then these events all of this can be found in the API documentation which is uh part of the The Notebook here so you can get all of that so let's keep going and look at a another chain this one's going to be a little more complex they're saying so right here we can already see different things right so we have our session name so it can be more explicit instead of having the the default and then we're actually adding in tags so that we can find this easier we've seen this in previous videos but look at this this is really cool we can actually add our metadata including our runtime and our platform and then any specific keys that we want so that's really cool that you can actually include the metadata for your run based on this API so in this round what we're going to do again this is just saying we can do whatever work we would want in this particular spot and now we're actually going to add events so we're going to have an event name of retry and an event name of new token so what will happen is we'll post a new request we'll Define our events and then we're going to append those events to a new call and which is an update call and it will be a patch so we're going to do the same kind of process of a post and then a patch as before but sorry for jumping all around but then we're going to include this events array right here all right so let's go ahead and run this now we're waiting for a request to come back all right and so what it's saying is again we're we're looking for these tags it included metadata and now we can actually look at our second run as well so let's go ahead and do that in Lane chain if we go back here we have our first run again this is in our default projects we go back to our projects we can already see that my rest project is running so this is what we defined in our session name let's go ahead and click that we have our input of Foo let's see if we got our patch so remember Foo is the first part that we're where we sent and then our patch was bar so we we can see already that we have our tags here those are the the parts of the array that we sent as well and let's take a look at our metadata awesome so we have our key our my key and my map my value and then we also have our runtime for our platform that we've logged inside the metadata for this run again this is really exciting and really cool that this is essentially linksmith is language agnostic so next what we're going to do is look at logging llm runs so we're still going to be taking our information of the of the through the post but now what we're going to be doing is defining what this llm is going to be doing so in this one we are again we're doing our post we're including our Json and this includes here so we're going to say our another our new name is the Run chat model or run I'm sorry my chat model run the Run type is an llm our inputs this is the kind of array that you know open AI would expect so we're defining our role as a user and we're asking a question about what's the weather in SF we have our model this is an open AI function so what we're doing is we're defining an array of a a function that we want to call back to in our code we'll actually have another follow-up video on this too but it basically means that we would be calling out to open AI then it would return with a function name and these specific parameters as answers so it's a way to kind of formulate like format your response but also then call a function after again we would have some work being done here and now what we're going to do is we are going to patch the information that we retrieve that we expect which is our choices we're getting an array so we have our index of the array we have our message which now we know we're an assistant we have our content and here is our function call that we could then put as a dynamic function using this name so as long as this function was in our code and here are the arguments that we would get back our argument is location and the value is going to be San Francisco California so let's run this and see what our next response is so we got 200 this is telling us to go you can see now because we didn't specify a session we're going to default so let's go check that out so we go back to projects we can also see that default is now at the top it's the most recent run here is our my chat model run we have our human interaction which would be the llm you can even see right here that the type is an olive in our Trace which is pretty sweet and then it's going through the arguments or not the arguments it's going through the response of our function call with the name and then our arguments is location and our value is San Francisco all right so let's keep going through uh there's a couple more examples next example which is a logging completions model and so what this is going to do is again we're going to be doing our post we're going to say that our name is my llm run again we're doing a run type of llm and we're putting in our inputs as a prompt we don't have an array here because we're following the uh the API for openai which is a prompt string telling you the engine which is DaVinci 3 our temperature our start time and then we would have some work here and then we're going to patch again the response so in the completion we would expect choices again we would have our array it's going to have the text that we're specifically looking for and then the reason for stop uh or the re the Finish reason is stop so let's go ahead and run this and again we're getting a response of 200 we're going to be looking at our default because we didn't Define a session name so let's do that so if we go back to default here we can see that my llm run is updated again we can see that this is an llm type saying hi there and then the completion my name is Paulie and I'm excited to talk to you all right so the next one is a nested run so in the nested run we need to have a parent run ID and then we're going to have an execution order all right so we're going to be doing something new we're going to create a run logger which is a class that's going to be managing our execution State order state so what we're doing here is we're putting in our ID our linksmith project again this is being defined in in my case for the M file we are going to have our class of run logger we're going to have a function that is our execution order we need to make sure that we have our run ID and then else it's going to be an execution of an execution order of one otherwise we're incrementing then we can go ahead and update the execution order and then we're going to post our run so the run the post run is literally the the request that we've been doing previously where it initializes the run with a post based on the Run ID so then we're going to say again we're going to have a name we don't have we do have a session name so this is going to go into our project we have our inputs data which we're going to pass and then we have our execution order which is coming from our logger class and our parent run ID and then of course our header for Elaine Smith API key then lastly we're going to have our patch which is going to look for errors or our output and then give the end time so let's go ahead and make sure we run that so we Define our class now in order to run this what we're going to do is we're going to call our logger and we're going to define a Fibonacci code as an example and as we're running this function we'll see that we're creating a parent run ID here we're put we're logging based on our logger with a post run on the Fibonacci sequence right so you could think of this as right here this is an example of us doing this in any kind of language essentially so we could just call Define a function we're defining our run ID we're logging it through our post to create a a new entry into langsmith and then here's our logic so we're saying if n is less than one where we have our result is in otherwise we're passing through this fibbit Nazi into this Fibonacci Fibonacci sequence and then here is when we're running the patch to actually update so remember we said that our result if you you can see it actually right here our outputs so here's our output for successful run and then if we have an exception for an error we can actually Define that as well just by including it as our error that goes to this Json so let's define this Fibonacci function and then let's actually run it so we're going to say Fibonacci of five so let me take a second to it for that to run so now it's come back so that means that both the both of the logs that we were trying to run have run successfully and now we can actually go and look at the trace of this Fibonacci so remember we actually defined a session so we're going to be looking for my rest project in this instance all right so if we go back here and we look at our if now we're back in Wayne Smith we go back to our projects you can see the my rest project but you can also see that the trace here so I misspoke it wouldn't be the the my rest project that we previously set because in our function we actually said grab the the Ling Smith project so it's saying if we didn't Define this environment variable which we did then it would be the my rest project but since we do have this our lane chain project environment variable it's actually going to be our in my case the tracing cookbook which we've been following so right here you can see the successful Fibonacci sequence and this is how it's actually chaining all the way through so you can actually see how it's going through a number and then result and then continuing down through our sequence until and then it's going back and it gives you all the execution and latency times which is super cool all right so our our next example we're actually going to take this same information but the this the last one we did was a success so remember we had our error that we wanted to test so as a uh single request we're going to run this now and sorry so now what we're going to do is we're going to run this here and we'll get the error specifically so we're taking that same function that we wrote previously we're going to say if it's less than zero then we have a negative number and negative numbers are not allowed and so we're going to try to trace with an error by saying Fibonacci to 0.3 so let's go ahead and run this we should see an example here and again we're going to be looking for our Fibonacci Sequence and let's take a look all right and so if we go to Lane Smith and we click on our project we can already see that we have our Fibonacci recursive and we have an input of 2.3 we notice that we have an error so we can click and we're getting our output print of a negative number not allowed and it's still giving us the trace so we can actually see the numbers that it was trying right so it's not and then there we go right here is where our negative number comes from so that's really cool it actually logs the area error as well all right so that's it for us today we went through a lot we looked at the capabilities of rest so we went through the Fibonacci example both the success and failure we went through a couple different nested logins or nested logs and as well as looked at different types of LMS and defining the sessions to see those in linksmith so I hope you're excited about this I hope you learned a lot please let me know in the comments if you have any questions don't forget to like And subscribe and happy learning

---

*Generated for LLM consumption from nerding.io video library*