# How to Use Chrome’s Built-in #AI: Translation, Language Detection & Summary

## Metadata

- **Published:** 12/11/2024
- **Duration:** 20 minutes
- **YouTube URL:** https://youtube.com/watch?v=8ybAFjH7bpM
- **Channel:** nerding.io

## Description

Discover the latest features of Chrome's AI Assistant, including real-time transcription, advanced language detection, and powerful APIs.
- AI assistance for styling
- Translator API for seamless multilingual support.
- Language Detector API for smart content customization.
- Summarizer API for concise content insights.


📰 FREE Code & Resources: https://sendfox.com/nerdingio
👉🏻 Ranked #1 Product of the Day: https://www.producthunt.com/posts/ever-efficient-ai
📞 Book a Call: https://calendar.app.google/M1iU6X2x18metzDeA

🎥 Chapters
00:00 Introduction
00:37 Web Summit AI
01:45 AI Assistant
03:30 Built-in
08:31 Translation
13:20 Language Detection
16:37 Summary

🔗 Links
https://www.youtube.com/playlist?list=PLNYkxOF6rcIAEVKJ98bDkQRkwvO4grhnt
https://developer.chrome.com/docs/devtools/ai-assistance/styling
https://developer.chrome.com/docs/ai/built-in#benefits-on-device
https://developer.chrome.com/docs/ai/built-in-apis
https://chrome.dev/web-ai-demos/translation-language-detection-api-playground/
https://chrome.dev/web-ai-demos/summarization-api-playground/

⤵️ Let's Connect
https://everefficient.ai
https://nerding.io
https://twitter.com/nerding_io
https://www.linkedin.com/in/jdfiscus/
https://www.linkedin.com/company/ever-efficient-ai/

## Key Highlights

### 1. Chrome's Built-in AI Capabilities

Chrome is integrating AI features like Gemini directly into the browser, offering new possibilities for developers.

### 2. AI-Powered Styling Assistance

Chrome offers an AI assistant for styling within developer mode, enabling real-time collaboration with AI for front-end development. Enable through experiment flags.

### 3. Gemini Nano API Access

Developers can access Gemini Nano through Chrome's exploratory APIs for tasks like translation, language detection, and summarization, all executed locally in the browser.

### 4. Origin Trials: Try AI Features Now

Chrome's Origin Trials program allows developers to test experimental AI APIs like translator and summarizer by signing up for a trial and enabling flags.

### 5. Local AI Processing Benefits

Running AI tasks locally offers advantages like enhanced security, reduced latency, and the potential for smaller, more efficient models. All processed with in-browser APIs.


## Summary

Here's a summary document designed to quickly communicate the core content of the video.

**Video Summary Document: How to Use Chrome's Built-in #AI: Translation, Language Detection & Summary**

**1. Executive Summary:**

This video explores the latest AI advancements integrated directly into the Chrome browser, focusing on the Gemini Nano APIs for translation, language detection, and summarization. The presenter demonstrates how developers can leverage these features locally within the browser via origin trials, enhancing security and reducing latency.

**2. Main Topics Covered:**

*   **Introduction to Chrome's AI Capabilities:** Overview of the Web AI Summit and Google's efforts to integrate AI (specifically Gemini Nano) into Chrome.
*   **AI-Powered Styling Assistant:** Introduction to using AI for styling in Chrome's developer mode, allowing real-time AI collaboration for front-end development. (Requires enabling experimental flags.)
*   **Gemini Nano API Access via Origin Trials:** Detailed explanation of Chrome's exploratory APIs (Translator API, Language Detector API, Summarizer API) and how to access them through origin trials.
*   **Local AI Processing Benefits:** Advantages of running AI tasks locally within the browser, emphasizing security, reduced latency, and smaller model sizes.
*   **Practical Demonstrations:** Step-by-step instructions on enabling experiment flags in Chrome, registering for origin trials, and implementing the translation and summarization APIs using sample code.

**3. Key Takeaways:**

*   Chrome is integrating Gemini directly into the browser, empowering developers with local AI capabilities.
*   The AI Styling Assistant in Developer Mode offers a co-pilot experience for front-end tasks.
*   Developers can access Gemini Nano for translation, language detection, and summarization through Chrome's exploratory APIs.
*   Origin Trials provide a way to test these APIs now, but require signing up for trials and enabling appropriate flags.
*   Local AI processing offers enhanced security, reduced latency, and potentially smaller, more efficient models. Each API (translator, language detector, summarizer) needs its own separate origin trial sign-up.

**4. Notable Quotes or Examples:**

*   "*...Google's been releasing a lot of different features that in Chrome you can actually leverage uh Gemini and other different apis in the browser...*"
*   (Regarding local AI benefits) "*...security if we think about the fact that we're sending data over the wire..."*
*   "*...basically in a few lines of code we're able to translate directly in the web...*" (Demonstrates the simplicity of the translation API.)
*   Translation API code example:  Uses Javascript to connect to the translator object and call the translate function.

**5. Target Audience:**

Web developers, front-end engineers, Chrome extension developers, AI/ML enthusiasts, and anyone interested in exploring the potential of client-side AI and the integration of Gemini into the Chrome browser. Specifically beneficial for those looking to use AI for translation, language understanding, and summarization.


## Full Transcript

hey everyone welcome to nerding IO I know it's been a little bit I've actually been working on a boiler plate and AI starter kit which I'll be releasing soon but today what we're going to talk about is some of the advancements that Google Chrome has been coming out with so the there's been the we web AI Summit there's been advancements in web uh NN and then there's also been different Google Chrome apis that allow you to access uh Gemini directly in the browser so we're going to go through all of that today and with that let's get started all right so a lot of what we're going to talk about has actually been inspired by the web AI Summit so this is a summit and Conference that Google put together but it has a lot of different contributors people like Microsoft hugging face uh and and different people doing kind of exploratory experiments but also some really cool things about why we would want to use AI on the client side uh in our applications so in upcoming video we're going to look at things like web NN we're also going to um continue hearing about Transformers JS which we have an entire playlist on if you if you haven't checked out or subscribed to uh this channel yet definitely should as well as things like web llm but today what we're going to look at specifically is the uh the state of AI that's actually built into Chrome and so what's been happening is Google's been releasing a lot of different features that in Chrome you can actually leverage uh Gemini and other different apis in the browser and so the first one is actually a utility and so it's uh this ability to use AI for styling and what you can do is you can actually in their uh in like the developer mode you can actually pull this up and you can let me see if I can get this a little bigger for everyone you can uh see that there's this AI assistance tab but there's also the ability to rightclick inside of the Dom and ask AI questions so if you click that it's just going to pull up this AI assistance tab you want to turn on the setting that allow ows you to do this which is pretty simple and then you can actually converse with this particular element as it's in your application and what's going on with that so I thought this was really slick for uh the ability to like develop with an AI almost like a co-pilot but a co-pilot in the browser again this is using Gemini um as an example so that's like one tool that out of the box you just have to enable and is great for development real quick everyone if you haven't already please remember to like And subscribe it helps more than you know also keep your eyes out for the boiler plate that we'll be releasing in the next two weeks it's an AR starter kit based on nextjs and some of the advancements in gen kit so with that let's get back to it but we're also going to look at is the uh some of the apis that they've been coming out with so if we look at this release where it's talking about Gemini n Nano for Chrome there's a few things that we're going to go through and the first is this uh get and early preview so you can actually sign up for access to get a early pre early preview program um where you can do like local prototyping we're kind of we're we're not actually going to apply for this but I'm going to show you how you can actually use some of the apis that are in origin trial so some of the things that are uh like the benefits for running uh AI locally is actually security if we think about the fact that we're sending data over the wire uh we have to be secure and sanitize all our information in two different locations this also on the front end allows for end to end encryption there are things like uh you know latency so we can have like a quicker snappier user experience and these models can be uh relatively small right like we're loading them in as an asset or we're using tools or concepts like web GPU to make it faster there's also this concept of being hybrid so maybe you have client first but a graceful fallback to server side if the browser doesn't support necessarily what uh you're trying to do so when we're actually going through and looking at these exploratory IDE apis and you can kind of see we're going to go through each one of these but we're I wanted to show like what this actually looks like so if we were to build an application the way this would work is you would have your app and you can actually hit the apis through uh an object like through and through JavaScript in an object and pull in the exploratory apis those are going to actually connect to Gemini Nano they have very specific use cases and then you can actually even like fine-tune things or have specific tasks that you want to do it's then going to go through the runtime and there's multiple different ways that we can kind of load in these the uh these models so the ones that we've looked at in this channel in the beginning probably like a year ago we were using like web assembly and more recently Transformers JS has been using web GPU so you can actually pull things in through web GPU now which is a lot faster but this is basically how the pattern would work is you would you would still have your front-end experience you a call through JavaScript these exploratory apis and then be able to hit your your AI run time so if we let's kind of take a look at what all this means so if we click on the status there's six different apis that they have that are either being discussed or are already able to uh be used and so there's this idea of An Origin trial or there's EP which is early preview program and so if you do the origin trial you can actually as long as you have the a version of I think it's like 137 to or 131 to 137 of chrome uh you can actually run these and so that's what we're going to do today is we're actually going to look at some of these how you would actually set this up and then uh in like a future video we'll actually look at how do you do the same kind of thing but through a Chrome extension so the origin trial means that you can actually get started today the Epp means that you have to sign up and if you get approved they'll give you access to these uh these um other apis so the three that we'll look at today are translator language detector and summarizer and so if you scroll down a little bit you can kind of see the the use cases for these and the idea being is that you can actually translate uh in the browser so you could think about the fact of right now you you have to take your translations and uh have you know multiple different locals uh associated with whatever the content is this would allow you to actually have live information that you could potentially change on the Fly uh or a forum or in a chat uh for instance so we're going to go ahead and take a look at this so if we look at the translator there's a a few things that we need to pay attention to like I was saying you need to have Chrome 131 13 or 137 and then we actually have to uh set up a couple of different things so if we are what we're going to be doing I actually already set up the the origin trial but the first thing you want to make sure you do is you come over here into Chrome and Chrome flags and you want to actually set up the translation API so I already have this enabled one thing that it's going to do is anytime you change it you see you're going to have to relaunch your Chrome browser so I'm just going to leave uh that as is and not have to worry about it you can also look for the other ones so writer is in here you can see writer API these ones will go through at a later date because we uh that's with the EP there's the language uh The Prompt API we'll look at this when we go through the Chrome extension and then there's the summarization as well as the writer so I would make sure you have all of these enabled then go ahead and relaunch your browser the next thing that you're going to need to do specifically for the uh the ones that are origin trial is actually sign up for an origin trial before we do this I just want to show you that you can actually see the uh so right here this is what we just did to enable that you can actually see the transl API so this is what the translation API is going to look like where you can actually call to translate you can actually see can translate if that's uh if it returns readily then that means that you actually have it uh working if it says no then it's it's not enabled in your browser essentially if you need to download this actually works uh fairly similar to like Transformer JS where you can actually see the progress of the model being loaded and then we can execute our action so this is the uh the translate obviously there's going to be some limitations and again this is all uh experimental so there's a handful of languages that they have and down at the bottom they actually have an example of how this would work so if you have the experiments turned on and you want to run this uh uh you can actually test it right here all right so this is a great way for you to just see it in action as long as like you have the the experiment turned on but what if you actually want to build something so we're going to take that same example we're going to do it locally and see how we can get this set up so the first thing is you need to set up for an origin uh trial now again I've already done this but we'll just kind of show what this looks like so basically what you're doing is you're just going to register for the API when it says web origin what's interesting about this is as you can see you can do https HTTP or Chrome extension again we're going to look at the prompt API and a Chrome extension in a later video uh you can also do an isolated app HTTP is uh only Local Host addresses and so if you wanted to develop this locally you have to do HTTP Local Host and then you're also going to want to choose your Port so in this case you could do like 80 or or 3,000 uh or whatever and then what we're going to do is we're just going to say 0 to 10,000 we're actually going to acknowledge uh that we know that this is you know a trial and uh it's also saying I need to be signed in so we just do that really quick [Music] so now that I'm signed in I can actually register this we're going to go ahead and uh take this information and figure out how can we actually add this so what I did is I actually took the translator code and I uh I set up a local version of this what's interesting about this code though is you need multiple things so you need both the translator API and the language detector API so you would need to go to the language detector API and actually go through the same process of setting up the flag going through and then signing up for the another uh origin trial each one of these trials you need to sign up uh each one of these these um apis you need to have a separate origin trial signed up for and agree to the terms so do that for translator language detector and uh summ summarizer API what's interesting though is like once you get this it's going to give you a token and so now we're going to actually look at the code so if we pull up cursor and we actually look at the trans late uh API we can actually see here that what we're do the way that we put in our origin trial is we have to have a meta tag is equivalent to this and then we put in our content right here as what our what we're using in order to actually run this locally what I did is I I literally just copied the HTML from the example uh same thing with the script here which is basically looking at the information from the Dom so getting our query selectors and then uh up at the top it's actually acknowledging do we have the translation or not and the detector right so then this detector is Tech detecting the initial uh language and then we're actually figuring out our confidence score and then uh right here oops is our uh tag to switch the languages right so now we're taking human readable we're looking at our uh submit and we are making sure that there are it's in between those languages and then we are actually doing the translation right here and that's it so basically in a few lines of code we're able to translate directly in the web so let's see what that looks like so all I'm going to do is launch this in a terminal and we will just do I'll go into translation and then I have something called serve so I'm just going to serve on 80 that's what I set up my original trial on and we're going to go ahead and look at this running in the browser so this is actually running on my Local Host it is using the uh origin trial uh token that I have it's giv me my confidence score it's allowing me to translate to different languages on the Fly and you saw how fast that was right it's not hitting a backend it's actually using uh AI Direct ly from Gemini in Chrome which is super cool so we're going to take a look at one more example which is the uh the summarizer so this same process right you're going to have to go through enable it and you're going to have to uh register the origin trial again you just click here gives you the ability to load and do the summarize or summarize so you can do local development this is the flag that you need to turn on which I'd already done you want to check if AI is available right so we're looking for a self. a and we're also looking at summarizer and then what we're going to do is we're going to come down here and again that's the download progress cool so this is how we're actually going to summarize it loved the fact that it included the option for markdown and this is where we're actually creating what that summarization is so uh and I'm kind of jumping around but what it's doing is it's going to take the article and then actually summarize the inner HTML based on the article so you can think about rather than processing every single like pre-processing or postprocessing your blog you can actually generate a summary on the Fly of your articles that are are being built uh on the browser now there's some benefit to having uh you know the SEO uh side of things where the back post-processing your article so that all your SEO is uh optimized but the interesting thing about this is that not only can you do this on on the browser but uh through the article but let's say you wanted to highlight certain sections and summarize those pieces you could now build in tools or build in Chrome extensions that allow you to do that directly in the browser without ever having to call your back end and all of this these interactions are free because you're accessing the uh the Google Chrome Gemini through the browser you're not paying anything uh to access that because the Nano version is included in the in in the browser so if we let's just take all the text on this page and we'll just do cool and then use it's showing how it supports Local Host it's giving us information here um and talking about some of the key points so we could obviously make this uh you know much simpler for for this to understand if we just did something like that and let's have it generate again and just see if it can get us us some key points cool so now we can actually see the markdown example too right this is showing you that it's that showing us that it's giving us bullet points and actually giving us the information directly in the browser you saw how fast that was again not going out to the back end at all today was some of the things that were released in the web AI Summit through Google as well as different apis that you can actually leverage in Google Chrome so that you can access things like Google Gemini directly in the browser without actually having to load any particular uh information from the back end with that happy nerding

---

*Generated for LLM consumption from nerding.io video library*