Chrome experimental AI features
blog.googleSoon you won't need to browse the web at all, Chrome will do everything for you: watch youtube ads, click on sponsored links, write positive reviews for restaurants buying ads from adsense, and write negative reviews for ones not advertising with google, fight with edge which browser is the default one. On the bright side, you will be able to enjoy more time offline.
Your comment reminded me of Douglas Adams' Electric Monk:
"The Electric Monk was a labour-saving device, like a dishwasher or a video recorder. Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; Electric Monks believed things for you, thus saving you what was becoming an increasingly onerous task, that of believing all the things the world expected you to believe."
I look forward to AI blocking ads after it gets tired of clicking through all of them.
> AI blocking ads after it gets tired of clicking through all of them
The future is (almost) now: SponsorBlock[1] skips embedded ads in YouTube videos, based on crowdfunded timestamps.
As a tab hoarder, I remember there were some attempts to implement rule-based tab organizer (using features like tab name, url, etc...) but most of them were only marginally useful for my case.
I wondered if generative models make any differences here so just tried it and a bit disappointed, it's consistently returning an error with a message "Tab groups suggestions are currently unavailable". It's just launched and the team might be experiencing lots of pages, perhaps I should try this again later.
Have you tried ToChunkA Smart TabS extension(https://chromewebstore.google.com/detail/tochunka-smart-tabs...)?
It organizes tabs and bookmarks by content similarity and does little more. Designed for tab hoarders in mind. :)
How did you use it? I'm trying with Chrome Beta but it doesn't seem to have the option to organize tabs.
I looked in Experiments and it's not there either.
It's updated to the latest version.
https://support.google.com/chrome/answer/14519765?visit_id=6...
AFAIK, this is geo limited to the US, at least for now.
Here are some better ideas from the top of my head:
- summarise an article
- find information on a given topic (free-form input text)
- full voice control ("click that link", "read that article", "find this")
- auto-submit a captcha
> "auto-submit a captcha"
we have come full circle
well, they're too hard for me to solve on my own
When I am prompted to 'try again' with a new 3x3 of low-res images I often wonder if there was a bit of 'crosswalk' I missed in one, or if this is just how they get me to annotate another set for 'bicycles' for free.
When they decide I'm a bot and just give me the low res stuff I give up accessing the website.
Well, we have been training them to do this for a while...
Take this tab collection, build a model or a RAG or whatever around them:
- Let me chat with a bot that knows the information from the collection
- Use the information to generate a summary
- Let me guide it in generating a well sourced article
Build a knowledge graph from the web
- Trace a source of information back to the originating point to help eliminate derivative blog spam
- Help moderate media bias and challenge echo chambers
Automatically recognize spam, scams, etc.
Let me describe something I need in text, return back links to shopping sites that sell that thing, if nobody has it, generate a 3d model, or more formal description of it and supply me with connections to let me farm it out to an additive manufacture, one-off makerspace place or something.
- Post resulting article on your spam blog
- The circle continues
Seriously, their first example seems useless to most people. Naming a tab group??? That doesn't take any time, little thought, and who does that regularly?
Summarizing an article seems like something everyone else can do OK. It's a huge avenue for bias (maybe that's why it's reasonably elided) but at least it's actually useful.
It's not just naming, but grouping as well. If you have 200 tabs and don't want to spend too much time on it, this can be very helpful.
How long until malicious pages fool this into placing themselves into, say, a banking tab group?
How would that affect the end user except they go into the banking tab group and go "Huh. This isn't a banking website" and close it?
Tab groups are a mental shortcut so you can spend less time figuring out the nature of the tab you're looking at or finding particular tabs.
If something automates their creation then there is absolutely an advantage to sites that can subvert the classification method, because users will start with the "banking site" expectation instead of no expectation at all.
Who has that many bank website tabs open? And that are still active, because of course bank websites log you out if you do nothing within a minute or two. And then create a tab group from those tabs, and then the bad actor correctly guesses which bank you use and has bypassed Safe Browsing (which also uses ML now) and you visit the tab and manually (because the password manager won't work) type in your credentials?
Worrying about this as a potential threat is on the same level as my bank "disabling" right click.
Replace banking with any other popular site. It was just an example.
There are already phishing websites which copy the look of the real website, and a user could easily mix the two up.
This is the only feature I'm excited about. I perpetually have 100+ tabs opened and have tried tab groups but eventually things get disorganized again. The ability to automatically group similar tabs, assuming it works, is going to be game changing.
To automatically group similar tabs I am using ToChunkA Smart TabS extension.
> Seriously, their first example seems useless to most people. Naming a tab group??? That doesn't take any time, little thought, and who does that regularly?
Funny, naming things, whether variables or groups of things is the main reason I use LLMs to date. Add in grouping as well and that handles something that puts me under a lot of cognitive load, because I can never shake the feeling I have ot yet manually grouped things optimally.
Maybe they are just testing AI with less used features first
1. This is actually a feature on Pixel through the assistant [0], surprised it's not on Chrome itself
2. That doesn't really seem like a Chrome feature? Belongs more on Bard.
3. That seems like a Google Assistant feature too, some of that actually may work on a pixel phone, though might be nice to have on desktop too.
4. Will never happen. Google themselves have a captcha product so defeats the point.
[0] https://support.google.com/assistant/answer/14163109?hl=en
I was excited to test it on my pixel 7 just to find out it's only available on Pixel 8. Why?....
...and only in the US.
> 1. This is actually a feature on Pixel through the assistant [0], surprised it's not on Chrome itself
It is only on the Pixel 8, not the previous models and their mid-range $ variants so they aren't giving it away for free just yet
I wonder if it's done locally or through the cloud. Some of the language around the release of Gemini Nano implies that it may be done locally?
https://store.google.com/intl/en/ideas/articles/pixel-featur...
I think for #2, they meant like AI-powered control-F / find in page.
Which is actually the first non-novelty AI tool I've heard someone pitch that actually sounded like a good idea. Way more visible failure mode than summarizing.
Watch an ad so i don't have to
AI theme generation really seems like a solution looking for a problem
The vast majority of AI development right now fits the solution looking for a problem mold. People are pushing hard for the adoption of LLMs in areas where the existing solutions are not only more predictable, but require equivalent or less effort to using an LLM.
At some point the hype will die down and we'll find out where these tools actually fit, but yeah right now it's madness.
It is a neat "it can do that" kinda thing but I also wondered when I need that.
Having said that chrome customization has always kinda bit me in the butt eventually when something changes and looks odd now and ... I just tend to avoid it altogether now.
Yeah honestly I just want my browser to be dark and not spam me with stuff when I open a new tab. I don't need some fancy picture.
The last thing I want is further telemetry from Chrome to Google. I'm so glad I switched to Firefox. This should be an API, not a browser integration.
an API to group your browser tabs..?
There is an extension for Firefox that does it.
https://addons.mozilla.org/uk/firefox/addon/tochunka-smart-t...
Maybe they meant an extension API, not an external service.
I had a thought while reading this, and I don't know if this would be the case but...
If it works by you hover over a link and Google gets the content in the browser behind the scenes and sends it to the mothership, where it's summarized and the summary then sent back to you to be displayed by the browser, then you may be accessing the linked page using your stored credentials, which give Google access to content they wouldn't otherwise have access to.
The same is true for translations in most browsers, right? At least I'm not aware of any browser that does it client-side/offline.
Edit: I stand corrected, Firefox does it offline! Thank you, Firefox team, this is awesome and I'll likely be using it more often now :)
Firefox does local LLM based translation. I kicked that project off in 2022 and it shipped last year.
Thank you, I find it really useful
> Unlike other browsers that rely on cloud services, Firefox keeps your data safe on your device. There's no privacy risk of sending text to third parties for analysis because translation happens on your device, not externally.
Firefox does it locally: https://www.mozilla.org/en-US/firefox/features/translate/
Oh, cool! I somehow misremembered that very announcement as being cloud-based (but using an open model).
Firefox does it offline
Sounds like a sneaky way to add your personal social media feed into their AI training data.
Edit: the suggestion that translation functionality already does this is valid though perhaps this expands the scope to data in the users default language?
I hope the Chrome devs will add some features like those in Arc browser, such as summarizing content while hovering over a link and pressing Shift.
That would be useful, but also probably quite expensive, if every chrome user use this feature?
Not if inferencing happens locally, e.g. with Gemini Nano.
Should be doable with a local model, but there might be some trade-off here. I expect it to roll out to Pixel users first where Google has a better control.
As long as laptops are still sold with 8GB of RAM, I don't see this happening.
Honestly not sure how much RAM that'd take but to me, it sounds like bloat
Desktop users are not going to be happy about a surprise local model running in the background, even a small quantized one.
So don't make it a surprise then. Besides, I'd be much less happy about my post-authenticated content being sent to a surprise cloud model...
I'm curious, why do you think that?
Because it seems like, regardless of the announcement, there will always be someone who has the most niche issue with it and manages to make assertions for an entire group of people while only really referencing their personal experience ("and all of the people they know").
I mean, I am the strongest local LLM advocate you will find. I have my GPU loaded with a model pretty much all day, for recreation and work. My job, my livelihood involves running local LLMs.
But it's intense, even with a very finicky, efficient runtime on a strong desktop. Local LLM hosting is not something you want to impose on users unless they are acutely aware of it, or unless its a full stack hardware/software platform (like the Google Pixel) where the vendor can "hide" the undesirable effects on system performance.
I think that's a reasonable generalization to make.
Fair but google does, _supposedly_ have a Gemini model meant to run on phones so it'd presumably be small enough that it wouldn't necessarily be a massive problem. Or, at least, we could get there eventually. Not arguing at this point, you're right. I just think over time we could get there
Running "smart" LLMs locally takes a lot of RAM, a lot of compute, and a lot of disk space.
It produces a considerable amount of heat unless it's run on an NPU, which basically doesn't happen on desktops at the moment.
Hot loading/unloading it can be slow even on an SSD.
Users often multitask with chrome in the background, and I think many would be very displeased to find Chrome bogging down their computer for reasons they may not be aware of.
Theoretically Google could run a very small (less than 2B?) LLM with very fast quantization, and maybe even work out how to use desktop NPUs, but that would be one heck of an engineering feat to deploy on the scale of Chrome.
Honestly that sounds extremely feasible, especially for a feature that isn't on by default. The one the parent comment references in Arc isn't on by default. Also chrome eating up system resources is already a meme and they've been working on using less by sleeping tabs.
I hope the Arc Browser team makes their Chromium based project as cross-platform as Chromium itself.
It's coming to Windows really soon! I'm not really holding my breath for Linux support though :/
Kind of amazing how unable to deliver Google seems to be here. Looking at Arc, a new player, and the kind of AI features they came up with, this here looks more like features developed by McKinsey rather than by someone with domain knowledge.
Love the textarea integration. I wish Chrome could do a better job of saving "drafts" and/or backing up text somehow. Adding long content is a constant worry for me to lose it somehow to an error or accidentally closed tab.
Absolutely. For the longest time I had this extension called "Comment Save" which saved anything you typed in a textarea. It doesn't seem to exist anymore, and I haven't been able to find a good replacement. I would also much rather have it in the browser than giving full permission to some third party.
The back button in Chrome sometimes help but I still lose long messages all the time.
Anyone else have AI fatigue yet? I haven't even tried GPT and I'm already sick of AI this LLM that.
You haven't tried what would only be called science fiction 4 years ago but are tired of it? The hype machine is grating, but do try a Gen AI model. I use it for code, for ideation, for various NLP tasks. It's at the very least moderately useful in various tasks, and extremely useful in some.
edit: tone
I get what it does, I've seen demos and it is no doubt impressive, but I have no personal use and don't see the point of giving them any data. I'm not limited in my current workflow and prefer official docs over the LLM interpretation of those same docs. The real satire are all the comments on any problem asking "have you tried GPT", more annoying than the Rust community.
What do you use it for that you think would benefit me?
GPT-4 is miles ahead of anything else (including DeepL) when it comes to translations, for one thing.
One example for which I use ChatGPT is tip of the tongue. I can't remember a word, but I can describe it in other ways. Google doesn't catch on those keywords, ChatGPT does.
It's also pretty good for generating pointers for a complex solution. Like I try to figure out something in the non-JPA old version of EclipseLink, so I ask ChatGPT. The generated code is very often wrong, but it often points me into a right direction.
I just use a thesaurus and hit one or more synonyms for the first example. The second isn't really a problem I encounter, some docs are terrible yes but I jump to definition or search the web.
I might've had more use when I was new to programming but my workflow these days is pretty solid and I don't see myself saving time by typing prompts and debugging output rather than just coding.
If it is too complex for me to understand I wouldn't really trust the output anyway and might spend a lot more time sanity checking the generated code and it might not be very useful in the end. I'll try someday if I really get stuck, I expect to get disappointed though.
Do you never find yourself using brand new tools or languages? My most common use-case for chatGPT is "explain this syntax <codeblock>" and "<language-construct in languageX>". Simple stuff that it pretty much can't get wrong. Much faster than Googling the same.
Constantly, but like I mentioned in the other 2 comments I just RTFM. Why would I need a chatbot to explain me something the devs put effort in explaining? New languages these days are pretty well documented. And you can read the entire C reference manual in an afternoon.
Asking GPT to summarize something like that is beyond lazy.
Side-note: stop using google as a verb, you're "searching", teach your peers that there are alternatives that are both better and won't ask you for your first born.
No it isn't beyond lazy, it's a more efficient use of my time. If I want in depth language features explained then I read the documentation. If I want to be reminded of specific syntax as rapidly as possible then I'll keep using ChatGPT and save tons of time over the course of a week. And if you refused to even google in such scenarios beforehand then that is a really, really inefficient use of time.
Side-note to your side-note: No. Stop acting like some holier than thou elitist. You're coming off as a luddite.
you are not alone, and ironically i work in this field lol
I have a chrome extension [1] which lets you re-write your selected text, or look-it up via ChatGPT using your own custom prompts. Gives you more control on what kind of suggestions or answers you want basically.
Won't help with rearranging/grouping tabs, but can definitely help rephrase text in input fields or looking up info.
Having a feature to summarise the website would be a much better than the first two. As I constantly find myself wishing for such a feature :/
I’ve been working on a project [1] to do just that from within a Chrome extension. The idea was that as an extension, it could make use of the context menu and feel more like a native feature of the browser. I’m always hesitant to link to my things from comments but in this case I think it’s a perfect fit for what you’re describing.
I suspect Google is afraid of getting sued by more publishers :/
The linked post links to another blog which is about this.
The Arc browser I use has that feature on hover.
I don't want any AI features in anything. Yet another reason to avoid Chrome. I have Chrome installed solely to watch netflix on Linux.
I really wish the first part of this article had an explicit "Here is how you get started" section. I just about missed it because it's a paragraph that links to a support article. If they want people to actually use this stuff, why not make turning it on front and center?
This feels like a VP at Google told the Chrome team they needed more AI.
Again features nobody cares about
The only "AI feature" I use in Chrome is the live caption one for French (which requires Chrome Canary). I use it to get automatic live caption while listening to French podcast since I'm learning the language. It's buggy as hell though, so if anyone has a suggestion on a replacement that would be much appreciated!
I wonder if this "Help me write" will give different suggestions from the Google Docs "Help me write" feature, or from the dozens of other help-me-write features that are cropping up these days within text-oriented webapps (e.g. Notion).
what i want is 'read aloud', like MS Edge
It's absolutely wild that this isn't the first thing anyone would make there. ChatGPTs talk mode is so good, I'd kill for the ability to listen to longform articles at varying speeds/voices.
Edge is honestly slowly turning into a better Chrome, and the better parts aren't even the LLM craze.
If they keep it up, this might actually threaten Googles browser dominance.
On the AI end though Microsoft aggresively pursues support for other AI providers (Mistral and Lamma both being on Azure API's now), Google tying themselves to Gemini seems to be tying themselves to the best they can do while Microsoft seems to be accruing the best they can get.
Serious question: What do you like about Edge?
I’ve tried it a couple of times (when Bing AI chat was still Edge-only) and was extremely put off by all the coupons, rewards, and other distractions.
Read aloud and Web captures are great.
I'm not daily driving Edge yet, but I do use it now and then and have been positively surprised a couple times.
Vertical tabs with grouping.
Anyone else think the whole idea of "tab management" tools is crazy? The way to manage tabs is to close them.
Or filtering and searching. Anything you can have a large number of should have an obvious filtering & searching UI. After how many decades of UI are we still so behind on basic usability design.
This is a feature in Chrome today.
If you don't limit yourself to only the closing one part of management, you might see the reason behind the craze
And if you'd like to read it later, bookmark it! You can organize bookmarks into groups, give them custom names, and they don't require the browser to be constantly hoarding multiple gigabytes of RAM.
Neither do the tabs, autosleep/unload exists, you know
And bookmark organization is worse since it's a separate UI breaking your mental connection to the physical layout that you spend most of the time with
Meanwhile I keep refreshing the Google Bard changelog site, waiting for Gemini Pro in Europe.
I switched to Brave. Slowly trying to get the number of Google products I use down to zero.
Another reason to use Firefox
The only AI feature I'm interested its an AI that lets me remove ads automatically, but Google by definition its not interested in such feature.
I don't want another microsoft edge where I have to press hundred buttons just to open website.
What are you talking about?
The first two are seemingly of questionable utility, but the 3rd feature (Help me write) is actually quite interesting.
As of late, most of my public written responses (bar HN) have had some sort of collaboration with ChatGPT, and I've often wondered about a native browser integration. For those of us who struggle with communication, this is an exciting prospect!
Ironically, the message "I'm interested in this place - do you allow dogs?" is a piece of decent business writing on its own and way better than anything I saw while trying to sublet a room. I would rather see AI suggest phrases like that rather than their proposed answer.
Now that I think about it, a Clippy that interviews you about needs and follows your browsing session to highlight stuff you like / don't like and propose questions to ask would be pretty sweet.
Yeah that suggestion is terrible. Really not looking forward to seeing this kind of writing everywhere.
I don't know. The screenshot that they use to showcase it makes me feel the opposite - it took a perfectly clear and concise question and dressed it up in a lot of unnecessary verbiage that the person on the other end will now have to unpack to get to the gist of it.
Or they could use an LLM to translate it back to clear and concise, I suppose. But then what is it even for?
I'm on the other end of the spectrum; I'm pretty worried that that feature will make it really easy to automate bot comments in-browser. It's sad that the internet as we knew it 3 years ago is gone forever and changes like this to push some team's OKRs means I won't be able to trust more online comments.
The Internet 3 years ago? Was that Internet that great?
No but it was (mostly) filled with real people. Sure there was a GPT3-generated article or two but they were pretty easy to spot.
But it was also largely filled with Business Insider-esque articles, which are as bad as LLM generated articles IMO.
I suck at plenty of things and when there is software (or another tool) to help me, I am happy to use it. Software that helps with communication, at least on this level, is a new frontier and therefore, as always, people feel a lot of uncertainty.
As someone who seriously utilizes this particular tool, what do you think of those issues? For example, do you feel like the result has your own voice? Your own specific, precise thoughts? Does it help or hurt growth in communication skill? How do those things play out in real application of the technology and what is the best way to use it?
Incidentally, communication is a strong point for me and therefore ChatGPT doesn't benefit me much in that respect. I hate to think that my skill has lost most of its value, but working in technology, I can hardly complain when it happens to me: Are communication skills even needed now, or how has that need changed?
The uncertainty you pointed out does seem to explain the downvotes I received.
For me personally, communication is arduous. I struggle daily to articulate what I want to say in a logical and efficient manner, let alone in a graceful or artistic one. I've noticed my vocabulary and communication skill has regressed as I get older, despite efforts to improve it.
Overall, ChatGPT has helped tremendously. I never had a written communication style that I was proud of, so I'm happy to assume its more generic tone of voice. If language fulfills its primary purpose, to get a point across, that's enough for me. Any kind of inherent artistic integrity is above my pay grade, so to speak.
> The uncertainty you pointed out does seem to explain the downvotes I received.
Yes, even on HN people despise anything that violates norms (unless it's promoted by a megalomaniac).
Thanks for sharing your experiences. It's great there's now a tool for that need.
I find it the exact opposite. Writing is joyful to me, whether I'm writing a short one-paragraph comment on HN or writing a thousand-word essay. I do not like the current crop of "Help me write" features because they take away the joy.
I see this may be a good replacement for general autofill features as well. Seemingly simple autofill tasks like filling e-mail, address, name, country etc... fields never "just works" for almost all sites since this relies on correctness of the target page implementation and devs usually never care of it. Large language models should be better on this task.
You don't need an LLM to figure out what a certain field is. It's like choosing the 18 wheeler to drive your kids to school...
I have a chrome extension (https://github.com/SMUsamaShah/LookupChatGPT) where i just added in-place text replacement option. Right click your selected text and chose your own prompt to refine the text your way.
Is there any good AI autocomplete tool out there? The only LLM tool I like currently is GitHub copilot. I basically want copilot for gmail + MS word, but every product I've found wants me to prompt an AI.
Gmail already has content-based autocomplete. The feature is called Smart Compose: https://support.google.com/mail/answer/9116836?hl=en
I also find the UX much better than typical LLMs that require you to write a prompt first; it simply suggests continuations of your sentences that you can accept or ignore, without requiring you to switch your mind between writing for your intended email recipient and writing a prompt for the LLM.