Dawn of a new era in Search: Balancing innovation, competition, and public good

Here's the docs[0].

It's basically a saved partial filter that you can use when searching. So you can e.g. say you only want to search reddit and have that saved as an easily-accessible button. But there's also predefined ones like "Forums" and "Academic", so that's the general idea.

The cool thing is you can enable a Lens when using the AI assistant with internet access, so its searches are also constrained to that Lens.

[0]: https://help.kagi.com/kagi/features/lenses.html

Yeah, most AI features are.

Thank you for the reply!

> it is about creating a searchable index of the web scale that has certain quality level that is simply impossible to do anymore for many reasons that would require another article to explain

I am both happy to take your word for it, and also very interested to know more. If you were to write that article then I would love to read it.

Or the Library of Congress, if it had the right appropriations. Google itself started with an NSF grant to explore the future of libraries.

Does CC publish the methodology for how they determine what to crawl. More particularly, how do they determine what not to crawl.

Yes OP is hilariously out of touch, the storage would be sub 1% of total costs.

I'd argue the minute they turn to an advertising model that subscriptions would churn overnight, but in the same vein as the don't be evil motto being dropped I nod at the scepticism.

When you use Google or another ad-powered search engine, you're saying that it's okay for some company to pay to bias the results away from your best interests.

When you use, say, Kagi, you become the customer; they have a vested interest in providing you with the best experience they can, because they know they're relying on your continued patronage to be able to keep competing with those other free search engines which you already admitted are pretty good.

The other three answers to your question were dumb, so I'll try to do better. Kagi has a free trial of 100 searches without requiring a credit card to sign up. Honest to god, try it. Just bookmark it and search once or twice when Google or Bing is giving you meh results.

As an engineer... googling for stuff is a good fraction of your job. It's quite reasonable to pay to improve that experience. The only real question is what is reasonable to pay.

Kagi allows you to permanently blacklist Pinterest from appearing in any search ever.

It also gives you exactly what you ask for. If you put words in quotes, you only get results matching that phrase. Same with +- modifiers, and all the other "advanced" search operators.

Meanwhile on any free engine, they often completely ignore your query to show you unrelated SEO slop. You can completely forget modifiers and advanced queried, they aren't even parsed. About the only thing that still works is the site: modifier. And I'm pretty sure they only keep that because 40% of google searches include site:reddit

As well, if Kagi can't find a result for your query, it returns nothing. Try searching something incredibly obscure on google or DDG. You get pages upon pages of results and they're all useless garbage or just straight up ads disguised as results.

That's why I pay for Kagi. It stays the hell out of my way and only gives me exactly what I ask for and absolutely nothing else.

You get what you pay for

Then you don't understand what you're getting. If you want to be targeted and misled by opposing incentives, that's on you.

Nothing stopping you from trying for one month to confirm.

You’re getting downvoted but I agree. I didn’t see the value and stopped after a while of paying. It is sad that basically any comment that isn’t pro-Kagi is downvoted but I guess that’s the Reddit-ification of this site.

Most people who complain about Google don’t even use it properly (e.g. PSE).

on the outside are people who could make websites but cant see the point of it. They are on Reddit on facebook on hn on twitter on youtube or they just don't bother. On platforms they make very crappy low effort content (with unearned audience) compared to what they could be making. There is room for casual chatter but almost anyone can make an effort to write down their perspective on something and with some practice enlighten someone or a lot of someones. One might become someone if only in the world of pokemon collectors.

Allowing users to re-prioritize search results directly harms their true product: ads

Organisating a directory of websites, or information about websites, by IP address/range is still common. For example, we can see news.ycombinator.com listed here:

https://www.robtex.com/cidr/209.216.230.0-24

https://www.robtex.com/ip-lookup/209.216.230.240

The above directory has been online for at least 18 years.

Here is one I can remember that disappeared:

https://web.archive.org/web/20090907060026if_/http://onsamei...

> view whatever was hosted at that address (default host in the case of virtual hosting). Not sure why these websites do not persist.

You probably answered your own question here--I imagine at this point the amount of things on the internet that aren't using some sort of virtual hosting are quite tiny.

Even ignoring everything else, with the IP address exhaustion and the push for SSL (can't get a certificate for an IP address), websites available directly at an IP address just aren't really a thing anymore.

I’ve had good results requesting fixes on https://kagifeedback.org/ , even minor/aesthetics stuff.

I do want their browser. Especially with chrome v3 going crapware route. At least there is brave.

Maybe Bing is managed by the same people that haven't managed to get Windows search right despite trying for many, many years?

Use DDG (Bing), append `!g` after you get nothing and search again? Do this with many many different `!`.

You have benefited my life immensely, so I will take the time to write the most helpful response I can.

*What is the nature of competition in the free market?*

The premise that Google’s success constitutes a monopoly that must be dismantled to protect consumers is rooted in a flawed interpretation of the market and the moral principles governing it.

In a free market, companies make money by offering the products and services demanded by customers. Google's dominance is not the result of coercion but its ability to meet customer demand effectively. Antitrust law declares that the choices of billions of individuals are wrong — not because those trading with the company have been treated unfairly (if they had, they wouldn’t have traded), but because those who know better through some mystical connection with the good declare it so using arbitrary standards.

There have been many competitors to Google, with at least one of them extremely well funded. There are other massive internet-scale companies that could compete with Google, though they’ve often failed in past attempts. Punishing the owners of Google for its success is to punish the very qualities that drive progress and prosperity.

You suggest that Google's control over the search index and its associated business models warrant government intervention to force “fair” access, because it would promote competition. Whatever else you mean by “fair”, you certainly don’t mean “terms that Google would find agreeable”. If one of the parties to a transaction is an unwilling participant, then it cannot be a fair transaction. We have names for this when the coercing party is anyone but the government: blackmail, protection rackets, theft, mugging, and so on.

*What about property rights?*

Simply put, Google’s indexes are the property of Google’s shareholders. Their property was created through decades of sustained investment and innovation. To force it to share its core technologies with competitors would be a gross violation of property rights—the very foundation of a free and productive society.

Your proposal to treat Google’s search index as an "essential facility" that must be shared “fairly” is a call for expropriation—the forced redistribution of Google’s property for the benefit of those whose sole claim to it is that they have not earned it.

The entire argument against Google is that they must be punished because they are successful. Kagi, even as a PBC, will one day be subject to the exact same arguments if it creates as much value as Google—someone will claim that some aspect of Kagi’s operations constitute a monopoly and Kagi should be forced to provide access to some data or service.

*What does it mean for the government to intervene?*

You claim consumers are harmed by the lack of choice in search experiences. Well, what about me? I love your product and pay for it. So do another 32,633 other subscribers. Apparently, we are free to choose your product, and I’m certain many of us advocate for your products to our friends and family. But your product isn’t for everyone. Maybe not even for most. If people want Google and free search, and no one’s rights are being violated, by what authority does anyone get to tell them they are wrong?

Consumers choose Google because they think it meets their needs, not because they are coerced into using it.

Government’s only tool is violence. It can force people to do something (rare, and usually ineffective), or force people to not do something (very common, and usually equally ineffective but with disastrous side effects). Government can’t create "more diverse competition" by forcing Google’s to share its property; competition is the result of companies striving to innovate and offer better products. Government can’t level the playing field by forcing the competitors to do better, it can only cut off the legs of the best players to bring them down to the level of the competition.

Were your proposal for the Google index actually implement, I speculate it would be a few short years until you would see the government declare all other company’s indices to be illegal and subject to regulation--after all, if its a public good, then it must be supported and defended by the public against threats. I would expect to see forced mergers of other indices into the Google index.

*What about non-moral arguments?*

I’m not going to go into detail, but based on my education and extensive past research, in good faith I assert the following to be true: there are no documented instances of long-lasting monopolies that are not perpetuated by governments; every antitrust intervention has made consumers materially worse off; government interventions nearly always have the effect of significantly reducing competition.

If you identify of a robust counter example to any of those claims, you could probably win a Nobel prize.

*What could you do instead?*

A judge has made an unjust ruling advancing an unjust law toward unjust ends--that doesn’t mean the conversation should shift to deciding on the best unjust punishments. That would be a contradiction: there is no such thing as the most-just-unjust punishment.

You could continue your education and advocacy of the true cost of “free” search. Take out an OpEd in NYT hammering Apple’s duplicitous claims of being privacy respecting while sending every customer’s most intimate thoughts to the Google monstrosity by default. Demonstrate the cost to privacy. Demonstrate how your product exposes resources that are hidden by the search giants. Show people the super powers they can have for a modest fee.

And build a better product. Hell, make a Kagi Phone—I’d preorder one even if it were years away from release—and free us from the Apple/Google hegemony.