New Obsidian Search Plugin: Combines Fzf, Telescope, OmniSearch and Embeddings
ssp.shSo what figure exactly is tested for 10k+? Is it 12k, 100k or more like 1m? I couldn't find.
The "exact" tests were made with 12k notes in my test vault and 10k in my personal vault. I haven't tested for 100k, but it would be interesting, do you have a vault or example of such a vault?
I know people hit the 10K really fast. 10K is the limit for which obsidian switches algorithms (personally, I don't understand how 10K db rows would be a problem, but ok). It is very easy to run into 100k:
- external sources like the documentation for your programming languages, frameworks etc that come as markdown
- if you store every entity as a note, like you do with obsidian bases. Think task tracking, habit trackers, book collections
- clippings, daily notes, project notes, zettels etc
So if I would design a search improvement, I would test the 100k, 200k, 500k, 1m mark with synthetic data.
Tested it on 100k notes, the fast find plugin works quite well. a bit slower, instead of a 3ms search result on a 10k vault, it takes up to 300ms in some cases. Indexing was fine, but Obsidian's own index took forever to finalize, and the whole Obsidian app was not really made for such big vaults. But my plugin seems to be fine :)
Congratz! And thanks for testing!
yeah good point, I will add more docs and try that, and I doubt it will have any problems.
it should be doing fine on any size, but I noticed OmniSearch hits limits around 10k notes for me, you could also use it way before and way over. But I guess if you have fewer than 4k, you might not need sophisticated search, except if you enjoy the added features such as preview and searching by title, headers, and semantics.