OpenAI adds PDFChat feature to ChatGPT
twitter.comI was very excited when this came out. The idea of "search this textbook and describe ..." or "search through this paper and implement X algorithm" sounded really interesting.
Admittedly, I don't know how the implementation works, but I was expecting it to be able to do a search on the pdf to find the relevant parts and answer requests, but my results have been really bad. It makes up stuff seemingly more when I ask using a pdf than when I ask it without a reference.
What is this? The ability for chatGPT to parse text in PDFs? Couldn’t it already do OCR on text in images?
It's way bigger than that! The new features will basically try to write python on the fly to do anything you want with several types of supported media. Example: OCR all the frames on an uploaded movie.
Depending on how well it works, it can be much better - accepting longer context sizes etc
Very recently it got vision which can be used for OCR on a single image, but that's massively inefficient and limited compared to what this is likely doing for longer documents
I tried using the vision feature for OCR & it was worse than Tesseract. At least for financial documents where you need exact numbers amounts. Will the new PDF feature be better? I’m not so hopeful.
Using the vision feature for OCR is like using an LLM for math: it might work, but we already have a lot of tools that are hyper-optimized for the task.
There is practically no chance the new feature uses vision because that'd be _insanely_ slow and expensive for any reasonably sized document. They're likely using Azure's LayoutLM derived tech to get out text, then using embeddings to answer on questions
Will it be better than Tesseract & other OCR tools?
I guess they are rolling it out gradually because mine hasn't changed. At least not on mobile web. Haven't looked at desktop.
What is up with them releasing big new features and not even posting about them? Like adding image generation, and things like the advanced data analysis were barely announced.
This pdf feature seems very useful, but there's no instruction on what it's doing under the hood or how to use it best.
Do those features (DALL-E, plugins, etc.) also work on the openAI developer playground? I can't find it anywhere
the free bing chat has had this for a while no?
Its been the case that most OpenAI/chatGPT features launch on bing chat first. Maybe part of the investment deal they have with Microsoft.