Settings

Theme

Rolling your own serverless OCR in 40 lines of code

christopherkrapu.com

127 points by mpcsb · 68 comments

Reader
17 threads
eapriv

Not sure what “your own” in the title is supposed to mean if you are running a model that you didn’t train using a framework that you didn’t write on a server that you don’t own.

  • ddevnyc

    I think in this case "your own" means under your control, rather than a service or license you pay for. "your own" as in ownership of artefacts, not as in being the creator.

  • RupertSalt

    Consider the source of the idiom: rolling your own cigarettes.

    Which involves taking some rolling papers, a pouch of loose tobacco (or whatever), and perhaps a little device if you're rich. As opposed to manufactured cigarettes, you're just doing some manual assembly for the end-product.

    You don't need to cultivate the plants or pulp any trees to roll your own.

  • ckrapu

    I originally tried to do this on my own server but my GPU is too old :(

    • LoganDark

      Slammed an A380 in my old server that doesn't even have a GPU power connector & it works pretty well for stuff that will fit on it. They're only like, $150 brand new nowadays; could be a decent option.

  • jen20

    It means "one that is yours" in the same way "running your own plex server" does not imply starting with building a silicon fab.

  • self_awareness

    <insert a random "i made this" meme>

  • croes

    And then call it serverless

  • nkmnz

    Not sure what "baking your own bread" means if you are using wheat grown by someone else in an oven that you didn't build that is run with electricity you didn't created from your muscles' force. You haven't even contributed to the nuclear fusion which created the oxygen for the water molecules you've been using! How dare you, standing of the shoulders of giants!

    • patmorgan23

      Is it "building your own oven" if you go to Lowe's, buy an oven, and installed it yourself? You've done some work, but your integrating a pre-built appliance into your kitchen, not built your own oven

    • ktm5j

      This is more like buying a loaf of bread from the store and saying you baked it yourself. They did nothing even close to making an OCR engine.

    • ranger_danger

      why are chefs baking bread? there's buildings to construct

voidUpdate

Wouldn't "Serverless OCR" mean something like running tesseract locally on your computer, rather than creating an AI framework and running it on a server?

  • cachius

    Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.

    • dsr_

      ~99.995% of the computing resources used on this are from somebody else's servers, running the LLM model.

    • locknitpicker

      > Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.

      Not quite. Serverless means you can run a server permanently, but you need pay someone else to manage the infrastructure for you.

      • Stefan-H

        You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.

        • locknitpicker

          > You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.

          That's not what serverless means at all. Most function-as-a-service offerings require developers to bother about infrastructure aspects, such as runtimes and even underlying OS.

          They just don't bother about managing it. They deploy their code on their choice of infrastructure, and go on with their lives.

          • Stefan-H

            A runtime is notably NOT infrastructure, had you said instruction set you might have landed closer to making a compelling argument, but the whole point is that AWS (and other providers) abstract away the underlying infrastructure and allow the developers to as I said, have "little care of the infrastructure it runs on". There is often advanced networking that CAN be configured, as well as other infrastructure components developers can choose to configure.

      • turtlebits

        Close. It means there's no persistent infra charges and you're charged on use. You dont run anything permanently.

        • dvfjsdhgfv

          It still doesn't capture the concept because, say, both AWS Lambda and EC2 can be run just for 5 minutes and only one of them is called serverless.

          • Stefan-H

            Unless the engineer takes steps to spin down EC2 infrastructure after execution, it is absolutely persistent compute that you're billed for whether you are doing actual processing or not. Whereas lambda and other services are billed only for execution time.

      • jwiz

        Depends if you mean "server" as in piece of metal (or vm), or as in "a daemon"

  • normie3000

    Thanks for noting this - for a moment I was excited.

    • xml

      You can still be excited! Recently, GLM-OCR was released, which is a relatively small OCR model (2.5 GB unquantized) that can run on CPU with good quality. I've been using it to digitize various hand-written notes and all my shopping receipts this week.

      https://github.com/zai-org/GLM-OCR

      (Shameless plug: I also maintain a simplified version of GLM-OCR without dependency on the transformers library, which makes it much easier to install: https://github.com/99991/Simple-GLM-OCR/)

    • mrweasel

      When people mentions the number of lines of code, I've started to become suspicious. More often than not it's X number of lines, calling a massive library loading a large model, either locally or remote. We're just waiting for spinning up your entire company infrastructure in two lines of code, and then just being presented a Terraform shell script wrapper.

      I do agree with the use of serverless though. I feel like we agree long ago that serverless just means that you're not spinning up a physical or virtual server, but simply ask some cloud infrastructure to run your code, without having to care about how it's run.

      • locknitpicker

        > When people mentions the number of lines of code, I've started to become suspicious.

        Low LoC count is a telltale sign that the project adds little to no value. It's a claim that the project integrates third party services and/or modules, and does a little plumbing to tie things together.

      • goodmythical

        >implement RSA with this one simple line of python!

  • esafak

    No, that would be "Running OCR locally..."

    'Serverless' has become a term of art: https://en.wikipedia.org/wiki/Serverless_computing

  • spockz

    Running it locally would typically be called “client(-)side”.

    But this caught me for a bit as well. :-)

  • mindslight

    Yep. That fraudulent term finally got me this time. Totally serverless except for that remote 3rd party server. Sigh.

kbyatnal

Deepseek OCR is no longer state of the art. There are much better open source OCR models available now.

ocrarena.ai maintains a leaderboard, and a number of other open source options like dots [1] or olmOCR [2] rank higher.

[1] https://www.ocrarena.ai/compare/dots-ocr/deepseek-ocr

[2] https://www.ocrarena.ai/compare/olmocr-2/deepseek-ocr

grimgrin

hi. i run "ocr" with dmenu on linux, that triggers maim where i make a visual selection. a push notification shows the body (nice indicator of a whiff), but also it's on my clipboard

  #!/usr/bin/env bash

  # requires: tesseract-ocr imagemagick maim xsel

  IMG=$(mktemp)
  trap "rm $IMG*" EXIT

  # --nodrag means click 2x
  maim -s --nodrag --quality=10 $IMG.png

  # should increase detection rate
  mogrify -modulate 100,0 -resize 400% $IMG.png

  tesseract $IMG.png $IMG &>/dev/null
  cat $IMG.txt | xsel -bi
  notify-send "Text copied" "$(cat $IMG.txt)"

  exit
brainless

I am working on a client project, originally built using Google Vision APIs, and then I realized Tesseract is so good. Like really good. Also, if PDF text is available, then pdftotext tools are awesome.

My client's usecase was specific to scanning medical reports but since there are thousands of labs in India which have slightly different formats, I built an LLM agent which works only after the pdf/image to text process - to double check the medical terminology. That too, only if our code cannot already process each text line through simple string/regex matches.

There are perhaps extremely efficient tools to do many of the work where we throw the problem at LLMs.

coolness

Slight tangent: i was wondering why DeepSeek would develop something like this. In the linked paper it says

> In production, DeepSeek-OCR can generate training data for LLMs/VLMs at a scale of 200k+ pages per day (a single A100-40G).

That... doesn't sound legal

  • Zababa

    HathiTrust (https://en.wikipedia.org/wiki/HathiTrust) has 6.7 millions of volumes in the public domain, in PDF from what I understand. That would be around a billion pages, if we consider a volume is ~200 pages. 5000 days to go through that with an A100-40G at 200k pages a day. That is one way to interpret what they say as being legal. I don't have any information on what happens at DeepSeek so I can't say if it's true or not.

Bishonen88

Tried adding a receipt itemization feature into an app using OpenAI. It does 95% right but the remaining 5% are a mess. Mostly it mixes prices between items (Olive oil 0.99 while Banana 7.99). Is there some lightweight open source lib that can do this better?

lkm0

So I'm trying to OCR 1000s of pages of old french dictionaries from the 1700s, has anything popped up that doesn't cost an arm and a leg, and works pretty decently?

ddtaylor

How does this compare to Tesserect?

apwheele

Question for the crowd -- with autoscaling, when a new pod is created it will still download the model right from huggingface?

I like to push everything into the image as much as I can. So in the image modal, I would run a command to trigger downloading the model. Then in the app just point to the locally downloaded model. So bigger image, but do not need to redownload on start up.

bovinejoni

That book is freely available from its author in pdf format already… but I guess it’s about the journey?

  • velcrovan

    If I had to guess, I would say that this method might be applicable to other books besides the one featured in the post.

  • ckrapu

    I wanted to let an LLM be able to grep and read through it.

sails

Always wondered how auth validation works on these. Could I use your serverless ocr?

jbs789

Why "rolling"? Is this a reference to baking or what's the origin?

fzysingularity

The cold-boot time on this model can hardly be called “serverless”

StackTopherFlow

The real question is why pick python 3.11

PlatoIsADisease

Uh... So I've been telling AI to write a single page html/js OCR app. And I'll include the pdf I want as an attachment.

I have 4 of these now, some are better than others. But all worked great.

zeroq

tl'dr version:

  step 1 draw a circle
  step 2 import the rest of the owl

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection