# Tiny OCR

In our paper reading group last month I presented DeepSeek’s [Context Optical Compression](https://arxiv.org/abs/2510.18234) paper which claimed amazing performance with a much smaller model (3 billion parameters) than the previous frontier models I had used for OCR (Claude Sonnet and Gemini Flash). Well, last week z.ai released an even smaller (0.9 billion parameters) [OCR model](https://huggingface.co/zai-org/GLM-OCR) called GLM-OCR. The fact that this could cost much less and potentially run faster prompted me to try it out on an old eval. Gemini 3 flash costs around 50 cents per million input tokens compared to 3 cents for GLM’s OCR if you buy it from them (which you don’t [have to](https://endpoints.huggingface.co/new?repository=zai-org/GLM-OCR)). On top of that I decided to run it locally on my MacBook Pro M3 MAX's CPU running pytorch (I tried using the Metal GPU but it ran out of memory, so I fell back to CPU). The model ended up taking a long time to run locally (about a minute per page). However, it performed the same as Gemini 3 Flash on a former project.

This eval has OCR as part of a multi-step process to extract entities.  
The pipeline extracts entities on a wide variety of documents anything from handwriting to a screenshot of a checkout cart at a restaurant.

Here’s how the pipeline works

1. Get input document
    
2. Convert the input document (PDF, XLS etc) into text **(this is the step where i dropped in the new model)**
    
3. Use an LLM (gpt 4o) to structure the raw text into lists of items
    
4. Do database search (RAG — we used HNSW on postgres) for each item
    
5. Have the LLM match each item to a specific Database item from the list we looked up
    
6. Do a QA step where we compare the original Document to the end list  
    

Heres a summary of the results courtesy of [Weights and Biases](https://wandb.ai/):

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1770841255252/d7cf7352-1761-44c4-8f40-4739fd06bfae.png align="center")

Here are some of the things the eval measured

1. did we get the correct number of groupings of items (called lists here — both found 3.75 lists per document)
    
2. did we get the right items (both had an accuracy of 65%)
    
3. was the item bucketed into the correct category (GLM had an accuracy of 76% and gemini had 75%)
    

As you can see the performance on the evals between the two models were very small. Since I ran GLM-OCR locally it took much longer than gemini (20 minutes for the run vs 4 and a half with gemini).

**Should you switch to GLM-OCR?** Based on these results, yes - especially if you're processing documents at scale. While my CPU-based local run was 60x slower than Gemini Flash, that's a hardware limitation, not a model limitation. Running GLM-OCR on a proper GPU setup (either self-hosted or through [z.ai](http://z.ai)'s API) should match or exceed Gemini's latency while costing 16x less (3¢ vs 50¢ per million tokens). For our use case processing thousands of receipts and invoices monthly, that cost difference is significant. I'm planning to move our production workload to GLM-OCR once I set up proper GPU infrastructure.

One final note, when we initially did this project I was shocked to find that actually the best tools for OCR were LLMs. The fact that now you can match the best OCR performance with a tiny, cheap LLM I find astounding.