DeepSeek-OCR: Contexts Optical Compression | Summary

This article explains the key points of DeepSeek-OCR: Contexts Optical Compression.

• 2025-10-21 (arXiv)
• Wei, Haoran, Sun, Yaofeng, Li, Yukun.
• DeepSeek-AI
• Paper
• Github

Read this article in Korean

Summary

• DeepSeek-OCR investigates whether document images can encode long text using fewer vision tokens, image representations consumed by a language decoder, while retaining enough information for optical character recognition (OCR), recovering text from images. On OmniDocBench, a document-parsing benchmark, its Gundam mode uses 795 vision tokens per page on average and achieves English/Chinese overall edit distances of 0.127/0.181, compared with MinerU2.0’s 0.133/0.238 using 6790 tokens; lower edit distance indicates fewer differences from the reference output.
• DeepEncoder combines local image processing, 16× token downsampling, and global image processing before a 3B mixture-of-experts (MoE) decoder, which activates a subset of its expert networks and approximately 570M parameters. On 100 English pages from Fox, a document-understanding benchmark, OCR reaches approximately 97% precision below 10× compression, defined as the reference text-token count divided by the vision-token count; the highest-compression bin reaches 59.1% at 19.7×.
• The experiments support efficient document transcription and parsing, but do not establish that compressed images preserve long-context reasoning or retrieval. The authors leave mixed digital/optical pretraining, training on interleaved digital text and image-rendered text, and needle-in-a-haystack testing, retrieving a target from a long context, to future work; progressive compression of older conversation history remains a proposed application.

1. Introduction

The paper motivates optical compression through the computational cost of long sequences in large language models (LLMs), models that process and generate text. It treats OCR as a measurable compression–decompression task: encode a document image into relatively few tokens, then reconstruct its text. Its contributions are a compression study, the DeepEncoder architecture, and an OCR system with structured image-parsing capabilities.

• Vision-language models (VLMs), models that combine image and text processing, provide the infrastructure for this approach. The introduction focuses on their ability to represent textual information efficiently.
• The authors report production throughput of 200k+ pages per day on a single A100-40G graphics processing unit (GPU) and 33 million pages per day on 20 nodes, each containing 8 A100-40G GPUs. These are reported production figures; the evaluation provides no controlled throughput comparison.

The Fox panel shows how transcription precision changes with page text length and a fixed budget of 64 or 100 vision tokens. The OmniDocBench panel compares parsing error with average token usage, separating the reconstruction trade-off from comparisons with other OCR systems.

2. Related Works

Related work addresses how vision encoders handle high-resolution images and how end-to-end OCR systems recover document content. The number of vision tokens needed for reliable text reconstruction connects these two areas.

2.1. Typical Vision Encoders in VLMs

The authors compare dual-tower, tile-based, and adaptive-resolution encoders. Their analysis motivates processing high-resolution detail locally, then reducing the token sequence before applying global attention, which allows tokens to interact across the full sequence.

• Dual-tower systems such as Vary process images through parallel encoders. The paper attributes deployment complexity and difficulties with pipeline parallelism, splitting sequential model stages across devices, to their separate preprocessing paths.
• Tile-based systems such as InternVL2.0 split images into small crops. They reduce activation memory, memory used by intermediate computations, but low native resolutions can fragment large documents and produce many vision tokens.
• Adaptive-resolution systems such as Qwen2-VL use NaViT, an encoder design supporting different image sizes and aspect ratios. The paper argues that full-image patch processing can require large activations and long sequences, increasing training and inference costs.

2.2. End-to-end OCR Models

End-to-end OCR replaces separate detection and recognition stages with a unified image-to-text model. The paper cites Nougat for academic-document transcription and GOT-OCR2.0 for parsing synthetic visual structures, alongside general VLMs with document capabilities. It asks how many vision tokens are needed to decode a document containing 1000 words.

3. Methodology

The method combines an image encoder and a text decoder with training data for transcription, layout extraction, structured visual parsing, general vision, and language modeling. Training first prepares DeepEncoder independently, then trains the combined DeepSeek-OCR model.

3.1. Architecture

DeepEncoder has approximately 380M parameters: an 80M SAM-base component, based on the Segment Anything image-segmentation model, followed by a 300M CLIP-large component, a vision encoder pretrained with natural-language supervision. A convolutional compressor connects them, and the decoder generates outputs from the compressed image tokens and a task prompt.

• The decoder is DeepSeek-3B-MoE, with 3B total parameters and approximately 570M activated parameters. These decoder counts are separate from DeepEncoder’s approximately 380M parameters.

The compression module precedes global attention, reducing the patch sequence before it enters CLIP. The decoder then receives the resulting visual features together with a task prompt.

3.2. DeepEncoder

3.2.1. Architecture of DeepEncoder combines window attention, attention restricted mainly to local image regions, with dense global attention after compression. SAM-base uses 16×16 patches; two convolutional layers reduce the token count by 16× before CLIP-large processes the resulting sequence. A 1024×1024 image produces 4096 patch tokens that become 256 tokens before global attention.

• Each convolutional layer uses kernel size 3, stride 2, and padding 1; channels increase from 256 to 1024.
• CLIP’s original patch-embedding layer, which converts image patches into input representations, is removed because CLIP receives tokens from the preceding components.
• DeepEncoder targets high resolution, low activation memory, few output tokens, multiple input resolutions, and a moderate parameter count. The paper provides no component ablation, a controlled comparison removing individual components, to isolate each architectural choice.

3.2.2. Multiple resolution support interpolates positional encodings, representations of token locations, to support Tiny, Small, Base, and Large modes at 512×512, 640×640, 1024×1024, and 1280×1280, producing 64, 100, 256, and 400 tokens respectively. Tiny and Small resize directly; Base and Large preserve aspect ratio with padding. Gundam combines local crops with a global view, while Gundam-master is obtained through continued training.

• Equation (1) counts valid tokens as N_valid = ⌈N_actual × [1 − ((max(w,h) − min(w,h))/max(w,h))]⌉, where w and h are the original image width and height. This calculation estimates tokens corresponding to image content rather than padding; the actual token count still determines the sequence budget.
• Gundam uses n local 640×640 views plus one 1024×1024 global view, producing n×100+256 tokens. The tile count is controlled within 2–9; when both image dimensions are smaller than 640, n=0 and the mode reduces to Base.
• Gundam-master uses 1024×1024 local views and a 1280×1280 global view, producing n×256+400 tokens. It is trained separately to avoid slowing joint training of the smaller modes.

The diagram distinguishes direct resizing, aspect-preserving padding, and local-plus-global views. Changing modes changes visual detail and preprocessing as well as token count, so the compression experiment varies both image fidelity and sequence budget.

Native modes span 64–400 tokens. Dynamic modes add 100 or 256 tokens per local view to a global-view budget of 256 or 400, making their total token count depend on the number of crops.

3.3. The MoE Decoder

DeepSeek-3B-MoE activates 6 of 64 routed experts, networks selected for an input, and 2 shared experts during inference. The decoder learns a nonlinear mapping from compressed vision representations to reconstructed text, with the source formulation allowing the vision sequence to be shorter than or equal to the text sequence. The suggestion that larger LLMs could acquire similar decoding abilities through specialized pretraining is a conjecture.

3.4. Data Engine

The data engine distinguishes traditional transcription, called OCR 1.0 data, from structured artificial-image parsing, called OCR 2.0 data. Under 3.4.1. OCR 1.0 data, the authors collect 30M PDF pages covering about 100 languages: approximately 25M Chinese/English pages and 5M pages in other languages. Coarse labels come from fitz, a PDF-processing tool, while fine labels interleave region types, coordinates, and recognized content.

• Fine annotations include 2M pages each for Chinese and English, labeled using layout and OCR models. For minority languages, a GOT-OCR2.0 model trained on extracted text patches labels regions after layout processing, creating 600K samples; different prompts distinguish coarse from fine supervision.
• The authors also collect 3M Word documents to produce image–text pairs, particularly for formulas and tables formatted in HTML, the markup language used for their table outputs.
• Natural-scene recognition uses 10M Chinese and 10M English samples from LAION and Wukong, image–text data sources, labeled with the PaddleOCR recognition system. Prompts control whether detection boxes are included.
• Fine layout annotations normalize coordinates into 1000 bins. Figure 5 illustrates the interleaving of document regions and their textual or structured content.

Each annotated region has a type label and normalized box coordinates preceding its content. The target combines spatial layout with transcription and structured content in one output sequence.

3.4.2. OCR 2.0 data includes 10M rendered chart images, 5M chemical image–text pairs, and 1M plane-geometry examples. Charts are mapped to HTML tables; chemical structures are mapped to SMILES, a textual representation of molecular structures; geometry is mapped to dictionaries describing line segments and coordinates.

• Chart rendering uses pyecharts and matplotlib, plotting libraries, and covers line, bar, pie, and composite charts. HTML-table supervision replaces the chart-parsing method OneChart’s dictionary format to reduce output tokens.
• Chemical data starts from SMILES in PubChem, a chemical database, and is rendered with RDKit, a chemical-structure toolkit.
• Geometry follows Slow Perception, a line-segment encoding method, with perception-ruler size 4. Translation augmentation moves the image while keeping its target structure centered in the coordinate system.

The chart target represents underlying values as an HTML table, while the geometry target records structural elements and coordinates in a dictionary. Both require recovering visual structure beyond transcribing visible text.

3.4.3. General vision data covers captioning, detection, and grounding, linking textual descriptions to image regions, to retain a general visual interface. Under 3.4.4. Text-only data, in-house pretraining text is included to preserve language capabilities. The final mixture is 70% OCR, 20% general vision, and 10% text-only data; text-only samples are processed to 8192 tokens, also the DeepSeek-OCR sequence length.

• The authors describe DeepSeek-OCR as a specialized OCR model rather than a general-purpose VLM.
• The report includes no ablation measuring the separate effects of the three mixture components.

3.5. Training Pipelines

3.5.1. Training DeepEncoder uses next-token prediction, learning to predict the next output token, with a compact language model, all described OCR 1.0/2.0 data, and 100M general samples from LAION. Training runs for 2 epochs with batch size 1280, sequence length 4096, AdamW, an optimizer with decoupled weight decay, a cosine annealing schedule, which varies the learning rate along a cosine curve, and learning rate 5e-5. In 3.5.2. Training DeepSeek-OCR, SAM and the compressor are frozen while CLIP and the decoder are trained.

• HAI-LLM, the training platform, divides the model into four pipeline-parallel (PP) stages: SAM/compressor in PP0, CLIP in PP1, and six of the decoder’s twelve layers in each of PP2 and PP3.
• Combined-model training uses 20 nodes with 8 A100-40G GPUs each, data parallelism, training replicated models on different data batches, of 40, global batch size 640, AdamW, a step-based scheduler, and initial learning rate 3e-5.
• Reported training speeds are 90B tokens/day for text-only data and 70B tokens/day for multimodal data.
• Gundam-master continues training on 6M sampled examples. The paper omits its detailed protocol because it follows the other modes’ protocol.

4. Evaluation

Evaluation covers text reconstruction under restricted token budgets, practical document parsing, and qualitative demonstrations. Fox measures the compression–precision relationship; OmniDocBench compares parsing errors across English and Chinese documents, tasks, and document categories.

4.1. Vision-text Compression Study

The compression study selects all 100 English Fox pages containing 600–1300 ground-truth tokens under DeepSeek-OCR’s tokenizer, whose vocabulary is approximately 129k. Tiny and Small encode resized pages into 64 and 100 vision tokens, respectively, and use the prompt “<image>\nFree OCR.”. Compression is the ground-truth text-token count divided by the vision-token count.

• With 100 vision tokens, the four bins from 600–1000 text tokens achieve 98.5%, 97.3%, 96.8%, and 96.8% precision at 6.7×, 7.5×, 8.5×, and 9.7× compression.
• For 1000–1300 text tokens, Small reaches 91.5%, 89.8%, and 87.1% at 10.6×, 11.3×, and 12.6× compression. Tiny reaches 59.1% at 19.7× in the 1200–1300-token bin, which contains only 4 pages.
• The authors suggest that complex layouts and text blurring contribute to degradation beyond approximately 10×. These explanations are not experimentally isolated.
• Output formatting does not fully match Fox’s reference format. The authors argue that this lowers measured performance, but provide no corrected score or evidence of lossless reconstruction.

Small maintains 96.8% precision in the 900–1000-token bin at 9.7× compression, whereas Tiny reaches 59.1% in the 1200–1300-token bin at 19.7×. The bins contain 4–28 pages, with only 4 pages supporting the highest-compression result.

4.2. OCR Practical Performance

On OmniDocBench, Small uses 100 tokens and achieves English/Chinese overall edit distances of 0.221/0.284, compared with GOT-OCR2.0’s 0.287/0.411 using 256 tokens. Gundam uses an average of 795 tokens and reaches 0.127/0.181, compared with MinerU2.0’s 0.133/0.238 using 6790. Gundam-M†200dpi reaches 0.123/0.157 using 1853 tokens; †200dpi denotes interpolation of the input image to 200 dots per inch using fitz.

• Table 3 reports edit distances for text, formulas, tables, and reading order, as well as overall values. DeepSeek-OCR does not lead every component metric: dots.ocr†200dpi has lower English text and table edit distances than Gundam-M†200dpi.
• Table 4 shows that resolution requirements depend on document type. Newspaper edit distance falls from 0.94 in Tiny and 0.744 in Small to 0.122 in Gundam and 0.099 in Gundam-M.
• Increasing resolution does not improve every category monotonically. Financial-report edit distance is 0.022 in Large but 0.289 in Gundam, as reported in Table 4.
• The authors associate newspapers’ higher resolution needs with their approximately 4,000–5,000 text tokens. This is their interpretation of the category results, rather than an isolated test of token density.

Gundam improves English/Chinese overall edit distance over MinerU2.0 with 795 versus 6790 average tokens. Individual task columns show that the best overall DeepSeek-OCR setting does not lead every metric; rows marked †200dpi also use an explicitly different image preprocessing condition.

Newspaper errors fall substantially under dynamic resolution, while books and slides have lower errors at small token budgets. Improvements are not uniform: the reported Gundam financial-report value is 0.289, compared with 0.022 for Large.

4.3. Qualitative Study

4.3.1. Deep parsing means making secondary model calls on images detected within a document. Figures 7–10 demonstrate chart-to-table extraction, natural-image description, chemical-structure conversion, and reconstruction of simple planar geometry. The shared figure prompt is “<image>\nParse the figure.”, following document parsing with “<image>\n<|grounding|>Convert the document to markdown.”.

• The chart example produces structured values that can be rendered as a table, while the chemical example converts molecular drawings into SMILES and renders the extracted structures.
• The geometry example reconstructs line segments and their arrangement. The authors identify dependencies among segments as a remaining challenge.
• These demonstrations provide example outputs without aggregate accuracy measurements for the secondary parsing tasks.

A financial-report page is parsed for layout, then an extracted chart is processed again to obtain table values. The example illustrates the secondary model call without measuring chart-extraction accuracy across a test set.

The model identifies a photograph within an article and generates a detailed description through the figure-parsing prompt. The output illustrates how the same prompt can yield natural-language description for a photograph rather than structured chart values.

Molecular drawings are extracted from a chemical document, converted to SMILES, and rendered as recovered structures. This demonstrates the workflow but provides no aggregate chemical-recognition accuracy.

An embedded geometry diagram is converted into a coordinate-based reconstruction and incorporated into rendered document content. The authors acknowledge that recovering dependencies among line segments remains challenging.

4.3.2. Multilingual recognition reports support for nearly 100 languages and illustrates Arabic transcription and Sinhala layout-aware extraction. 4.3.3. General vision understanding illustrates image description, object detection, grounding, and language completion. The report provides examples rather than a quantitative multilingual or general-vision benchmark.

• Different prompts select plain transcription or layout-aware output for multilingual documents.
• The authors omit a separate supervised fine-tuning (SFT) stage and state that the model is not a chatbot; some capabilities require completion prompts, inputs that the model continues. This does not mean its OCR training lacks supervised targets.

The Arabic example illustrates plain transcription, while the Sinhala example includes region annotations and extracted content. These examples demonstrate prompt-controlled output formats without establishing accuracy across the nearly 100 reported languages.

The examples include locating text and objects, describing images, detecting objects, reading scene text, and completing language. They illustrate retained task interfaces and completion behavior without a quantitative general-vision evaluation.

5. Discussion

The discussion proposes rendering conversation history older than k rounds into images and progressively shrinking older images to reduce token consumption, where k is an unspecified conversation-age threshold. Figure 13 relates this proposed mechanism to fading human memory and reduced visual clarity at increasing distance. The authors characterize this approach to ultra-long context processing as early-stage work requiring further investigation.

• The OCR experiments demonstrate a reconstruction trade-off as compression increases. They do not test whether progressive information loss preserves useful conversation facts, retrieval accuracy, or downstream reasoning.
• The report does not measure end-to-end latency or memory savings for a conversation system that renders, encodes, and uses compressed history.

The aligned scales draw an analogy between memory age, viewing distance, and lower-resolution document representations. The proposed reduction of older history’s token budget is conceptual; no conversation-memory experiment validates the analogy.

6. Conclusion

The conclusion presents DeepSeek-OCR as preliminary evidence that a compact decoder can recover text from a small vision-token budget, with practical use in producing pretraining data. It explicitly states that OCR alone is insufficient to validate true context optical compression. Planned follow-up work includes digital-optical text interleaved pretraining, needle-in-a-haystack testing, and other evaluations.

Appendix

• The supplied 22-page report contains no source appendix; pages 20–22 contain References. Code and model weights are linked in the abstract at http://github.com/deepseek-ai/DeepSeek-OCR. The supplied version is arXiv:2510.18234v1, dated 21 Oct 2025.

Brief Thoughts

Gundam provides a concrete document-parsing result: lower English and Chinese overall edit distances than MinerU2.0 with 795 rather than 6790 vision tokens per page. The broader context-compression claim remains untested: the reconstruction study covers 100 English pages, its highest-compression bin contains 4 pages, and the report includes no downstream long-context retrieval or reasoning experiment. The category results support validating resolution choices on the intended document distribution, since dynamic resolution helps newspapers but does not improve every category.