Experiment
One missing connection
updated
September update: Astra and Fable 5.1 on a much larger diagram.
I made a test to understand the current capabilities of SOTA LLMs in visual content understanding, including the recently published DeepSeek OCR with a bunch of people claiming that it solves the context length limitation, memorization, and maybe your back pain too.
The prompt was simple: “What’s the difference between these two diagrams?”
Show the difference
The arrow from Node 6 to Node 8 appears in A but is missing in B. Node 6 still connects to Node 7 in both.Results
The largest models have mixed results (one-shot):
| Model | Response |
|---|---|
| GPT-5 | Found the missing connection. |
| Sonnet | Said the diagrams were identical. |
| Gemini | Said the diagrams were identical. |
| Qwen | Invented a missing connection elsewhere. |
DeepSeek OCR has also failed the task, although I had to stitch the images because I couldn’t input two of them at the same time.
Check it out for yourself (tested on 12GB 3060).
Update · 13 September 2026
I kept benchmarking every new model release on a much larger pair of screenshots of the CAP system diagram, with one redacted edge. Astra and Fable 5.1 are the first ones to notice the difference.
Show the difference
The green “realize interest” arrow from Interest in the Debt Tokens block to Fee Auction appears in A but is missing in B. Both Astra and Fable 5.1 identified it.See the model responses
