AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox

METAL LAB

liustack/modlens

★ 3,616+809 this weekTypeScript

A plugin that lets text-only AI models like DeepSeek 'see' pasted images by turning them into structured text data

ModLens is a plugin that gives text-only chat models such as DeepSeek and GLM the ability to understand images. When a user pastes an image, it is sent to a vision engine and converted into structured JSON evidence (transcribed text, layout, entities) that the text model can read and answer from. It installs as a single plugin or skill folder, works across multiple AI coding tools, and can use free or existing logins instead of requiring a new paid vision API.

What it does

  1. What it does: Text-only models like DeepSeek and GLM cannot read images; ModLens intercepts a pasted image, runs it through a vision engine, and hands the model back structured JSON evidence (full text transcription, layout regions in reading order, and lists of entities/relations) instead of raw pixels.
  2. How it works: It offers ten possible vision sources total, including six built-in providers (Gemini API, any OpenAI-compatible endpoint, Anthropic, the free Antigravity CLI, Claude Code, Kimi Code) and reuse of four other local agent CLIs (Codex, OpenCode, Pi, Grok) the user already has logged in, chained together so if one fails the next is tried automatically, recorded in a 'meta.attempts' log.
  3. Installation is minimal: on DeepSeek Harness it's one command-line plugin install; on other supported agent tools (Claude Code, Codex, OpenCode, Pi) it's one skill folder with no changes to existing configs, and uninstalling means just deleting that folder.
  4. Result/proof shown: In demo runs on a text-only DeepSeek-V4-Flash model, it correctly read a tweet screenshot's exact engagement numbers (5.4M views, 1.6K replies, 5.7K reposts, 116K likes) and accurately parsed a dense scatter plot comparing 128 AI models, including axes, log scale, and highlighted regions.
  5. Speed/cost varies by source: built-in API providers answer in about 5-10 seconds, while reused agent CLIs take about 15-45 seconds; a free Gemini API key is the recommended zero-cost starting option.

Why it matters

Many capable coding/chat AI models are text-only, so anyone pasting a screenshot, chart, or UI mockup into them normally gets nothing useful back. This plugin removes that barrier without requiring users to switch models or rebuild their existing agent setup, and its 3,616-star popularity suggests real demand for cheap, plug-in vision support.

Terms in this repo

  • vision engine · An AI service or model that can actually process and understand images
  • OCR · Optical Character Recognition, extracting text from an image
  • failover chain · A backup order where if one service fails, the next one is automatically tried
  • OpenAI-compatible endpoint · Any API server that follows OpenAI's chat request format, so tools built for OpenAI can plug into it too
  • skill/plugin folder · A self-contained set of files that adds a capability to an AI agent tool without editing its core config

Repository description (English)

The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics). | 全网最强 DeepSeek Harness 外挂视觉插件，为 DeepSeek、GLM 等纯文本模型外挂视觉能力，粘贴图片即得结构化 JSON 证据（OCR、版面、语义）。

Open on GitHub

Trending repos

All repos →

Latest from METAL LAB