Modly: Open-Source App Converts Photos to 3D Models Using Only a GPU

Converting images into 3D models has traditionally been the domain of cloud services or specialized software. Getting a result meant uploading files to a server and waiting for processing, and commercial use often required paying separate fees. Modly moves this entire process onto the user's own graphics card, aligning with the growing "local-first" AI tool trend. As METAL LAB reported on the 11th in Unsloth Releases Desktop App That Even Handles Local Training, open-source desktop apps that push model execution and generation work down to local GPUs have been appearing one after another this summer.

How to try it

The starting point is downloading the installer for your operating system from the Releases page of the github.com/lightningpixel/modly repository. For those who prefer not to install anything, instructions are also provided for cloning the repository and running it directly.

1. Download and run the installer for Windows, Linux, or Apple Silicon macOS from the Releases page.
2. Open the app, go to the 'Workflows' tab, and check that the default workflow's three steps—Image → Generate Mesh → Add to Scene—are properly linked.
3. Go to the 'Generate' tab, select the workflow you just confirmed, and click the 'Generate 3D Model' button.
4. If the result looks off, check the error log under the 'Settings/Logs/Errors' menu.

To use a wider range of models, go to the 'Models' page, click 'Install from GitHub,' and enter the HTTPS address of the extension repository. If the extension provides model nodes, you'll need to separately download the model body or variant; process extensions, on the other hand, are ready to use as soon as installation and setup are complete.

Possible use cases include feeding in a single product photo to create a 3D preview for e-commerce, or turning a sketch into a prompt to produce a draft game asset. Turning a person's face or portrait photo into 3D form to create an avatar prototype also falls within the app's feature set.

A CLI agents can call without the UI

Modly ships with a stdlib-only CLI that lets a running instance of the app be invoked without a human operating the screen. There are five standard commands—health, model, workflow-run, capability, and process-run—and among them, the more user-friendly generate command internally calls POST /workflow-runs/from-image to run the job, then polls for progress and, on request, exports the finished mesh file. The resulting JSON also includes recovery metadata such as workflow-run status or workflow-run cancel.

The legacy command is a compatibility layer wrapping the old /generate/* endpoints, while dev serve-api and dev ensure-server are development commands that only spin up the FastAPI backend, so they don't guarantee that the Electron desktop bridge is ready. experimental comfy-image and experimental generate-from-workflow are auxiliary features for orchestrating ComfyUI workflows externally, and are classified separately from Modly's standard agent protocol. Related documentation is reportedly organized in tools/modly-cli/SKILL.md within the repository.

Comparing recent local desktop AI apps

App	Key Feature	Supported OS	License
Modly	Image/prompt → 3D mesh generation	Windows, Linux, macOS (AS)	MIT
Unsloth Desktop	Local model execution/training, image/video/audio	macOS, Windows, Linux	Open source
LabLLM	Mini LLM pretraining, SFT, LoRA, DPO	macOS (M1+) only	MIT

All three apps share the trait of running locally without a cloud account and of being open source, leaving room for customization. However, they target different tasks: Modly focuses on 3D mesh generation, Unsloth Desktop on general-purpose model execution and training, and LabLLM on training small language models.

License and community

Modly is distributed under the MIT license, but with a condition attached: anyone who forks the project to build their own app must retain credit to the original author in the app's UI or documentation. The development team is taking updates, bug reports, and feedback through its Discord server.

Editor's take

Modly's message is clear. 3D content creation is moving out of a domain locked behind cloud subscriptions and professional software licenses, into one where all you need is a single graphics card. 3D mesh generation now appears to be following the path image-generation AI walked two years ago — starting out on cloud APIs, then converging, once open-source weights accumulate, into desktop apps that run on local GPUs.

Putting local image-to-3D tools into actual practice tends to reveal the same divide every time — it's workflow flexibility, not mesh quality, that ultimately determines adoption. Modly's choice to connect workflows like nodes, and to let models and processes be extended purely through GitHub repositories, reads as a design meant to avoid lock-in to any single model and to let the community keep adding new ones. Opening up a separate CLI fits the same logic — it signals that the team already had in mind scenarios where a coding agent, rather than a human clicking buttons, batch-converts hundreds of images into 3D as a background job.

In Korea, plausible use cases for this tool include e-commerce product 3D previews, draft asset work at game studios, and pre-production steps for small-scale makers doing 3D printing. That said, since output quality depends heavily on local GPU performance, a safer strategy for anyone needing commercial-grade results would be to use it only for prototyping and refine the final output with professional tools. With the extension ecosystem still in its early stages, how quickly and how extensively the community rolls out new model extensions over the coming weeks will determine whether this app sees real-world adoption.