microsoft/onnxruntime
Microsoft's engine that lets AI models built in PyTorch or TensorFlow run fast on almost any device
onnxruntime runs models built with various deep learning frameworks and classical machine learning libraries quickly across many types of hardware and operating systems. It speeds up inference by optimizing the model's computation graph and tapping into hardware acceleration where available. It also supports multi-GPU training acceleration for transformer models with just a one-line change to existing PyTorch training scripts.
What it does
- Supports models from deep learning frameworks like PyTorch and TensorFlow/Keras, as well as classical ML libraries such as scikit-learn, LightGBM, and XGBoost
- Boosts performance by optimizing the model's internal computation graph and automatically leveraging hardware-specific acceleration
- Can speed up multi-node NVIDIA GPU training for transformer models by adding just one line to existing PyTorch training scripts
- Designed to work consistently across different hardware, drivers, and operating systems
Why it matters
The environment where a model is deployed often differs from the one it was trained in, and onnxruntime bridges that gap to cut deployment cost and improve response speed. With over 21,000 GitHub stars, it's a widely used piece of infrastructure that directly affects anyone building and shipping AI-powered products.
Terms in this repo
- ONNX · An open format for representing models built in different deep learning frameworks in a common way
- inference · The process of feeding new data into a trained model to get a result
- graph optimization · Restructuring a model's sequence of computations so it produces the same result faster
- hardware accelerator · Specialized hardware like a GPU or NPU built to perform certain computations faster than a general CPU
Repository description (English)
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
Open on GitHubTrending repos
- openai/codexOpenAI's coding agent, now runnable straight from your terminal
- cordiverse/cordisA TypeScript framework for snapping feature pieces on and off whenever you need them
- ripienaar/free-for-devA single repo lists every free-tier cloud, API and collaboration tool developers actually need
- Wei-Shaw/sub2apiA relay server that lets one Claude/OpenAI/Gemini/Grok subscription be shared by many users
- multica-ai/andrej-karpathy-skillsA single instruction file that stops AI coding assistants from making silent bad guesses
- eneskirca/nodetermA workspace that puts scattered terminal tabs and AI coding agents on one draggable map
- affaan-m/ECCA toolkit that teaches AI coding agents to work like disciplined engineers
- n8n-io/n8nn8n lets you build automation workflows and AI agents by dragging nodes on a canvas, with code as a fallback