yanght27/GPU-Perf-Playground
A hands-on learning path that takes you from writing your first CUDA kernel to running vLLM, DeepSpeed and multi-GPU parallel training
GPU-Perf-Playground is a self-study project built by one developer on a personal RTX 4070 laptop, organized into 29 small step-by-step tasks called Tickets. It walks through GPU operators (CUDA C++, Triton, PyTorch), profiling tools (NCU, NSYS), inference frameworks (vLLM, SGLang, TensorRT-LLM, ms-swift) and training setups (PyTorch, DeepSpeed, ms-swift SFT/LoRA), ending with a comparison of parallelism strategies like data/tensor/pipeline parallelism. Every ticket comes with a runnable script and a written lecture note, so results can be reproduced rather than just read about.
What it does
- 29 incremental 'Tickets' (T00-T28) guide a beginner from basic GPU setup to full AI infrastructure topics
- Same computation (e.g. vector add, GEMM, attention) is implemented five different ways -- plain PyTorch, CUDA C++, Triton, cuTile, CuTe DSL -- so readers can compare approaches side by side
- Covers profiling with NCU/NSYS, then moves to real inference engines (vLLM, SGLang, TensorRT-LLM, ms-swift) and training setups (PyTorch, DeepSpeed with ZeRO, ms-swift LoRA fine-tuning)
- Finishes with a comparison of parallel training strategies: DP, DDP, ZeRO, FSDP, TP, PP, SP, CP
- Each ticket ships a one-command reproduction script and a matching lecture markdown file in docs/lectures/
Why it matters
For anyone trying to learn GPU performance engineering or AI infrastructure from scratch, most material is scattered across papers, docs and blog posts with no clear order; this repo packages a tested, ordered curriculum with runnable code on consumer hardware. It's useful as a self-study checklist or teaching reference rather than a production library.
Terms in this repo
- CUDA · NVIDIA GPU에서 직접 병렬 연산 코드를 작성하는 프로그래밍 모델
- Triton · GPU 커널을 파이썬과 비슷한 문법으로 작성하게 해주는 컴파일러
- NCU/NSYS · NVIDIA Nsight Compute/Systems, GPU 코드 실행을 세밀하게 측정하는 프로파일링 도구
- vLLM/SGLang/TensorRT-LLM · 대형 언어모델을 빠르게 서빙(추론)하기 위한 프레임워크들
- DeepSpeed ZeRO · 메모리를 여러 GPU에 나눠 저장해 큰 모델을 학습 가능하게 하는 기법
- DP/TP/PP/SP/CP · 여러 GPU에 모델과 데이터를 나누는 서로 다른 병렬화 전략들(데이터/텐서/파이프라인/시퀀스/컨텍스트 병렬)
Repository description (English)
GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构
Open on GitHubTrending repos
- cathrynlavery/diagram-designA skill that makes AI coding tools draw magazine-quality diagrams instead of generic rounded boxes
- public-apis/public-apisA giant crowd-curated directory of free APIs for developers
- semantica-agi/semanticaAn open-source graph infrastructure that lets AI agents show their work, not just their answers
- cactus-compute/needleA 14MB AI model small enough to run tool-calling on a phone or watch, without internet
- unslothai/unslothA desktop app that lets you run and train AI models on your own computer, no coding required
- macro-inc/macroAn all-in-one workspace where email, chat, docs, tasks, and CRM are cross-linked and share one AI memory
- harry0703/MoneyPrinterTurboAn open-source tool that turns a single topic or keyword into a finished short video, complete with script, footage, subtitles, and music
- basecamp/omarchyA ready-made, opinionated Linux setup built by DHH
Latest from METAL LAB
- Google Discover adds chatbot that adjusts your feed based on spoken preferences
- OpenAI Closes In on Anthropic Again in Enterprise Spending Share
- Meta Unveils First 10 Tasks in WildArtifactBench, a Benchmark for AI Agents
- Musk: "Optimus + Grok will one day handle healthcare for all humanity"
- 35% of Web Pages Published Since ChatGPT Show Signs of AI Authorship