이미지: METAL LAB 생성
Summary
- Microsoft launched MAI Code 1.1 Flash, a code model for GitHub Copilot, touting a 25% improvement in token efficiency and costs cut to a quarter of the previous version
- It scored 62.9% on Terminal Bench 2.1, far below DeepSeek V4 Flash-0731's 82.7%, and was also more expensive
- The move is seen as a continuation of Microsoft's recent strategy of replacing OpenAI and Anthropic models with its own MAI models in Copilot, prioritizing cost savings
- 신모델
- MAI Code 1.1 Flash (마이크로소프트, 깃허브 코파일럿용)
- 개선점
- 전작 대비 토큰 효율 25% 향상, 비용은 4분의 1 수준
- SWE-bench Verified
- 72.6% (전작 71.6%, 클로드 하이쿠 4.5 69.8%, GPT-5.4 미니 69.2%)
- Terminal Bench 2.1
- 62.9% — 딥시크 V4 플래시-0731(82.7%)에 크게 못 미침
- 출력 토큰 가격(백만 토큰당)
- MAI 1.20달러 vs 딥시크 V4 플래시 0.28달러
- 개발자 코드 수용률
- 전작보다 4%p 상승, 재방문율 9% 증가(마이크로소프트 발표 기준)
On paper, it looks like a win
On the 12th, Microsoft released "MAI Code 1.1 Flash," a code-generation model for GitHub Copilot. The company said token efficiency improved by 25% and costs dropped to about a quarter compared to its predecessor, MAI-Code-1-Flash (released last June). Microsoft also said the rate at which developers actually adopted code generated by the model rose by 4 percentage points.
The training method stands out as well. Microsoft said it trained the model using "hundreds of thousands of reinforcement learning environments within GitHub Copilot." Reinforcement learning is a training method in which a model writes code itself and improves by receiving scores based on outcomes, such as whether tests pass. This means feedback drawn from real development environments was used as training material.
The picture changes once you look at the benchmark table
The problem lies in what's being compared. The benchmark table Microsoft published places the new model side by side only with its predecessor and lightweight models from Anthropic and OpenAI. Within that comparison, MAI Code 1.1 Flash edges ahead slightly. But when DeepSeek's open-source model, DeepSeek-V4-Flash-0731, is added to the same table, the gap widens sharply.
| Benchmark | MAI Code 1.1 Flash | MAI Code 1-Flash (previous) | Claude Haiku 4.5 | GPT-5.4 mini | DeepSeek V4 Flash-0731 |
|---|---|---|---|---|---|
| SWE-bench Verified | 72.6% 73 | 71.6% | 69.8% | 69.2% | Not disclosed |
| Terminal Bench 2.1 | 62.9% 63 | 51.7% | 49.4% | 60.7% | 82.7% 83 |
Terminal Bench 2.1 measures a model's ability to complete tasks by executing commands in an actual terminal environment. In this category, DeepSeek led by nearly 20 percentage points.
Price is also worse
It's not just performance. Microsoft's model is also more expensive per token. Some point out that when usage efficiency is factored in, the actual gap could be even larger than the numbers in the table suggest.
| Model | Input (per million tokens) | Cached input | Output |
|---|---|---|---|
| DeepSeek V4 Flash | $0.14 | $0.0028 | $0.28 |
| MAI Code 1.1 Flash | $0.20 | $0.02 | $1.20 |
| Claude Haiku 4.5 | $1.00 | $0.10 | $5.00 |
According to the outlet The Decoder, Microsoft avoided this kind of direct comparison in its official announcement, instead emphasizing vague improvement metrics such as a "4% increase in code survival rate" and a "9% increase in return usage." It noted that the raw benchmark data was buried inside the model card.
Praising open source while avoiding it
DeepSeek V4 Flash is an open-weight model released on April 22. Anyone can download its weights to run on their own servers, or fine-tune it directly through supervised learning or preference optimization on platforms like Together AI.
Together AI begins supporting fine-tuning for DeepSeek V4 Flash 0731
Microsoft has recently sent several signals suggesting support for the open AI ecosystem. But its actions point the other way. It previously restructured Copilot to reduce reliance on OpenAI and Anthropic models in favor of defaulting to its own MAI models — a decision that prioritized cost reduction even at the expense of performance. The Decoder views MAI Code 1.1 Flash as part of the same trend, arguing that the reason Microsoft continues investing resources in its own model, despite lagging in both performance and price, ultimately comes down to margin management.
So what changes
For Copilot users, the structure that lets them choose among multiple models within the app remains for now. But most users simply stick with the default. If Microsoft locks in its own model as that default, it could create a situation where a model that lags in both performance and price still captures market share by default. With the cost-effectiveness of open-weight models already proven through benchmarks, how far Big Tech's push for closed models can go remains to be seen — something that will become clearer through future changes to Copilot's pricing policy and default model settings.
