NVIDIA's 300 Verified Skills Lift Correctness by 41 Points

The term "skill" may be unfamiliar. An NVIDIA certified skill is a packaged, signed capability description — essentially an instruction manual that tells an agent what a specific product does, when to invoke it, and how to call it. Even a strong model can waste steps or burn tokens hunting for the right tool when facing an unfamiliar library, and skills were designed to reduce exactly that trial and error. This lightweight open format was originally proposed by Anthropic, and in early August AWS also released its entire policy-verification pipeline for Bedrock packaged as a skill.

Filtered through three stages

A skill goes through three stages before release. Stage 1 is a safety and structure check. It verifies schema and frontmatter format, scans for possible prompt injection or data leakage, and runs static checks on secrets, personal information, licensing, and script quality. Stage 2 is redundancy analysis, using embedding similarity to filter out repeated instructions within a single skill as well as overlapping skills across the entire catalog.

The real test happens at stage 3. Using the open-source framework Harbor, the agent is actually run twice inside an isolated sandbox — once with the skill installed, once without. The prompt, model, task input, and grading criteria are all held identical, with only the presence of the skill changed. The difference between the two results is the skill's contribution — the "Skill Lift." This comparison is repeated across two different agent harnesses to increase reliability.

The numbers

These figures are based on a benchmarks.json snapshot (commit 738d79e) as of August 12, 2026. The baseline scores for five categories, without any skill installed, were as follows.

Evaluation Category	Baseline Score Without Skill (out of 100)	Lift After Installing Skill
Correctness	46	+41
Discoverability (finding the right tool when needed)	42	Figure not separately disclosed
Effectiveness (goal achievement)	39	+39
Efficiency (reaching the goal without waste)	43	+35
Security (safety regression)	97	Negligible change

Separately, NVIDIA's blog stated the overall average lift was 31 points (including Security), and 39 points when Security was excluded. Since the 41/39/35 figures highlighted in the tweet and the blog's aggregate figures use different methodologies, the two sets of numbers are presented separately by source rather than merged. The direction, however, is consistent — adding a skill improves not just whether the answer is correct, but the entire process of reaching the goal.

How to try it — evaluating your own skills

NVIDIA has released this entire measurement process as an open-source tool called SkillEvaluator. Installation instructions and documentation are available in the GitHub repository. Anyone who builds a skill can run the same evaluation before distributing it.

1. First, build an evaluation dataset with the command skillevaluator create-eval-dataset ./my-skill --full. Adding the --full flag generates an evals.json file that includes not just explicit cases but implicit, contextual, and negative cases as well.
2. After reviewing the dataset, run the actual comparison with skillevaluator tier3 evaluate ./my-skill --agents codex --env-mode docker, which runs and grades the skill both with and without installation.
3. Use the resulting scores and Skill Lift values to identify exactly where the skill helps in practice — or where it actually gets in the way.

For those who want to use the skills themselves, NVIDIA distributes plugins for Claude Code, Codex, and Cursor, and the main catalog is available from the github.com/nvidia/skills repository. The same catalog can also be downloaded from Skills.sh, ClawHub, and Hermes Hub. The original material does not mention any country or plan restrictions.

Editor's take

The description of NVIDIA as not a chatbot competitor but a pickaxe seller once again holds up in this announcement. The difference this time is that what's being sold isn't GPUs, but "a manual that makes an agent actually use your product correctly." Signal-layer chatter recently noted that the community project Hermes checks PII, leaked secrets, unicode smuggling, and licensing using SkillEvaluator when installing skills — unverified information, but it lines up with a trend of this tool being adopted outside NVIDIA's own ecosystem. AWS attaching six skills to its Bedrock policy verification points in the same direction — how to hand agents instructions for tool use is becoming a problem each vendor is now solving separately.

Previously, the approach was to dump an entire library's documentation on an agent and let it figure things out. What these numbers show is that this approach tops out in the 40s out of 100 on a correctness basis — a passing grade of less than half. If simply adding a single skill pushes correctness up to around 87 points, that means the bottleneck was never model capability — it was how well the instructions were written.

For teams in Korea building agents on top of the NVIDIA stack, there's something to do right away: check Discoverability and Efficiency scores with SkillEvaluator before releasing a skill. Correctness depends heavily on the model's own capability, but these two categories are areas that can be directly improved through skill design, making them a faster return on investment. Conversely, pouring effort into something like Security, which is already close to 97, is closer to a waste of resources.

What happens over the next few weeks seems fairly predictable. Competition is likely to intensify as AWS, NVIDIA, and community hubs (Skills.sh, ClawHub, Hermes Hub) each build out their own verification layers around Anthropic's Agent Skills format. Once skills become something version-controlled and benchmarked like code, who verifies them trustworthily first is likely to become the next competitive battleground.