Hardening STB libraries using AI agents

stb is a collection of single-header C libraries used for a variety of tasks, such as decoding images, fonts, audio, and many other things. However, these libraries have a history of security issues, and many of them are not addressed quickly, making them unsuitable for environments where untrusted input is expected.

Meanwhile, in the last few months, there has been a lot of discussion around the use of LLMs for security research, especially after the introduction of Claude Mythos by Anthropic, followed by the short-lived Claude Mythos 5.

And while I don’t have access to Mythos, I still believe currently available LLMs can be useful for security research and hardening, so here I’m gonna use them to harden stb libraries.

The agent flow

After some experiments, I was able to come up with an agentic setup that allows us to discover and fix security vulnerabilities, you can check out the AGENTS.md used for this.

The main advantage of this flow is that it ensure that the bugs being found are real security issues, and not just false positives, by validating the bugs before patching.

This flow goes through 3 different phases:

• Discovery
• Validation
• Patching

Phase 1: Discovery

In this phase, the agent searches for potential security issues and documents them in BUGS.md.

For the discovery process, there are three different techniques our agent can use:

• Web search: The agent can look up known bugs and vulnerabilities online from different sources, such as GitHub issues and CVEs.
• Fuzzing: The agent can use or implement tools and scripts for fuzzing to test the target against different sets of malicious inputs.
• Static analysis: The LLM can read the source code and flag suspicious sections that may contain bugs or security issues.

Phase 2: Validation

Since LLMs are well known for hallucinations and for reporting security vulnerabilities that do not exist, it’s generally a good idea to validate the bugs that have been flagged earlier, since we do not want to patch bugs that do not exist.

For that, we let the agent implement some automated tests (unit tests or E2E tests) that would let us validate both the vulnerability and the patch for it.

Phase 3: Patching

After we validate the bugs we discovered earlier, it’s time to fix them.

Here, the agent can simply patch these bugs with the minimal amount of changes, then re-run the previously implemented tests to validate the fix.

The results

I have tested multiple AI agents, models and subscriptions, including Opencode, Codex, GitHub Copilot and Hermes.

Suprisingly, all of them gave impressive results, including the free models from Opencode.

At the time of writing, this is the number of bugs that have been found and fixed in stb:

Performance Impact

So I did some benchmarks to measure the performance impact of the patches that have been done, the performance regressions for these libraries was negligible, except for stb_truetype.h.