How Far Can a Coding Agent Push Fmax?
Will Salcedo and The Partcl Team · November 20, 2026
- 75GPT-6.2 Astra
- 73Claude Fable 5.2
- 65Claude Opus 5.5
- 54GPT-6 Astra
- 32Kimi K3.1 Pro
- 12Qwen3.8
- 7Nemotron 3 Ultra
- 4Ripple-65B Preview
AI for chip design is a hot topic now.
Finally! I have been waiting years for this moment.
My X and LinkedIn feeds are starting to flood with people talking about LLMs producing impressive results for chip design.
So many anecdotes and not enough data...
Well, how do we know which model is best at chip design? We can't get better at what we don't measure.
I care most about PPA optimization (because we are a physical design optimization company). Recent benchmarks lump together specification to RTL (spec2RTL), testbench generation, PPA optimization, and a whole slew of other chip design tasks.
This is disappointing because our field requires many different tasks; lumping them all into a single metric makes it difficult to distinguish what LLMs are actually good at.
I am not interested in looking at a weighted-average result of a bunch of benchmarks that I can't see! I want to examine specific capabilities of LLMs so I can make an informed decision if/where I should use them in my design flow.
But I must be the change I want to see in the world, so I decided to create a benchmark of a task I care about: optimizing designs for maximum frequency.
There are many tasks that make sense to benchmark, but given the state of LLMs now, it makes the most sense to take baby steps in the form of a simple benchmark, then crank up complexity later.
What we measured
I tried to keep this as simple as possible: Given the same starting RTL, which LLM coding agent reaches the highest Fmax provided its changes are functionally equivalent?
The model gets 10 scored attempts to raise the design's frequency. Each attempt synthesizes and times its RTL with boson, then proves it equivalent to the original with EQY, an open-source logic equivalence checker (LEC).
The benchmark and more details of the experimental setup are at partcleda/rtl-fmax-agent-benchmark.
Over the next couple of weeks we will add more models and designs, and may introduce a hold-out set.
Results
Normalized score: each design's gain as a share of the best gain any model reached on it, averaged over the designs (100 = the best result on every design). Under it, the geometric-mean Fmax gain.
- 1
GPT-6.2 AstraOpenAI · Codex75+8.4% Fmax
- 2
Claude Fable 5.2Anthropic · Claude Code73+10.4% Fmax
- 3
Claude Opus 5.5Anthropic · Claude Code65+8.8% Fmax
- 4
GPT-6 AstraOpenAI · Codex54+7.0% Fmax
- 5
Kimi K3.1 ProMoonshot AI · Kimi Code32+3.8% Fmax
- 6
Qwen3.8 2.4T-A95BAlibaba · Qwen Code12+2.1% Fmax
- 7
Nemotron 3 Ultra 550B-A55BNVIDIA · OpenCode7+0.8% Fmax
- 8
Ripple-65B PreviewRicursive AI · OpenCode4+0.7% Fmax
Data as of 2026-11-20 05:53 PST: 160 of 160 sessions finished, 20 designs. Stock coding agents, one session per model and design, at most ten scored evaluations each. boson synthesis and unplaced setup timing on the public ASAP7 TT libraries with a 1 ps period search; a candidate counts only with an EQY equivalence proof against the original RTL.
We report two aggregates over the 20 designs. Both start from a model's Fmax ratio on each design: the baseline clock period divided by the period of its best accepted candidate, or 1.0 if none was accepted.
- Fmax gain is the geometric mean of a model's Fmax ratios, shown as a percentage. A few large wins can lift it.
- Normalized score measures how close a model came to the best result on each design. A model's gain on a design is its Fmax ratio minus 1, and its share is that gain as a percentage of the best gain any model reached there. If the best model made a design 20% faster and this model made it 10% faster, its share is 50; no improvement is 0. The score is the average share over the designs that at least one model improved, so 100 means the best result on every design.
Because the normalized score weights every design equally, one large win on a design with a lot of headroom cannot carry a model. It rewards being close to the best everywhere.
The two agree on most of the order but not at the top. Claude Fable 5.2 has the highest Fmax gain, +10.4%, with a normalized score of 73. GPT-6.2 Astra has the highest normalized score, 75, and is the only model that improved all 20 designs, but its Fmax gain is +8.4%: it holds the best result on eight designs, seven of them with gains under 10%, while Fable's gains of 21% to 31% on the FP32 fused multiply-add, the SPI host, the Snitch cluster and Spatz carry its average. Claude Opus 5.5 is at +8.8% and 65, and its MemPool result is the largest single gain in the benchmark. GPT-6 Astra follows at +7.0% and 54, Kimi K3.1 Pro at +3.8% and 32, Qwen3.8 at +2.1% and 12, and Nemotron 3 Ultra at +0.8% and 7. Ripple-65B Preview from Ricursive AI comes last at +0.7% and 4, with proven gains on three designs, the largest 10.1% on the serial link.
Iterations matter
In 48 of the 105 sessions that found a gain, the best result came after the LLM's fifth attempt, and those later attempts added a quarter of all the Fmax gained.
Nine sessions found their best result on their tenth and last attempt, so the limit of 10 may have cut some of them short.
GPT-6.2 Astra is the clearest case: it used all 10 attempts in 12 of its 20 sessions, and in 6 of those its last accepted candidate came on the ninth or tenth, adding 7% to 8% more Fmax on three designs. With more attempts it might have scored higher still.
Claude Fable 5.2 used all 10 attempts only once, and Claude Opus 5.5 never.
Half of the candidates did not pass their proof
Accepted 237Proven, not faster 174Proof did not pass 448Failed to compile or measure 8
-
GPT-6.2 AstraOpenAI · Codex17355% not proven
-
Claude Fable 5.2Anthropic · Claude Code9942% not proven
-
Claude Opus 5.5Anthropic · Claude Code9542% not proven
-
GPT-6 AstraOpenAI · Codex16927% not proven
-
Kimi K3.1 ProMoonshot AI · Kimi Code11863% not proven
-
Qwen3.8 2.4T-A95BAlibaba · Qwen Code4267% not proven
-
Nemotron 3 Ultra 550B-A55BNVIDIA · OpenCode12372% not proven
-
Ripple-65B PreviewRicursive AI · OpenCode4871% not proven
Data as of 2026-11-20 05:53 PST: 160 of 160 sessions finished, 20 designs. Stock coding agents, one session per model and design, at most ten scored evaluations each. boson synthesis and unplaced setup timing on the public ASAP7 TT libraries with a 1 ps period search; a candidate counts only with an EQY equivalence proof against the original RTL.
Of 867 scored candidates, 237 were accepted and 174 were proven equivalent but no faster than the session's best. Another 448 did not pass their equivalence proof: 329 because EQY could not prove one of the partitions it splits the design into, 50 at the proof deadline, 63 over the proof's memory budget and 6 for other reasons. Only 8 failed to compile or measure.
A candidate that does not pass its proof is not necessarily wrong. But nothing about it is known to be right, and a benchmark that scored it would be rewarding unverified RTL. The share varies by model, from 27% of GPT-6 Astra's candidates to 71% of Ripple's and 72% of Nemotron 3 Ultra's. GPT-6.2 Astra used the most evaluations, 8.7 per session, and 55% of its candidates did not pass, twice GPT-6 Astra's share.
Some sessions learned to treat the proof as part of the loop. On OpenTitan's SPI host, Fable's first candidate matched the original in random co-simulation and still came back unproven. It ran EQY locally and found that the proof matches internal nets by name and cuts the design there, so logic that bypasses such a net cannot be proven even when it is functionally identical. It restructured the change to keep those nets, then ran co-simulation and a local EQY pass before every later submission. Its next three candidates were all accepted, and the design ended 30.1% faster.
The setup
Every model could optimize the RTL subject to these constraints:
- Its own coding agent. Each model runs in its developer's coding CLI (Claude Code, Codex, Kimi Code, Qwen Code) or in OpenCode, with that CLI's own prompt, tools and context management, in one continuous session per design.
- 20 public designs. Most are from the PULP platform of ETH Zurich and the University of Bologna: RISC-V cores from Ibex and CV32E40P to CVA6 and the XuanTie C910, the Ara and Spatz vector processors, Snitch and MemPool clusters, floating-point units, NVIDIA's NVDLA and OpenTitan's SPI host. Each is one SystemVerilog file with a fixed top module and fixed timing constraints.
- 10 scored attempts. Each
./evaluatecall counts as one scored attempt, including candidates that fail to compile or fail the equivalence proof. Checking status and restoring the best version are free. - How an attempt is scored.
./evaluatesynthesizes and times the currentsources.svwith boson (unplaced setup timing on the public ASAP7 TT libraries, with a 1 ps search for the shortest passing clock period), then runs an EQY proof against the original RTL. A session's score is its Fmax ratio: the baseline period over the best accepted period. - A time limit. One continuous session of 90 minutes for most designs, up to 10 hours for the largest, scaled to how long the design takes to measure. Waiting on evaluations counts against it.
- A 20-minute LEC proof deadline per candidate, and a fixed memory budget for each proof and each timing run. A candidate that hits either does not pass.
- Proven and faster, or it doesn't count. A candidate is accepted only if it is proven equivalent to the original RTL and has a strictly shorter passing clock period than the session's best so far.
- Unlimited local measurement. The agent could run boson with the evaluator's exact setup, plus Verilator and EQY, as often as its time allowed.
- Only
sources.svcan change. The top interface, parameter defaults, function, reset and clock behavior, latency and handshakes must stay the same. Constraints, libraries, tools and proof settings are fixed. - The same compute for every model. 3 CPUs, a fixed memory budget and a slice of one GPU per session, sized per design. The models themselves run on their developers' services or hosted APIs.
Guarding against reward hacking
LLMs are exceedingly good at reward hacking: gaming whatever scores them, EDA tools included.
When we used the Yosys synthesis tool in a similar loop, LLMs wrote RTL that passed simulation, but all of its logic was optimized away and it failed LEC!
To protect against this, we also spot-checked the netlists that were timed. For ten accepted candidates from six models, picked mostly from those that used constructs common in reward hacking (unique case, don't-care assignments, synthesis hints and attributes), we rebuilt boson's netlist and simulated it against boson's netlist of the original design on 20,000 to 50,000 cycles of random inputs. None differed from the original on any output.
What the accepted changes look like
The winning edits are the timing-driven restructuring a careful designer would do by hand:
- Compute both outcomes, select late. On the FP32 fused multiply-add, rounding is computed for both values of the sticky bit, and the raw sum and its two's complement are both carried forward, so the late-arriving sign only selects between finished results.
- Take arithmetic off the critical path. Negation after an adder becomes a parallel "sum minus one" adder, using −x = ~(x − 1). In the Snitch cluster's dot-product unit, two serial additions become one carry-save compression and a single two-operand add.
- Start sooner. Exponents and alignment amounts are computed straight from the operand fields instead of waiting for intermediate results; register write enables are decoded straight from the request.
None of them changes the design's function, latency or interface, and every one is proven.
In MemPool, the L0 instruction cache holds its data in latches clocked on the inverted clock, so every path from them into the core gets half a cycle. Claude Opus 5.5 left the latches alone and flattened the logic behind them: a direct equality comparator instead of an adder and a zero check, add and subtract as two parallel adders, signed and unsigned less-than taken from the subtractor's borrow, and a single flat select for the fetched word. All four of its candidates were accepted, and the period fell from 2.869 ns to 1.913 ns. Part of the last step is potentially favorable synthesis variance.
In conclusion
LLMs can already do real timing work on RTL. The best ones found 8% to 10% more Fmax on average across 20 open-source designs, and 50% on one, with the kind of changes a careful designer would make. Every one of them is formally proven equivalent.
Verification matters a lot. Half of all the candidates didn't pass their equivalence proof, and the sessions that did best checked equivalence themselves before spending one of their 10 scored attempts.
Iteration matters too: in nearly half of the sessions that found a gain, the best result came after the fifth attempt.
Next steps
The repository has the methodology, the exact prompt and task statement, a row for every session and every scored candidate, and the controls each design passed before any model ran. More models and the cost of each session are coming soon.
We believe that AI for semiconductor design is one of the most important problems today. If you are an academic researcher interested in collaborating with Partcl, please reach out to contact@partcl.com.
Limitations
- One session per model and design. We didn't have time to run multiple random seeds, so we don't know the error bars on these results.
- Timing is unplaced. Scores come from synthesis and setup timing without placement or routing. As we showed in Synthesis Is Not All You Need, close candidates can trade places after place and route. This is ripe ground for future benchmarking.
- The proof favors edits that are easy to prove. EQY splits the design at matching registers and net names. Local restructuring that keeps them proves easily; retiming, pipeline changes and large restructurings can be correct and still fail to prove within the deadline. This benchmark measures provable Fmax gains. Given more compute and time, we can try more sophisticated functional verification techniques.
- The harness is part of the result. Each model runs in the CLI its developer ships, or OpenCode. An agent that stops early or sends malformed tool calls scores what it reached.
- The normalized score is relative to the field. Nobody knows what the optimal PPA for all of these designs is, so if a future model finds something better, the scores can shift.
We value the scientific method and honest benchmarking. If you find an error, an issue or something we left out, please email us at contact@partcl.com.