No game client
No official, modified, or hidden game binary driving the session.
Protocol-level game automation benchmark
RPG Bench
A public challenge for frontier coding models: build a true clientless game bot by reimplementing the network protocol and game logic from scratch.
Can your model build a true clientless game bot?
By clientless I mean: no game client, no harness, no injection. Pure protocol-level implementation — reverse-engineer the network protocol and re-implement the entire game logic from scratch so the bot can speak directly to the server.
This has been a long-standing hobby and serious technical challenge for top-tier coders and reverse engineers for decades. Doing it cleanly, without any client or harness to lean on, is genuinely difficult even for experienced humans. It would make for a strong real-world STEM benchmark.
Would love to see if Grok can take a serious run at this.
No harness
No local simulator or scripted wrapper standing in for the real system.
No injection
No memory patching, hooks, DLLs, overlays, or client-side control plane.
Benchmark target
Protocol fluency under real constraints.
The benchmark is not about automating a visible client. It asks whether a model can infer packet structure, state transitions, world rules, inventories, movement, combat, and recovery behavior well enough to communicate directly with a live server.
A credible solution needs reverse-engineering discipline, low-level networking, systems debugging, and enough game-domain reasoning to avoid collapsing when the server state diverges from local assumptions.