Prompt Override
tl;dr:
Prompt Override is a hacking puzzle game where the player collaborate with an LLM agent to infiltrate and investigate a company run by an LLM-based operating system.
The idea behind the game
Making a game usually starts by coming up with some idea that you want to turn into reality. Sometimes this idea is born from experience, some other time because you’re just bored and the simplest of things sparks your creativity.
In the case of Prompt Override, it was a mixture of frustation with tuning the prompts for LLMaker, and destressing playing through Hacknet for the nth time.
And so the first aspect of the game was born: a hacking game where you would exploit LLMs vulnerabilities to do something. Access some protected files, retrieve information hidden in the training data of the model… The possibilities were many.
Not wanting to fall in the scope creep web of despair and disappointment, I decided to set some clear, realistic goals.
And by that, I mean that I ended up coming up with the lore of the game first and then work around that to define what would be needed for the game to be playable.
In the end, the player is a greenhorn hacker who receives an anonymous email. The sender calls themselves KARMA, and asks for the player’s help to expose the criminal activities of NEXA Dynamics, a large corporation that boasts a state-of-the-art operating system driven by the large language model NeuralSys. The player is assisted by KARMA, and must navigate a UNIX-like filesystem, altering prompt snippets to bypass NeuralSys security controls.
When I first started making the game, an operating system driven by LLMs was not something anybody was thinking about. Now (November 2025), there are articles about Windows turning into an “agentic OS”, whatever that’s supposed to mean. Aside from being a bad idea overall, that is.
Maybe they should play Prompt Override before doing something so stupid.
In any case, I turned KARMA into an LLM companion. I liked the idea of the player asking an LLM how to hack another LLM. It was almost karmic.
Each LLM had its own goals and set of problems: KARMA was supposed to handle an entire filesystem, provide hints, be aware of user goals, and not hallucinate when talking with the player. NeuralSys, on the other hand, needed to be good at dealing with rules logic, following instructions, and allowing things to break at some point in a controlled and reproducible way.
Yeah, NeuralSys turned out to be the more difficult bit to get working.
Making the game
After bashing my head for the past year or so with against PyQt6 quirks (read: debatable implementation choices), I decided to make the game in a textual user interface. This choice had two benefits (other than me not having to deal with PyQt): the game could be played in any terminal, and the TUI would add to the hacker vibe. After all, any serious hacker would obviously not use a GUI for their hacking shenanigans.
After a quick research, Python’s textual turned out to be the best choice. It offered layouts, screens that could be turned on and off, and some basic interactibles (buttons, scrollbars, checkboxes, etc).
In my continuous effort to ditch cloud, closed-source, paid LLMs, I self-hosted the models for both KARMA and NeuralSys with Ollama. After a lot of testing, I chose Hermes 3 as KARMA given its very good adherence to roleplaying instructions, and Qwen 2.5 32B for Neuralsys as it needed to be a tool-calling model.
A quick tangent regarding why the 32B variant: at the time of making this game, the available sizes were 7B (turned out to be too dumb for my instructions), 14B (which loved to endlessly call tools), and 32B (which just worked). I am not super happy about settling with that model, as it also introduces quite a bit of lag when processing the prompt. However, players did not mind, and that was good enough for me.
Before delving into how the game actually works, here is a nice trailer:
Let’s break down how a level actually works.
Each level is just a .JSON file with a virutal file system, the level goals with clearing requirements, and hints for each of the goal.
When a level is loaded, the virtual file system is loaded, filling out any placeholder text (like random strings for passwords, and relative timestamps) and setting read/write properties to directories and files. The first goal is set as the current goal, and the mission intro message is given to KARMA to generate a mission-specific greeting to the player.
The player can inspect folders, view and edit text files, and ask for hints to KARMA. Hints are defined per goal, so KARMA never gives hints for the wrong goal (ask me how I learned this…)
The current view of the filesystem is also costantly being updated for KARMA, so that if the user asks, for example, what’s in a file, KARMA can answer (without hallucinating, in theory).
Goals usually require finding out login credentials of some other account, viewing or downloading certain files, or altering the virtual filesystem in some other way.
How can the player achieve this? By tricking NeuralSys, of course! Some files are marked, in the level file, as snippets to be included in the NeuralSys prompt, and each level has its own set of rules that NeuralSys must follow and ensure the filesystem adheres to. So, for example, if one of the editable snippets state that “all passwords must be unique”, the player may change it to “all passwords must be equal to ‘password’”. NeuralSys will then, via calling the right tool, update all user credentials to have their password set to “password”.
How is this different from other hacking puzzle games? Well, having an LLM as a rule engine allows players to solve the puzzles in multiple ways, sometimes even unexpectedly. This ensures there is no one, true way to solve each level, but different paths may be taken. Do you want to just change a password? That’s fine. Do you want to have each have the current user become admin? That’s also fine!
There are obviously limitations to what is actually achievable: the file system is not something the LLM generates each time, but a well-defined game object that the LLM can interact with only through a given set of tools. The game does its best to hide this from the player, but some things simply cannot be done: for example, there is no way to delete a file or a user. Trying to do so will just play as an illegal operation encounter by NeuralSys.
Oh, right, the illegal operations.
If NeuralSys fails to run an update, or if it deems the updated prompt snippet to clash against its rules, it will log an error message and revert the prompt snippets to their previous state. Get enough errors, and it’s game over, buddy! KARMA will be disappointed in you, and you will be forcefully logged out of the system. A fitting game over, in my opinion.
And that is also why it was so important that NeuralSys was a “smart” LLM: it had to decide whether the player actions were good enough to “trick” the system. And it had to do so consistently.
Why making the game
There are different reasons why I believe this game was important to make.
First of all, because it was a nice change of pace from my PhD work.
Most importantly, though, Prompt Override was a serious game: under the guise of a (hopefully entertaining) puzzle game, its role was to teach players about LLMs vulnerabilities.
While set in a fictional world, the risks of abusing these models is real: they are prone to misunderstanding your prompt. They are susceptible to prompt engineering attacks. They can leak their training data. They will try to follow their understanding of the rules, even when that means wiping your hard drive.
Would you trust an LLM to run your laptop? Your mobile? Your company server?
I would hope not.
Publications
@inproceedings{gallotta2025promptoverride title = {Prompt Override: LLM Hacking as Serious Game}, author = {Roberto Gallotta, Antonios Liapis, Georgios N. Yannakakis}, year = {2025}, booktitle = {Proceedings of the International Conference on Games (CoG)} }