Scriptorium
A dungeon crawler where you draw runes to attack, read by a neural network I trained on players' drawings.
What I did
- Led design and programming on a team of 5
- Designed rune-drawing combat: players draw runes to attack
- Trained multiple iterations of a CNN in Python (TensorFlow, Keras) to recognize player-drawn runes
- Retrained the CNN on playtesters' drawings after messy runes were misread
Problem
Players draw their attacks by hand, and the network has to read sloppy drawings quickly enough that combat still feels responsive.
Goal
Make drawing a rune the attack itself, with a neural network reading the drawing.
Iterations
-
- Before
- Hand-drawn runes were recognized by an earlier version of the CNN.
- What we found
- In playtests, messy runes were misread.
- What changed
- Retrained the CNN (TensorFlow, Keras) on samples drawn by players.
Design
In Scriptorium you draw to attack. Instead of pressing a button to cast a spell, you draw a rune on screen, and a neural network I trained decides which spell it is. The HUD shows how closely your drawing matched a known spell. It is a first-person dungeon crawler with a narrative and a lot of blood.
Process
I trained several versions of a convolutional neural network (CNN) in Python with TensorFlow and Keras. In playtests, messy runes kept getting misread, so I retrained the network on drawings from the people playing. The retrained network was built on how players actually draw.
Technical notes
The game runs in Unity 6 and C#. The recognition model is trained in Python (TensorFlow, Keras).
Challenges
Integrating neural network recognition for drawing input while maintaining responsive gameplay was extremely challenging.
Outcome and lessons
Scriptorium reached a playable demo. It taught me a lot about putting machine learning into a game and about processing player input in real time.
