bex edmondson | Blog

Hexbuilder: Devlog #3

Sun, 25 Jan 2026 00:00:00 GMT

Catch up on the Hexbuilder progress starting here!

Announcement

I’m happy (and slightly scared) to publicly commit to releasing Hexbuilder this year! I’m more excited about this project then I have been about any other, and I think have the ability, capacity and resources to get it there. I’m not sure of the specifics of how yet, but saying I’m going to is the first step. So: I’m going to!

Now that’s out there, on to what’s changed since my last update…

Screenshots

Section titled “Screenshots”

Updates

Section titled “Updates”

In my last post, I talked about adding storage and continuing to improve information accessibility, and that I was considering adding a concept of resident happiness. I have updates on all three of those, plus more!

Storage

Section titled “Storage”

We have storage! Resource quantities are now capped, with caps increasing based on which buildings are present in your town. This has added a lot of needed complexity to the game and provides good motivation for players to build more. Also, due to the added restrictions I’ve been able to delineate very early gameplay decisions that will teach players the basics.

And I added a couple of new buildings to fit the theme, like this stockpile!

Information accessibility

Section titled “Information accessibility”

I spent a lot of time improving which information was displayed and where before my last post, but there was still a lot left to do! I’ve added a lot and improved a lot of existing visuals, and refactored things to simplify implementing UI as I go in the future.

Adjacency effects

Section titled “Adjacency effects”

Adjacency effect visuals got a big visual improvement with this fancy effect:

It took about 3 days to get working right, but I’m really pleased with the result!

Toasts

Section titled “Toasts”

A new notification system that lets the player know all sorts of information! I originally used the Toast Party plugin, but it wasn’t as flexible as I needed and was written in GDScript, so I ported it over to C#. And then completely refactored it about three times! I ended up with something really clean and flexible - I even used this system to display a currency change animation¹.

Resident happiness

Section titled “Resident happiness”

Easily the biggest gameplay change since last time: residents gained sentience! Now each one can have a range of happiness values, affected by a number of different needs.

Needs get assigned under specific conditions and have a list of satisfaction requirements. These are communicated in the reworked residents popup, and provide a sort of rudimentary tutorial or achievement system - a bit like wants in The Sims.

Resident needs don’t currently have much of a direct gameplay impact yet, though that should be very straightforward to change if I think it’s needed. Changes in happiness do get communicated to the player through the new [[#Toasts|toast system]] though so they’re quite front-and-center for motivation purposes. There’s a lot I want to do from here to add depth to this system, so this will be something I work on soon.

Big changes

Section titled “Big changes”

I got a whole lot done other than following up on what I mentioned last time! Here are the three biggest headlines:

Worker counts have more meaning, part 2

Section titled “Worker counts have more meaning, part 2”

Worker counts gained a purpose in my last post, and in this one I added something small but powerful - maxxing out worker capacity at a building can now provide a production bonus, indicated by this nice little shine effect:

And I added this feature to buildings using…✨components✨!

Componentising building data

Section titled “Componentising building data”

A huge overhaul of how building data is stored! I used a pattern that I’m very familiar with from previous work, and broke each data chunk into its own class which inherits from AbstractTileDataComponent. Each CustomTileData instance then has just one array of abstract components, and at runtime these are mapped into a dictionary with the component type as the key. There are some minor limitations around component type inheritance, but overall I’m way happier with this system. It’s much, much more flexible and readable, and makes adding new types of building data very easy.

Locked cell visuals

Section titled “Locked cell visuals”

I figured out how to grey out individual cells using alternate tiles, so the ugly hexagonal overlay is now gone!

What’s next?

Section titled “What’s next?”

Making the map more dynamic by adding map interactions, e.g. clearing forest tiles, growing grass
Adding more resident needs and building adjacency effects
Adding more buildings
Solidifying the order in which different resources show up in early game
Adding adjacency requirements to new buildings, e.g. ensuring a smelter must be built next to a mine

Footnotes

Section titled “Footnotes”

Admittedly, this is kind of a hack. But it works! ↩

Hexbuilder: Devlog #2

Tue, 23 Dec 2025 00:00:00 GMT

Catch up on the Hexbuilder progress starting here!

Screenshots

Section titled “Screenshots”

What have I done since the last post?

Section titled “What have I done since the last post?”

I have made a LOT of changes since the last time, but I want to get this post up before the changelist grows so much as to be overwhelming. So here’s the quick-fire round :)

3D Modelling

Section titled “3D Modelling”

I started some very rudimentary modelling in Blender a little bit! Began by merging existing assets to switch up some things, but then I had to get brave and actually build something as there was no existing asset for the weaver building I wanted to make. So I made a loom! Very simple but I think it fits nicely with the art style of everything else, and it communicates what it needs to. Plus I learned a lot for the next time I need to do something like this :)

My biggest slowdowns in Blender have all just been knowing where things are. I fully expect to get a lot faster with time, and depending on if I feel like I hit a plateau somewhere, I’ll take a detour and do some tutorials to learn things a bit more “properly”. But my current haphazard picking up of things as I need them is working alright for me at the moment at least!

New buildings and currencies

Section titled “New buildings and currencies”

I built out some upgrade paths a little bit with some new buildings, including some that produce the new currency I added this month: fabric!

Showing info everywhere

Section titled “Showing info everywhere”

The unlocked cell popup has been completely overhauled, and now has a ton of information on it. All buildings show the default production rates for the selected building, plus more specific info depending on the kind of building it is. For example, if it’s a workplace, this popup now shows the effects provided to any relevant buildings around it, and any effects received from surrounding buildings as well. Also you can assign and unassign workers from this popup as well, in the same way that you can from the overall workplaces popup - much more convenient for one-off worker assignments!

I also added an encyclopedia popup - basically a list of all available buildings and what they do. I added this because in order to make progression desirable, you need to know that progression exists! So I thought that I should prioritise showing buildings that you have yet to unlock - and as a nice visual touch, I also wrote a very quick greyscale shader that I’m using to show which buildings are locked.

Tool improvements again

Section titled “Tool improvements again”

As I was introducing more building upgrade paths, I thought it would be useful to be able to visualise them. So now I can! It was very straightforward to add this into the existing placement tool I made, and it’s made things much faster.

I also added a shortcut to be able to jump to the focussed view of a tile that’s in the upgrade tree. A small but very convenient upgrade!

Worker counts mean something now

Section titled “Worker counts mean something now”

Until this point, having more than one worker at a workplace had no effect on anything. This is still the case, but I’ve now laid the groundwork for allowing production rates and adjacency effects to change based on worker count.

Adjacency effects are now based on the number of workers in neighbouring workplaces, rather than the number of adjacent workplaces overall. Building production¹ is a bit more complex, and not yet implemented - the idea is to have a baseline production at a workplace for one worker, and then an increase for each subsequent worker. I’m also considering a double increase for the last worker to make maxxing out a workplace feel extra fun, though that might be a bit overpowered - I’ll have to try it and see!

Godot Updates!

Section titled “Godot Updates!”

I moved to Godot 4.6 beta 1 for the native Wayland game embedding! No more blurry engine fonts! And the new theme is just icing on the cake - the dark grey is so much nicer on the eyes.

Obviously being on a beta version comes with a few bumps in the road - e.g. turning on a specific setting in the font import window seems to break my theme at the moment, still narrowing down the full cause for a bug report at the moment - but it’s SO worth it. I’ll likely keep on the latest beta version until 4.6 comes out properly.

What’s next?

Section titled “What’s next?”

My main focus from a design perspective will be adding some gentle pressure to pursue the progression that I’m building out. Currently it’s pretty easy to find a balance where all your resources are increasing, and there’s not a whole lot of reasons to expand the map and build more if that’s where you are. So changes must be made!

Storage

Section titled “Storage”

Firstly I’m thinking of limiting storage based on buildings, which will mean players are incentivised to keep expanding the map so they can build capacity to hold enough resources for building higher-level buildings. There are already some buildings that I have implemented that could do with more purpose, like the lumber pile. Also, adding storage to buildings like houses gives me more parameters that I can use for upgrades, without scaling up resource production really fast.

Resident Happiness

Section titled “Resident Happiness”

The other way I’m considering adding pressure is to add a happiness score, continuing the theme of focussing on residents as motivation. I’m thinking that each resident should have an individual happiness level, influenced by…well, I’m not sure yet. Here are some ideas I’ve had so far:

Neighbouring buildings to their residence can provide happiness
After a certain number of turns, happiness drops if their residence isn’t above specific level
Residents make timed requests for the player to build specific buildings or produce a certain number of resources, and happiness drops if a request isn’t fulfilled
If a resident is at minimum happiness, their housemates and colleagues’ happiness starts decreasing as well
Depending on the number of total tiles unlocked (or some other similar measure of player progression), residents require specific buildings built to remain happy

I’m still not sure if I’ll implement any of these, but at the very least I’ll be thinking about it. Watch this space!

Anything else?

Section titled “Anything else?”

Improving progression some more! Continuing to add more buildings, particularly improved versions of existing buildings rather than just new buildings.
Making even more information visible: in particular, unlock requirements for future buildings need to be in the encyclopedia.

Footnotes

Section titled “Footnotes”

note: this only applies to buildings with workers! Other buildings’ non-adjacency-bonus production is just flat ↩

Hexbuilder: Devlog #1

Sun, 30 Nov 2025 00:00:00 GMT

If you haven’t read devlog #0, I would strongly recommend reading that first so this post makes sense!

Screenshots

Section titled “Screenshots”

Finding some direction

Section titled “Finding some direction”

Since my last post, I have made some pretty big steps forward in determining player motivations in Hexbuilder. I had a very helpful conversation with my sister (thanks Beth, if you’re reading!), and as a result decided that since cities are made of people, my city builder needed more emphasis on its residents. Before, I had people set up as one of the resources/currencies that could be produced and spent by buildings, and since then I’ve completely removed that, and now individual residents are individual entities.

Now, I have a set of manager classes that handle residents and what they each do. Residents are created and assigned to housing and workplaces by the relevant manager class, and the presence of residents directly determines workplace production at the end of a turn. Currently, workplaces must have at least one worker assigned to them, though I’m considering revisiting that later - for now, if a workplace has no workers then I’ve added a little ”!” over the workplace’s info UI. Also, adding more workers doesn’t do anything yet, but that definitely will change in future.

Timed Jobs

Section titled “Timed Jobs”

I also created a manager class for what I’m calling “timed jobs” - something that means a resident can’t be working anywhere else that turn, but only exists for a set number of turns and frees up the residents afterwards automatically. Currently, the only timed job is the process of unlocking a new tile, but I could see this expanding to cover things like foraging on a tile or repairing a building. One thing I don’t love is how similar a lot of the code is between the WorkplaceManager and TimedJobManager, but I’m not going to refactor thise to make a shared base class unless I work on something else that needs it - there’s enough to do elsewhere!

Names

Section titled “Names”

One last thing I did was to give the residents silly names, randomly picked from a list that I wrote one evening. This was partly so each resident was easier to keep track of when debugging, partly because I was procrastinating, and partly because making up silly names is extremely fun and surprisingly hard to stop once you’ve started!

UI improvements

Section titled “UI improvements”

One part of building out all of the above was adding in a few info screens. These did double duty first as helpful debug tools for myself, but also will serve as a way to get information to the player.

The UI is still very much placeholder-filled, but I brought in a free theme to make everything a bit easier to read and navigate: Clashy, by Wadlo.

I also set up a way to add UI to show about cells over the top of those cells, and added those for about workplaces and timed jobs.

What else did I do?

Section titled “What else did I do?”

Set up better resource production values for the buildings that I’ve been testing with.
Added an event dispatcher class that I had from another project, allowing me to dispatch and listen to events.
Set up keyboard shortcuts for accepting options in specific popups and closing popups.
Added a common base class to popup classes to handle the above in one common location.
Refactoring lots of things, notably the cell unlocking to allow cell unlocking via timed jobs (as mentioned above).
Made some usability improvements to the tiles tool, mainly alphabetising building names in the places they’re listed.

Had a couple of tries at fixing a confusing null ref that’s appearing in the tiles tool code after Godot rebuilds the project. I haven’t yet gotten rid of it fully because I’m not sure what’s happening, it’s something to do with the data associated with a graph node being re-assigned after build. But I think I did at least reduce the frequency that it appears, so depending on how annoying the effects of this error are, I might leave it as-is.
And as a little bonus to myself, I tweaked the locked tile overlay to remove the irritating gaps that showed when the texture was tiled.

Next steps

Section titled “Next steps”

I need to upgrade the unlocked cell popup to be more specialised and show more info for workplaces and buildings. I also want to show the effects that a building has on surrounding tiles, and to do that I’ll need to center the building in the screen while this popup is open.
I want to separate out the concept of terrain from the concept of buildings. I partly set it up like this for simplicity, because all the models have terrain embedded in them, but I think I need to separate them in order to have more flexibility when it comes to building out the building tech trees.
Talking of which, I want to add more to the upgrade paths for some of the buildings, and visualise that somewhere in the UI. I think I might also need to add a tool for this, though I may be able to deal with things as they are.

Hexbuilder: Devlog #0

Sun, 09 Nov 2025 00:00:00 GMT

Hexbuilder is the name¹ of a personal project I’ve been working on here-and-there for the last few months. As such, I’ve already done a decent chunk of work on it, but I decided that for accountability, I should start posting about it publicly! So this post is to summarise what I’ve done so far and give context for future updates.

What it is

Section titled “What it is”

Hexbuilder is a turn-based, city builder, resource management game. As the name implies, the map is laid out on a hex grid. The central mechanic is based on building specific things on tiles: each tile you can build will consume and provide different resources when each turn ends. Tiles also have effects on neighbouring tiles, e.g. a smelter next to a mine will produce one more stone per turn.

Unlocking new terrain tiles to build on also costs resources. Additionally, there are restrictions on which types of tiles can be built on which types of terrains, e.g. a mine can only be built on a hill or a mountain.

What I’ve done so far

Section titled “What I’ve done so far”

Created the first draft of the hex map
Implemented tile purchasing and tile placement
Processed the 3D tile assets I’m currently using into 2D assets for use in Godot’s tile map system
Set up a data structure to hold game config data for each tile
Implemented turns, including resource quantity changes based on existing tiles
Implemented tile adjacency effects
Built a tool to more easily view and change which tiles can be placed on which terrain types
Expanded the tiles tool to view and change tiles’ adjacency effects on other tiles
Quick and dirty first pass at camera movement
Ultra basic placeholder HUD with a next turn button and currency display
Even more basic placeholder popups for unlocking a tile and building a tile
Noise-based terrain generation
First draft at river detection in generated terrain

Screenshots

Section titled “Screenshots”

What’s next

Section titled “What’s next”

The main problem I need to solve is…well, fundamentally that I’m not a game designer! Developing games - well within my skill set. But designing is a whole other matter. I’ve got the mechanics, and I have them functioning to a reasonable degree, but there are bigger questions that I need to figure out. For example:

what is success?
what is failure?
is there even a fail state at all?
what is progress?
what is the goal?

…and so on.

Such nebulous, open-ended questions! Ordinarily, my job is based on having the end goal provided to me, and then figuring out the best way to make that happen, balancing considerations like performance, architecture, speed, and whatever else. Finding the answers to questions like the above is such a different way of thinking - I need to decide the end goal, and how players should reach it. This seems so difficult! I have no idea how game designers do this every day!

To be honest, I’ve gotten to this point with previous projects quite a lot, where I have the central mechanic but none of the bigger progression questions answered, and in the past I’ve just moved on to something else. But I’m more determined this time around, so I hope to post devlog #1 with at least some of the above questions answered.

My very early thoughts are around including some kind of tech-tree style system to answer some questions around progression. I think I could use the tile placement settings to make buildings upgradable, and then separately I would need to determine how buildings get unlocked.

I also would like to modify the tiles to be more flexible. Currently, the building tiles contain the terrain, which means that I’m quite limited in determining which buildings can go on which terrain, and I think this might end up limiting me a fair amount when it comes to the tech tree concept as well. What I have currently is workable, but I may start looking into this so that I have some room to play around and see what feels fun.

I have some other things that I know I’ll need to do eventually (like making buildings destroyable), but I don’t want to get bogged down in minor improvements when such big questions are looming over me. I’m doing my best to stay focussed on solving the big questions first - stay tuned to see if I manage that!

Footnotes

Section titled “Footnotes”

or at least, the placeholder name until I think of something better ↩

Batch Rendering Models with Blender

Wed, 10 Sep 2025 00:00:00 GMT

In this article, I’ll explain how to use Blender 4.0 to create renders of multiple models using Blender scripting. I’ll be using Kenney’s Hexagon Kit as the 3D assets, and rendering one image per model with one click of a button.

Setting up

Section titled “Setting up”

Start by opening a new Blender project and importing all the models you want to render. In my case, I imported the GLB models.
Move all the models into one Collection so we can find them easily later. Rename the collection to something memorable - you will use this name later.

If you’re using GLB models as I am in the example, your models might appear grey at this point. Don’t worry - they will still appear correctly in the final render!
At this point, you can set up any lighting and reposition any of the models.
Set up the camera.
1. Position: The easiest way I found was to position the editor view in the correct position, and then use View > Align View > Align Active Camera To View.
2. Render Size: Select any object in the scene. Select the Output tab in the Properties area, and change the dimensions of the output images here.
By default, Blender won’t render images with transparent backgrounds. To set the background as transparent, select the Render tab in the Properties area, expand the ‘Film’ dropdown and check the ‘Transparent’ box.
Check the results of the render using this menu option:

You may want to only render one object when testing the final result. Toggle off other objects using the camera icon in the Outliner area:
Once you’re happy with the render result, switch to the Scripting layout using the top bar and make a new script. Make sure you save your new file with the .py extension: Blender will not do this for you automatically.

Scripting

Section titled “Scripting”

Full Script

Section titled “Full Script”

batch_render_models.py

1
import bpy, os
2

3
def set_render_self_and_children(object, show):
4
    object.hide_render = not show
5
    for child in object.children:
6
        child.hide_render = not show
7

8
dir_path = '/full/path/to/desired/output/directory'
9

10
objects_to_render = list(bpy.data.collections["YourCollectionNameHere"].all_objects)
11

12
for object in objects_to_render:
13
    set_render_self_and_children(object, True)
14

15
    for other in objects_to_render:
16
        if object == other:
17
            continue
18
        set_render_self_and_children(other, False)
19

20
    bpy.context.scene.render.filepath = os.path.join(dir_path, object.name + '.png')
21

22
    bpy.ops.render.render(animation=False, write_still=True)

Breakdown

Section titled “Breakdown”

1
import bpy, os

Importing Blender’s Python module, as well as the os module to use in file path manipulation

3
def set_render_self_and_children(object, show):
4
    object.hide_render = not show
5
    for child in object.children:
6
        child.hide_render = not show

Defining a function with two parameters: an object in Blender, and a bool value indicating whether it should be rendered
Setting the object’s hide_render value to the inverse of the show argument
Iterating through the object’s immediate children and setting their hide_render values to the same; if your models have more complex structures, you may want to modify this to iterate through more than one level of children

10
objects_to_render = list(bpy.data.collections["YourCollectionNameHere"].all_objects)
11

12
for object in objects_to_render:

Using Blender’s API to access all objects in the collection created during the setup, and putting those objects into a list
Iterating over the list; then, for each object in the list:

13
    set_render_self_and_children(object, True)
14

15
    for other in objects_to_render:
16
        if object == other:
17
            continue
18
        set_render_self_and_children(other, False)
19

20
    bpy.context.scene.render.filepath = os.path.join(dir_path, object.name + '.png')
21

22
    bpy.ops.render.render(animation=False, write_still=True)

First, enabling rendering for this object using the function defined above

13
    set_render_self_and_children(object, True)
14

15
    for other in objects_to_render:
16
        if object == other:
17
            continue
18
        set_render_self_and_children(other, False)
19

20
    bpy.context.scene.render.filepath = os.path.join(dir_path, object.name + '.png')
21

22
    bpy.ops.render.render(animation=False, write_still=True)

Turning off rendering for all objects except the one we’re currently looking at. In detail:
- Iterating over the list of objects again
- Doing nothing if we find the object we’re looking at in the outer loop
- Otherwise, preventing the object from rendering with the function defined above

13
    set_render_self_and_children(object, True)
14

15
    for other in objects_to_render:
16
        if object == other:
17
            continue
18
        set_render_self_and_children(other, False)
19

20
    bpy.context.scene.render.filepath = os.path.join(dir_path, object.name + '.png')
21

22
    bpy.ops.render.render(animation=False, write_still=True)

Setting Blender’s internal render filepath value to the target directory plus the model’s name and PNG extension
Note that (unintuitively) you don’t pass an output path into the render function, the output path is set here completely separately

13
    set_render_self_and_children(object, True)
14

15
    for other in objects_to_render:
16
        if object == other:
17
            continue
18
        set_render_self_and_children(other, False)
19

20
    bpy.context.scene.render.filepath = os.path.join(dir_path, object.name + '.png')
21

22
    bpy.ops.render.render(animation=False, write_still=True)

Calling Blender’s render function
Ensuring that the rendering process doesn’t pull data from the animation ranges in the scene by setting animation to False
Setting write_still to True so that the render writes to an external file at the path set earlier

Example Results

Section titled “Example Results”

Before

After

Making, Breaking, and Uncomplicating Analytic Events

Sun, 10 Aug 2025 00:00:00 GMT

Or: how I built a system that prevented almost all our analytics bugs, how I came up with the idea in the first place, and the key things I learned along the way.

Context

Section titled “Context”

Let me set the stage for the context of this story. When I began working at Trailmix, the game that would become Love and Pies was still over a year from release. As such, the team was small and our playtests even smaller, which meant that our player data gathering at the time consisted of watching over another teammate’s shoulder.

A few months after I joined, Trailmix hired their first data team member, who was given the gargantuan task of building out the whole data pipeline from scratch. They began setting up a system where the game build would send analytics to a Playfab server, and then that data would later be injested into Google BigQuery for analysis.

What are analytics and why are they important?

In games, analytic events consist of pieces of data about a player and what they're doing. Analytic events are important in free-to=play games because it's almost impossible to make a game sustainable without at least knowing where your players are spending money, which players are spending money, and where they're dropping out of the game completely.

Due to the obvious capacity constraints on the data team side and a similar lack of time on the dev team side, devs began setting up analytics events for our features in a pretty ad-hoc manner, usually without much forethought, consistency or testing. This is a pretty normal situation for small teams who are working fast, but in turned out that Love and Pies had longevity! So these hacky analytic event implementations made it into production. And then stayed there. For quite a long time.

And as is often the case with hacky implementations, it was only a matter of time before something went wrong.

What went wrong

Section titled “What went wrong”

One Monday morning about a year after Love and Pies’ global release, a marketing team member came over to the devs in quite a frantic state. They told us that the game hadn’t been sending any analytics events about monetisation or attribution for at least three days. This was quite terrifying to hear, because four days ago we had released an update, and during those four days we had been featured by the Google Play store. This meant two things:

We were missing crucial data for a huge number of new players.
We had broken something in analytics, even though we didn’t think we had changed anything to do with it.

We began digging straight away, and before long a teammate discovered that they’d caused the type of bug that is every dev’s nightmare: they had refactored a system that they thought was totally unrelated to analytics, and through a series of non-obvious dependencies that routed through some historical hacks, they had actually completely broken the monetisation and attribution analytics system.

They of course immediately began working on a fix, our QA team dropped what they were doing to start testing, and before the end of the day we had submitted a hotfix that solved the issue. But the end result was that we permanently lost the earliest, most important monetisation data for all the new players that came in from the Play store featuring, and we had to delay the next release by more than a week.

Problem solved?

Section titled “Problem solved?”

After we had verified that the fix had worked, we now needed to stop it from happening again. Devs, marketing, data, and production all crowded into a small meeting room and began to figure out how this could’ve happened to begin with, and what we could do about it in future.

It’s at this point I must admit that I had very little to do with the whole scenario up until this point. At the time, I was relatively junior in comparison to everyone else in the meeting, and I think I’d been invited with an expectation that I’d learn something rather than contribute.

Listening to the conversation that ensued was definitely interesting. The developer who had made the breaking change was clearly frustrated with themself and put the blame on entirely on their own shoulders. The QA lead, who at this point was the entirety of the QA department (!), blamed themself for not catching it as well, and said that going forwards, they would just manually check those specific analytics every release.

I was fully aware of how overworked our singular QA tester was, and it seemed to me that adding more to their workload was not only unfair, but also unrealistic! From where I was standing, nobody here was really at fault exactly: it wasn’t the dev’s fault that they didn’t notice the breakage in what should’ve been a totally unrelated system, and I felt that it was really just chance that it didn’t happen to me instead. And I couldn’t blame QA either, as they were already so overworked and definitely had enough new things to be testing for each release. But the main thing that stood out to me was how this really didn’t solve the underlying problem. Sure, if these specific analytics broke in future, that MIGHT get caught. But there was still only one line of defence there, and it was one that relied on overworked people working even more. And it didn’t fix the true problem that analytics could break due to changes that seemed unrelated, and nobody would know unless they specifically went to check.

I wasn’t the only one thinking this way: the head of the data team became increasingly frustrated as no systematic changes were suggested, and the meeting concluded with QA agreeing to the extra workload. I left the meeting knowing that there must be a better solution out there.

Finding another way

Section titled “Finding another way”

Once I got home, I opened up my laptop and simply googled “json validation”. Lo and behold, JSON Schema came up as the first result: a way to describe and define JSON data, using the JSON format! JSON Schema is widely used in several industries, and libraries existed that could be used to validate data against schemas. I dug a bit further, and discovered the Python library json-schema-for-humans, which generates nicely-formatted Markdown from JSON schemas.

I knew I was on to something good - I could solve the automation, inconsistency, and comprehension problems all at once with the tools I’d found. So I set up an experiment:

First, I put together a quick example schema in a git repository for an existing event.
I broke out some of the data that was frequently shared by other events into a subschema and brought it in with an include, to prove that we could reuse parts of schemas for new events which meant that making new schemas would get more efficient over time.
I copied out a few examples from our real analytics as test cases.
I wrote a script that brought in the test cases and used the event name to pick the right schema. Then it ran the event data through the validator, feeding the result into Pytest to generate a report at the end.
Finally, I added a Github action that automatically ran on push to generate Markdown pages, automatically rendered by Github to be a lot more readable than the raw JSON Schema file.

It worked great as a proof of concept. I made a few tweaks like bringing in a custom schema template so I could get inter-schema links working in default Github markdown rendering, and then I brought it to my boss. Happily, he saw the potential immediately, and set aside time for me to start implementation!

Goals

Section titled “Goals”

After talking to my boss, I talked to the data team, QA and some of the other devs to narrow down my key goals for when this tool would roll out to the wider team.

Goal 1: A clean slate

Section titled “Goal 1: A clean slate”

I wanted to make sure that I had no test failures. Allowing failures as the default state would mean that the team would become used to failures being the default state, increasing the likelihood that “true” failures - that is, test failures indicating that something is actually wrong - would be disregarded along with the baseline failures that ran than a state that required an action. It was also important to build a system with those who would maintain it day-to-day so that we could return to that clean baseline as soon as possible. Having a clean slate to fall back on avoids the risk of diluting the response to all failures, which would compromise the fundamental purpose of having a validation system.

Goal 2: Minimal effort

Section titled “Goal 2: Minimal effort”

I wanted to hand the content of the system over to the data team, and the monitoring of the system over to QA. QA were happy to take on logging issues if they were flagged up automatically, so automatic notification went into the specification as well. The data team were happy to take ownership of the documentation as JSON Schema would allow them to keep all the information they needed about the events in one central place. They were especially looking forward to easily outlining events in a consistent and easy-to-understand format, so devs could implement those events with minimal reworking.

Goal 3: Automation

Section titled “Goal 3: Automation”

I wanted the system to run validation automatically, requiring zero effort from anyone if all the tests were passing. It’s hard to argue that people should do something if they don’t get any new information most of the time! And it’s natural to become complacent and prioritise other tasks if that’s the case as well.

Implementation

Section titled “Implementation”

I began by copying my rough prototype into a new work repository, and set up authentication with Playfab so that I could pull data from storage. I set up my validation script in a job on Teamcity on our build machines, so that I could automatically pull the most recent 24 hours of data. Importantly, I didn’t pull data from our live build: I pulled the data from the store that our test builds sent to, so that we could catch issues before they went live, while QA was running other tests.

Teamcity has a Pytest integration already, so I hooked that up to the validation report for nicely-readable results. Then I wrote up some draft schemas for all the events and ran the job.

And…there were thousands of failures. I was expecting this to a degree, because I knew my draft schemas weren’t perfect and I would need to modify them to actually capture the intended structure of each event, but at least I had a starting point!

Running out of time

Section titled “Running out of time”

So I started going through each schema and correcting errors. Unfortunately, as I dug into the code for each event, I discovered some major flaws in the data we were sending. There were even more inconsistencies than I expected: many things worked but were convoluted to read or oddly structured for no apparent reason; several things were broken and sent nothing; and worst of all, a few things sent data that was outright wrong. Fortunately, the fact that it hadn’t been flagged meant that the wrong data likely wasn’t being used, but then that meant we were spending money storing and processing data that we just weren’t using!

So suddenly my task had expanded to cover fixing or changing several events, many of which came from code in fundamental systems. But I knew it was necessary in order to have a consistent baseline to begin testing from. Ultimately, I came to the end of the allotted time without solving all of these issues. Everyone I spoke to could see the potential, but there were still many important events that weren’t covered by a thorough schema, and there were some major bugs that either I didn’t have time to fix, or the QA team didn’t have time to test thoroughly enough, or the data team didn’t have capacity to send across an exact specification for. I wasn’t happy to start running the validation while there were still so many failures because I knew that we’d become accustomed to seeing those failures and not notice when things were actually broken. So I put the whole thing on the backburner for the time being.

Rollout

Section titled “Rollout”

However, the story doesn’t end there. I continued to push for more time while working on other things, and while the automated testing wasn’t running properly yet, the documentation aspect was absolutely present. I booked a meeting with each new data team member that joined to show them how it worked and explain the potential. The data team took on the job of writing schemas for new events as they requested implementation from the dev team, which meant that the documentation stayed up to date. During the wait for more dedicated time, I also had the ability to manually run the validation process, with the results only visible to me. As I had spent so much time digging into each of the events, I could see through the noise and tell when there was something truly wrong, so I flagged up relevant issues when things changed, and sometimes got the opportunity to fix other issues as side effects. Obviously this wasn’t sustainable though!

Eventually I did get more time - two years later! I threw myself into ironing out the final issues, and over the next couple of weeks, with the help of some dedicated data team members (if you’re reading this, Sonya and Ankit, thank you so much - you were both invaluable!), I got to a point where we had no failures. I set up nightly automation and handed monitoring over to the QA team.

The system continued to evolve after the initial hand-off:

Added the ability to run validation against events from only specific builds, or only from builds that were made from specific branches
Allowed the user to specify a time range to gather events for validation
Filtered daily testing to just test events against main and release builds to reduce noise from in-development builds
Integrated Slack notifications when tests completed
Organised a system to make sure the team wouldn’t become accustomed to the validator reporting failures, where the QA department would monitor the test results and assign any issues to a designated “on-call” developer, and then follow up on those issues until the tests passed again

What could’ve been better

Section titled “What could’ve been better”

While I did achieve most of what I wanted, there are lots of things I didn’t get to. Here are some things I wish I’d had time for:

Publishing the readable generated markdown documentation to a password-protected website, to make accessing the documentation more straightforward for CS agents and others without Github access - the current solution was to manually download the markdown files and send a zip file to those who needed it, who then opened the files in a program that rendered Github-flavoured markdown
Making the schemas more thorough, adding better validation for field values with regexes or enums for example, or checking combinations of values in different fields using JSON Schema features like if/then/else
Periodic validation against live events - currently the validator only runs against data sent from test builds due to the cost of pulling the large amount of data that is sent from players, but this was never really analysed to see if the cost would be worthwhile
Creating a test plan that intentionally went through and sent instances of each event for filtering through the validator
Creating a set of test data that would be sent through with each round of tests to make sure the validation itself hadn’t broken

And most important thing I felt the system was lacking was…well, unfortunately I actually don’t think that this system fully solved the original problem, where a change caused an event to not send at all. I did implement a set of tests to monitor whether an event was present in each round of testing, but I couldn’t flag that as a failure without compromising goal 1: not all periods of 24 hours would contain every type of event just because not all systems were tested every 24 hours.

Compounding this issue was the fact that the Slack integration for Teamcity test reports was lacking, to say the least. The message sent through was not customisable at all, meaning that I couldn’t add any information about how long it had been since each event had been tested: so even if I had time I implement some sort of persistent data store to track the most recent test for each event, I wouldn’t have been able to send that to Slack with the other test results.

Ultimately, the best I could do in the time I had was to mark a test as “skipped” if there was no instance of an event in that run’s data. This meant that the presence of an event would be at least tracked somewhere, without polluting results with false failures. This is not ideal at all: people need to proactively seek out that information instead of having it be passively monitored, they would need to seek out that information for each event one by one, and would have to manually go back and check the last time when that test had last reported a status other than ‘skipped’. My options were to compromise goal 1, a clean slate, over goal 3, automation, and ultimately I settled on keeping goal 1 as my gold standard. But I never came up with a solution that felt right to me. If you have ideas or if you’ve solved this problem before, please feel free to reach out on social media - I’d be very interested to hear your perspective!

But did it work?

Section titled “But did it work?”

That said, I think overall this system did solve a lot of problems! I can recall many situations where the validation system flagged a problem before changes went live, and at the time I left the team, the QA department had been successfully and consistently managing the validation system for over a year. The changes I implemented on the road to the first clean slate also cleaned up a lot of code and made it clearer and easier to use.

Surprisingly though, I think the biggest benefit of this system wasn’t actually the validation at all, but the documentation. Members of the data team had to frequently interrupt their work to ask the dev team for clarifications on our obscure and cryptic analytics. But thorough and up-to-date documentation solved that problem completely! And as the documentation also functioned as test specifications, it avoided the most common documentation pitfall and was kept up-to-date.

And it was also really useful documentation! I often see documentation from devs that’s mired in technical language and is more confusing than useful for end-users.But as this was written and maintained by the same people that were using it, it was focussed only on what they needed to know! Other departments benefitted from the improved clarity as well: customer support agents were more able to understand the context of players’ messages as they could understand the data sent by each player without having to wait for devs to interpret it; the product team were able to use the raw data if they needed to as well; and analytics were more present in everyone’s minds, which meant they were much less likely to be skipped over when planning features.

Key take-aways

Section titled “Key take-aways”

Solve problems at the source, not at the symptom
You can’t solve problems by adding work to already-overworked people
“Do the exact same thing again, just better” is not a good solution
Automate testing wherever possible
Make tests that alert relevant people when something goes wrong
False failures lead to a boy-who-cried-wolf situation: avoid them at all costs or real failures won’t trigger an appropriate response
It’s always better when documentation users are responsible for accuracy and upkeep, because they’re invested in its reliability and they’re engaging with it regularly
Prototypes will help people understand what you’re trying to solve and why you think this solution is a good one, and it will keep your solution in their minds when they’re involved in planning new tasks and allocating time for them

bex edmondson | Blog

Hexbuilder: Devlog #3

Announcement

Screenshots

Updates

Storage

Information accessibility

Adjacency effects

Toasts

Resident happiness

Big changes

Worker counts have more meaning, part 2

Componentising building data

Locked cell visuals

What’s next?

Footnotes

Hexbuilder: Devlog #2

Screenshots

What have I done since the last post?

3D Modelling

New buildings and currencies

Showing info everywhere

Tool improvements again

Worker counts mean something now

Godot Updates!

What’s next?

Storage

Resident Happiness

Anything else?

Footnotes

Hexbuilder: Devlog #1

Screenshots

Finding some direction

Timed Jobs

Names

UI improvements

What else did I do?

Next steps

Hexbuilder: Devlog #0

What it is

What I’ve done so far

Screenshots

What’s next

Footnotes

Batch Rendering Models with Blender

Setting up

Scripting

Full Script

Breakdown

Example Results

Making, Breaking, and Uncomplicating Analytic Events

Context

What went wrong

Problem solved?

More than just one problem

Finding another way

Goals

Goal 1: A clean slate

Goal 2: Minimal effort

Goal 3: Automation

Implementation

Running out of time

Rollout

What could’ve been better

But did it work?

Key take-aways