What Happens to a Studio When Builds Are Unpredictable
Nobody talks about what unpredictable builds do to people.
They talk about build times. They talk about failures. They post in forums about 4-hour cooks and shader compilation crashes. But the real damage isn’t in the logs. It’s in the behavior — the quiet ways a team reshapes itself around a pipeline it can’t trust.
Builds are the nervous system of the studio. When the nervous system is unreliable, the whole organism adapts — and the adaptations look so normal that nobody questions them.
Until you do.
People Stop Committing Freely
The first thing that changes. Nobody notices.
Engineers start batching commits. Not because it’s good practice — because each commit is a roll of the dice. Trigger a build, wait 90 minutes, find out if it worked. Why trigger three builds when you can batch your changes and only gamble once?
Artists push at end of day and check results in the morning. Friday afternoons go quiet. Nobody commits after 4pm because a broken nightly means a wasted Monday — three artists’ work sitting in limbo, a producer recalculating the milestone, an engineer debugging at 9am instead of building the game.
I saw someone on r/devops describe their team’s response to flaky builds perfectly: “Devs have learned to just click rerun and grab coffee.” That’s the adaptation in a sentence.
The death spiral starts here. Fewer commits means larger commits. Larger commits mean bigger blast radius when something breaks. Bigger blast radius means worse failures. Worse failures mean more fear of committing. The loop tightens.
The team is rationally optimizing for a broken system. And that rational optimization makes the system worse.
Engineers Run Local Builds “Just to Be Safe”
If you can’t trust CI, you build locally. Just to check. Just before you commit.
This seems responsible. It’s a waste. The engineer waits for a local build that takes 30 minutes to 3 hours — on a machine that isn’t the CI environment, with different caches, different SDK versions, and different state. A local build passing tells you almost nothing about whether CI will pass.
At Ayelet Studio, we caught ourselves doing this. Three engineers, each running local builds before committing to Son of Thanjai. That’s potentially 9 hours of compute time per day for something that should be zero. And the local builds didn’t even predict CI outcomes — different environment, different result. We were paying the cost of two builds and getting the reliability of neither.
When CI has cried wolf enough times, the team stops trusting it — even when it’s telling the truth. So they build their own shadow validation. Local builds. Manual checks. “I’ll just run it myself.”
The Studio Develops a “Build Mood”
Green build Monday: the studio is productive, the standup is short, people are building the game. Failed build Monday: the standup is 20 minutes of pipeline debugging, the producer is tense, artists are waiting, engineers are triaging instead of creating.
The team’s energy is coupled to infrastructure. The producer checks Slack before coffee — not for messages, for build status. The vibe of the entire day is set by a Groovy script and a Jenkins job that may or may not have survived the weekend.
This isn’t a metaphor. At Ayelet, we had a producer who could predict whether the Monday standup would be productive based on whether the build status Slack bot had posted a green checkmark before 8am. When it hadn’t, she started the meeting differently — more cautious, more buffer time, more “let’s see where we are before we plan.” The whole team’s posture shifted based on a single boolean: did the build work?
Iteration Dies Quietly
When each build is a gamble, you test fewer ideas. You don’t try three versions of the combat system because each version needs a build, and each build might fail, and each failure costs a day of debugging. So you ship the first version that compiles. Not the best version. The first one that survived the pipeline.
Studios that ship polished games iterate relentlessly. They try things, test them in-engine, throw them away, try again. That loop depends on fast, reliable builds — not fast alone, but reliable. A 30-minute build you trust is worth more than a 10-minute build that fails randomly.
This is the cost nobody puts in a spreadsheet. You can’t measure the game you didn’t make because the pipeline wouldn’t let you iterate. You can’t count the ideas that died in someone’s head because “it’s not worth triggering another build for that.” But if you’ve ever shipped something and thought we could have polished this more if we’d had time — ask how much of that time the pipeline ate.
QA Gets Garbage
QA installs a build that “passed.” But passed what? The one that succeeded on the third re-run? The one from the shared drive that might be build 247 or 249? The one an engineer packaged locally because CI was down?
Bugs get filed against stale builds. Engineers investigate issues that exist in the build QA grabbed, not in the current code. And the worst version: QA finds a real bug, but nobody trusts the report because “that build was probably bad anyway.”
When the pipeline’s credibility is gone, everything downstream loses credibility too.
A Clarification
If your builds are slow but predictable — same time, same result, every run — you have a speed problem, not a trust problem. Different diagnosis. Speed is fixable with better hardware, better caching, better parallelism. Trust is a deeper problem. This post is about trust.
Why This Happens
Every behavior above traces back to one root cause: the pipeline doesn’t understand what it’s building.
Generic CI — Jenkins, GitHub Actions, TeamCity — runs commands in sequence. It doesn’t know what a .uproject file is. It doesn’t know that a code change needs recompilation but a texture change only needs re-cooking. It doesn’t know which caches are still valid and which are stale. So it guesses. Rebuilds too much or too little. Produces different results from the same inputs. Breaks in ways that aren’t reproducible.
Non-determinism in a build pipeline isn’t a bug. It’s the natural consequence of forcing a tool that doesn’t understand the build to orchestrate it anyway. Stale DDC entries, race conditions between stages, environment drift between runs, hidden state from previous builds — all of these are symptoms of a CI tool that’s blind to the structure of what it’s building.
We had three co-founders with ten-plus years of enterprise DevOps experience each. We couldn’t make Jenkins deterministic for Unreal. Not because we were bad at Jenkins — because Jenkins wasn’t built to understand what we were building.
If you think you can solve this with better scripts, you’re wrong. We thought so too. We spent six months trying. The problem isn’t skill. It’s category.
What Changes When Builds Are Predictable
When every commit produces the same result — when the build either works or tells you precisely why it didn’t — the behaviors reverse.
People commit more often. Smaller batches, faster feedback. Nobody runs local builds because CI is trustworthy. The producer doesn’t check Slack at 6am. QA trusts what they’re testing. The team iterates — tries ideas, tests them, throws them away, tries again. The build stops being a conversation and becomes infrastructure. Invisible. Boring. Exactly what it should be.
At Ayelet, after we replaced Jenkins, the first thing we noticed wasn’t the speed. It was that artists started committing twice a day instead of once. Nobody told them to. They just stopped being afraid.
The worst part isn’t that unpredictable builds waste time. Time is visible. You can count hours.
The worst part is that the team adapts. They get good at working around it. And once they’re good at working around it, they stop seeing it as a problem. That’s when it becomes permanent.
Every studio we’ve talked to that fixed this said the same thing: “We should have done it a year ago.”
None of them said it was about the build times.
If any of this sounds familiar — abrahamk@speedrun.ci
Never miss a ue5-main update.
Get them delivered straight to your inbox.
