When we think we make performance better, we don't have any way to prove it. We're blind firing into the dark, hoping we hit something. That's terrible. I've been talking for a long time about how we need to have a benchmark, and we need to run it on the same computers, on every build, over and over again. That way we can compare a changeset to a previous one and have hard evidence that one set of code is faster than another.
So this is what we've started doing. We set up a bunch of servers with hardware points we want to target (out of whatever was lying around). So we have a
low spec one, a
mid-range one, an
iMac, and a couple of others all running the benchmark on every version of Rust we build and posting the results to the website.
Then we can go on and see if results are consistent across builds.
So if we update Unity, we don't need to ask around on reddit if performance is better or worse, we already know if there's any change.
Share Your Benchmarks
You can get involved too: just open the game, open the console and type in benchmark. It'll run a bunch of tests then open a webpage with your results. You can share the URL with your friends or with us to see what kind of performance you're getting. Here's my
work PC. If you keep your results together you'll be able to compare your results across versions, and flag up any performance changes with us.
The game's performance has been totally shitty recently, and has gotten worse. This system was important for us to add-- not only because it tracks positive changes--but it should also allow us to identify and rollback changes that make stuff worse.
The results of the tests are shown in milliseconds, where lower is almost always better. We'll add more benchmarks over time: when we want to prove something to be slow, when we want to see if an optimization has worked, etc. Here's a quick explanation of the current tests.
Player Model
Render and animate hundreds of player models. This speed tests our animation code, rig and mesh skinning.
PlayerModelRebuild
When a player model is shown for the first time, or when a player changes clothes, we have to rebuild the model. This could be a potentially slow process, especially when a bunch of sleepers spawn in, so we want to keep an eye on the performance of this.
Procmap.Gen
How long it takes to generate the procedural map? This is the huge majority of loadtimes on Rust right now. We want to improve this, we don't want it to get worse. This is also very hardware specific in terms of times, ranging from 30 seconds to 3 minutes.
ProcMap.Normal
Render the procedural map with grass and decor, but no trees. It's surprising to me how slow this is, because we previously assumed that bad frame-rates were down to the buildings. If it's slow even without buildings, then we have problems.
ProcMap.noGrass
Lots of people complain that the grass is the cause of performance issues. This test is to prove that right or wrong once and for all.
ProcMap.NoDecor
Decor is things like bushes, flowers, driftwood. This test is similar to the grass test, we want to see how much these things affect performance, and whether they should be enabled or not.
ProcMap.Rain
Does the rain lower frame rate? It has historically, is that still the case? This test seems to suggest that having rain on improves the frame rate... which might be because of fog.
SkinnedMeshCollider
This test shoots a bunch of invisible bullets at a bunch of player models to test the speed of the hitbox generation. This is a potentially slow operation, and the same code is used for a bunch of different things that can potentially be called every frame.