130 rounds of Legend Quickplay as a host: An analysis of the game's RNG

Miscellaneous Fatshark Lounge
1

As the title suggests, I have been keeping track of the last 130 rounds I hosted on Legend difficulty, quickplay. Most games were played in blocks of 3 to 5 each if that matters, but since I never hosted without keeping track, so the “data” if you’d call it that is pretty authentic, I’d say. I give you the rundown first and the complete sequence afterwards.

First things first: Why did I go through the trouble of doing this? Mostly because I was curious how “legit” the game’s rng worked. As I have stated numerous times, I am rather sceptical about the game’s RNG mechanics. People have claimed it lies within the nature of RNG that certain results seem biased by perception, but I had a gut feeling that this is simply not the case here. Now, of course I know that 130 rounds is not yet statistically relevant. I chose 130 as a number because there are 13 maps, so roughly speaking, each and every map should have been played about 10 times - which, surprisingly, does hold true for the majority of maps, except for one: Skittergate. Skittergate is dramatically overrepresented and again, almost comically exactly by a factor of three when compared to the expected result of 10 times it should have come up. I have a few hypothesis about why that is so, but without further ramblings, here is are the results in alphatecial order:

Against the Grain 6
Athel Yenlui 13
Convocation of Day 9
Empire in Flames 10
Festering Ground 10
Fort Brachsenbrücke 9
Halescourge 5
Hunger in the Dark 5
Into the Nest 13
Righteous Stand 6
Screaming Bell 8
Skittergate 31
The Warcamp 5

I have had so many people claim that Skittergate seems to pop up more often in quickplay than other maps, yet most have been “shut down” with “this is how RNG works”. Well, again, I know this is not statistically relevant just yet, but after 130 rounds played, having one map pop up 3x more often than the others should at least lead to further investigation, especially considering how almost all other maps behaved exactly as one would predict.

My hypothesis as to why that is (just a stab in the dark, but it would make sense): The quickplay RNG doesn’t just randomly pick a map out of a pool of 13, but selects one ACT out of the three and then selects one map out of that act. Since Skittergate belongs to each and every act as a final mission, it would make sense that it has a higher chance to pop up and since there are 3 Acts, it would fit perfectly.

Make of this what you will, but I just thought I’d share my intelligence with you.

Here comes the whole sequence (You can call me biased on starting at Skittergate, but the idea to keep track came before that round even started :wink: ). There are some other very strange “regularities” and doubles and even triples have a very strong representation. Enjoy:

1 Skittergate
2 Festering Ground
3 Skittergate
4 Skittergate
5 Skittergate
6 Into the Nest
7 Empire in Flames
8 Into the Nest
9 Fort Brachsenbrücke
10 Fort Brachsenbrücke
11 Convocation of Decay
12 Skittergate
13 Skittergate
14 Into the Nest
15 Hunger in the Dark
16 Fort Brachsenbrücke
17 Athel Yenlui
18 Empire in Flames
19 Convocation of Decay
20 Athel Yenlui
21 Skittergate
22 Skittergate
23 Athel Yenlui
24 The Warcamp
25 Skittergate
26 Skittergate
27 Righteous Stand
28 The Warcamp
29 Hunger in the Dark
30 Festering Ground
31 Skittergate
32 Skittergate
33 Into the Nest
34 Skittergate
35 Athel Yenlui
36 Into the Nest
37 Fort Brachsenbrücke
38 Athel Yenlui
39 Skittergate
40 Screaming Bell
41 Against the Grain
42 Empire in Flames
43 Empire in Flames
44 Against the Grain
45 Festering Ground
46 Convocation of Decay
47 Screaming Bell
48 Righteous Stand
49 Athel Yenlui
50 Righteous Stand
51 Empire in Flames
52 Festering Ground
53 Into the Nest
54 Righteous Stand
55 Empire in Flames
56 Screaming Bell
57 Skittergate
58 Against the Grain
59 Skittergate
60 Skittergate
61 Into the Nest
62 Into the Nest
63 Righteous Stand
64 Athel Yenlui
65 Athel Yenlui
66 Athel Yenlui
67 Screaming Bell
68 Screaming Bell
69 Into the Nest
70 Hunger in the Dark
71 Into the Nest
72 Fort Brachsenbrücke
73 The Warcamp
74 Against the Grain
75 Screaming Bell
76 Festering Ground
77 Against the Grain
78 Convocation of Decay
79 Empire in Flames
80 Festering Ground
81 Into the Nest
82 Convocation of Decay
83 Righteous Stand
84 Fort Brachsenbrücke
85 Skittergate
86 Halescourge
87 Convocation of Decay
88 Festering Ground
89 Skittergate
90 Hunger in the Dark
91 Athel Yenlui
92 Empire in Flames
93 The Warcamp
94 Halescourge
95 Into the Nest
96 Hunger in the Dark
97 Halescourge
98 Screaming Bell
99 Screaming Bell
100 Empire in Flames
101 Festering Ground
102 Skittergate
103 Fort Brachsenbrücke
104 Convocation of Decay
105 Athel Yenlui
106 Halescourge
107 Skittergate
108 Skittergate
109 Empire in Flames
110 Into the Nest
111 Skittergate
112 Skittergate
113 Skittergate
114 Athel Yenlui
115 Against the Grain
116 Festering Ground
117 Skittergate
118 Skittergate
119 Fort Brachsenbrücke
120 Skittergate
121 Convocation of Decay
122 Skittergate
123 Fort Brachsenbrücke
124 Convocation of Decay
125 Skittergate
126 Halescourge
127 Festering Ground
128 Skittergate
129 The Warcamp
130 Athel Yenlui

14 Likes
2

Is it possible you (and/or the other members of your QP) are not completing Skittergate as consistently as the other maps? Robin has said that map RNG is weighted on map completions (seemingly to discourage people from bailing on or intentionally failing certain maps).

Robin [developer]
It’s likely because you’re not completing the maps. The random weights in the map rotation is based on map completions. It’s pseudo random so you won’t get the same map over and over. But to not encourage people to just quit out of maps that are hard it only factors completions and not started maps. So if you always quit out of a certain map or you always fail on a certain map, you’re going to see that map quite often in the quick play rotation.
Also, it puts a very severe penalty on the map you played last (even fails) so that map will be pretty unlikely to get again unless you have a very big defecit on that map.

2 Likes
3

If that is the case, then my findings tend to disprove that, as I am only skipping Skittergate if it comes multiple times in a row, but I do so with almost any map. Skittergate definitely is not my least favourite map, that award goes to Athel Yenlui. It is also not my least completed map. I’d also like to stress that the system is kinda weird as getting the same map over and over again increases the chances of me skipping it because it gets stale, which in turn increases my chances of getting it again, which increases my chances of skipping it, which increases my chance of getting it again.

2 Likes
4

Pretty fascinating. I actually almost never see skittergate in quickplay… Its been long enough that I was starting to wonder if it got removed or something.

But I’ll see patterns where we get the same map over and over again as well… If there’s supposed to be protections against that, they ain’t workin!

1 Like
5

I’ve been in premade parties on discord where skittergate pops up 3-4 times in a row. And by that, I mean we complete skittergate, get our loot. Roll back into Quick Play with the same people on the same classes and get skittergate again. Finish it again, and guess what we get again?

It’s mental… lol

2 Likes
6

This just strengthens my suspicions that there is something wonky going on with the RNG in general, making it skewed towards certain results. I’ll try to elaborate.

First, for those who don’t know how RNG in games and other computer stuff usually works, a simple rundown (Wikipedia helps more). Computers (and humans too, for that matter) are bad at true randomness. In computing, it can essentially only be achieved with external, specialized hardware or a service that uses some. More common is pseudo-random number generation: First, take a “seed” of something that’s at least somewhat unpredictable, often something happening inside your computer like system clock with precision in at least thousandths of a second. Then, use an algorithm of some kind to generate the end result from that seed.

Now, in this game, the RNG affects a lot of things: The maps you get in quickplay, the enemy spawns, route choices in certain maps, loot (in several ways), item Properties and Traits… It’s likely that all of these use the same RNG (with tweaks for certain purposes, certainly), as coding several different ones would just be extra work and extra space required. This means that if there is unintentional skewing in the RNG results, it’s likely to affect everything. Which, indeed (based, admittedly, on anecdotal evidence from the forum users), is what seems to be happening. There has been reports of both Trait and Property rolls skewing towards similar results on a lot of rolls, far more duplicate items than should be appearing by expected probabilities, similar density (either a lot of stuff or hardly any) through the whole run on a map and even throughout the day, and Quickplay map queue. Before any of this can be confirmed, though, more experiments on different aspects of the RNG is needed.

If this is true, there are a few possibilities how this can be. The seed method can be faulty, producing biased results even after filtering through the RNG algorithm. Reports of the host’s CPU affecting spawn density would suggest this, as (by the devs’ comments) there shouldn’t be anything in the AI code to be affected by CPU speed. The algorithm could also be faulty, tending towards certain results altogether (like a skewed die) or, as suggested somewhere else, skewing towards previous results. Or, of course, a combination of both.

Do not take any of this as truth. Humans are very good at detecting patterns, and will find some where they don’t exist. This also makes us, as a species, bad at probabilities. It bears repeating: We need a lot more testing, enough to get at least some statistical significance, to validate any of these speculations. A coordinated effort by the community can do it, if we really want to find out things. Personally, my resources in-game are limited, but others have far more dusts and red items to test for bias in Property and Trait rolling, and far better abilities to pay attention to things like enemy spawn frequencies. Different people with different hardware setups need to test these, too.

6 Likes
7

For real, Skittergate come around way too often. Getting the map twice in a row is nothing out of the ordinary:expressionless:

8

I can go 20-30 games without seeing Skittergate. I get war camp back to back a lot though.

1 Like
10

I ran my pure RNG QP simulator 4 times for 160 QP matches. Here are the results:

[Album] 4 x 160 QP simulations

Based on these simulations, I think it’s safe to say that getting any particular map (Skittergate in your case) 31 times in 160 matches is unusual. Not unthinkable for pure RNG, though. I’d have to spend quite a bit more time to work out how rare a number that high should be if we assume RNG.

It’s my understanding (and I may be recollecting this wrong) that the map-weighting algorithm used by Fatshark takes the relative number of map completions into account. Not failures per se, although map failures (and skips) obviously don’t tend to lead to completions. But if we take FS at their word, then the best way to even out one’s quickplay maps (with the current system) might be to play what comes up but skip the map (suicide) if it’s a double and you just completed it. There definitely is the potential for a negative feedback loop to become established, it seems: e.g. if the game thinks you ought to be playing skittergate, and you keep skipping it (because you don’t want to play the same map during the same session) then it’ll keep coming up more often, leading to even more reliable skipping, etc.

At the same time, “gut feelings” on this kind of thing are the whole reason it took so long for statistics be invented. Humans suck at this stuff. For instance, doubles and even triples on a 20-sided die aren’t all that rare. Additionally, part of the issue with “getting to the bottom” of the map selection algorithm is that any weighting based on map completions is going to change from QP group to QP group. This makes collecting data that could definitively identify the strength of the map weighting (and thereby how well the RNG is working) essentially impossible unless you run solo matches or keep the same crew for the whole experiment.

11

necromancy3

1 Like
12

Yeah – my bad! Look like Haxorzist gets the primary necro credit, though :stuck_out_tongue:

2 Likes
13

Well, if the devs love pure RNG so much that they ditched their “no red dupes” policy from VT1, why the hell do they bias the RNG that chooses the maps? Bail out my ass - just make good maps - everyone hates DLC maps now and tend to skip them every time, and the game shoves them in our faces. Nice job.

14

Sorry posted here by accident. Was just looking something up for another thread. :sob:

15

HERESY!

16

https://pa1.narvii.com/6464/56916b593f5310844c98883399c1be33423a5689_hq.gif

5 Likes