I just wanted to get some clarification on what you guys are looking for in a Beta. I do appreciate that you’ve been putting patches into beta before releasing them. I think that is a good process going forward. However, there have been several instances where bugs have been reported and/or 100% negative feedback has been given about a certain thing.
Here are two instances I can think of to clarify my point:
Huntsman bow change caused the bow to break when ulting or using speed pot. This bug was reported many times and it was released with no fix;
The feedback on the Beastmen change was almost unanimous that Beastmen were too rare and that we should find a middle ground.
Given this information (and numerous other examples I left out) I am just curious as to what you guys are looking for in a Beta? Are you just making sure the branch doesn’t have some game-breaking bugs? I think it would be nice to get a little peak behind the curtains to gain some insight as to what the development process is.
P.S. This is not a slamming FS post. Please don’t flood the comments with tears complaining about FS.
What? I legit have no clue what you are talking about. I am talking about the feedback in the weekend beta feedback forum that is now closed. As far as the actual frequency of beastmen, it dropped from about 50% chance to start maps as beastmen to about 10% chance (halescourge outlier). As far as I can tell, it never becomes beastmen unless the map starts as beastmen. I’m not terribly fussed about it, really. Nice break from beasty boys. However, given the feedback that was in the weekend beta forum, I am just curious as to what the purpose of the patch betas are.
I was just going to suggest that the feedback forum here was pretty un-used during the beta (not entirely, and was taken in to account!!). A great deal of the conversations we happened across stated that things were much more balanced in terms of faction represtentation. Of course, playing one or two games just isn’t enough to get a full feel for the changes so there was a lot of knee jerk reactions from some folks. We also looked at data, what was being killed in what volume, and found that things were much more balanced during that test (and beyond in to 2.2.1). Had there been truly an overwhelming negative response to the changes, we’d have looked at adjusting that before it went live, as the beta was very specifically for that feedback.
As for the huntsman thing, yeah, wasn’t ideal at all, but it was one of many issues raised. I forget which beta it was, but my memory says it too had another focus. I’m sure folks can remind me!
Hardly a surprise given the short timespan combined with the actual content of it.
I am interpreting this as a partial answer to the original question about the purpose of betas beyond this one specific short one. It’s refreshing to see you say it like it is, and my opinion is us players & commenters here in the forums would do good with some hard facts from your side.
Which comes back to the first quote - it wouldn’t hurt if you told us what is happening and/or what to expect. It should be obvious that you can’t form a deep understanding of a new patch in 3 games, but (and I am sure I am guilty of this too) discussions here start & continue based on about that or even less. Knowing a beta is for telemetry collection mainly (along with obvious bug reports that reproduce & came with the beta) might make interactions here more fluid.
Performance has been troubling me lately, and thus is on my mind. I don’t know the kind of telemetry you collect specifically about performance, but I am assuming it is mainly averages with some exceptions. Before I started fiddling with game settings & hardware I had decent FPS but during hordes my frametimes were horribly variable which made 50-70 feel 20-25.
What I would like to say is that I hope you look at all the data with the fact in mind that there is a sampling rate to the telemetry that acts as a band pass filter, unless specifically implemented to collect specific high/low frequency events. And I know you do, but we hear so little back from the decision making (why x bug gets no attention, why y gets instead, z wasn’t even an issue, etc) that it can feel you aren’t aware of (or taking seriously) some things that are common for many players.
So there were a lot more beastmen kills before than skaven or northlanders? Interesting.
Well, I’m glad things are more balanced now, but one thing I think should be considered is, precisely, outliers. Like I said in my feedback thread, last night I played 5 games in a row and didn’t see a single beastman. While that clearly is just bad rng and an outlier case, I strongly feel that outliers like these shouldn’t ever happen. Perhaps a more weighted form of RNG, or a minimum/maximum of numbers to be taken into account during the faction distribution, something.
And in the opposite direction, some people are still complaining about too many beastmen even with the changes from the beta.
It’s certainly a tricky subject because the randomness of vermintide is what aids its replayability, if things become predictable/vary too little, then it becomes stale. But there IS such a thing as TOO random, or in this case, situations that, no matter how rare, shouldn’t happen.
Maybe if there was a minimum of, like, at least 1 zone in the map has to belong to each of the factions allowed to be in said map, and also a maximum, it’d help keep both outliers from happening, while still maintaining the RNG factor.
It’s not an outlier at all, depending on the map. That is a fairly normal outcome for the majority of maps. At least as far as testing has shown thus far.
I appreciate the response. Note, I’m not salty at all about the beasty boys taking a vacation. From my point of view, a 2-day beta where nothing was changed from inception of the beta to release indicates that FS isn’t really looking for feedback as to the content. Perhaps that’s just an incorrect assumption. Given the response (or sometimes lack thereof) to feedback in several (not all) betas, I was beginning to suspect the betas are simply to test the update to make sure it doesn’t introduce new game-breaking bugs. Which, if that’s the case, I think it’s definitely a good thing. I just think it would be good for participants to fully understand the scope of the beta.
Edit: I would be curious as to what your data says about race balance.
It certainly helps find any common game breaking bugs or crashes! We use betas in different ways, depending on the beta. We could be clearer up front about what we are expecting of them.
I think beta’s like this are fine, it adds a little buffer incase things really break hard.
About the feedback, i’m sure they read feedback but it’s just very hard to respond on every feedback thread that gets made and doesn’t really add anything i think. If things really are gettinf alot of feedback, like deed exploit or spear push for example, like alot of posts or big threads that get alot of responses. I’m sure they act on that and we have seen that they do read it in the changes they made recently.
Also at the end of the beta and after the evaluation of the data, one post with the results would be a courtesy towards the testers, instead of, like NOTHING.
I mean how is it not clear what needs testing in a beta? Wasn’t it clear what they needed feedback on for this mini beta they did?
Yes they could say something like in the patch notes after the beta like: ‘after reading feedback and comparing it with our data we changed the beastmen spawning rates’. But if feedback gets given on the beta and fs introduces it into the game i’m sure they read it. If we are talking about bugs, sure, they should get adressed and fixed before the beta ends and patch goes live.
Well, sort of no, because separate feedback forums get instantiated with them. During WoM beta it was clear and communicated, but when these smaller betas are for collecting telemetry and game breaking bugs then creating a feedback subforum maybe confuses things and gives the impression that feedback beyond bugs is wanted.
I don’t know, maybe we are desperate and to try control things our way because we like the game but perhaps not some decisions and thus having another kneejerk reaction
But if they release a beta branch, and in that beta they have reduced beastmen spawning rates, how is it not clear they want feedback on that. Not talking about bugs.
Yes i did, maybe i missunderstood what he meant here:
And ofcourse they use beta’s to collect data on specific things, i’m very sure every company looks at data and runs beta to collect data in a controlled way. companies take data over 1 guy saying there are too much beastmen by playing 1 map. As i quoted above i might missunderstand but they looked at feedback but also at data, and if there was a huge negative response like the deed fix for example, they would revert it. am i missing something?
I was just going to suggest that the feedback forum here was pretty un-used during the beta (not entirely, and was taken in to account!!). A great deal of the conversations we happened across stated that things were much more balanced in terms of faction represtentation. Of course, playing one or two games just isn’t enough to get a full feel for the changes so there was a lot of knee jerk reactions from some folks. We also looked at data, what was being killed in what volume, and found that things were much more balanced during that test (and beyond in to 2.2.1). Had there been truly an overwhelming negative response to the changes, we’d have looked at adjusting that before it went live, as the beta was very specifically for that feedback.