Context of Steam Charts Data

[zesban]

Your selection strategy was highly biased (Only GBX titles)

I picked them out to avoid a bias, if I cherry picked out specific games that would be a bias. I dont know if I have the time or resources to do a proper study of this. But I can add more games that show a similar path. I did look at some PVP focused AAA games and they also followed the same path.

It’s still a bias, just not that bias. And a bias still affects margin of error and confidence.

If you don’t have the time, that’s totally understandable, but you have to present the data with all caveats, or it’s a statistical lie.

Also, even after sharing more PVP focused AAA games the sample population size will still be too small and that will affect margin of error and confidence, though it’d be better than the data now.

Further, only looking at PvP focused AAA games is an issue, because it doesn’t fully model Battleborn unless they also include a PvE mode that is required for improving PvP performance, and can be played as PvE multiplayer focused experiences as well. Otherwise that too will affect margin of error and confidence. Either way, it’d be better than the data now.

That’s part of the problem with comparing Battleborn to other games in the first place. It’s such a niche game. It’s in a newly emerging genre and it mixes modes of play that haven’t often been mixed before, and that brings unique problems along with it. Part of the margin of error here is always going to be due to this fact.

It’s gonna be a lot of work to get to a point of honesty. Good luck!

And again while the data will be much more honest, it still won’t be useful to the players that just want solutions. So expect them to continue to point that out.

[zesban]

One more dip into this pool.

From what I can gather,

@Derch believes that he was providing a set of data to show that some of the assertions commonly made around here (about Battleborn’s performance), are not necessarily accurate/fair in context of the broader market.

@Zesban believes that @Derch did not actually “provide context,” instead he provided a picture that paints Battleborn’s situation as being rosier than it actually is, which he believes is misleading and dangerous – especially for a mod.

@Derch is attempting to point out that he was trying to present the information neutrally, and that he presented the information in the first place to help better frame the conversations we have around here (about solutions), with a more accurate lens.

@Zesban in turn, doesn’t think this matters, because pain is pain and we should spend time fixing the pain points – rather than trying to justify them.

I doubt very much that @Derch disagrees with this.

Beyond that, I fear that we are slipping into pedantic arguments over experimental design and statistical significance, which I absolutely adore, but isn’t really getting us anywhere.

We’re all trying to make the game better, it’s hard to see how, “quibbling over commas,” helps that.

I agree. Which is why I said:

Basically the best way to help Battleborn’s community is not to debate or apologize within the community, it’s to try to engage others in to the community as players or even fans, and to provide feedback to the developers on the challenges you encounter while trying to do so. That way they can aggregate that feedback and use it to effectively steer their game to something that will attract large amounts of players.

I’m continuing to participate in working with the data and statistics because that’s what this thread is about, but it really is just statistical analysis at this point, and most people will see it as “quibbling over commas,”. But since we have this mountain here, we’ll climb it.

[Derch (Where's lucky?)]

I’ll add some of these to the OP

GTA5
-51.55% month 2
-49.71% month 3

Fallout 4
-58.4% month 2
-41.76% month 3

Witcher 3
-56.06%
-55.70%

Dark Souls 3
-61.78%
-65.22%

Call Of Duty Black OPs 3
-30.12%
-17.7%

The Division
-59.43%
-63.05%

Doom
-60.76%
month 2 not finished

Metal Gear Solid 5: The Phatom Pain
-65.01%
-62.7%

Total War: Attila
-44.69%
-39.08%

Dark Souls Prepare to Die edition
-44.73%
-57.71%

Rise of the Tomb Raider
-55.39%
-46.85%

Dark Souls 2
-44.02%
-2.38%

Fallout New Vegas
-26.93%
-35.11%

Just Cause 3
-44.92%
-47.19%

Xcom enemy unknown
-62.39%
-31.91%

Saints Row 4
-73.79%
-60.19%

Left 4 dead 2
-32.72%
-22.81%

Age of Empires 2 HD edition
-44.96%
-34.31%

I think thats enough AAA games to kinda show. Not all do that most call of duties seem to do well and others of course but I’m just trying to show that drops on the first few months are not uncommon for AAA games.

Not saying anything about the size of the pc player base. Just that the drop has context.

[sbspalding (sbspalding@gmail.com)]

There is no such thing as a perfect model.

It doesn’t exist.

There are better and worse models, and which we end up using in practice often has to do with how much time we have to devote to the task at issue, and the resources at our disposal.

It also has to do with the quality of the available information, and how important the question we are trying to answer is in the grand scheme of things.

I’m not going to train up a neural network or whip out a Monte Carlo simulation to win an argument on a video game Forum (not saying either would be used in this case, just providing examples of overkill). It doesn’t make sense, nor does it provide enough additional value to be worth the effort.

To claim that every, single point made with data has to be perfectly unbiased is to claim that almost no study done in the 21st century is valid. And it’s to say half of the field of Data Science is barking up the wrong tree.

While it’s very bad to present information with a clear bias (without identifying the bias), it’s worse to completely dismiss a point because it’s not a perfectly designed study – especially if the stakes are low, and the person presenting the information did not think that his assertions would be under Peer Review.

[Derch (Where's lucky?)]

I’m continuing to participate in working with the data and statistics because that’s what this thread is about, but it really is just statistical analysis at this point, and most people will see it as “quibbling over commas,”. But since we have this mountain here, we’ll climb it.

My hope here is to help spread some of this so when these numbers get thrown around that more people know what they really mean.

Again I’ve heard so many talks on here, youtube, twitch, reddit, etc… The sky is falling xx% of the player base left, again now some people might now of known to expect around a 60% drop in the first month or more.

Or the 10,000,000 vs 1,000 numbers I’ve seen tossed around.

Its about sharing better information for the community. Much like so many of us did for borderlands, that wasn’t really player base or numbers like these but x “x gun sucks” or “this is impossible” A lot of us went out of our way to help those people.

This is a different kinda of help. But if we as a better informed community can stop some of the sky is falling talk but offering context some people might be willing to give the game a shot or not stop playing.

There is the group effect of when you hear something is dead or loosing you don’t want to be a part of it.

[zesban]

I’ll add some of these to the OP

GTA5
-51.55% month 2
-49.71% month 3

Fallout 4
-58.4% month 2
-41.76% month 3

Witcher 3
-56.06%
-55.70%

Dark Souls 3
-61.78%
-65.22%

Call Of Duty Black OPs 3
-30.12%
-17.7%

The Division
-59.43%
-63.05%

Doom
-60.76%
month 2 not finished

Metal Gear Solid 5: The Phatom Pain
-65.01%
-62.7%

Total War: Attila
-44.69%
-39.08%

Dark Souls Prepare to Die edition
-44.73%
-57.71%

Rise of the Tomb Raider
-55.39%
-46.85%

Dark Souls 2
-44.02%
-2.38%

Fallout New Vegas
-26.93%
-35.11%

Just Cause 3
-44.92%
-47.19%

Xcom enemy unknown
-62.39%
-31.91%

Saints Row 4
-73.79%
-60.19%

Left 4 dead 2
-32.72%
-22.81%

Age of Empires 2 HD edition
-44.96%
-34.31%

I think thats enough AAA games to kinda show. Not all do that most call of duties seem to do well and others of course but I’m just trying to show that drops on the first few months are not uncommon for AAA games.

Not saying anything about the size of the pc player base. Just that the drop has context.

This is interesting. It shows that Battleborn performed worse in the first month than every game presented so far, and usually by a wide margin. That’s even worse than the initial picture provided. Of course, there’s a lot missing from this picture, but the sample population sure is much bigger.

This does show that there is something seriously wrong with Battleborn, though it doesn’t highlight what. A lot of theories are going around, but causation is a lot harder to determine than correlation.

But a 76% drop off, compared to all of these other games, demonstrates that Battleborn has a serious problem.

[sbspalding (sbspalding@gmail.com)]

Holy crap, people who like Dark Souls 2, really like Dark Souls 2…

A second month drop of 2% is ridiculously low…

[zesban]

There is no such thing as a perfect model.

It doesn’t exist.

There are better and worse models, and which we end up using in practice often has to do with how much time we have to devote to the task at issue, and the resources at our disposal.

It also has to do with the quality of the available information, and how important the question we are trying to answer is in the grand scheme of things.

I’m not going to train up a neural network or whip out a Monte Carlo simulation to win an argument on a video game Forum (not saying either would be used in this case, just providing examples of overkill). It doesn’t make sense, nor does it provide enough additional value to be worth the effort.

To claim that every, single point made with data has to be perfectly unbiased is to claim that almost no study done in the 21st century is valid. And it’s to say half of the field of Data Science is barking up the wrong tree.

While it’s very bad to present information with a clear bias (without identifying the bias), it’s worse to completely dismiss a point because it’s not a perfectly designed study – especially if the stakes are low, and the person presenting the information did not think that his assertions would be under Peer Review.

Yep, that’s correct. Which is why I wasn’t asking the analysis to be a perfect model.

Instead, as you well know, because there is no perfect model, when you present yours, you have to include why it’s not perfect, how imperfect you think it is for those reasons, and clearly state a reasonable margin of error. Otherwise you’re being irresponsible, and telling statistical lies.

It’s completely fair to dismiss a claim of honesty when that hasn’t been presented, and that’s all I’ve dismissed. =)

[Derch (Where's lucky?)]

It also did better in the second month, and also how many of games like that had to go agaisn’t a game or company like overwatch/blizzard.

Another thing to keep in mind, most of those are highly successful games.

If BB can stop the bleed and start adding players it could do well.

BB was
-76%
-21% but this month isn’t over so we will see how this comes out

[sbspalding (sbspalding@gmail.com)]

Fair play. I think I understand your point, and you know my position, so I’m going to duck out of this.

[jYorkElder]

@Derch,

First, love your youtube channel. Thanks for that btw.

Second I agree that we need to stop the bleed.
It would also help to bring back some of those we lost over the last couple of months.

Lootmeggedon is a start but we need more cowbell.

just sayin’.

[Derch (Where's lucky?)]

Community can change things.

At one point people said the Bekah sucked, people said the Pimpernel sucked. We can help change things.

[zesban]

It also did better in the second month, and also how many of games like that had to go agaisn’t a game or company like overwatch/blizzard.

Another thing to keep in mind, most of those are highly successful games.

If BB can stop the bleed and start adding players it could do well.

I think the point is just that, some of those are highly successful games, and that’s what this data shows. Unless you measure success in terms of units sold, in which case, that data isn’t included, but I’d guess some of those games are highly successful in that regard as well. However, then I’d be interested to see what BB’s performance was like on that metric.

Otherwise we can also say that Battleborn is not a highly successful game. And if it isn’t, why isn’t it? What’s the problem?

Your claim is that it’s because of market competition. But like I said, proving causation is much more difficult than proving correlation. You can easily say Battleborn had a precipitous fall in its first month and Overwatch came out at the same time, so there may be a relationship. But you can’t say it’s because of Overwatch that Battleborn had a such a fall without proof of causation. It’s a valid theory however.

[Derch (Where's lucky?)]

Otherwise we can also say that Battleborn is not a highly successful game. And if it isn’t, why isn’t it? What’s the problem?

I’m not saying its highly successful, but its too early to say it’s not or won’t be.

I’m not saying its because of that but if you point out one reason you should point out others there are many factors. Just trying to remain in the middle.

I’m not going to say the sky is falling, but I’m not going to say everything is wonderful. There are issues, but its not hopeless. It’s early.

You can easily say Battleborn had a precipitous fall in its first month and Overwatch came out at the same time, so there may be a relationship. But you can’t say it’s because of Overwatch that Battleborn had a such a fall without proof of causation.

Absolutely, but like I said earlier if an average AAA game drops 60% on the first month, BB comes out and right after it you have Overwatch open beta and release. Well you should expect it to do worse than average. How much worse, I don’t know maybe 70% instead of 60%.

Again none of this helps the player base issue on PC, but when people keep saying BB lost 80% of its base you can put it in context and let them know its not as bad as it seems.

The real issue will be in the next few months weather than can turn it around. That remains to be seen, there is a lot in the pipeline as we have been told. Can those things help bring in new players or bring back old, we will see.

[zesban]

Otherwise we can also say that Battleborn is not a highly successful game. And if it isn’t, why isn’t it? What’s the problem?

I’m not saying its highly successful, but its too early to say it’s not or won’t be.

I’m not saying its because of that but if you point out one reason you should point out others there are many factors. Just trying to remain in the middle.

I’m not going to say the sky is falling, but I’m not going to say everything is wonderful. There are issues, but its not hopeless. It’s early.

I agree, it’s not hopeless by any stretch.

The only reason I jumped in to this thread was that there was a lot of attacking the neutrality and validity of the claims. And rather than shouting down anyone, I wanted to get at why that was happening.

I know your intention was just to provide more data. But others didn’t perceive it that way, and some good old data stewardship would go a long way toward alleviating that.

If we only measure Battleborn’s success by the data we have here, player retention as a percentage in the first 30 and 60 days, then it’s performance put it well outside of being a highly successful game. That shouldn’t be the only way success is measured, and I pointed out another common way of measuring it. But it’s an important component of a multiplayer focused game, so it counts for something.

But as the sample population grows Battleborn continues to have the worst player retention of all titles in the first 30 days. And that’s not good. In fact, it’d probably be a pretty decent challenge to find a game that lost more players, as a percentage, in its first 30 days than Battleborn did.

[Derch (Where's lucky?)]

It was mainly to shed light on the 2 big numbers I saw getting thrown around everywhere

BB lost 80% of its players and that’s unheard of

and

Overwatch has 10,000,000 players BB has 1,000

What I showed should help bring those 2 into context.

BB lost ~80% of its players since it came out, sure but that isn’t so far out of normal for AAA games. Sure the first month is worse than most but the 2nd is better than most. This says nothing about the player base but just that 80% number.

The second number I don’t think anyone here has complained about.

That is what the op is about, nothing more. This isn’t meant to be some great massive study, just context for 2 numbers people have been talking about.

A lot of posts here seem to want this thread to be about something other than that. But it is not.

[otsuexodus]

For those tuning in late:

The winner of the debate so far seems to be Sbspalding, for exiting the thread at the opportune moment.

Can we put a bow on this one and call it done, yet?

[Derch (Where's lucky?)]

The conversation is fine, if you don’t have anything to add to it then you shouldn’t.

I’ve enjoyed the back and fourth with @zesban, more and better info has been pulled out for it. There is nothing wrong with that, in fact its a good thing.

2 sides both disagreeing and agreeing on things is how we get to a better place. We are not fighting or insulting each other but just going for clarity and more information.

I’m finding it to be a wonderful debate. I like to be challenged, it makes me better.

[Elaura (Ninja Truffle)]

I’m sorry, I’m not getting into a debate about how Battleborn is doing. My only intent on participating in this thread was to decry those who seem to insist on giving @Derch a position on this very debate, simply because he posted graphs comparing Battleborn’s performance with other Gearbox titles

Moderators are not moderators because they love a particular game. Moderator status here is across all of Gearbox’s gaming forums and is given to people who are able to be fair and impartial in enforcing the rules. I know of many people who have been around a long time who are not moderators here and it isn’t because they just weren’t very good at spouting the party line.

As for the rest of your post and those following, @sbspalding is a better man than I, because I’m not going any further into it than the following: I’m not an apologist, I’m not a cheerleader, I’m not wearing blinders. As I don’t own stock in GBX, there are only two sales that matter to me in the slightest and my husband and I have most certainly gotten our money’s worth. We played PvE until we finished it and now he plays PvP on PC and has no difficulty getting games.

These are the facts of my position. FWIW, I liked the OP because I appreciated the work he put into it and that he did his best to stay neutral while presenting the numbers in a way that someone who didn’t have a single extra brain cell to dedicate to math or statistics after passing the requisites in college.

Enjoy your climb.

EDIT: [quote=“sbspalding, post:94, topic:1539992”]
so I’m going to duck out of this.
[/quote]

I spoke too soon.

[jYorkElder]

More math and graph thingees.

Yesterday and today. What a difference it makes