Inaccurate data could lead you in an entirely wrong or counterproductive direction. And the tragedy is that most customer surveys are inherently inaccurate, unreliable, or misleading.
www.forbes.com
WotC surveys make a number of those mistakes.
Do they now?
1)
Sampling and Reliability: WoTC is making no mistakes here. They are getting a plenty large sample size, and they know exactly who they are getting. Self-selected survey takers ie the most passionate and engaged section of the fan-base, those they need to please the most to ensure long-term success.
2)
Inherent Bias in Questions: WoTC is making no mistakes here. Their questions are perfectly neutral
3)
Allow Logistical Hurdles to Disrupt Participation: Okay, granted, they could have a mistake here. Their surveys have gotten longer and longer, and the questions are all essentially the same. There is a fatigue factor at play. Though it is also mitigated by the self-selection of passionate people who will work through the survey anyways. But, I can give this to you, it isn't like this point has been made by either of you before right this moment.
4)
Conflating Attitude and Behavior: Not a mistake WoTC is making, they aren't asking anything like either of these
5)
Ignorance of Outliers: HAHAHAHA, yeah, WoTC isn't making this mistake any time soon. They are WELL aware of their outliers.
6)
Confirmation Bias: Very little evidence of this at all, so can't say it is a mistake WoTC is making.
7)
Correlation for Causation: Doesn't apply to the questions being asked, so can't be a mistake they are making
Finally, "
and only a rigorous and critical look can definitively improve your results." Something we can say with confidence WoTC has done over the last decade. So, in summary... they may have made ONE of those mistakes.
Because playtests are super stable and don't shift.
Playtest 7 hasn't changed since I downloaded it. And it won't until... WoTC changes it. Seems like a completely different situation.
No. I talked about them getting percentages for each category independently and only as a broad X percentage voted this way, which doesn't give a hard satisfaction rating. i.e. you can say that 44% of survey takers said they were satisfied, but you can't say that there was an 80% satisfaction rating, because they can't know how satisfied the customers were with their votes of "satisfied" or "very satisfied."
You might be "very satisfied" at 66% and I might be "very satisfied" at 88%
You don't need a "hard" satisfaction rating, whatever that means. And, yes, you CAN say that there was an 80% satisfaction overall. That math is completely possible to do, using the techniques developed by the field. It is not actually impossible, as has been demonstrated by the people in the field... doing exactly that all the time, across disciplines.
Because we are not wrong and you haven't actually countered what we are saying. You keep misunderstanding things and trying to counter things we aren't saying with links and statements that aren't accurate about what WotC is actually doing.
Good God!
They CAN'T do that. It's not possible, because none of their specific satisfaction percentages are accurate. It's quite literally impossible for them to know how many of the "unsatisfied" and "very unsatisfied" customers want the idea scrapped completely and how many want another iteration of the ability.
Just repeating "They use the entirety of it!" doesn't counter diddly.
IT
IS
NOT
IMPOSSIBLE
You keep insisting it is. It isn't. They are capable of taking all their results and getting a single percentage. You are wrong about this.
Yes they are. Not directly, but if they are saying that if satisfaction hits X percentage they will give it another go, they are asking it indirectly. If they weren't, they wouldn't be doing it.
Asking a question directly is what they are doing, then using the answer to that question to determine step two. You can't declare their questions are wrong because they are using the answer to determine something, and then declare they are "indirectly" asking a question they are not asking.
You keep reading into their survey things that are not there, to justify your declarations that was is there is flawed. That isn't how this works.
And yet they announced a percentage of satisfaction that if hit, will result in trying to make a successful incarnation of the ability. You've just admitted they don't have accurate data to determine that. Thanks for finally conceding one of the points that
@mamba and I have been making.
Yes, they announce the percentage of "We like this". Then they use that data to determine if they should reiterate something. BUT THEY DO NOT ASK IF WE WANT THEM TO REITERATE IT. That isn't the goal of the survey. That isn't what they are gathering data on. I've conceded this point before, because it is a non-point. The only reason you two think this is a victory for you is you keep intentionally misrepresenting what WoTC is actually doing.
