Psychology has been using various feedback terms and graphs for decades and game devs have been using those too. Some of those terms are specifically used to make it easier to save the true meaning of a layman’s feedback on complicated topics.
For example: If you want to have a “DPS test” where players are testing various weapons and how they perform you can make testers use specific terminology. If a weapon has to be buffed you can let testers give their feedback in those kind of answers:
Q: How much has item X to be buffed to feel good to use?
A:
- Not at all = The weapon is in a good spot
- A little bit = 5-15%
- A bit = 15-25%
- Quite a bit = 25-40%
- A lot = 40-60%
- Immensely = more than 60%
By using terminology instead of trying to let people interpret math you bridge a gap and make the dimension of the test more tangible. People often cannot interpret math, they can calculate, but most do not have a real feeling for mathematical consequences, especially not when it comes to video games, where there are so many other factors that go into the potency of items.
Another great, but way more elaborate way of beta testing is to give the testers direct access to in-game tools to balance an item themselves. The problem is that you need to communicate a lot for this to work as many people tend to overshoot it and make their test items too strong.
You can do a combination of both too, where you combine the “qualitative” testing method, which needs direct feedback from the testers, but therewith gives the most valuable data with the “quantitative” testing method to put both methods result up against each other and find a middle ground between satisfactory gameplay based on personal adjustments and basic game balance.
I would assume that Gearbox will probably do it similarly to how they adjusted the drop rates of Borderlands 2: They will let a large group of people test multiple weapons with various modifications and then gather the results to then adjust the weapon according to the quantity of the feedback given. Quantitative research is cheaper and easier to make and usually gives results that are good enough to implement in cases like these. It’s rare that a qualitative approach is necessary.