They can be seen as a hindrance by the devs if there was no clear definition of what is a bug. If the guidelines are clearly defined, that is not case. Even if the definition of a bug is never the same as the beginning and at the end of a project.
-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
Are QA employees seen as a hindrance when a game is developed?
- Thread starter fireflame
- Start date
Wholesome thread, some good posts in here.
My biggest pet peeve is "how did QA not find this?!" for really obvious bugs, what gets me is they do not understand the QA process and that QA do not decide what gets fixed or not.
I've personally had that one thrown at our team around a specific bad bug that was 100% and very obvious that publishing would refuse to act upon, despite our team flagging it 10+ times through dev.
Oh another thing people don't realise is that games are riddled to the ceiling with bugs weeks maybe even days before they're finished. Sometimes most of the baddies get fixed, sometimes they don't. Priority vs. time.
My biggest pet peeve is "how did QA not find this?!" for really obvious bugs, what gets me is they do not understand the QA process and that QA do not decide what gets fixed or not.
I've personally had that one thrown at our team around a specific bad bug that was 100% and very obvious that publishing would refuse to act upon, despite our team flagging it 10+ times through dev.
Oh another thing people don't realise is that games are riddled to the ceiling with bugs weeks maybe even days before they're finished. Sometimes most of the baddies get fixed, sometimes they don't. Priority vs. time.
Op, I can guarantee you that without QA department at Bethesda, you would find 10 times more bugs in their games. There are still many that require a dedication and actual gameplay intent that QA testers do not even have time to get into. (Depends on their development model and dev cycles.)
And with online some bugs are corrected post launch or many times between gold release and release date(day 1 patch).
Usually there is the disc version which gets submitted to first party for cert. Then a Day 0 patch, which usually replaces the digital version / reorder / preload version, and a Day 1 patch to bring the disc and the Day 0 versions on the same baseline.
stares into space despondent thinking about a myiad of sonic games
QA, particularly embedded QA, is an invaluable part of the development team. I've never encountered any condescension or antipathy towards the QA team. At worst I've seen external QA incentivized by volume, not quality, and there had to be some process changes in order to get them aligned and providing the most valuable sort of bug reports.
QA is hugely important, but no big project is ever going to fix everything QA finds. You're talking about millions of lines of code with infinite permutations of input coming in from players. Perfection is impossible. The end of every project is going to be triage, deciding which bugs are worth spending the limited remaining time to fix, and which will just have to ship. When a game is really really buggy, it's because it just needed more time than was feasible to give it. A six month delay is millions and millions of dollars in salaries that have to be paid for people working that project, so there comes a point where you really do have to just cut your losses and accept a subpar product if you're not making something that's guaranteed millions of sales. So yeah, I imagine towards the end that process can become somewhat adversarial, between testers and coders who think some bug passes muster to be fixed versus producers and managers who think something else should take precedence, but no one's out here saying QA is useless or should be cut. That'd be insane. Then you would ship with all of the bugs, instead of just the bugs we chose to deprioritize.
Bethesda proves you can. How they get so many sales of their in house games have with how much of a technical mess they always are I will never know.
There are some developers where QA/Support people are viewed as a lower form of life in a caste system.
The game developers are gods and rockstars and can do no wrong. While everyone below that, QA/Tech/CS/IT/etc, are expendable losers who shouldn't even be allowed to talk to the gods above them.
The game developers are gods and rockstars and can do no wrong. While everyone below that, QA/Tech/CS/IT/etc, are expendable losers who shouldn't even be allowed to talk to the gods above them.
Speaking from personal experience as a developer (not in games), QA is so absolutely fucking crucial and such a godsdend that it actually makes me sad that, at least in my area, it is an undervalued position that is paid like shit and often considered easily replaceable.
In my current team it's seen with a much better attitude (my team's manager comes from QA, even) and it makes such a positive difference.
However deadlines are usually a thing (not very stringent in my current company which makes development take slightly longer but be exponentially more robust) and I can imagine that when dealing with the extremely tight deadlines of game development a whole lot of QA findings get ignored.
In my current team it's seen with a much better attitude (my team's manager comes from QA, even) and it makes such a positive difference.
However deadlines are usually a thing (not very stringent in my current company which makes development take slightly longer but be exponentially more robust) and I can imagine that when dealing with the extremely tight deadlines of game development a whole lot of QA findings get ignored.
I never suggested turning a blind eye, thanks for the implication though.
I'm saying there's an attitude shift.
Day 1 of a project during some embedded testing, you find a huge issue, you're hyped. You just did your job and probably saved everyone a load of effort way down the line.
Day 1328 of a project while testing Release Candidate 17, you find a huge issue, you are very much not hyped. The project is already a year overdue, it's been mismanaged and bloated beyond all recognition, there's feature creep on feature creep. Nobody will thank you, nobody is happy about it, you just want it to be done.
I have a friend who worked at EA Tiburon back around the early ps360 era and he said they had weekly quotas. I thought I'd love to be a game QA tester until he told me about how it was working there.
Remember when Sony ran a reality show contest, and the winner would get a job as a game tester?
In all seriousness, it really depends on the company and its attitude towards testing. I used to work for a game developer that had such a good testing department, other companies would ask to use it. Most testers liked their jobs and got really good at it (and they often ended up moving on to other jobs in the company, like designers or producers). We would embed a tester on the development team from really early on, ramp up to a few full time testers by the end, and by the time the game shipped we had hundreds of bugs in the database that were deemed either too minor or too dangerous to fix (bugs were rated A,B,C or D - A is a crash bug, B isn't a crash but major enough the game can't ship with it, C is minor and D is a suggestion - we couldn't ship with any A or B bugs).
Note that this was the era of game cartridges, where games could not be patched after shipping. I think attitudes towards testing changed overtime as game consoles went online and patching became a thing, thus making extreme testing before launch less important.
In all seriousness, it really depends on the company and its attitude towards testing. I used to work for a game developer that had such a good testing department, other companies would ask to use it. Most testers liked their jobs and got really good at it (and they often ended up moving on to other jobs in the company, like designers or producers). We would embed a tester on the development team from really early on, ramp up to a few full time testers by the end, and by the time the game shipped we had hundreds of bugs in the database that were deemed either too minor or too dangerous to fix (bugs were rated A,B,C or D - A is a crash bug, B isn't a crash but major enough the game can't ship with it, C is minor and D is a suggestion - we couldn't ship with any A or B bugs).
Note that this was the era of game cartridges, where games could not be patched after shipping. I think attitudes towards testing changed overtime as game consoles went online and patching became a thing, thus making extreme testing before launch less important.
Last edited:
Where I work (a developer of one of the big publishers) the (embedded) QA is encouraged to give feedback. Obviously finding bugs is their job, but also preventing bugs devtesting changes before they're submitted - as well as give general feedback about the feature. If something doesn't feel right the QA usually picks up on it while a designer might not. The QA also decides the priority of the various issues (as much as they can) - which means they do have influence on what gets fixed and what doesn't.
There's also no quotas. That stuff is ridiculous.
Some places treat their QA like absolute shit, but they're just as important as anyone else during game dev so you just gotta find a company that treats them as such. The pay is still shit either way though haha.
There's also no quotas. That stuff is ridiculous.
Some places treat their QA like absolute shit, but they're just as important as anyone else during game dev so you just gotta find a company that treats them as such. The pay is still shit either way though haha.
Fallout 76 and Bethesda's extensive mod community says you still can't.
Not in our case. QA are seen as a part of the team just like anyone else. I mean, it's a bummer if a new issue is found just when you thought you're done, but that's on the whole team - not the QA. And QA is also very much welcome to give feedback, just like anyone on the team.
The cause for a released game being buggy is probably not the quality of the QA or how much feedback they've given - It has more to do with how much time QA was given for tests, which tests and issues were prioritized and which issues were approved for release.
The cause for a released game being buggy is probably not the quality of the QA or how much feedback they've given - It has more to do with how much time QA was given for tests, which tests and issues were prioritized and which issues were approved for release.
As a dev, QA are anything but a hindrance in the studio I work in.
We work on more than one platform (mobile), so it's hard to prevent all bugs on my own while integrating things. Thanks to them, all grounds are covered.
They also know the game inside out and have a better underestanding of many features that don't fall under my umbrella, so their knowledge of the project is absolutely invaluable.
We work on more than one platform (mobile), so it's hard to prevent all bugs on my own while integrating things. Thanks to them, all grounds are covered.
They also know the game inside out and have a better underestanding of many features that don't fall under my umbrella, so their knowledge of the project is absolutely invaluable.
I'm not a game developer but I do work in software. Over the past few years, we've been getting rid of dedicated QA people entirely. My experience is that they were mostly useless.
Nowadays, we design all of our features to be as simple and single-task oriented as possible. We also require a level of unit and acceptance tests. The idea being that we understand our code well enough to know what it should do and what's considered a failure scenario. Our tests run every time we deploy our code and any failure will break the pipeline. This prevents an unrelated code change from breaking something unexpected.
In practice, we aren't perfect. It's difficult to prepare for every scenario and some tests are difficult to make. It's pretty annoying when it takes longer to build a test than it takes to write the functional software. And, unfortunately, there's always pressure to disable certain tests when there's a demand to go production. We do our best though.
Nowadays, we design all of our features to be as simple and single-task oriented as possible. We also require a level of unit and acceptance tests. The idea being that we understand our code well enough to know what it should do and what's considered a failure scenario. Our tests run every time we deploy our code and any failure will break the pipeline. This prevents an unrelated code change from breaking something unexpected.
In practice, we aren't perfect. It's difficult to prepare for every scenario and some tests are difficult to make. It's pretty annoying when it takes longer to build a test than it takes to write the functional software. And, unfortunately, there's always pressure to disable certain tests when there's a demand to go production. We do our best though.