Are QA employees seen as a hindrance when a game is developed?

[fireflame]

I have read that one of the reasons some games are poorly optimized or have bugs when they are released is that the feedbacks of QA are overlooked and seen as a hindrance, a potential obstacle in the race for release and profits.

To what extent do QA feedbacks matter? I suppose that a company like Bethesda has QA, and yet a lot of their games in the past had numerous issues at launch, so can we assume in their case QA feedbacks was ignored?

This old article from Kotaku described the status of QA, and it looked like a tedious and not rewarding job.

I feel however dismissing quality assurance is something that can, and normally,would backfire. If you don't care about the quality of your product, obviously, the reputation of your company will be quickly tainted and you will lose customers unless your marketing is really powerful.

 

[ILikeFeet]

it's more that devs don't have the time to tackle everything QA finds

 

[Imran]

Some devs really welcome it and some don't. Depends on a lot of factors, including how implementable fixes and suggestions are, whether the QA is internal or external, etc.

I've definitely heard some stories of QA and developers not getting along at all.

 

[Big-E]

QA's job is to find bugs. The dev team decides what is worth fixing or not. It is a pure cost analysis at that point. Does fixing this mean financial sense? If the answer is no, it gets shipped.

 

[Z-Beat]

it's more that devs don't have the time to tackle everything QA finds

Click to shrink...

Basically this. No one likes having everything they worked on get sent back and told it's broken

Things like Sonic 06 prove that you can't skip on QA

 

[mrmoose]

This obviously changes in modern development environments where roles are often intertwined, but they often have different purposes.

Devs develop new features and fix the bugs. But developing new features is usually the focus.
QA finds bugs.
Operations tries to make sure that stuff is done in a sustainable way.

So often the latter two can cause things to slow down, which is where I think this hindrance thing you're talking about comes from. But even traditionally, a good dev team will appreciate a good qa team because it's much better to catch things early.

 

[Iucidium]

it's more that devs don't have the time to tackle everything QA finds

Click to shrink...

QA's job is to find bugs. The dev team decides what is worth fixing or not. It is a pure cost analysis at that point. Does fixing this mean financial sense? If the answer is no, it gets shipped.

Click to shrink...

/Thread.

 

[808s & Villainy]

As a developer I'm both thankful that they find bugs before it goes off to the client while simultaneously annoyed at them for trying something random and finding a bug (even though they're not the reason it's there)

 

[Velocity:Design:Comfort]

Only at Netherrealm Studios

 

[elenarie]

the feedbacks of QA are overlooked and seen as a hindrance

Click to shrink...

Perhaps by morons and arrogant people that think they know better.

it's more that devs don't have the time to tackle everything QA finds

Click to shrink...

This.

 

[Tygre]

The answer is, as almost always, "it depends".

Some studios have fully integrated Test/QA departments. They interface and communicate directly with developers, they're involved from the earliest design stages, their feedback is not just welcomed it is proactively sought. Test/QA are equal and respected partners in the overall development process.

Some studios have no internal Test/QA. They have as little to do with whatever outsourced contractors they hire as possible, they would never imagine sharing anything until it's practically finished, they don't want any feedback other than "doesn't brick a PS4, will pass cert".


I will say that even in the most positive environments, there still comes a point when EVERYONE just wants a project to finish. At that point even Test/QA just want to stop finding showstoppers and move on. At that point it's very easy to start feeling like a hindrance yourself, never mind what anyone else thinks of you.

 

[Deleted member 6562]

I feel however dismissing quality assurance is something that can, and normally, would backfire.

Click to shrink...

Oh it does, and that gives us comfort. :) Seriously though, I feel my job is done if I've found the bugs. If (whoever is in charge) decides to release it knowing the issues, that's on them. I don't test games anymore, but really it's the same at any software company.

 

[stuckpixel]

Everyone loves the QA guy who finds a few bugs. Everyone hates the QA guy that finds a lot of bugs.

 

[Gentlemen]

It hinges on the management culture. If management sees QA as an equal partner in the output of the engineering team, then they'll set a much more reasonable cutoff for what constitutes a showstopper bug and agree to dedicate resources to a stable build over filling out the list of 'nice to have' features before ship date. If management sees QA as a disposable herd of script-following contractors then you get what you sometimes get, a long list of buggy features in an unstable release.
In any case if QA doesn't have the ear of the person in charge of meeting the milestone dates then QA's feedback risks gets trampled like a drunken tourist in Pamplona.
I wouldn't trust any of the largest AAA studios today to treat QA like an equitable partner.

 

[MrCibb]

I don't think any good developer would ever see QA as a hindrance and not pay attention to the feedback. Grumbling about a large amount of bugs to fix I get, but that's QA doing their job. As the OP says, if you want a quality product then you want those bugs found and fixed and most devs do. QA is very important.

It's just that games have deadlines and so there comes a point where you have to say "No, we can't fix that 1cm of grass floating anymore.", and then it becomes "No, we can't fix that dodgy animation if you enter the mission from a certain direction", etc. etc. The QA is still doing their job and are valued for it, but somebody's got to make the call when they can't keep fixing everything. The game has to come out, so sacrifices get made.

 

[Deleted member 43084]

Throwback to Tales from the Trenches about horror stories in gaming QA.

http://trenchescomic.com/tales/post/9810

 

[A Path Finder]

What? No. QA and functionallity testers are super important to the quality of the game I am working on. If a bug does not get fixed then that's because there was no time and other higher priority bugs came first.

We use QA for more than testing, sometimes ask for opinions on difficulty and gameplay as well. Always get great feedback since they are gamers and passionate, it's invaluable.

 

[Zhengi]

A game has to go to the platform holder like Sony or Nintendo for additional testing. If there are too many bugs, the game gets kicked back to the publisher for additional testing and bug fixing.

So QAs are treated fairly, at least at the place I worked at for a couple of summers when I was in college.

 

[Technesis]

QA find bugs but when its comes to what get fixed and when it's beyond them. Throw bug quotas in there and it becomes a toxic mess.

 

[Agentnibs]

I've done QA for a few years now but not game QA.

My general experience is that no one is ever really happy to see QA, in the sense that we find issues that means more work for the dev team. That being said everyone knows the importance and value of the QA team.

Actually whats funny is that in my experience the QA team is often much smaller than the dev team, so often that means they fix bugs faster than we can find new ones or retest old ones. So at the end of the day we both hate each other lol

 

[DiipuSurotu]

In software development in general the only people that are seen as more of a hindrance than QA are the documentation people, if they even exist

 

[fireflame]

I don't think any good developer would ever see QA as a hindrance and not pay attention to the feedback. Grumbling about a large amount of bugs to fix I get, but that's QA doing their job. As the OP says, if you want a quality product then you want those bugs found and fixed and most devs do. QA is very important.

It's just that games have deadlines and so there comes a point where you have to say "No, we can't fix that 1cm of grass floating anymore.", and then it becomes "No, we can't fix that dodgy animation if you enter the mission from a certain direction", etc. etc. The QA is still doing their job and are valued for it, but somebody's got to make the call when they can't keep fixing everything. The game has to come out, so sacrifices get made.

Click to shrink...

but then I wonder what happens regarding games like Oblivion. They still have many bugs when released, so I wonder if this is because they can't afford caring about QA, or the game had even more bugs before release?

 

[MarioW]

There are a couple of levels to this.

Speaking from experience, when QA is first introduced to frontline developers either internally or via a publisher, it can certainly be the case that testers can be seen as "just creating more work" when something is already "done". However, developers do come around and see QA as a valuable part of the process, especially for helping hunt down obscure issues and verifying work.

It can also be the case that management will decide to ship a game, despite known issues raised by QA. It is going to be incredibly rare where the QA function itself has the authority to hold a game back from shipping. So some of those situations where gamers might say "how did their QA miss this" is a misdirected comment when in actuality the critique should be "how could the company ship knowing they had this".

That being said, games are very complicated pieces of software and can be incredibly hard to test in a comprehensive way that exposes all known issues ahead of time. The majority of console games will have dozens of testers logging a few hundred hours each at best. On release, you suddenly have thousands or tens of thousands of "testers" playing in a much wider variety of ways, so through orders of magnitude more eyes on the game you have the potential for the audience themselves to expose issues that managed to slip past the QA team.

 

[Don Fluffles]

As a designer, QA is crucial to a game's quality. From what I've heard, smaller, less "corporate" devs are more welcoming.

Bigger corps, however, just tend to separate testers from devs out of fear that they'll influence the game too much.

 

[Expy]

No. Time constraints are the hindrance. The dev and qa teams work within those constraints as much as possible.

Time is money.

 

[Gakidou]

Yeah QA are tremendously important.

The thing to understand about devs is on a personal level people can tend to get quite narrow minded into their own tasks. They just want to do their part of the job well and have the appearance of quick and efficient production on their end. So they don't like that QA gives them more work to do, or if the bug report is written poorly or they disagree with a judgment call. It's kind of a shoot the messenger scenario though. Deep down every good dev knows QA is important. Personally I try never to be underappreciative of QA, and always add a little thankyou or a "good spot" comment if they find an especially difficult or obscure bug. I always encourage them to even mention petty or subjective bugs and even their own opinions.

Also regarding commercial games that are always "very buggy" on release, you ain't seen nothing! Games like oblivion i'll wager were 10x as broken and buggy a few months before completion. And yeah the truth is it's generally production who decides when bugs stop getting fixed, rather than devs no longer getting any feedback from QA.

I definitely think QA deserves to be treated better, it's true that most companies treat them like unskilled labour to try and farm for cheap. But there are a lot of myths as well. Devs do appreciate and respect QA, it's not a "dead end" career either it can get you involved in proprietary tech and be a legit way to get further into the industry, though there's a lot of competition on that level.

 

[Deleted member 48897]

Perhaps by morons and arrogant people that think they know better.

Click to shrink...

Management! *jazzhands*

But seriously, if an end user finds a bug within the first week of a product shipping, then either QA found that bug and it was ignored, or they didn't have a QA process.

 

[Sankara]

based on some stories I've heard, they are seen as disposable and somewhat sub-human

 

[Technesis]

Management! *jazzhands*

But seriously, if an end user finds a bug within the first week of a product shipping, then either QA found that bug and it was ignored, or they didn't have a QA process.

Click to shrink...

Or repro rate was like 1/100 and the end user was that 2. Unlikely but possible lol.

 

[Monarch1501]

I'll weight on this by saying that both our internal and external Montreal QA teams are invaluable in shaping the final end product. It's easier for me to go and talk to our own QA team to find bugs but I'm always actively cooperating with any QA members to try and see if there's any bug with anything I've done or to see if there are any easy steps to reproduce them.

Sometime I'll ask to waive some of them (I don't ultimately decide what get waived) since it doesn't hinder too much of the experience or because we don't have time but every bug reported gets eventually treated.

And if you're crawling under tons of bugs - don't blame the QA team for finding them since you were the one creating them.

 

[Bitmap Frogs]

Or repro rate was like 1/100 and the end user was that 2. Unlikely but possible lol.

Click to shrink...

This, right here, is the worst.

 

[MrCibb]

but then I wonder what happens regarding games like Oblivion. They still have many bugs when released, so I wonder if this is because they can't afford caring about QA, or the game had even more bugs before release?

Click to shrink...

The game definitely would have had more bugs before release, 100%. Makes you think how bad it was! In the final few months alone a game can see so many fixes. But no matter what the end result is, whether it's buggy or not, QA are valued and an important part of the development. A dev may shake their fist when they have thousands of bugs to fix, but if you want a quality product it's got to be done and any developer worth his salt knows that.

Chances are, in regards to games like Oblivion, a lot of the bugs in the final release were likely logged but QA's job ends there for the most part. The developer needs to have the time and scope to fix it after deciding it even is worth fixing, and afterwards allow time for additional QA to be done to ensure there's no regression. They also need to be able to ACTUALLY fix it. Some bugs can be ingrained into the game, they might require a potentially big change to an in-game system due to an over-sight nobody thought about, so the bug might get reluctantly passed for just being a bit too dangerous to start messing with. But, QA did their job in finding it. Nobody would, or should, be faulting them there.

 

[srylain]

When I had my two month stint as QA, at the end of one project we were told that because no one is going to play the game for more than 200+ hours like we did that they wouldn't be finding the bugs we were finding. So a lot of the last week bugs just got declined, which to be fair most of them were just extremely small visual things. And because the game ended up getting released for an event and then delisted not too much later, they really didn't expect anyone to play it that long.

 

[CortexVortex]

Unfortunately you will never have enough time to ship a game or release a content update with all bugs fixed.
If we release an update with only one open A bug (which is the worst kind of bug in our system) I'm already kinda happy with the quality.

Sometimes it just producers who don't want to change an release but sometimes you have a nice marketing deal with Apple or something and for that reason the update has to be released on Day X.

That does not mean they don't appreciate what QA does though.

 

[Deleted member 41183]

Depending on scope, there can be anywhere from a few hundred bugs to tens of thousands, so it's about balancing time/cost/reward.

In the case of some games that have bugs for years and across re-releases, it's possible that fixing that bug could cause other, possibly worse ones, or the bug you see as an end user is the result of a fix that is stopping other, possibly worse, bugs from happening. Each developer/publisher will have different standards, too, which is a function of the budget they have for QA (both testing and fixing).

But people are human; sometimes QA misses stuff. And sometimes the developers don't (or can't) fix stuff they probably should. I know some devs who've told me there's nothing more frustrating than not being able to fix stuff they know isn't perfect.

 

[mullfuchs]

QA has saved my ass multiple times and I have an immense amount of respect for anyone who does QA, either in games or elsewhere.

 

[Bitmap Frogs]

Just casually dropping a non affiliate link here...

https://www.amazon.com/Perfect-Soft...3692/ref=mt_paperback?_encoding=UTF8&me=&qid=

 

[Nome]

I don't hate QA, QA hates me 8-)

 

[ToddBonzalez]

I work in AAA games. QA are an essential part of the process. Good QA testers are a godsend toward the end of development when the team is focused on ironing out as many bugs as possible.

 

[Kroz]

I worked QA on many projects. It really depends on the bug. Sometimes you get devs that want tons of bugs, even the most minor issues.
Then there are devs that will ignore fixing major issues despite them being logged by the QA team.
You also have to consider that some bugs are too time consuming to fix under strict dev times. Sometimes fixing a minor issue will spawn 10 more in its wake.
At times it feels like you're work isnt being looked at by devs or taken seriously but you gotta remember things are often crazy on the dev side.

A lot of people dont fathom how hard/stressful making a game can be for any part of the team

 

[bastardly]

i never personally worked QA, only helped at times, but its literally their job to break the game, and i've never known devs to ignore bug reports ever.

and every game that was ever made or will be made will be launched knowlngly with bugs, its just matter if any of those known ones will break the game. i always remembered how much we all laughed at grandmas boys when they were celebrating "congrats, we found all the bugs", uh yeh sureeeeeee. qa jobs is never done and often thankless, but it is vital

 

[UnholyChocolate]

99% of QA is great and super helpful, but some give a vauge explanation with no repro and continuously bounce bugs back changing what the issue is.

I think he best bug I received was that some text was upside down on a wheel.

 

[Kapryov]

Didn't QA reports for some Sonic game leak online at one point, containing numerous bugs that were still present when the game shipped?

Anyway, at the end of the day the devs have a time limit, so they need to choose which bugs they prioritize. Some bugs may not even be easy to replicate and fix in decent time.
For all the stupid bugs you find and wonder how it shipped with them, think about all the possibly much worse bugs they chose to squash instead.

 

[elenarie]

Didn't QA reports for some Sonic game leak online at one point, containing numerous bugs that were still present when the game shipped?

Click to shrink...

You're saying as if that is something strange. All games ship with crap load of bugs.

 

[Kapryov]

You're saying as if that is something strange. All games ship with crap load of bugs.

Click to shrink...

No it just stuck out when I saw the thread title.
I remember it appearing and people being like "they knew all along!" etc

Maybe it was Sonic Adventure, back before the days of console patches. I need to go look this up now...

Edit: it's probably this: http://info.sonicretro.org/Game_Development:Sonic_Adventure#Tester_Feedback_database
They're right, Twinkle Park should have multiplayer.

Edit 2: this is gold! this is a reported bug:
"In Casinopolis, in the Shower Room, with Sonic, if Sonic takes a shower, Sonic appears to take a shower with Tails."
I feel so sorry for developers that need to sort through this kinda stuff

 

[TheEchosOfTheCyborg]

They seem to be considered some of the least important and most expendable by higher ups by higher ups at least from the stories I've heard, similar to Community Managers, least paid, least respected by a management that thinks anyone can do it.

It's like Sound Mixers and Sound Editors in the film industry, the many stories I've heard about them being belittled there both inside and outside their industry as "Not important" (yeah go watch a movie with terrible sound mixing and editing and then go say that) and it reflects a lot of what I see which game industries QA and Community Managers.

 

[TestMonkey]

I will say that even in the most positive environments, there still comes a point when EVERYONE just wants a project to finish. At that point even Test/QA just want to stop finding showstoppers and move on. At that point it's very easy to start feeling like a hindrance yourself, never mind what anyone else thinks of you.

Click to shrink...

If your testers turn a blind eye to a showstopper at any point in development, they are garbage QA and should be sacked immediately. Write it up and if the producer wants to waive it in the database it's on their ass.

 

[Jarhab]

As someone with both QA and developer experience, I can say that I've never seen QA as a hindrance. Games would be broken as shit without them. If a game ships buggy, it's because it was released prematurely. Testers find tons of bugs but there are limits to how many devs can actually fix when they have deadlines to meet. Once you reach beta, bugs start to get triaged and minor or rare ones are waived. Eventually, the only bugs that get addressed are critical ones because fixing anything else is too much of a risk.

 

[Dead Man Typing]

QA can be seen as an obstacle at the end of a project.
The build has to pass internal QA playthrough, pass internal compliance and then pass the first party submission process.

QA can also be seen as a great asset when organised properly.
I've been a part of many projects that wouldn't have shipped on time if QA hadn't been organised properly.
Finding show stopping bugs in games that take days to play through is not easy work.

At the end of the day, someone has to play the game like an end user.
You'd rather it was someone who isn't going to get angry that they spent their money on your game.

 

[Deleted member 12833]

I have to do UAT at my company and pretty sure both QA and the devs hate me. Sorry, not sorry