• Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
  • We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.

Captain of Outer Space

Come Sale Away With Me
Member
Oct 28, 2017
11,340


Rami's Twitter thread in text form since it's pretty long:
OK FULL STOP ON THE DISCOURSE PLEASE

Just to explain: Sony and Microsoft are -not- responsible for the state of Cyberpunk 2077 on PS4 and Xbox. "Cert" isn't something that ensures games are 'good' - it ensures games do not brick it or disable critical functionality.

Now, industry contracts prohibit me from speaking about the certification processes and requirements of any specific platform, so let me tell you about sort of the general process and concept of 'certification'.

Certification doesn't mean that your game is free of graphical bugs, free of performance issues, free of glitches, or even that it's functioning properly. Cert means the game should not mess up your console, or your ability to use your console, or break rules & trademarks.

Certification is stuff like "don't render critical stuff off-screen", "display warnings if your internet connection is lost", "showcase the correct button labels", "ensure unplugging & plugging back in your controller doesn't crash the game" - stuff like that.

Certification is NOT stuff like "the textures pop up five seconds late" or "objects are floating" or "the game is glitching and my character is T-posing out of the car roof with no pants". That has nothing to do with cert, and even if that comes up in cert, that's not cert's job.

Certification as a process is a giant (GIANT!) list of rules and considerations you can access. You submit your game, wait (a) week(s), and get a list back of failed criteria. If testing finds a lot of failed criteria, they might stop testing midway & return a partial list.

You then get an opportunity to fix the problem, or request a "waiver". You generally cannot continue the submission process without all of the testing criteria having been marked in the backend as a PASS or a WAIVED.

For a "waiver", the developer basically argues why they believe it is fair to be excluded from a requirement. The platform then agrees or disagrees dependent on the game, the situation, the urgency of clearing cert, & promises by the dev to fix fails in a (day 1) patch.

Sometimes, what you can get a waiver for feels counter-intuitive. For example: correctly labeling your buttons is (as far as I've seen) an absolute 100% fail cert - one might argue that it's really not that important that the game uses proprietary language - but it is.

Anyway - after that, you submit again, cert checks against the giant list of rules again, but skip testing on the rules that you've gotten a "waiver" for. If all is clear, you get the ability to set a launch date, and you're ready to go. You go through cert for patches, too.

(All of this is through the worst interface you will ever see in your life. No exceptions. I've seen 1980's DOS interfaces controlled by obscure key combinations that were easier to navigate. At large studios there are specialized people that manage these release processes.)

While I'm describing a generalized certification process above, that doesn't mean that it isn't mostly relevant in this case. In the end, cert is a measure for making sure your console doesn't brick, but the publisher of the game is solely responsible for the quality of the game.

If you make a game that's so buggy that your character spawns T-posing naked flying through the air while everything explodes at once & the only way to progress is hope the NPC you need to talk to got flung exactly your way - but it adheres to all the cert rules, it passes cert.

It is then up to the publisher to say, "OK, good, we've hit all the certification requirements, but the current state of the game does not warrant a launch - we should delay, or cancel". They fix things, and go through cert again.

Platforms do not have a magic "this doesn't adhere to our standards of quality" stamp". They don't "sign off" on your game being good. They leave the decision up to the publishers on whether they want to shoot themselves in the foot with a subpar project launch.

So all I'm saying is no responsibility for the state of Cyberpunk 2077 is on the platforms for "signing off". The responsibility is on those who decided to publish & launch, and only those who participated in that decision - not the devs, not QA, not cert, not the platforms

That's all. Discourse continue.
Thought this might be a useful explainer with the way the discourse is going at the moment.
 
Last edited:

Heliex

Member
Nov 2, 2017
3,109
As somebody who works in certification (not cyberpunk pls dont kill me) I can tell you that you would be BAFFLED at what can be permitted to slip by the publishers if you're close enough to release.
 

Dineren

The Fallen
Oct 25, 2017
3,482
It's probably for the best that platform holders don't try to decide how buggy is too buggy. That said, I don't have a ton of sympathy if people blame them when they don't allow refunds of any kind on their digital platforms.
 

mute

▲ Legend ▲
Member
Oct 25, 2017
25,088
Yeah I was under the impression console cert was just making sure that a game didn't cause anything to the hardware they would have to start issuing warranty repairs on or similar. As much as I'd like to think they check for a baseline level of quality too.
People actually think platform holders provide free QA service for devs...?
Doesn't submitting cost money?
 

Minthara

Freelance Market Director
Verified
Oct 25, 2017
7,928
Montreal
Isn't it QAs job to check on these as well?

QA doesn't fix bugs. In fact, as a QA tester at every level, your bugs coming in the last months before release are often marked as "Will Not Fix" because the developer wants to get their product out the door.

QA has no power in the situation at all besides flagging "Hey, this is bad". In fact, if the devs forget a bug is logged, they often try and shirk the blame on QA and QA has to prove it was logged and that it was marked as won't fix, else they risk losing their jobs.

The power dynamics between programmers and QA is beyond fucked up.
 
Oct 25, 2017
5,620
This forum moves fast.

On one hand I appreciate the amount of threads in this game but on the other I understand the mods/site management wanting to keep things more categorised.

Hand on heart, I honestly do not understand 99% of the mess around the game. The 1% I do understand leaves me with a lot less respect for the developer and yes the platform holders on which the game is released as well.
 

CRIMSON-XIII

Member
Oct 25, 2017
6,173
Chicago, IL
QA doesn't fix bugs. In fact, as a QA tester at every level, your bugs coming in the last months before release are often marked as "Will Not Fix" because the developer wants to get their product out the door.

QA has no power in the situation at all besides flagging "Hey, this is bad". In fact, if the devs forget a bug is logged, they often try and shirk the blame on QA and QA has to prove it was logged and that it was marked as won't fix, else they risk losing their jobs.

The power dynamics between programmers and QA is beyond fucked up.
Woah. I don't even have words for this.
 

Deleted member 8468

User requested account closure
Banned
Oct 26, 2017
9,109
QA doesn't fix bugs. In fact, as a QA tester at every level, your bugs coming in the last months before release are often marked as "Will Not Fix" because the developer wants to get their product out the door.

QA has no power in the situation at all besides flagging "Hey, this is bad". In fact, if the devs forget a bug is logged, they often try and shirk the blame on QA and QA has to prove it was logged and that it was marked as won't fix, else they risk losing their jobs.

The power dynamics between programmers and QA is beyond fucked up.
Is it the programmers, or is it a management problem?
 

FullMetalTech

The Fallen
Oct 25, 2017
2,426
Brooklyn, New York
QA doesn't fix bugs. In fact, as a QA tester at every level, your bugs coming in the last months before release are often marked as "Will Not Fix" because the developer wants to get their product out the door.

QA has no power in the situation at all besides flagging "Hey, this is bad". In fact, if the devs forget a bug is logged, they often try and shirk the blame on QA and QA has to prove it was logged and that it was marked as won't fix, else they risk losing their jobs.

The power dynamics between programmers and QA is beyond fucked up.
Thanks for insight guys. Man they dropped the ball on this game so hard fo sure.
 

Minthara

Freelance Market Director
Verified
Oct 25, 2017
7,928
Montreal
Is it the programmers, or is it a management problem?

Little bit of both sometimes. I've seen developers (at the programmer level) literally berate QA testers because they wrote a bug saying that "Issues should be fixed by X, Y and Z" and even making the suggestion of how a fix should be implemented. In fact, I've seen the rhetorical response "Are you a dev?" Or a request to have that person removed from the project soon after a whole bunch of times.

Yes, management and the production side make the final decision, but the relationship can be just as fucked up between a programmer and a QA tester, especially when that programmer feels like the tester is "challenging" them by bugging a lot of things in their domain. I don't absolve people who behave like that of their behaviour.
 

MatrixMan.exe

Member
Oct 25, 2017
9,499
Is it the programmers, or is it a management problem?

It's a mix, but actually I'd say Producers play a big part in this moreso than programmers. QA will know doubt raise bugs and issues, take that to the QA Lead who will then discuss it with the Producer. The Producer is then within their right to delay an upcoming milestone because of these issues or, if they won't impact the certification process, just say fuck it, let's roll with this build. Especially if upper management tell said producer that they cannot afford any delays.

Not to shift the blame on one person, but providing some context.
 

inpHilltr8r

Member
Oct 27, 2017
3,251
programmers only have so much time

bugs get waived, and production knows exactly who sits in those meetings

after that it's all power politics bullshit
 

Evolved1

Member
Oct 27, 2017
2,619
This forum moves fast.

On one hand I appreciate the amount of threads in this game but on the other I understand the mods/site management wanting to keep things more categorised.

Hand on heart, I honestly do not understand 99% of the mess around the game. The 1% I do understand leaves me with a lot less respect for the developer and yes the platform holders on which the game is released as well.
Congratulations on not reading the OP, I guess?
 

androvsky

Member
Oct 27, 2017
3,506
It passed cert. So no.
I remember Wipeout HD getting delayed 6 months because zone mode failed an automated epilepsy test. Of course, it's a Sony title so as publisher they could have felt the extra care was warranted given the content rather than as the platform holder, but I vaguely remember an implication that the test was a common one.
 

Minthara

Freelance Market Director
Verified
Oct 25, 2017
7,928
Montreal
Doesn't submitting cost money?

Cert and pre-cert (hiring a specialized team to do mock cert checks prior to the actual thing) costs lots of money. People don't work for free and cert is often outsourced since it's only checked for a few weeks at certain periods of time in the game development lifecycle.
 

ultracal31

Member
Oct 25, 2017
4,616
Woah. I don't even have words for this.

and thats a decent team too

if a dev team really don't like you then prepare for passive aggressive comments on your bugs


I don't know if submitting builds cost money on consoles (I work on mobile, where submitting a build is free) - But let's just say QA cost way way more then whatever these platforms would charge :P

Submitting does cost money, quite a bit
 

Viale

▲ Legend ▲
Member
Oct 25, 2017
11,616
This forum moves fast.

On one hand I appreciate the amount of threads in this game but on the other I understand the mods/site management wanting to keep things more categorised.

Hand on heart, I honestly do not understand 99% of the mess around the game. The 1% I do understand leaves me with a lot less respect for the developer and yes the platform holders on which the game is released as well.

So, to recap, you came into this thread, admit you don't understand much about the "mess" of this game and then outright ignore the words of someone who does know what they're talking about regarding the platform holders role in this.

There are plenty of things to dunk on the platform holders about (like sony's refund policy), the state of 2077 at release is not one of those things.
 

Cess007

Member
Oct 27, 2017
14,107
B.C., Mexico
People saying "well, then QA should have done it's work"
Me:
Screen_Shot_2016-08-01_at_12.34.21_PM.0.0.png


I can only imagine what QA at CDPR must have been feeling right now. I pour one for you, friends.
 

Lemony1984

Member
Jul 7, 2020
6,716
I remember Wipeout HD getting delayed 6 months because zone mode failed an automated epilepsy test. Of course, it's a Sony title so as publisher they could have felt the extra care was warranted given the content rather than as the platform holder, but I vaguely remember an implication that the test was a common one.

That doesn't sound like it was a certification thing to me but I'm not sure. I'm not sure how a cert process would even run a test on every piece of the game to make sure it didn't pass the epilepsy tests, whatever they would be.

Would be interested if anyone knows more on that, obviously.
 

Edward

▲ Legend ▲
Avenger
Oct 30, 2017
5,112
Question about the bricking thing being part of the process didn't blops brick some ps5's? How did activision get it through if that's the case.
 

element

Member
Oct 27, 2017
920
QA doesn't fix bugs. In fact, as a QA tester at every level, your bugs coming in the last months before release are often marked as "Will Not Fix" because the developer wants to get their product out the door.

QA has no power in the situation at all besides flagging "Hey, this is bad". In fact, if the devs forget a bug is logged, they often try and shirk the blame on QA and QA has to prove it was logged and that it was marked as won't fix, else they risk losing their jobs.

The power dynamics between programmers and QA is beyond fucked up.
I worked on a project where the producer just arbitrary picked a date and closed all C and D severity bugs. Just select all and "Won't Fix". No triage. No concern of quality. Just 'we don't have time' and we can't have 600 open bugs in the bug database.

The idea of 'zero bug release' or ZBR is comic in how development cuts corners.

But as others have said, QA gets tons of crap when bugs ship and I'll say 99.9% of the time those bugs are logged and a choice by production or game dev (designer, programmer, etc) not to fix it.