Ok clearly this hasn't done much for my rather poor image with RTG :/
Ha! No worries Lee, you don't have a poor image with us. Anyone who's stuck around for this long, contributing to the forum, being an active member of the community, and who hasn't become overly negative or simply burned out — we don't mind the odd criticism or ten! — gets a star in our book.
My statement wasn't intended as a big criticism, I think maybe you miss read the first part of that, I completely understand it must be a small testing team, I think I made that understanding pretty clear. Hence my saying, its not like you have an AAA EA style super lab to check everything. The 'awful lot' part was unfair, but I wasnt really picking my words all that carefully, so sorry about that one.
Hopefully I can spare my self the wrath (again), i'll try be abit more specific. So from my point of view, as a fan (that loves the game dearly) who played the game on 1.0, I did wonder how on earth the performance issues with europolis could have been missed. It was borderline unplayable on anything but a top spec system, even I was getting drops below 20fps. It was across the board and the same for everyone.
I was also quick to praise how fast that got fixed, and have gone out of my way to speak up about things your self and RTG have done well (which is a lot), for what my opinion is worth anyway.
The lazor eyes again I would have thought would have been obvious as it effects a wide range of amd cards (the popular r9 series mostly), its not like you can just not see it right ?
Also the HoAW door you could walk right though (which was how I ended up finishing the game the first time), would also seem an obvious thing to have checked. It was a door after all. Players WILL try and open it, and end up passing though it.
For me (I have done some game testing, and other stuff in my time) the above would be fairly obvious stuff to check, and from a personal point of view I wonder how they could have slipped passed.
But the team has nothing but my up most understanding on how hard it is to test games. Obviously I have no idea if testing is internal, or if it goes out to some one else (or another company for that matter).
It was less about the music.
And I love the game so far, its brilliant.
Most of the things you describe — not all, but most — are simply due to a relatively limited test park: i.e. our development hardware. The game ran fine for us with version 1.0. No, performance wasn't GREAT, but we could run it on 'awesome' on most of our PCs (our Macs were definitely slower, but by no means unplayable) with a decent frame-rate and few issues. Heck, I ran the game in 4K on our two-year old meeting room computer…while recording Fraps video.
The performance issues encountered by many (definitely not by all) took us by surprise. We'd heard from partners and friends of ours who'd played the game prior to launch who experienced no such problems. And maybe all of them were running on high-end hardware with 64-bit Windows and plenty of RAM? And maybe all of them were fine with frame-rates dipping to the low 20s? I don't know, but the fact is that we didn't experience or hear about these performance issues until after launch.
As soon as we received bug reports and detailed system specs, we were able to identify problem areas; mostly related to memory. There's not much we can do with the frame-rate, however, since Unity doesn't offer a lot in terms of multithreading support. The strain is largely on the CPU, and not the GPU. This isn't ideal, but with a third-party engine and no access to the source code, we're stuck — at least until we can port (or consider porting) to Unity 5.0.
In regards to AMD cards, it was just bad luck. We don't have a whole hardware park of different setups available to us. We have our development PCs. Those PCs did not exhibit those problems. With a bigger budget and/or more time, we could have either bought more test rigs or outsourced to a test lab — not something we can afford right now, and not something I'm particularly comfortable doing — but we had to make do with what we had. And we didn't see or catch this bug until it was too late. There are many, many larger games — games from companies with huge QA departments — that end up in the same boat. Shipping a game on three platforms, with a host of different configurations, is incredibly difficult and ambitious. I'm amazed that we haven't seen MORE problems…but part of that is using Unity, which takes care of most (though not all, in turns out) compatibility issues. That's why we chose it, so that we didn't have to worry about that part of development. It obviously didn't work 100%, but I think it's fair to say that MOST players did not have any issues with graphical glitches.
The door you could walk through in HoAW… Yeah, no explanation for that. We spent a lot of time running up against collisions — walls, objects, doors — everywhere in the game…except right there. We even plugged some collision holes in the House weeks and days prior to launch. How we missed that, I'll never know, but it happens. Did you play (or read about) the new Assassin's Creed? That's from a company with probably a THOUSAND testers. They missed stuff, too.
As a small company with a pretty big (and ambitious) game running on a third-party engine, I think we shipped a remarkably good product. It wasn't bug free by any measure of things, but few games of this size and complexity are. More QA would have helped, but that was outside our scope. We did a lot of testing, and we missed a few things — more than I would have liked, but there we go; I was a tester, too — and then there were things we simply couldn't catch, because we were on the 'wrong' hardware. Them's the shakes. We fixed things VERY quickly after launch, our tech guys worked tirelessly, even when it was hard to reproduce bugs, and right now we have a solid, solid foundation to build subsequent episodes on.
It is unfortunate when new bugs are introduced in patches, but it happens, and in this case it was easy to miss it for those on the team who've spent hours and hours playing and replaying. We'll fix this, too. And soon.
The reason I reacted strongly is that I've worked in game development now for twenty years (eek) and I know how incredibly hard and exhausting it is to test a game. Every time we make a build, every time we fix a TINY, tiny thing, something else might break, and we have to sit down and play through the entire game from start to finish, with all different choices & consequences accounted for. And not just play it: we have to play it THOROUGHLY. This is hard, hard work and stuff slips through the cracks, no matter how many dedicated testers you have. I think Chapters is one of the least buggy games I've worked on, and this is without a single dedicated tester. Everyone on the team contributed (and continues to contribute) to the test loops, and we're lucky to have one of the best testers I've ever worked with in charge of that process, but even then it's pretty fucking amazing that we managed to pull it off.
So yeah, that's why. Appreciate the effort, applaud the result, and realise that game testing is one of the most backbreaking and complex tasks imaginable. Games are a nightmare. They break, all the time. And testing games is far from being a dream job: it's hard labour. And, most often, a task that gets little to no recognition.