Videogames, politics, science, all the important things in life.

Why game reviews suck and why we’re stuck with them.

Posted by nfinit on November 12, 2009

The way we do game reviews right now is kinda fucked up.

As most people get their review information from websites instead of print media, there’s undue incentive for these sites to release their reviews as early as possible. Most games take upwards of 6-10 hours to complete; and that’s not taking time needed to learn the game itself. So for a game like Call of Duty 6: Modern Warfare 2, which may have a 12 hour long main story, you probably need to at least double that to actually complete the game proper.

The way game sites see it, having a game review for your site show up after the game itself is released is pretty much useless. Game sites are in competition with each other for your page views, and all it takes is one glowing or one damning review to convince your average reader either way. So review sites need review copies, and need those copies ASAP.

For the most part, review sites and print magazines are beholden to the publishers to get these review copies in reviewer’s hands in time for a review to come online. This is why the lead time for a game review is a pretty good indication of a game’s quality and/or marketing effort– if you don’t see reviews come out before the game’s release, chances are the publisher doesn’t have a lot of confidence in that game’s quality and refused to ship out review copies. This works a lot like Hollywood– If you don’t start seeing movie reviews the week before the movie’s actual release, there’s a pretty good chance you’re better off not watching that movie:

Columbia Pictures would rather you not be warned too soon

(A neat bit from the most recent Giant Bomb podcast that inspired this post in the first place– Giant Bomb’s Brad Shoemaker was visiting New York City for an EA press junket when he learned that the street date for CoD6: MW2 was broken worse than Carla Nash’s face. Shoemaker, being a game reviewer for Giant Bomb, knew that his site had not yet received their review copy went to Chinatown, bought a copy at considerable mark-up, shipped overnight said copy back to Sausalito, California. whereupon Jeff Gertsman spent the next two days feverishly playing through CoD6: MW2 in order to get their review up before the game’s actual street date.)

If a gaming site gets their review copy late– or worse, not at all– they’re sorta fucked.  Aside from the sudden expenditure of being forced to pay for games to review, they’ve lost valuable lead time.  If you put a game review up in the time it takes a game to be released, played, then reviewed, chances are your review is now largely meaningless.  You also have to consider that most games do 90% of their business within the first two weeks of release– it’s not like keeping a review score up on your site weeks or months after release is doing you any real good.

And as with anything involving the games industry and a potential abuse of power, publishers will make the most openly corrupt use of these review copies possible.  Take for instance the recent revelation that Ubisoft has threatened to withhold review copies from sites who do not give Assassin’s Creed 2 favorable scores.  Luckily, Ubisoft only makes about 2 worthwhile games a year, but what if this decree had come down from a company from whom people actually buy games from, like EA?

I mean, this is still totally going to be good, right? Right? ….shit

Of course, there’s a third option for review sites, but it’s a Faustinian bargain at best– reviewing pre-release builds.  This is fraught with danger, as best highlighted by Dave Halverson of Play magazine handing a 95 to Sonic 2k6 based on impressions from a pre-release build and promises from Sega that “the game would be fixed”.  The resulting scandal– the game’s Metacritic aggregate stands at 45%– resulted in Play magazine abandoning review scores altogether, although it’s unclear if Dave Halverson still reviews pre-release builds.  But you have to assume this still happens, there’s just no way for print media to get review scores out in time if they were playing release candidates for every game they review.

The simple truth is there’s no real way to trust a review score.  Early scores are likely based on unfinished code, or the game is finished but the review copy itself comes with strings attached from the publisher.  Even scores that come out closer to he review date aren’t entirely trustworthy– there’s just no way to make an informed judgment on a 20+ hour game that you’ve only been exposed to for 48 hours of nonstop gaming.

The whole system is fucked and there’s no good way of fixing it that doesn’t involve dumping Metacritic and it’s disproportionate influence upon publishers.  But there’s little chance of that– in many ways, Metacritic is shaped by the publisher’s themselves.  It provides the only reliable infodump for game reviews that gamers have access to, and at the same time publishers know how to game the Metacritic system to serve their own ends.  A review that doesn’t appear on Metacritic may not appear at all in the minds of mainstream gamers, and the publishers know how to manipulate Metacritic scores on a high level.  In the publisher’s minds, they have created the perfect tool– the one feedback source they will pay any attention to, while at the same time shaping it to their will.  It’s an echo chamber.

So we need another source for game reviews– and we need a source that the publishers can’t game.  The only way of keeping the publishers from manipulating review scores is by removing scores entirely.  If you sit and actually read a review instead of skimming over it until you get to the score, you can get a pretty good idea if the reviewer has formed an informed, unbiased, professional opinion.  That part’s not hard– it’s the second part to getting this working that’s going to kill us– we have to accept the idea that review scores released at or before release date are inherently untrustworthy.  And that means actually waiting a week or so to buy the game itself.

Of course, exceptions can be made for especially sweet pre-order bonuses.

And that may well prove impossible.  Publishers and retail have forced gamers into a mindset that they need games at release, that they need to be “part of the conversation” that they need to have these games in hand before the online community of said game dissipates entirely.  Which is madness.  I’m not sure how we get past that point with anything but sheer willpower.


4 Responses to “Why game reviews suck and why we’re stuck with them.”

  1. Theodosia Skye said

    Yes. Captions are back.

    Oh captions, never go away again!

  2. yakety sakul said

    Tbh there’s really only a handful of games where you really need a review. Most publishers: Valve, Bioware for example are consistant enough that a simple yeah its not crap is enough.

    Otherwise I think we’ve played enough games to know what you’re getting into. Now of course new properties especially those from new developers are harder to judge on first sight.

    Maybe a review system that says: here’s the genre do you like that? And then a head nod either way

  3. nfinit said

    I always liked the idea we had on Plat for Buy/Don’t Buy/Rent.

  4. LoganStormbringer said

    While i rarely buy games at release i also like to avoid the review with the score as well. I like the Gametrailers/Giant Bomb approach which is to peruse as much content as possible instead of just watching a review.

    Also as i’ve gotten older i kinda already know whether i want to buy a game or not before a review even comes out. Its weird because my ideal job would be a reviewer of Video games. I mean i have never in my life even checked Metacritic

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: