[H] System Evaluations - Awards System?

[Chris_Morley]

We have big things in store for our Systems Evaluation Program, and part of our goal is to adequately database and sort our information for quick access by our readers.

Choosing a numbers-based scoring chart has its ups and downs, but I think we're doing pretty good with our category score and bottom line score.

However, we were toying with the idea of boiling our metrics down into a high impact 'award' that reflects our bottom line conclusion.

One system that I have floated to Kyle, and he suggested that we get your feedback on, is the following:

1) Each system is awarded either 'Not Recommended', 'Recommended', or 'Highly Recommended', and is determined by The Bottom Line Score. (IE, 7 or below is 'Not Recommended', 7.5 to 8.5 is 'Recommended', and 9 or higher is 'Highly Recommended'.)

2) After an arbitrary number of 'Recommended' or 'Highly Recommended' awards in a row, an OEM can be awarded a 'Trusted OEM' status. For instance, it could be after 3 in a row 'Highly Recommended' awards, or 5 in a row 'Recommended' machines. 'Trusted OEM' status should be a very big deal, and it should carry a lot of weight both with the reader and with the OEM.

3) If any 'Trusted OEM' sends a system that is awarded a 'Not Recommended' status, it has its status as a 'Trusted OEM' pulled and they have to start over. Remember, with our 2-4 week turnaround time and the scheduling of an OEM's review systems, an OEM who has its 'Trusted OEM' status yanked could take up to a year to get it back.

We'd really appreciate your thoughts on this.

 

[Tanky]

It is great that someone came up with that idea. It would make it that much easier to reccomend OEM's to people using that system, and it would also encourage OEM's to build better and support better knowing that they could earn an award from an enthusiast site such as this. However, I think that they should only get it if they earn highly reccomended raings more than three times in a row. Reccomended is simply not good enough for me. Even if it is a budget system, I want to know that I am getting the most for my hard earned dollars. One thing that I think should have a higher weight in the overall score is the tech support score. If someone buys an OEM system, than 9 times out of 10 it is because they do not have the knowledge to repair/rebuild the system themselves (like me...).

Just my 2 cents...

 

[PCJ]

It's a great idea.

However, remember one thing:

DON'T FACTOR IN PERFORMANCE.

If a rig has a celery, 512mb and onboard video, then so be it.
As long as they provide a kickass warranty and such, they should still be able to gain any award. After all, the costumer knows that he's buying a celery - or a hardcore overclocked FX-57 rig. You can't help him there. However, you can help him with the stuff he won't know beforehand, like noise, support, stability,...

edit: By all means, show performance, and factor it in when you determine value, but leave it out of the seller reputation.

 

[Tanky]

I agree. Definately leave out performance when it comes to the rating. Whether its a $399.00system or a $4000.00 system, the performance doesn't matter.

 

[Chris_Morley]

We don't judge performance beyond 1) does it perform as others similarly configured should and 2) how does it perform for the price, ie bang for your buck. A machine that is not targeted towards gamers and is priced as mom's email machine will not get knocked for being exactly what it is...an inexpensive internet computer. Velocity Micro's ProMagix DCX kicked the hell outta the Gateway 9310XL in the benchmarks, but Gateway's machine scored higher due to its reliability.

So you guys think that the 'Trusted OEM' status should be granted after 3 or so 'Highly Recommended' system evaluations? 'Recommended' shouldn't cut it?

 

[robberbaron]

Yeah, at least 3. Awards like that seem to be given out too cheaply these days. I think [H] should set an example by saving the award for the best of the best.

 

[Chris_Morley]

Yeah, at least 3. Awards like that seem to be given out too cheaply these days. I think [H] should set an example by saving the award for the best of the best.

Yeah, it should take at least 6 months for a company to qualify for the award, and it should be easily taken back if they misstep, and equally long for them to get it back.

 

[ashmedai]

If [H] is going to say that an OEM is worthy of trust, it should definately be because of their performance over time...also agree with PCJ that performance shouldn't be a factor in establishing the reliability/quality of an OEM's products. As far as "Trusted OEM" status it should just include trust factors...reliability/stability of the system, customer service/tech support, RMA turnaround, et cetera.

As far as slapping a "best" sticker on something, if there were monthly/quarterly/etc round-up style reviews it would be appropriate to say "best OEM system March 2005" or something like that. Not just slap best and a gold star on something, reviewers in general kiss too much ass. If I'm reading a product review I want to know what it does right, what it's strong points are, but even more I want to know what it does wrong. Then again the [H] tends to suffer from that much less than some places. Which is why y'all need to be extra careful slapping "Trusted OEM" on something, [H] is a trusted reviewer so watch out not to hand it out too lightly.

 

[Jason_Wall]

Yeah, I dig that system. 3 HR's for the [H]-verified 'Trusted OEM' status, and as soon as they put out a NR system, that status is rescinded. With the readership this site has, that could be quite the gold ring for OEM's to shoot for.

Nothing too easy, but nothing so impossible as to make the OEM's think that they have nothing to gain by applying the [H] factor. I'm all for exposing the shortcutters and scam artists in the industry, but I think carrying the reputation that we give fair, but thorough analyses will go a lot further than the reputation of being a bunch of ballbusters.

Naturally, with every 'Recommended' verdict dealt out, it will have to be stated as to what it is recommended for. As has been said, premium gaming box vs. grandma's email and ebay machine. Looking forward to hitting high gear on this!