Search:  

 
 
   All ForumsHot TopicsGallery






how-to block ads


 
Forums » Tech and Talk » Computer Games and Clans » PC gaming Tech » Intel Cheats in 3D Performance Benchmarks
Search Topic:
Uniqs:
622
Share Topic:
RSS topic:
toggle:
flat / full
normal / watch
Posting:
Post a:
Post a:
New ATI Reference Drivers 9.10 »
« Core i5-750 OC Preview  
page: 1 · 2
AuthorAll Replies


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL

Intel Cheats in 3D Performance Benchmarks

This isn't terribly game related, since no gamer in their right mind would use an integrated Intel chipset as their main graphics card, but it's still interesting.

»techreport.com/articles.x/17732
--
It takes two to lie, Marge. One to lie, and one to listen.


Mchart
Super Joe

join:2004-01-21
Gurnee, IL
·AT&T Yahoo
·RoadRunner Cable

I don't see what the problem is. It's not like they are reducing video quality to get the FPS gains. They are doing something that actually probably took quite a bit of work to pull off and it's cool if you ask me. GPU overloaded? Offload some of that work to the CPU. Doesn't seem like cheating to me.
--
THIS IS SPENCER. MISSION ACCOMPLISHED - I HAVE JOE. RETURNING TO BASE.


TK421
Premium
join:2004-12-19
Canada

Driver optimizations are common and should be considered fine if they can sqeeze a few more FPS without losing image qaulity. AMD, Nvidia, and Intel all do it.

The problem here though is manipulating a benchmark result for marketing purposes.

How do Intel's driver optimizations affect the competitive landscape? To find out, we assembled an AMD 785G-based system that's pretty comparable to the G41 rig we used for testing: Athlon II X2 250 processor, Gigabyte GA-MA785GPMT-UD2H motherboard, the same Raptor hard drive, and 4GB of DDR3 memory running at 800MHz with looser timings than the Intel system. We even disabled the board's dedicated sideport graphics memory, forcing the GPU to share system RAM like the G41.

With the Futuremark-approved Catalyst 9.9 drivers, the AMD 785G-based system scored 2161 in 3DMark Vantage—nearly the same score as the 2132 3DMarks the G41 gets when it's playing by the rules, but well below the 2931 the score the G41 posts with optimizations enabled. (Renaming the Vantage executable on the AMD system had no notable effect on benchmark scores.) The app-specific optimization gives the G41 a definitive lead in 3DMark Vantage.

Here's the tricky part: the very same 785G system managed 30 frames per second in Crysis: Warhead, which is twice the frame rate of the G41 with all its vertex offloading mojo in action. The G41's new-found dominance in 3DMark doesn't translate to superior gaming performance, even in this game targeted by the same optimization.

That's no great shock, really. We've seen Intel's integrated graphics solutions thoroughly outclassed by rivals from AMD and Nvidia on multiple occasions.

All of which brings us back to the perils of using 3DMark Vantage as a substitute or proxy for testing real games. Those perils are well established by now. PC makers and others in positions of influence would do well to re-train their focus on real applications—especially for testing integrated graphics solutions, which have no need of advanced graphics workloads based on the latest version of DirectX to push their limits. 3DMark's traditionally purported role as a predictor of future game workloads makes little sense in the context of these modest IGPs.


In summary, Intel violated Futuremarks explicit rules prohibiting targeting 3DMark optimizing by recognizing the executable (proven by renaming the executable for testing). Intel gets a higher 3DMark score than their competition yet even with specific real game optimizations still can not get half the real performance!

Nothing really shocking but it demonstrates again that as techreport says: "3DMark's traditionally purported role as a predictor of future game workloads makes little sense in the context of these modest IGPs."


SKYHN
Lu.. Lu.. Lulululu
Premium
join:2001-09-16
99999
reply to knightry
Nothing that ATI, AMD, Nvidia, even Matrox have not done before.
--
Do you douche with coconut juice?


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL

reply to knightry
The problem is that they've done something specifically banned by 3DVantage.

And just because other companies have done it means it should be acceptable? This seems like a dangerous precedent.
--
It takes two to lie, Marge. One to lie, and one to listen.


Mchart
Super Joe

join:2004-01-21
Gurnee, IL
·AT&T Yahoo
·RoadRunner Cable

Precedent? It was done first by Nvidia YEARS ago and ATI.

I have absolutely NO problem with a company doing this if they can improve the performance without decreasing visual quality. There should not be any complaints if the visaul quality is not decreased.

Repeat after me. There is nothing wrong with increasing performance if visual quality is not decreased.

And FYI, no one gives a hoot or a holler about 3DMark anymore. Considering that Intel's fixes also work in Crysis and other games - Cool. It means the customer is getting more for his money.
--
THIS IS SPENCER. MISSION ACCOMPLISHED - I HAVE JOE. RETURNING TO BASE.

Joe12345678

join:2003-07-22
Des Plaines, IL

reply to knightry
said by knightry See Profile :

This isn't terribly game related, since no gamer in their right mind would use an integrated Intel chipset as their main graphics card, but it's still interesting.

»techreport.com/articles.x/17732
apple may start putting this carp in the new minis.


elios

join:2005-11-15
Springfield, MO
reply to knightry
its only a cheat if it ONLY does it on the benchmark
if it does it with games to its not a cheap
its not a bad plan really with multicore cpus your going to 1-2 cores not fully loaded what not use that power for some thing?


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL

said by elios See Profile :

its only a cheat if it ONLY does it on the benchmark
if it does it with games to its not a cheap
its not a bad plan really with multicore cpus your going to 1-2 cores not fully loaded what not use that power for some thing?
Look, I'm not saying for them to use it in games is a bad thing, but using it on the Vantage Benchmark is strictly prohibited (whether they do it for games too or not).

quote:
With the exception of configuring the correct rendering mode on multi-GPU systems, it is prohibited for the driver to detect the launch of 3DMark Vantage executable and to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection. Optimizations in the driver that utilize empirical data of 3DMark Vantage workloads are prohibited.
--
It takes two to lie, Marge. One to lie, and one to listen.


elios

join:2005-11-15
Springfield, MO
rename the benchmark see if it does it
if it does its not cheating
if it doesnt its cheating

pretty easy imo if it DOES IT IN EVERY THING its not cheat
if its ONLY doing it based on detecting what ap is running then yes its cheating


Mchart
Super Joe

join:2004-01-21
Gurnee, IL
·AT&T Yahoo
·RoadRunner Cable

Well, it has to do it based on detecting an application as they have written application specific profiles for each application.

If you say 'cheating' in these terms that specifically is implying that the graphics are being degraded to achieve better performance - Like nvidia and ATI did a while back. However, Intel is not doing this. They are offloading some of the work to the CPU.

End of discussion.
--
THIS IS SPENCER. MISSION ACCOMPLISHED - I HAVE JOE. RETURNING TO BASE.


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL

said by Mchart See Profile :

Well, it has to do it based on detecting an application as they have written application specific profiles for each application.

If you say 'cheating' in these terms that specifically is implying that the graphics are being degraded to achieve better performance - Like nvidia and ATI did a while back. However, Intel is not doing this. They are offloading some of the work to the CPU.

End of discussion.
Look, I'm sorry to belabor the point, but Vantage is very clear about this. I quote again, my point being bold:

quote:
With the exception of configuring the correct rendering mode on multi-GPU systems, it is prohibited for the driver to detect the launch of 3DMark Vantage executable and to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection. Optimizations in the driver that utilize empirical data of 3DMark Vantage workloads are prohibited.
Hell, I don't even care they they did it, but they broke the rules, which is cheating by my books.
--
It takes two to lie, Marge. One to lie, and one to listen.


Mchart
Super Joe

join:2004-01-21
Gurnee, IL
·AT&T Yahoo
·RoadRunner Cable


1 edit
said by knightry See Profile :

said by Mchart See Profile :

Well, it has to do it based on detecting an application as they have written application specific profiles for each application.

If you say 'cheating' in these terms that specifically is implying that the graphics are being degraded to achieve better performance - Like nvidia and ATI did a while back. However, Intel is not doing this. They are offloading some of the work to the CPU.

End of discussion.
Look, I'm sorry to belabor the point, but Vantage is very clear about this. I quote again, my point being bold:

quote:
With the exception of configuring the correct rendering mode on multi-GPU systems, it is prohibited for the driver to detect the launch of 3DMark Vantage executable and to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection. Optimizations in the driver that utilize empirical data of 3DMark Vantage workloads are prohibited.
Hell, I don't even care they they did it, but they broke the rules, which is cheating by my books.
And?

I don't care. Nor does most anyone else. If the quality isn't being changed - No one cares. You didn't bother to bold the part of the quote that actually matters - And thats the line that says if the quality is altered to get said quality gains.
--
THIS IS SPENCER. MISSION ACCOMPLISHED - I HAVE JOE. RETURNING TO BASE.


elios

join:2005-11-15
Springfield, MO
·Mediacom

reply to knightry
also who says there detecting any thing?
ever think MAYBE thats just how the driver works high load on GPU -> off load to CPU?

really try it out rename the benchmark to some some thing random and see what it does

then try the same with a game if its still off loading its just how works

Margolis
Premium
join:2003-11-24
Saint Louis, MO
·Charter Pipeline


1 edit
reply to knightry
intel didn't cheat. This is a VERY BIASED article. the original article snipped only a small part of the 3dmark rules that they thought would back up their biased claims.

The very first rule from 3dmark concerning cheating:

It is prohibited to for the driver to change the rendering quality level, rendering technique or replace or alter any shaders or other parts of the workload from that what is requested by 3DMark.

and they ignore the most important part of the quote they do use.

to alter, replace or override any quality parameters or parts of the benchmark workload based on the detection
they are NOT replacing any quality parameters or any of the workload. They are just offloading parts to the cpu to help out.


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL


1 edit
reply to elios
said by elios See Profile :

also who says there detecting any thing?
ever think MAYBE thats just how the driver works high load on GPU -> off load to CPU?

really try it out rename the benchmark to some some thing random and see what it does

then try the same with a game if its still off loading its just how works
This is what the writers of the article did. They renamed the EXE and lost the performance increase.

After a closer read, I see the point of some of the others here. Since they didn't affect the quality, it seems like the change they made should be okay. Though I'd still like to hear 3DMark's statement on whether they feel the modification is allowed. After all, the test is supposed to be a stress-test of the GPU, which can easily be argued to not be accurate if some of the processing of the GPU is off-loaded to the CPU.

edit: fixed typo
--
It takes two to lie, Marge. One to lie, and one to listen.

Margolis
Premium
join:2003-11-24
Saint Louis, MO
reply to knightry
the thing is, ms is doing this for games also, not just benchmark programs like 3dmark. If they only did it for 3dmark and never helped other games it would be one thing, but they are helping other games perform better also.


knightry
I'm Not Fat, I'm Festively Plump

join:2002-05-06
Oviedo, FL

said by Margolis See Profile :

the thing is, ms is doing this for games also, not just benchmark programs like 3dmark. If they only did it for 3dmark and never helped other games it would be one thing, but they are helping other games perform better also.
True, though only for a very very small subset of the games out there. I feel like it would be more fair if their offloading were something more generally applicable, not just for the few they've picked.
--
It takes two to lie, Marge. One to lie, and one to listen.

Margolis
Premium
join:2003-11-24
Saint Louis, MO
·Charter Pipeline

said by knightry See Profile :

said by Margolis See Profile :

the thing is, ms is doing this for games also, not just benchmark programs like 3dmark. If they only did it for 3dmark and never helped other games it would be one thing, but they are helping other games perform better also.
True, though only for a very very small subset of the games out there. I feel like it would be more fair if their offloading were something more generally applicable, not just for the few they've picked.
It may not work for all. So they are testing and tweaking what they know it works for. I'm sure as time progresses they will add more games to the list.


Dream Killer
Graveyard Shift
Premium
join:2002-08-09
Forest Hills, NY
clubs:
reply to knightry
I don't see what the problem is. Nothing is wrong with tweaking drivers for a unique executable. Both ATi and Nvidia tweaks driver profiles for each game. It's like tuning a race car's settings to the race track before each race.
-
Forums » Tech and Talk » Computer Games and Clans » PC gaming TechNew ATI Reference Drivers 9.10 »
« Core i5-750 OC Preview  
page: 1 · 2


Saturday, 21-Nov 09:28:28 Terms of Use | Privacy Policy | Hosting by www.nac.net - DSL,Hosting & Co-lo | feedback | contact
over 10 years online! © 1999-2009 dslreports.com.
page compression OFF
Most commented news this week
· [115] Verizon Again Hints At Metered Billing
· [84] Spain Declares Broadband A Legal Right
· [82] Will AOL's Implosion Ever End?
· [80] There's Still No Evidence That Metered Billing Is Necessary
· [74] Verizon To Be Tested By Unofficial Droid Tethering
· [73] Femtocells Are A No Show
· [71] Deploying FTTH Without Digging Things Up
· [68] Weekend Open Thread
· [66] Verizon To AT&T: The Truth Hurts
· [60] Chicago Tribune Visits 'Comcast University'
Most people now reading
· Hacking.....seriously, how easy is it to get hacked? [Security]
· 3.x Feral Druid - Bear Tanking Guide [World of Warcraft]
· You get ONE WoW wish... what would you do? [World of Warcraft]
· Town to photograph EVERY car that enters and leaves [Security]
· Connecting to Google Voice Via SIP [VOIP Tech Chat]
· RG Firmware update to VDSL2 this morning [AT&T U-verse]
· Removing Wood Paneling on Walls.. Alot of It [Home Repair & Improvement]
· Extjs grid combo box. [Webmasters and Developers]
· [ Classes] Druid tanking: rotation and glyphs [World of Warcraft]
· Framed for child porn 151; by a PC virus [Security]