i7 920 vs i7 860 direct test comparison

[Snow Crash]

Good idea! So we just run a batch of X WUs and check the completion times right?

Yes

If so, make sure to include enough WUs in the process so that highly multithreaded machines won't be half idle. Which presents a problem.. 24 WUs would be a good number for SP rigs, it's divisible by 8 (Core i5/i7), by 4 (i5 750 or older Quads), and by 12 (Gulftown). However, my Dual Gainestown as well as the Quad Opteron (16 Threads) would have an "idle problem" there.

So I'd suggest 24 "test"-WUs and just leaving the multi sockets out? Since most people crunch on single sockets.

Got it ... I messed up my calcs ... nice catch. Yes, 24 is a much better number of WUs to test although going to 48 gets us to an even divisor across all current and near future core count (1,2,3,4,6,8,12,16) but I can see where crunchers are going to be hard pressed to go down for 12 hours as it is so I think your suggestion of 24 is great.

I will be working on a basline for 24 HFCC WUs over the next couple of days and I will be trying to pick out WUs that are more or less uniform so there is not too much raw idle at the end. Oh ... Oh ... I need to go talk to Mr. Wizard ... why not build the set from the same exact WU ??? This way we could scale up or down as much as necessary to handle overall runtime and core count linearly. Looks like I'll have some crunching downtime next week as I work out the details of how to make this happen.

As a start, what elemtents do we want to capture?
CPU
vCore
vTT
CPU model
CPU Stepping
CPU Multi
CPU BLCK
QPI
Thread count (HT on/off)

DIMM
vDIMM
Amount
Speed
Channels (1, 2, 3)
Timings

HDD
SSD

Graphics

110 / 220

Headless

[jcool]

Sounds good Snow Crash. HDD vs. SSD shouldn't make any difference, but we can include it regardless.

When measuring efficiency, we should also include rating, make and model of the PSU used - that way we can roughly calculate the net power draw of a certain rig w/o PSU losses (SPCR has great efficiency tables for their tested PSUs at different load levels, for example). Also any other components drawing power should be mentioned, for example, a watercooled rig with 2 pumps and 10 fans will draw a lot more than an open air test bench-like setup with just 2 fans or so.

[Fitseries3]

great read.

posted a link to this on TPU.

[123bob]

Agreed that an average runtime of a fixed number of the exact same WUs would be the most accurate way to test the things we are in control of. Namely, the machine doing the crunching.

The "back half" of WU processing, the WCG initial scoring algorithm, your "wingmen", the quorum scoring algorithm, the timing of a days data, we have zero control of.

The fixed WU group is a better way than my post #46 here. What I figured there was to take a large enough sample to try to wash out variances in the WUs themselves. The exact same WUs would eliminate that variance. Truthfully, that post was the one that counted, to me, to show that the 860 was in the same "ballpark" with the 920. The fixed WU group would provide perhaps a faster, more accurate picture. The loss of some WUs would be in the name of science, even though I don't like losing any WUs.

However, the method of that post #46 would appear to be an alternative if you don't want one of the rigs to lose production while crunching the test batch. The trick to this would be to use as large a sample group as you can stand to process. I used excel to process the data. Now that the sheet is set up, it's not too bad to cut and paste pages of WUs into it. We would have to use some "uncertainty bars" around that type of data though.

I hesitated to post score totals, knowing that the "back-end" variances would come into play. I'm happy to see that it sparked the conversation in this thread about what the test metric should look like.

I will start a fixed WU run sometime today. I'll report here if I come up with any issues making this happen.

Regards,
Bob

[123bob]

great read.

posted a link to this on TPU.

Yo fits, good to see you. Hope all is well.

Bob

[fallwind]

It'll be interesting to see if a machine will crunch transplanted WU's! In reality only the second machine will have to be taken offline for the test. The first machine can crunch the WU and return them. On the second one you'll have to disable network activity, crunch the WU's and collect the data, and then copy back the original BOINC folders for that machine. It can then continue on it's way.

Personally, I'd transplant the entire "BOINC" and "Program Data" folders over to the second machine. BOINC should recognize the hardware changes at startup and then run the benchmark. For the sake of accuracy, we can exclude and WU's that were "in progress" when the transplant took place. Also the last 7 WU's should be excluded since the CPU is not running 8 threads while crunching those.

Here's a cool idea: save a copy of the BOINC folders for future use! Whenever you want to test some mew hardware, you'll have a reference set up WU's to crunch to compare it to other hardware. You could even zip it up and email it to other members. I'll have a Gulftown up and running shortly and would be more than happy to run the test.

[123bob]

OK, I did it.

Here's how.

The 860 rig is the "host" machine.
The 920 is the "client" or "target" machine.

1. I suspended crunching on both.

2. Aborted the current running WUs (Yes, I tried REAL hard not to scream...

3. I aborted all "waiting to run" WUs on the 860 rig, except for 24 HFCC WUs. These are the test WUs.

4. "Updated project" so the queue was cleared. Only 24 units sitting there.

5. I aborted ALL the WUs on the 920 rig. (With the same screaming as above. I think the rig screamed louder though....)

6. "Updated project" so the entire queue was cleared. No WUs in this rig.

7. I copied the entire "BOINC" folder, under the app data folder on the 860 rig, on to a USB stick.

8. I made a backup of the BOINC folder of the original 920 rig so I can recover it later, I hope.....

9. I copied the BOINC folder from the 860 rig into the app data folder of the 920 rig.

10. I disconnected the network cable from both rigs.

11. Rebooted the 860 rig and resumed crunching. It started on the test units.

12. Rebooted the 920 rig, it then showed the test WUs. Ran benches, which turned out the same as reported before, then resumed. It is now crunching test WUs.

We'll see in about 12 hours. Probably more like tomorrow morning since I ain't staying up until 4 am here....though I may get up early...

So, the cut and paste data will come from the Boinc mgr screens of both rigs. I hope it formats well for import into excel.

May the crunching gods forgive me for blowing 24 WUs worth of time here.... The test WUs from the 920 rig are garbage after the test....

I have these 24 units still on the stick for future use if needed.

Bob

[Lu523]

Looking forward to the results of this test.

[123bob]

Here's a cool idea: save a copy of the BOINC folders for future use! Whenever you want to test some mew hardware, you'll have a reference set up WU's to crunch to compare it to other hardware. You could even zip it up and email it to other members. I'll have a Gulftown up and running shortly and would be more than happy to run the test.

The only issues I see here is that the "app data>BOINC" folder contains your BOINC key. It also appears to contain another "authenticator" key in the "lookup_account.xml" file.... I do not know if that's like your account authenticator key. We know what that means, if it is.

Could they be stripped out? I think they could. That would mean the person receiving the WUs would have to edit the files to put their own keys in it. I think it would run after that...We know for sure we can change BOINC keys. That's how I'm contributing to Scott's account.

I was unable to make the 920 rig see the test WUs when I tried to copy just the WU "projects" folder by itself. It appears to need the other data in the main folder to discover it's status.

Bob

[DAK1640]

123bob, your commitment & dedication to this analysis is second to none.

[my87csx481]

Looking forward to the results of this test.

123bob, your commitment & dedication to this analysis is second to none.

Ditto.

I'd just like to express my gratitude for you guys putting this together.

I'm on the verge of getting a new job (just waiting for them to get the drug screen results back), which may hopefully allow me to get another rig going (I've been gone for quite a while). My electricity costs nearly 20 cents per KWH so the results will help me make a better decision when I'm able to put another rig together.

Thanks a lot to all involved.

[INFRNL]

You are insane , but your dedication to get this right is priceless

I do not think I need to go that far.

I got my x3440@2.8ghz@1.2v now till I get a new cooler in this week.
ram is the same, only diff is this rig only has 1 gpu

Got 1 920@2.8ghz@1.2v. same ram, but 2 gpus

I am currently only running HFCC on both as I think those are what we agreed upon; if not, thats what I am sticking to for my comparison. I basically want to monitor for a few days to a week to compare. Hopefully I get usable results

The only thing I can see about this way is that each rig has to make sure its wu's are all turned in before last update. I have noticed in the past that sometimes the machine does not like to update as often as it should. I will try to keep an eye on this to compare with.

I will let you know my results

[[XC] Hicks121]

Good thead guys!! Keep up the good work....I cant wait til I can get into the newer stuff again.

[123bob]

You are insane , but your dedication to get this right is priceless

I do not think I need to go that far.

I got my x3440@2.8ghz@1.2v now till I get a new cooler in this week.
ram is the same, only diff is this rig only has 1 gpu

Got 1 920@2.8ghz@1.2v. same ram, but 2 gpus

I am currently only running HFCC on both as I think those are what we agreed upon; if not, thats what I am sticking to for my comparison. I basically want to monitor for a few days to a week to compare. Hopefully I get usable results

The only thing I can see about this way is that each rig has to make sure its wu's are all turned in before last update. I have noticed in the past that sometimes the machine does not like to update as often as it should. I will try to keep an eye on this to compare with.

I will let you know my results

Yup, HFCC. The test WU of choice..

Good to see you got some clock on the new rig. I'll be curious to see how well it clocks later. So, for now at least, you have the 2.8 data point, I have 3.7, and Emu has 4 gig.

BTW to all, if I was to re-connect the network cable with "suspend network activity" selected, would that also block uploading WUs? If not, this test has to be run with the cable unplugged. Otherwise you'll be letting the bogus WUs from the "client/target" machine hit the work servers. That couldn't be good.

I'm going to stay unplugged for the duration just to make sure.

Bob

[Otis11]

If you say suspend network activity you'll be good. It does not connect at all if that is selected.

Good work, keep it up!

[fallwind]

Simply awesome! Good luck Bob, here's hoping everything goes smoothly.

[emuexport]

Looks like we will be testing something else now!

I'll probably get onto this tomorrow the way things are travelling at the moment.

[123bob]

OK, drum roll please....I have results....Interesting results, I might add...

REMEMBER, an IDENTICAL COPY of 24 WUs were run on each machine.


First the mandatory screenies.

The 920 IDLE CPUz shot...



The 920 BOINC Data Screen.



Add up the hours/minutes/seconds as individual columns and you get total 85:658:702.

This is easier math than converting all to decimal hours at this point. I could not get a good cut and paste out of boincmgr to go right to excel. Had to do it manually. Arrrgh. SOMEONE IS MOST WELCOME TO CHECK MY MATH. I did this bleary eyed at 5am here...




The 860 IDLE CPUz shot. The only difference I see here is on the memory page for NB freq, whatever that is. The 860 is dual channel, the 920 is triple of course....



The 860 BOINC data screen.



Add up the hours/minutes/seconds as individual columns and you get total 85:642:697 for the 860.

Now, it gets interesting. 24 WUs were run.

For the 920, converting to decimal hours we get 96.161666 hours. Divide that by 24 and you get 4.006736 Hours/WU.

For the 860, converting to decimal hours we get 95.8936111 hours. Divide by 24 and you get 3.9955671 Hours/WU.

Note the 860 is doing WUs FASTER than the 920.....

How much? Well, .0111688 hours, or .6701322 minutes, or 40.207 seconds.

So the 860 is about 40 seconds average FASTER per WU than the 920.

I think I'd still call them even, but in no way is the 860 "crippled" compared to the 920.

So at least at the 3.7 gig data point, the 860's better power efficiency and similar score to the 920 makes it a good choice for pure crunching. If you need to go Gulftown later, then 920 might be your choice.

I'm also finding the X3440 choice INFRNL made to be interesting for farm rigs....Might have to get a dozen of those.

Bob

FURTHER INFO: Remember I copied the 920's original BOINC app data folder in order to restore it at the end of the test? Well, I'm happy to report that it worked. The 920 rig is crunching away again on it's own folder. It pizzed off the server due to dumping the WUs, but I have 8 units crunching and I'm sure I'll get more when the WUs complete and show the server that it's OK.

[v0dka]

Nice work, I agree completely with your conclusion.

Only difference between te 920 and the 860 is the lower NB speed and command rate? Even if small, the effect of that might explain the difference and / or a bit more.

[jcool]

Awesome work Bob

Although I'm afraid you'll have to run that again on the 920 - "NB Speed" is uncore, which is important and makes a difference for crunching, as it is the area where the L3 cache resides.

It should by all means be identical for a real comparison. To fix this, just increase the Uncore mult from 16x to 18x on the 920. The 860 doesn't have adjustable uncore multipliers.

[v0dka]

And while you're at it, put command rate at 2 as well.

[123bob]

I'll think about it. I have a day job you know.....

[jcool]

Yes, it's not that important really. I suspect the 920 would be a little faster with higher uncore, but when it comes down to it, crunching performance is virtually identical between those 2 - which is what I always said
Still, awesome to know for sure.

[123bob]

Gave it some thought. I'd only need to re-run the 920.

Tried the uncore bios multi at 18, no boot. I think it was too high. Had to clear CMOS to recover.

I suspect this board multiplies by two, so I should need x9 in my bios. I'll see.

I'll update as soon as I get everything back where it was.

Bob

[jcool]

Huh? Naw, 18x uncore should work with 8x dram... there is no 9x dram divider, next would be 10x which wouldn't work with 18x uncore (for Bloomfields, uncore must be 2x dram speed or higher).

Try higher VTT and Vdimm for higher uncore.

Edit: Ah, you mean the board only shows 9x for a real uncore mult of 18x? Weird.. but then again, it's MSI, so anything is possible
See if it has any "preview" option for uncore speed, 16x is 2960, 18x should be 3330 at your BCLK of 185