Optimizing # ATI GPUs Crunching for Best Production Efficiency

[haschioz]

depends on your cpu oc. i rund 0,5cpu(4200mhz) threads on a 7950(1000mhz) and average ~800k per day. so i dont know. on the same clocks your 7970 should do 930k. cpu time would need to be more than twice as high to make it worse (mine is still not maxed out, nor is my 7950. but the lowest utilisation is higher). there is no way you calculate your ppd when you change threadcounts (cpu bottlenecks or doesnt but average gpu utilisation will be higher. it just depends on what's prevalent) i think that there is something to oldchaps idea but i am no microelectronics expert (1 compute unit, 4 texture units and 64 stream processors form one "calculation unit")

[Snow Crash]

I experimented with HT on and HT off and found that if the number of GPU threads does not exceed 2x the number of cores I get better run times with HT off. Usually I don't have all my GPU threads synced so there are not more requests for CPU activity than I have cores. HT off always processes faster, allows me to OC higher and temps stay lower. I've done this with a 7858 and also with multiple NVIDIA cards I the same rig.

SR Hunter - you may be the exception in that you are trying to run more total GPU threads faster than anyone else and hence saturating your CPU. You might actually be better with HT than not

[sRHunt3r]

Thanks for the input guys, I changed it to 6 per gpu and .66 on CPU, trucking along fine at just over 2 mill ppd and CPU is down to 80% usage. I was tinkering around today and thought I had bricked my 7970 flashing the bios but I was able to recover it and she's running fine at 1200/977 .... I gotta admit a was quite pissed at myself for awhile there!

[jeanguy2]

Thanks for the input guys, I changed it to 6 per gpu and .66 on CPU, trucking along fine at just over 2 mill ppd and CPU is down to 80% usage. I was tinkering around today and thought I had bricked my 7970 flashing the bios but I was able to recover it and she's running fine at 1200/977 .... I gotta admit a was quite pissed at myself for awhile there!

SR, 977 seems like an odd number for your video memory speed... How did you end up selecting it? What GPU voltage are you using to run 1200/977 ? What is your GPU temp at load and with what fan speed?

Thanks,
Jean-Guy

[Johnmark]

sRHunt3r, glad to hear you didn't brick the 7970! Been there done that, know how u feel
When you say "1200/977" are you referring to CPU@1200 MHz and mem.@ 977 MHz ? If so, that seems to be a very low speed for the 7970. Would you be so kind as to share your GPU voltages ?

[sRHunt3r]

My xfx7970 has the second version pcb without the dual bios and with a hard lock on voltages, I have not found a way around it yet. So those speeds are the min and max for the stock voltages, as far as the mem clock 977 is the lowest spot I can go to without them going back to default

Sorry the fan is set on variable but my gpu stays around 50-55c fully loaded.

[haschioz]

2mill ppd? are you sure, if so i would really regret not getting an 7970.

[sRHunt3r]

No the triple 7950 rig gets over 2 mill not the 7970 rig, sorry for the confusion.

[bluestang]

you should be the closest with 950k. it should actually be doable with about 1200 on the core.

I feel a little puzzled. If you are able to get nearly a million with an OC 7950, you are at not far from my level (a little over 1Mppd) but I run a 7970. I should be able to pull more from my 7970. I have to check again about my overclockings.

Gettin' close, but most likely not...



EDIT: Wish I had a 3930k setup!!!

[jeanguy2]

Great work Bluestang....!!!

You are already matching the PPD production of non OC basic 7970. Pump up your CPU to 4.8 GHz & you'll break the million PPD.

[EtaCarinae]

Great work Bluestang....!!!

You are already matching the PPD production of non OC basic 7970. Pump up your CPU to 4.8 GHz & you'll break the million PPD.

Guys all this is fine but take care with all the Overclockings, I already destroyed once a video card. So now I am a little conservative. Maybe a little less ppd but for longer time.

[jeanguy2]

Guys all this is fine but take care with all the Overclockings, I already destroyed once a video card. So now I am a little conservative. Maybe a little less ppd but for longer time.

Eta, I am just curious....

What video card and how exactly did you destroyed it by overclocking. Overclocking does indeed present an inherent added risk factor. However, it is nomally OK if temps and voltages are respected and overclock is raise in small steps.

Thanks,
Jean-Guy

[Johnmark]

Gettin' close, but most likely not...



EDIT: Wish I had a 3930k setup!!!

Bluestand I wish I knew what you where doing to get such great numbers!
The only real difference I see in our setups is that I'm running Asus cards, the 7970 is a Matrix ROG so it's OCed out of the box and running at 1230 GPU/6600 Mem, CPU@4.6GHz. No points going up in smoke due to invalids, etc.
The 7950 is running at 1150/6000, CPU @ 4.6GHz, each machine is now running 10 threads of GPU only. Had a few hiccups the first couple days with setting things up etc. however the last two day's have run rock solid.

Here are my results
7950


7970


What the heck am I doing wrong? The rest of you getting the same results as bluestang?

[OldChap]

The cards tell only half the story .... What rigs are they running on? If these are on one rig then you have your answer right there

[bluestang]

Johnmark,

My setup is Cruncher #1 in my sig and I'm currently running 12 WU's.

I did fall the past couple days in points...

12/10/2012 --- 0:011:22:11:45 --- 930,893 --- 2,104
12/09/2012 --- 0:011:20:29:00 --- 938,541 --- 2,105

...and today isn't looking any better (only 419,761 with 961 WU's as of this mornings WCG stats) as I've been ripping some blu-ray discs over the past couple days. I'm going to give it a couple more days to settle in with the 12 WU's. I may then go back to 10 WU's and see what the difference brings again.

EDIT: Also, I'm on BOINC 7.0.38, my BOINC data directory is on my Samsung F3 1TB RAID0 disks, and my pagefile is on my Crucial M4 64GB RAID0 disks...if that helps at all.

[jeanguy2]

Johnmark,

I can offer some comments on your 7970 performance. I have a 7970 that operates on a system similar to yours (details in sig below). This is my worrking system so it is not fully dedicated to crunching (estimate ~ 20 to 21 hours crunching. We can however do some results versus point comparaison:

You: 966,170 / 2317=415.19
Me : 913,642 / 2071 =440.15

Suggest you check your points claimed versus given. Your points per results are lower than mine. My claimed points are 55.6/65.2 (low/high). Given points are 59.9/63.2 (low/high). If your results do not match mine, suggest you used SnowCrash added count label to your app_info file and then fine tune it to get your estimated completion time about 20-30 seconds above your highest actual completion time on that rig. I average a 6:44 actual WU completion with a 7:23 estimate set. If this works you should get an extra ~ 80K PPD.

I also feel that your daily return may be low but cannot confirm since I do not run my 7970 full time crunching. Double check your BOINC log to insure that you do not have a scheduled OS task or some application that takes over the CPU and/or video card while it run ( auto backup, anti-virus update& the likes).

Good luck,
Jean-Guy

[Conquistador SW]

Just to compare,

I am getting ~880.000 ppd with a HD7970 at 1025MHz running 8 threads on a 3770K 4.7GHz. Trying 12 threads now and according to calculations I will get above 920.000 ppd and possibly more, but I have to wait a couple of days to get a more accurate prediction.

So I think my 7970 does a little bit worse than bluestang's 7950, but a lot better than the Johnmarks 7970 which is clocked much higher. Don't know why . I did nothing special on my rig. Just the latest BOINC manager and beta ATI drivers and the standard app_info, that's all.

EDIT: looking more closely at Johnmarks 7970, the last couple of days the runtime is decreasing but the ppd is increasing. Maybe you have some older WU validate from the period you had problems and maybe it is still ramping up?

[jeanguy2]

Johnmark,

Follow this link to post # 4454 by Eta who runs a few 7970s. http://www.xtremesystems.org/forums/...Thread/page179

It contains pertinent data regarding the 7970 productivity. Your top of the line ASUS 7970 seems to match the returns for his top factory OC cards. Your PPD is somewhat lower. As conquistador says let it run for a day or two to insure your are getting the results for the actual card settings. If your number are still low after that and if temps are OK and there is room for a little extra GPU voltage I would be tempted to add a little to see if it helps.

[Johnmark]

Thank You for all your replies!!

OC, both these machines are dedicated headless 2600k's with 8 gigs, of RAM,high speed HD's(just cause) on Asus boards. ALL they do is crunch. Of all 8 socket 1155's only the 980x is a shared machine. The rest reside in various parts of the house keeping things warm and cleaning the air while they are at it. They make great air purifier/heaters lol

Bluestang thanks for the added info. The 7970 runs a SSD drive and the 7950 a WD Black. I see the 7970 is at 524,884/1,217 so far today so maybe it's just taking some time to get the wu's thing straightened out, I've been at 10 on both machines for two days now.

jeanguy that is fairly normal to get more point for slower results, don't understand why that is just the way it is. And honestly I care less about points, only want to optimize the actual WORK done for WCG. I'll have a look at the link you posted as i don't recall reading it. I wonder what actually constitutes work being done, as per WCG?

Conquistador, I look forward to hearing how increasing your WU's affects your production. There where about 135 pages of pending units to valid so that's probably what you noted, and my run time is down because I reduced the number of WU's to 10. Tried 16,14,12 and now at 10, I've read the more wu's the more run time, but not necessarily more points.

I'm beginning to think the only real issue is the wu bunching up causing the CPU usuage to go to 100% and the GPU to drop for short periods. I look at the 7770 card which is running on a 2600k as well and it produces pretty much the same results every day, but the CPU is never 100% and the GPU is alway's 100%.

[OldChap]

Just to compare,

I am getting ~880.000 ppd with a HD7970 at 1025MHz running 8 threads on a 3770K 4.7GHz. Trying 12 threads now and according to calculations I will get above 920.000 ppd and possibly more, but I have to wait a couple of days to get a more accurate prediction.

So I think my 7970 does a little bit worse than bluestang's 7950, but a lot better than the Johnmarks 7970 which is clocked much higher. Don't know why . I did nothing special on my rig. Just the latest BOINC manager and beta ATI drivers and the standard app_info, that's all.

EDIT: looking more closely at Johnmarks 7970, the last couple of days the runtime is decreasing but the ppd is increasing. Maybe you have some older WU validate from the period you had problems and maybe it is still ramping up?

With this you definitely have to let it run. I would say 4 or 5 days at each different setting.

Remember this is using all of your hardware, albeit at different times, too. A Great GPU clock is just the start. A Great CPU clock is a good thing sure but after reading that this is dependent on system ram I have a feeling that based on the numbers I am reading that good ram bandwidth makes a difference.

It is not easy but you need also to get your separation managed. Whilst this does not need to be 100% perfect re-alligning every 2-3 days will do some good, at least it will once you get above 14 instances running

I will edit this post in a short while (after the update) with my rig statistics then I think you will see what I mean about my 4-5 days comment. I am trialing adding more instances having, I think, seen that 14 is worse than 16.

My rig is MVG + 3770K@4.8 Samsung green at 9,10,10, 1t @2000. Card is ASUS HD7950-DC2T-3GD5 = Stock 900/1250 runs 1100/1250

[Johnmark]

A Great GPU clock is just the start. A Great CPU clock is a good thing sure but after reading that this is dependent on system ram I have a feeling that based on the numbers I am reading that good ram bandwidth makes a difference.

OC, what are you running for voltage on your 7950?

Hmm, I seem to recall being told RAM speed didn't make much difference with the CPU only crunching, could this have changed? This has also been stated about the GPU RAM as well, so I've never tried to OC the RAM on any of my crunchers.

To be honest I never really understood how CPU RAM speed could not make a difference as all the work done at the CPU level is done via CPU/RAM ticks on the stack ??

[nanoprobe]

FWIW here are some of my 7970 results for comparison. Settings on card are core @ 1100Mhz. memory @ 700MHz. Running 10 tasks on i7-2600k @ 4.2Ghz. 8GB. 1600MHz. memory.

12/11/2012 0:010:02:23:45 957,362 2,159
12/10/2012 0:009:17:33:39 944,168 2,142
12/09/2012 0:009:18:16:59 933,830 2,111
12/08/2012 0:010:04:03:38 977,678 2,219
12/07/2012 0:009:19:54:05 950,571 2,179
12/06/2012 0:010:04:43:13 993,106 2,252
12/05/2012 0:009:22:07:14 966,077 2,211
12/04/2012 0:010:01:31:18 968,644 2,220
12/03/2012 0:009:20:31:34 942,281 2,158
12/02/2012 0:009:16:17:13 920,671 2,119
12/01/2012 0:010:09:48:52 1,006,090 2,328
11/30/2012 0:010:02:03:20 990,502 2,326
11/29/2012 0:009:15:11:33 947,249 2,233

Average-> 961,402 2,204

Wish it would show spacing when I copy and paste from WCG.

[Johnmark]

Dang, another machine kicking my butt

You say your running memory @700MHZ, so on my Asus 7950 which has a stock mem clock of 5000 MHz ( 1250 MHz GDDR5 ) that I should use Asus Tweak and crank her as low as she will go ?

[OldChap]

nano: Yeah I have asked a number of times for [tables] to be available on this forum.

~8.5% increase compared to mine. I would expect ~14% based on shaders alone so one would have to put the difference down to cpu and mem clocks

Taking into account shaders and cpu clocks together in a rather simplistic way one might expect something in the order of 102% better for the 7970 but we know that it is not that easy. my gut feeling is that your numbers are about right for your settings nano

I will test memory some more soon.

Johnmark: I don't remember volts I will check tomorrow....bedtime here now.

[Johnmark]

Sweet Dreams OC !