Multi thread GPU

[Bun-Bun]

Ok I grabbed the app_info for ATI from one of the other threads going around but...

How do you guys keep them going? Everytime I walk back to my computer with two 7850's (running two threads per card so four total) one card is hung up and I have to restart BOINC to get it going again. I figure its hanging up when windows does its stupid timeout issue (display stop responding blah blah blah BS). I am thinking that putting in a cheapo video card for display duties would solve that. But anyway.

Also after it gets through all its WU's it refuses to download anymore. I have to clear app_info for it to grab a new set of WU's which it also grabs CPU WU's as well then. I have tried reseting the project but it does the same thing.

[ggdh]

I can't help you with your problem, but i must ask how you run 2 threads per card? Is it the way how all should work or you have to do some magic and set it up this way? Yesterday I tried my GTX580 as a warm up before SCW and was doing 1WU every ~2min20sec. Is it right?

Sorry again for bombing your thread

[Rob_B]

As I have posted in the other thread, it doesn't look like it is a Windows or driver problem. I have taken out my 7770 that was keeping getting stuck on WU's and put in my 5850 without out changing anything else. And that card is running 2 WU's without problems. The 7770 doesn't want to run 2 WU's for more than one hour in the other cruncher either.
When I have some time I may try underclocking and overclocking the 7770 and run older drivers to see if that help.

I can't help you with your problem, but i must ask how you run 2 threads per card? Is it the way how all should work or you have to do some magic and set it up this way? Yesterday I tried my GTX580 as a warm up before SCW and was doing 1WU every ~2min20sec. Is it right?

Sorry again for bombing your thread

Not sure about nVidia cards, I don't have any. But 2:20 sounds right.

All info you need to run 2 WU's on an nVidia card is on this page in the HCC GPU Launched thread: http://www.xtremesystems.org/forums/...aunched./page3

[Mydog]

To me it seems like the cards acctually likes a more even load. While running 4 WU's on my 7970 three of them often got stuck/hung but after switching to 6 I've not seen that at all. I'm only running GPU WU's so if you guys are on a combo of CPU/GPU I'm not sure what Your problem is but I would recomend trying GPU only for a while.

Bunbun you need to set Windows power setting to "high performance"

[BlindFreddie]

... it refuses to download anymore ...

You may be bumping into WCG's newly-imposed Limits on Jobs In Progress (link is to a post over at WCG by knreed).

It seems that StoneAgeMan and a few others have managed to bust the fastest servers that Big Blue can muster for WCG's use.

[nanoprobe]

It has been my experience that when you find stuck WUs that card has recovered from a driver crash. The driver restarted but the WUs didn't.

[Snow Crash]

IF ... anyone wants to turn off the Windows gpu watchdog to make sure it is not windows resetting the driver ...

turn off bugcheck
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\DisableBugCheck
=1

you have to add EaRecovery key and then set the value to stop that nonsense
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
=0

[OldChap]

I switched to running 2 per last night then checked on it a couple of hours later and all OK. 12 hours later I see what I was not expecting ... that the wu's still seem to be in sync on each card that is to say both starting and stopping at the same time so I paused one on each to put a 1 minute offset between them in the hope that they get the whole GPU to work for themselves when the other is running CPU.

Those of you that ran multi per card can you tell me what you guys did about this?

[Rob_B]

I tried to get them to start out of sync a few time when I ran that on my 7770. And if it ran long enough without getting stuck they would end up running in sync again.

My 5850 is still running 2 WU's after 20 hours and then start within 3 sec of each other.

[Rob_B]

IF ... anyone wants to turn off the Windows gpu watchdog to make sure it is not windows resetting the driver ...

turn off bugcheck
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\DisableBugCheck
=1

you have to add EaRecovery key and then set the value to stop that nonsense
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
=0

Are these existing keys that I need to edit or do I add these keys myself. I just looked on my cruncher and can't find any of those in the register. This is on Windows 7.

[Snow Crash]

I am also on Win7 x64 and yes, if they do not already exist you will need to add them.
I would like to be clear, I don;t know this has anything to do with the 7770 locking up but the keys above are related to windows watching and interuppting GPU processes if Windows does not think they are not responding quickly enough. My thought is that if we see a "lag" what does Windows think of that? I also do not know how long is too long before Windows decides to intervene but I have been running with these keys since they we first posted many many months ago without issue in daily driving and crunching 24x7 GPUGrid.

[0ne.shot]

I can't help you with your problem, but i must ask how you run 2 threads per card? Is it the way how all should work or you have to do some magic and set it up this way? Yesterday I tried my GTX580 as a warm up before SCW and was doing 1WU every ~2min20sec. Is it right?

Sorry again for bombing your thread

That's about the same time I'm getting with my GTX 570 @ 800 MHz core. You have to create an app_info.xml text with notepad and place it in the programdata/boinc/projects/worldcommunitygrid.com folder. I wasn't getting the best results so I'm just running on default without app_info and CPU set to 25% to run 1 GPU WU and 1 CPU WU. My cache is full of "nvidia_hcc1" WUs, but only one at a time. I hope this helps.

[Bun-Bun]

To me it seems like the cards acctually likes a more even load. While running 4 WU's on my 7970 three of them often got stuck/hung but after switching to 6 I've not seen that at all. I'm only running GPU WU's so if you guys are on a combo of CPU/GPU I'm not sure what Your problem is but I would recomend trying GPU only for a while.

Bunbun you need to set Windows power setting to "high performance"

I am running GPU only. Without app_info I haven't been having any issues.

IF ... anyone wants to turn off the Windows gpu watchdog to make sure it is not windows resetting the driver ...

turn off bugcheck
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\DisableBugCheck
=1

you have to add EaRecovery key and then set the value to stop that nonsense
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Watchdog\Display\EaRecovery
=0

Holy crap you are amazing. This is a problem many people have with windows 7 and both Nvidia and Ati cards.

What happens is the new video stack in Windows Vista/Seven runs seperate from the kernel so that video driver restarts are possible without a ful reboot. Thus preventing many video related BSOD's. However as a part of this if the driver doesn't respond to windows fast enough windows automatically restarts the driver and throws and error when actually nothing could be wrong. It is really annoying on some systems. Some systems have higher internal latency (can't remember what that is called right now) that cause this to trip.

So with loading up the GPU's so much I am sure it is causing the driver to delay in responding to windows causing it to trip. So if these registry keys actually disable that stupid timeout driver restart then you are a god send



EDIT:

Copy and paste the following into notepad and save it as a .reg file and run that to add the following to the registry.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog]
"DisableBugCheck"="1"

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Watchdog\Display]
"EaRecovery"="0"

Now is there a registry key to disable the check that makes windows "detect" poor video performance and disable aero? Happens at work all the time and I haven't found a workaround for it. Computer is running fine but windows randomly decides it needs to disable aero.

EDIT2:

Two 7850's running 2 threads at once clocked 900MHz core on a Q9650 OC'ed to 3.825Ghz in a maximus ii formula complte WU's in about 2min 50s with 50-80% CPU utilization.

And ATI seems to handle regular use while gruncher better than Nvidia. My nvidia systems are unusable while crunching where this system I can browse forums and do whatever while crunching.

EDIT3:

Found the microsoft article

http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx

Even says in there the code is complex and could cause compatibility issues with drivers. I am surprised this isn't a more widely known fix to the 1000's of people having issues with this.

[bluestang]

Wonder if this reg fix will help the BSODs I get caused by "atikmdag.sys" ?

Running 12.9 beta, may go back to 12.4 as I have read that is the most stable.

[Bun-Bun]

It's worth a shot. The driver timeout in windows does cause bsod's.

[Rob_B]

At the moment I'm running 2 WU's on my 7770 with the registry changes and it is running for a few hours now without problems.

[Plan.B]

I can't help you with your problem, but i must ask how you run 2 threads per card? Is it the way how all should work or you have to do some magic and set it up this way? Yesterday I tried my GTX580 as a warm up before SCW and was doing 1WU every ~2min20sec. Is it right?

Yes, about 2:20/wu for the gtx580 is what mine are doing.

[Bun-Bun]

Well no more errors but I come home to an empty queue. It still wont get new tasks with my app_info in place.

10/20/2012 4:15:46 PM | World Community Grid | update requested by user
10/20/2012 4:15:50 PM | World Community Grid | Sending scheduler request: Requested by user.
10/20/2012 4:15:50 PM | World Community Grid | Requesting new tasks for ATI
10/20/2012 4:15:52 PM | World Community Grid | Scheduler request completed: got 5 new tasks
10/20/2012 4:15:52 PM | World Community Grid | Resent lost task DSFL_00020-41_0000049_0065_1
10/20/2012 4:15:52 PM | World Community Grid | Resent lost task GFAM_xMRC_CMxtl257_PfSUB1_0044925_0095_1
10/20/2012 4:15:52 PM | World Community Grid | Resent lost task DSFL_00020-40_0000033_0399_1
10/20/2012 4:15:52 PM | World Community Grid | Resent lost task DSFL_00020-31_0000014_0340_1
10/20/2012 4:15:52 PM | World Community Grid | Resent lost task SN2S_AAF21681_0000081_0575_3
10/20/2012 4:15:52 PM | World Community Grid | [error] App version returned from anonymous platform project; ignoring
10/20/2012 4:15:52 PM | World Community Grid | [error] App version returned from anonymous platform project; ignoring
10/20/2012 4:15:52 PM | World Community Grid | [error] App version returned from anonymous platform project; ignoring
10/20/2012 4:15:52 PM | World Community Grid | [error] No app version found for app dsfl platform windows_intelx86 ver 625 class ; discarding DSFL_00020-41_0000049_0065_1
10/20/2012 4:15:52 PM | World Community Grid | [error] No app version found for app gfam platform windows_intelx86 ver 612 class ; discarding GFAM_xMRC_CMxtl257_PfSUB1_0044925_0095_1
10/20/2012 4:15:52 PM | World Community Grid | [error] No app version found for app dsfl platform windows_intelx86 ver 625 class ; discarding DSFL_00020-40_0000033_0399_1
10/20/2012 4:15:52 PM | World Community Grid | [error] No app version found for app dsfl platform windows_intelx86 ver 625 class ; discarding DSFL_00020-31_0000014_0340_1
10/20/2012 4:15:52 PM | World Community Grid | [error] No app version found for app sn2s platform windows_intelx86 ver 620 class ; discarding SN2S_AAF21681_0000081_0575_3

This is my app_info

<app_info>
<app>
<name>hcc1</name>
<user_friendly_name>Help Conquer Cancer</user_friendly_name>
</app>
<file_info>
<name>wcg_hcc1_img_6.56_windows_intelx86__ati_hcc1</name>
<executable/>
</file_info>
<file_info>
<name>hcckernel.cl.6.56</name>
<executable/>
</file_info>
<app_version>
<app_name>hcc1</app_name>
<version_num>656</version_num>
<platform>windows_intelx86</platform>
<plan_class>ati_hcc1</plan_class>
<avg_ncpus>1.0</avg_ncpus>
<max_ncpus>1.0</max_ncpus>
<coproc>
<type>ATI</type>
<count>.5</count>
</coproc>
<file_ref>
<file_name>wcg_hcc1_img_6.56_windows_intelx86__ati_hcc1</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>hcckernel.cl.6.56</file_name>
<open_name>hcckernel.cl</open_name>
</file_ref>
</app_version>
</app_info>

delete app_info and it downloads WU no problem

[Snow Crash]

Let's try to clean up before using app_info.
Stop BOINC
Remove app_info
Restart BOINC
After WCG finishes downloading the "lost tasks" set WCG to No New Tasks
Abort all tasks
Stop BOINC
Add app_info back in
Restart BOINC and allow WCG to get new tasks

For the moment I would suggest setting your Min Buffer to .35 days and additional buffer to .01
Press update and take a look at the event log.
Is BOINC requesting ATI tasks?
Is it getting any?
If not please post the messages from the point you click Update until you see it did not get any.

[Bun-Bun]

I have taken out app_info. let it download everything. Aborted everything, I have reset the project even.

With app_info in it tried to download for ATI and gets those 5 lost tasks still. As long as app_info isnt there it downloads and process's fine.

[bluestang]

Just a thought...is that ZeroCore crap on these 7000 series cards causing any of these lockup/BSOD/crashing/etc issues?

[Bun-Bun]

My cruncher just worked through its worklog again. Sitting empty. Force update gives me the exact same results as above.

Remove app_info and it starts working again.

[=[PULSAR]=]

I'm having the same issue, I wonder what it is...

[pirogue]

Have you tried just running 1 WU at a time with the app_info?
<count>1</count>

Running it that way should at least point to whether or not the app_info is causing the problem.

[werdwerdus]

Have you tried just running 1 WU at a time with the app_info?
<count>1</count>

Running it that way should at least point to whether or not the app_info is causing the problem.

excellent suggestion, +rep