POEM: GPU Project Status

[RSS]

As many of you have noticed, our GPU project has been re-enabled after the update last week.With the apparent error of our last release being fixed, I'm still not satisfied with the failing rate of GPU tasks, especially on NVIDIA cards.I think there are currently three main issues:1. Incompatibility to old AMD GPUs.We're suspecting our latest version of poemcl won't run properly on cards of the Radeon 4000 series, which only have a limited OpenCL support.If you are using such a GPU, and it works however, please let me know which OS and driver version you are using. If there is no working setup, we will exclude the Radeon 4000 series in future releases.2. Multi-GPU NVIDIA hosts still fail when using more than one graphics card for poemcl tasks.If you have attached a host with multiple NVIDIA cards to POEM@HOME, please configure your client to use only one of them for our project, see http://boinc.berkeley.edu/wiki/client_configuration headword exclude_gpu.This is a well known problem which should have been fixed with the last release, but obviously it is a sticky one :(3. There is a new error, which may be related to incompatible OpenCL library versions.If your host uses only a single GPU, but still fails with poemcl tasks, please download one of the following files (depending on your OS):boinc.fzk.de/userdocs/temp/pcl_201_test_linux.tar.gzboinc.fzk.de/userdocs/temp/pcl_201_test_win.zipExtract the files to a new folder, run the executable (if it works, this may take some minutes), and post the content of stderr.txt in this thread.Thanks for your help!I hope we can improve the application soon with your feedback.

More... (http://boinc.fzk.de/poem/forum_thread.php?id=1069)

[zombie67]

Yes!! Back to work on my 250m MM.

[FourOh]

Let's hear some runtimes/production on the new Poem OpenCL app!
Right now I only have it running on one GPU, but I'll be adding others soon:
HD 6950 Running 2wu/gpu with .5cpu/wu Runtime +/- 29,000 for 39,000 credits/day
HD 7750 Running 2wu/gpu with .5cpu/wu Runtime +/- 21,000 for 53,500 credits/day
HD 7790 Running 2wu/gpu with .5cpu/wu Runtime +/- 10,800 for 104,000 credits/day
HD 7950 Running 2wu/gpu with .5cpu/wu Runtime +/- 10,100 for 111,000 credits/day (underclocked to 825Mhz for heat control)
HD 7950 Running 3wu/gpu with 1 cpu/wu Runtime +/- 12,280 for 137,000 credits/day (OC'd to 1000Mhz)
GTX 580 Running 2wu/gpu with 1 cpu/wu Runtime +/- 12,900 for 87,000 credits/day
GTX 750Ti Running 2wu/gpu with 1 cpu/wu Runtime +/- 14,900 for 75,000 credits/day

Update: Comparing some initial results, mid-range 7xxx series cards appear to have a distinct efficiency advantage over the higher end cards for the new app. The 7750 & 7790 are getting roughly double the credits/watt than a 7970.

Update 6/29: Wow, very disappointed with the performance on HD 6950 - I figured there would be a hit due to the slightly older architecture but 27% slower than 7750 and half the production of the 750 Ti? That's bad!

Update 8/11: Got my 7950 running in a cool environment and overclocked comfortably at 1000Mhz, resulting in a 20% production gain.

[zombie67]

I am running .5 GPU and .5 CPU with my 7970s. 7800-8400 seconds per task.

[Al]

I don't know if it's changed, but Poem use to be a cpu project disguised as a gpu project. In the past it was better to reserve a full core per wu on the fastest processors you had available. The gpu also benefited by being in a x16 slot if at all possible because of the amount of data transfer.

[zombie67]

I think it has changed. Now it is more like a normal GPU project, as far as I can tell. At least on AMD GPUs.

[Al]

Cool, that will be a nice change. Too bad they didn't fix the dual nv problem while they were at it.

[Ron Shurtz]

Let's hear some runtimes/production on the new Poem OpenCL app!
Right now I only have it running on one GPU, but I'll be adding others soon:
HD 7750 Running 2wu/gpu with .5cpu/wu Runtime +/- 21,000 for 53,500 credits/day

I ran it on a GTX570.
2WUs/gpu with .5 cpu
Runtime about: 16000 secs/WU
Credits/day: 70,200

[FourOh]

Beware of Poem GPU heat! I just logged in remotely to check on my 7790 and it was running (stock clocks) at 99C!

[Al]

Well, that's actually a sign that it is using the gpu, never had heat issues previously.

[Al]

I'm trying .25 cpu and .25 gpu on my 7970s. Run times are 14,500 seconds for 4 wus at a time. Aprox 155k/day based on these run times.

[Shadowlurker]

Is anyone getting work? I can't tell if they just don't have any for AMD or if I need something I don't have in order to get them

[FourOh]

Is anyone getting work? I can't tell if they just don't have any for AMD or if I need something I don't have in order to get them

I've been using BOINC Updater set to 1 minute intervals to get a cache of work. Once I have 8 or 10 downloaded I set it to 15 minutes.

http://www.setiusa.us/showthread.php?3067-BOINC-Project-Updater

[Bryan]

I posted in the wrong thread ... Standings.

Do you guys drop the memory speed or leave it alone?

[Al]

I left mine stock, but I don't know if it helps or not.

[E-30]

I will post my 650ti times when I get on to validate

[E-30]

650ti oc from 1033 to 1161 gpu clock and ram 1350 to 1362 ran one 13975sec and about 13019sec on the sec one.
640gt stock 19600sec

[E-30]

i need some one to send me the fire to set up two to ron on my 650ti i am only seeing about 65% used

[EmSti]

Cool, that will be a nice change. Too bad they didn't fix the dual nv problem while they were at it.

I bet my work around for the old version works on the new version, I will need to try some day. The original reason I figured out how to get multiple clients to run on the same machine was to ignore all but one NVidia gpu and run Poem on it. One client for each gpu, each only able to see one NVidia.

I know it getting to be an old on-going theme of mine, but I tell you the multiple client thing solves interesting problems.

[EmSti]

Notes on the new GPU app so far:
Project updater needed to get wus flowing
Only running on the R9 295x2, increased from 1 wu per r9 290x gpu to 2 wus per gpu and it took no more time. Double the credit same amount of time. Keeping about 4 cpu threads free for the 4 wus. Completing 4 wus every 2.25 hours (11,555 credits an hour, on collatz the card produces ~130,400/hr).
Usage is still less than 100% on the GPUs, I haven't tried 3 or more wus.
Power usage and heat are a lot less than collatz (not hard to do).

3 wus per GPU with 4 CPU cores free: 3 hours and 2 to 3 minutes, so about 13,000 credits/hour on the 2 gpus. Utilization of the GPUs is closer to 100%. Tomorrow I will try 4 wus on the dual 290x. Then I will try 7970s or test getting 2 NVidia cards in the same machine to run.

[zombie67]

Great info. I really appreciate the work you're doing.

[Bryan]

Here are some times for 3 versus 4 WU

7970
11500 3WU 146.5k/day
15500 4WU 144.9k/day

GTX 780
11000 3WU 153.1k/day
14500 4WU 154.9k/day

There is virtually no difference between 3 and 4 at a time except WuProp goes up faster with 4 - the reason I changed :D I'm running the clocks at 1050.

I'm reserving 4 threads in all cases. The 7970 could be cut back to 1 or 2 and shouldn't have any effect ... VERY little CPU usage. The GTX 780 however IS using a full thread per WU.

[Slicker]

The GTX 780 however IS using a full thread per WU.

and nVidia doesn't care.


nVidia has decided that since there is nothing in the OpenCL specs about whether kernels need to truly run asynchronously, unless there are more OpenCL command queues instantiated than there are CPU cores, it will run the kernel in synchronous mode which means the CPU will wait for the kernel to finish. That uses 100% of the CPU even though it really isn't doing anything but waiting. While many have complained about it, the excuse thus far has been.... OK. There hasn't been any and that's pissing many people off. Of course, nVidia would rather everyone use CUDA since that is proprietary and requires nVidia hardware. The CUDA apps use asynchronous kernels so my guess is that it is a marketing ploy. First they push OpenCL to get AMD to abandon CAL and Brook+ and then they change the drivers to encourage people to use CUDA. AMD's OpenCL compiler still has lots of bugs so nVidia doesn't feel threatened there. That, and AMD seems only interested in gamers now.

[Fire$torm]

This is why I have never liked nVidia. As a company, they are and always have been..... Assholes.

[Christopher Herr]

I can only second Slicker's and F$'s rants about nVidia; their cards/chips are much more expensive than AMD's and at the same time a lot slower when it comes to brute force TFLOPS. Add to that their crappy policy with OpenCL: a standard they actively took part in its design in the Khronos Group IIRC, in contrast to AMD, and then crippled on their own hardware due to crappy marketing reasons.
Well, I guess they argue someone has to keep buying their totally overpriced Quadro and Tesla product lines running CUDA... :mad:

And they still claim to be the world market leader in visual computing professional equipment, somehow I find that hard to believe... But then again it could be the traditionalists in businesses and research using GPU computing equipment for construction/design never buy anything else than NVidia equipment. :confused: :eek:

Sent from my personal Android slave using Tapatalk Pro :P

[E-30]

Well the poem gpu app is running on my phone 640 and 650ti with no problems

[cineon_lut]

I bet my work around for the old version works on the new version, I will need to try some day. The original reason I figured out how to get multiple clients to run on the same machine was to ignore all but one NVidia gpu and run Poem on it. One client for each gpu, each only able to see one NVidia.

I know it getting to be an old on-going theme of mine, but I tell you the multiple client thing solves interesting problems.

My 7970 is looking for work. I couldn't find the app_info.xml anywhere in the searches. Can someone post theirs?

Thanks!

[EmSti]

App_config.xml is what is needed. I believe I have one posted in the app_config.xml section, sorry not connected to home machine or I would post it here.

[cineon_lut]

App_config.xml is what is needed. I believe I have one posted in the app_config.xml section, sorry not connected to home machine or I would post it here.

<dumb look on face> There's an app_config.xml section? </dumb>

EDIT: Thanks. All the searches I did, even advanced searches looking in posts and everything didn't show that thread. :/

[EmSti]

Sounds like you found it, for others http://http://www.setiusa.us/forumdisplay.php?167-Project-App_Config-Files (http://www.setiusa.us/forumdisplay.php?167-Project-App_Config-Files)

[EmSti]

now that I am back to the hotel, I have access. Here is my current app_config.xml for poem
<app_config>
<app>
<name>poemcl</name>
<gpu_versions>
<gpu_usage>0.33</gpu_usage>
<cpu_usage>0.33</cpu_usage>
</gpu_versions>
</app>
<app>
<name>poempp</name>
<max_concurrent>4</max_concurrent>
</app>
</app_config>

[Fire$torm]

EmSti, That's for your AMD cards or Cuda cards?

[EmSti]

AMD, the CPU part may need to be tweaked, the value doesn't matter one that machine. The gpu work is being done in its own client and only BU is running on the CPU, so I am not sure what CPU % is needed.