DistrRTgen app_info file

[Maxwell]

I recently put in a new CUDA card, and put it on DistrRTgen to get to my 1M there. As I was futzing with it, I noticed how horrifically inefficient the app is. So, I spent a bit of time getting an app_info file running for the project - it is running the stock app, but has set the CPU priority to high and runs 2 WUs at once.

This is working on Win7x64, 6.10.58. Here it is, if anyone is interested:


<app_info>
<app>
<name>distrrtgen</name>
</app>
<file_info>
<name>distrrtgen_3.20_windows_intelx86__cuda23.exe</name>
<executable/>
</file_info>
<file_info>
<name>cudart.dll</name>
<executable/>
</file_info>
<file_info>
<name>charset.txt</name>
</file_info>

<app_version>
<app_name>distrrtgen</app_name>
<version_num>320</version_num>
<plan_class>cuda</plan_class>
<avg_ncpus>0.75</avg_ncpus>
<max_ncpus>1</max_ncpus>
<flops>1.0e11</flops>
<coproc>
<type>CUDA</type>
<count>0.5</count>
</coproc>
<cmdline>p3 b-1</cmdline>
<file_ref>
<file_name>distrrtgen_3.20_windows_intelx86__cuda23.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>cudart.dll</file_name>
<copy_file/>
</file_ref>
<file_ref>
<file_name>charset.txt</file_name>
<copy_file/>
</file_ref>
</app_version>
</app_info>

[zombie67]

How much does it improve performance?

[Maxwell]

How much does it improve performance?
I didn't get much of a baseline in terms of credits (pending, impatience, etc). But I'm watching the GPU load. Running one at a time, the typical load was about 50% - now it's >95%. Completion times have not noticeably increased, either. I'll post my credit total at the end of the day.

[zombie67]

I only have a handful of tasks validated so far, but this is what I found:

GTX 580:
o Normal: Run time averaged 70 seconds and pay averaged 51. That works out to about 63,330/day.
o With app_info: Run time averaged 133 seconds and pay averaged 61. That works out to about 78,865/day.

GTX 295:
o Normal: Run time averaged 352 seconds and pay averaged 1231. That works out to about 30,234/day.
o With app_info: Run time averaged 524 seconds and pay averaged 61. That works out to about 40.057/day.

The odd bit is around the credits awarded. Normally, they vary between 48 to 54 per task on the 580. But with the app_info, they are all awarded the exact same 60.74 credits. Similar situation with the 295, all exactly 60.74.

[Maxwell]

That's really interesting - I am not seeing the same behavior. I'm seeing huge variation in the number of credits awarded, from ~40 to ~142, with an average of 60.79 (damn close to your fixed value).

Since you did it, I went in and calculated numbers, and I'm seeing a similar gain. My number of normal WUs is fairly small, but I was getting 48k/day on a 570 normal, and 59k/day with the app_info file.

[zombie67]

Just curious, why did you pick .75 CPU per? With my GTX 295, that is taking 3 of my 4 CPUs...not leaving much for crunching. Can this be lowered at all, without impacting the performance? Anyone feel like doing some experiments?

[Maxwell]

I used 0.75 because the default was 0.85, and 0.75 seemed rounder - completely arbitrary, but I wanted to stick close.

For the record: At exactly 06:00 UTC 27 March 2011, I changed it over to 0.25. Anything reported after that time has the new app_info system...

[zombie67]

I used 0.75 because the default was 0.85, and 0.75 seemed rounder - completely arbitrary, but I wanted to stick close.

For the record: At exactly 06:00 UTC 27 March 2011, I changed it over to 0.25. Anything reported after that time has the new app_info system...


I think the "default" is based on the stats of the GPU. The faster the GPU, the higher the percentage of CPU required (to keep up, I assume). For example, the default of my 295 is .69, and for my 9500GT PCI is .27.

[Maxwell]

After dropping the CPU usage down to 0.25, I haven't seen a difference. I'm calculating around 58k/day now, well within the margin of error for the number of WUs I've run.

[Maxwell]

After dropping the CPU usage down to 0.25, I haven't seen a difference. I'm calculating around 58k/day now, well within the margin of error for the number of WUs I've run.
I think you're wrong, Maxwell...

After running it for a while at 0.25, I was noticing a severe lack of GPU loading on average - it was hovering around 55% on average. I changed it back up to 0.50 CPU/WU, and (running two WUs) this leaves 1 core dedicated to feeding the GPU. Since I made this change, my GPU load is above 95% on average. Yay!

[Fire$torm]

Hey, out of curiosity is anyone still using an app_info file for this project?

[Al]

Don't see the need, they can't keep up with us as it is. I currently have 700+ wu's pending.

[zombie67]

Don't see the need, they can't keep up with us as it is. I currently have 700+ wu's pending.

Yeah, but that's a different issue, and will be resolved shortly. It shouldn't stop anyone from crunching, as the credits will be awarded eventually. No crunching time or credits lost.


About the app_info: It used to be that the app under-utalized the GPU. By using the app_info to run 2 tasks per GPU, you got almost 100% utilization, and better over all credits/day. However, they recently updated the app. Now the tasks run at near 100% GPU load, which is great. No app_info needed any more.

[Al]

I haven't stopped crunching and I won't, but it doesn't change the fact that it's true....they can't keep up with usand I feel it will be an ongoing issue. Just sayin'

[zombie67]

I haven't stopped crunching and I won't, but it doesn't change the fact that it's true....they can't keep up with usand I feel it will be an ongoing issue. Just sayin'

Only just recently have they encountered an issue, since they increased their credits to compete with PG. But it's not been an ongoing issue or anything. The last time this happened (something like 6-12 months ago), they immediately upgraded their HW. They just today posed that they would be upgrading their HW again in two weeks. Why do you feel it will be an ongoing issue? What have you read that I haven't? Maybe I missed something.

In any case, that has nothing to do with GPU utilization, which is what this thread is about. We should be maximizing utilization regardless of the amount of work. The faster we crunch distrrtgen tasks, the faster we free up the GPUs to crunch other projects when distrrtgen is out. It all about maximizing utilization. An half-used GPU is a crime against humanity. ;)

[Cruncher Pete]

Only just recently have they encountered an issue, since they increased their credits to compete with PG. But it's not been an ongoing issue or anything. The last time this happened (something like 6-12 months ago), they immediately upgraded their HW. They just today posed that they would be upgrading their HW again in two weeks. Why do you feel it will be an ongoing issue? What have you read that I haven't? Maybe I missed something.

In any case, that has nothing to do with GPU utilization, which is what this thread is about. We should be maximizing utilization regardless of the amount of work. The faster we crunch distrrtgen tasks, the faster we free up the GPUs to crunch other projects when distrrtgen is out. It all about maximizing utilization. An half-used GPU is a crime against humanity. ;)

I like that statement about crime against humanity. :) Keep on crunching and have a nice day...

[Duke of Buckingham]

Is there any app_info for ATI to use at DistrRTgen to run multiple tasks?

I will start thinking on bunker some Dirty, just for precaution ...

Bunkered Duke
http://www.notmytribe.com/wp-content/uploads/2008/12/military-bunker.jpg

[zombie67]

I didn't find that an app_info.xml was necessary. GPU load was very close to 100% naturally.

[Duke of Buckingham]

Maybe descending a point on CPU use to 99% that could be true. I had instability problems on the pass but I didn't learn tho lowered CPU use to feed the GPU as it should. Thanks Z. The Asteroid man.
http://upload.wikimedia.org/wikipedia/commons/0/0a/Asteroid_2004_FH.gif

Very Crazy Duke searching for apps, where none is needed.

[zombie67]

Okay, now that I am crunching this project again, I am seeing only 88-94% utilization on my 7970. I think there may be an advantage to using an app_info.xml after all.


Soooo, does anyone have an app_info.xml for ATI/AMD?

Edit: Nevermind. Reserving an extra core solved it. The app already reserves .989 of a core, which I would have thought was plenty. But reserving another full core has the GPU utilization pegged at 97-98%.

[Zytozux]

Does anyone have an ATI app info file for this project I could use? I finally figured out you have to check "run test applications" in the project settings (╯°□°)╯︵ ┻━┻