Motherboard & PCI-E slots

[Philadelphia]

I've given some thought to building a computer for gpu crunching only. By that I mean I'm not interested in a cpu (other than is necessary for the gpu's) with a gazillion processors or threads, they are too slow compared to a gpu (I think anyway).

The best I could find was a AMD board (http://www.newegg.com/Product/Product.aspx?Item=N82E16813128508) that appears to have 6 pci-e slot (2 x 16, 2 x 8 and 2 x 4, whatever they mean, lol).

What MOB would one consider buying for as many gpu's as one can put on it?

[Mike029]

I've given some thought to building a computer for gpu crunching only. By that I mean I'm not interested in a cpu (other than is necessary for the gpu's) with a gazillion processors or threads, they are too slow compared to a gpu (I think anyway).

The best I could find was a AMD board (http://www.newegg.com/Product/Product.aspx?Item=N82E16813128508) that appears to have 6 pci-e slot (2 x 16, 2 x 8 and 2 x 4, whatever they mean, lol).

What MOB would one consider buying for as many gpu's as one can put on it?

I'm running into a bit of a problem on my main cruncher. I have the 5970 and a 5870 in a MB with AMD x2 cpu. The gpu's are running moo and eating up both cpus to run at 96 to 98 pct. I would recommend that you have at least as many cores as gpus on any crunching rig.

[coronicus]

I would def stay with AMD with the abilty to get the 6 real cores versus intel with the hyperthreading making it hard to control what gets to use a core and which one gets to use the hyperthread which i just gave up trying to tweek after a while... I guess it all depends on da money.

Dual pcie is easy to find and also much cheaper on the motherboard and casing side. Also alot easier to cool the gpu by making sure there is extra room between the pcie.

3 pcie your looking at spending much much more on the motherboard and also requiring a beast of a casing with enough room and slots on the back to handle that one extra card. Also cooling the video cards will require much more air flow specially since they will be up against each other (noise)...

Anymore PCIE would put you into some kind of water cooling situation on the gpu to fit more then 3 standard high end video cards.

My two cents save money on the dual pcie motherboard with extra room between em a nice quad or even the 6 core and spend that extra money on the dual GPU like the AMD 6990 or the nvidia version which im not sure which one is better but i guess it comes down to which projects your the most interested in and which one will work or perform the best.

[Fire$torm]

I'm running into a bit of a problem on my main cruncher. I have the 5970 and a 5870 in a MB with AMD x2 cpu. The gpu's are running moo and eating up both cpus to run at 96 to 98 pct. I would recommend that you have at least as many cores as gpus on any crunching rig.

+++1 AMD quads can be had for very little money. I recently grabbed an Phenom II 840 for my Uncle's new system. Retail sale at Micro Center was like $45~$49 before rebate. He was very happy that I didn't have to spend a lot of his money... :P

[Mike029]

+++1 AMD quads can be had for very little money. I recently grabbed an Phenom II 840 for my Uncle's new system. Retail sale at Micro Center was like $45~$49 before rebate. He was very happy that I didn't have to spend a lot of his money... :P

SCREAM next time you see that deal and I'll upgrade. I fell for the x2 BE and wanted to oc the heck out of it. Only gpu's could care less about an oc'd cpu.

[Mike029]

+++1 AMD quads can be had for very little money. I recently grabbed an Phenom II 840 for my Uncle's new system. Retail sale at Micro Center was like $45~$49 before rebate. He was very happy that I didn't have to spend a lot of his money... :P

SCREAM next time you see that deal and I'll upgrade. I fell for the x2 BE and wanted to oc the heck out of it. Only gpu's could care less about an oc'd cpu. This only applies to Moo and PG I believe. I say make sure you have the room to crunch whatever gpu project you want.

[DrPop]

@Philadelphia - yes, I a think that is the best mobo for doing what you want. Gigabyte is top notch and rock solid, stable. What you want is a LOW Watt 6 core AMD CPU in there, specifically the Phenom II X6 1035T. The reason is, it only pulls 95W! Which is sweet for a 6 core, and leaves you more watts free for the big GPUs in the system.
It also O/C's like it's on steroids. I got one for Kat's rig to replace her aging quad core, and it O/Ced to 3.6GHz - that on a low end $45 MSI mobo, even! The chip is well worth the $, that was a 1GHz O/C! :D By the way, look for one on Ebay - I got a "Dell refurb" CPU from a seller there that had tons of them for $99. I think what that means is it was a "used pull"...best bang for the buck CPU I ever bought. Nearly matches my 1100T that I paid twice that much for. EDIT: There is one on Ebay right now for $119, not sure if that is the best price you can find, but that's a pretty decent deal for that kind of O/C.

[Philadelphia]

Personally, I'm not concerned withe the casing, in fact, it wouldn't even have to have one, one of those you see on 'you tube' I'm just thinking of a MOBO that has the most gpu slots, that could handle several (???) 5xxx or 6xxx ( or equivalent nvidia card) on it.

[DrPop]

What exactly do you mean by the "casing"? You mean a computer case? Yes, I crunched for a long while without one, but it can be nice to have one with a big fan in the side blowing in over the GPUs. They get hot. ;)

[Philadelphia]

What exactly do you mean by the "casing"? You mean a computer case? Yes, I crunched for a long while without one, but it can be nice to have one with a big fan in the side blowing in over the GPUs. They get hot. ;)

That's what I was referring to, the physical case.

[DrPop]

OK, yeah you can easily crunch without a case. If it gets too hot, just point a large "house" fan at it! ;)

Looking at Mobos that will allow you to run 4 big GPUs . . .

The other AMD option to save some money is this MSI mobo (http://www.newegg.com/Product/Product.aspx?Item=N82E16813130274) - it is the 890FX chipset instead of the 990FX, and I personally don't see how you could beat the Gigabyte mobo, but if you are not ever going to put a high end CPU in there, then can't see how it would matter too much. The one thing the Gigabyte has going for it, is that Gigabyte is legendary for having stable mobos at huge power draws, (which would be a big thing for a huge GPU cruncher) and also, if at any time AMD gets their stuff together and puts out a future revision of the Bulldozer worth buying, it would run it. The MSI board *might* run it, but that depends on how long they update the BIOS for the older 800 series chipset.

On the Intel side, there is always this MSI mobo (http://www.newegg.com/Product/Product.aspx?Item=N82E16813130595R) or this Asus mobo (http://www.newegg.com/Product/Product.aspx?Item=N82E16813131714), but you will be (depending on model) spending more on the mobo & CPU to get a Sandy Bridge i7. However, the CPU will do more work for the same 95Watts, so it's totally up to you how you look at it. ;)

[John P. Myers]

Lol every time i look at that Intel MSI board DrPop just linked at the bottom of his last post, it gets cheaper and cheaper. Now $29 less than what i paid for it. But that board is god. Hands down.

Of coarse i still feel Gigabyte makes better ones :) This was just an exception :p

As was already mentioned, you will need a fairly powerful CPU to keep several powerful GPUs busy. All GPU work units use some amount of CPU power to crunch, plus the various data manipulation tasks involved with keeping the GPUs a workunit to crunch on. IIRC, 6 PCIe slots is the best you're gonna get on an AMD mobo. 8 for Intel. As for the hyperthreading on the Intel chips, just turn it off. Chip runs a few degrees cooler like that anyway. But getting a cheap 6-core AMD CPU should be able to handle whatever you stuff in those 6 slots and still run FreeHal and WuProp.

[STE\/E]

I'm running into a bit of a problem on my main cruncher. I have the 5970 and a 5870 in a MB with AMD x2 cpu. The gpu's are running moo and eating up both cpus to run at 96 to 98 pct. I would recommend that you have at least as many cores as gpus on any crunching rig.

That's strange, I'm running Moo on 4 different Box's & not using a single CPU Core, 2 of the Box's are running the 5970's ... Are you running some sort of App file ???

[Bryan]

I'm with Ste\/e on this, I don't use up much CPU in the 6990 machine running Moo.

I seem to recall that way back when there were folks with machines that used quite a bit of CPU. It was a combination of processor, driver, GPU etc. Originally they had set the wu to reserve a full core/thread but they dropped that once they found out that it was machine specific. That is shaking cobwebs but that is the way I recall the situation.

[Slicker]

I'm with Ste\/e on this, I don't use up much CPU in the 6990 machine running Moo.

I seem to recall that way back when there were folks with machines that used quite a bit of CPU. It was a combination of processor, driver, GPU etc. Originally they had set the wu to reserve a full core/thread but they dropped that once they found out that it was machine specific. That is shaking cobwebs but that is the way I recall the situation.


I find that dedicating a core to Moo increases the GPU credits way more than what I lose from the CPU core. In other words, I tell boinc to use 87.5% of the processors on my i7. Maybe it depends upon the specific GPU though. FYI, the i7 contains a 6970.

[STE\/E]

I checked 1 of the 5970 Box's & didn't see where the MOO Wu was using any CPU Resource @ all. I set to use 3 Cores on the Quad Core & will run like that for awhile to see if the MOO Wu speed increases or I get more Credits ...

[John P. Myers]

I find that dedicating a core to Moo increases the GPU credits way more than what I lose from the CPU core. In other words, I tell boinc to use 87.5% of the processors on my i7. Maybe it depends upon the specific GPU though. FYI, the i7 contains a 6970.

Exactly true. The amount of CPU usage BOINC reports from GPU projects is only what it's told to report by the app, not what it's actually using. The app setting this number low though (such as 0.05 CPUs) causes BOINC to reserve a lower amount of your cores, freeing them up for other tasks. The issue comes in where the GPU app suddenly needs more than 0.05 of your CPU, but it's busy with other things since BOINC didn't reserve the cores. This slows down your crunching.


Don't know what happened but the problem 5970 that had been running the MOO Wu's in the 14-15 Min Range for the last 2+ Days now has gone back to the 18-19 Min Range again. Probably must be something to do with changing the CPU Usage to 3 & then back to 4 again, still ran the Wu's in the 14-15 Min Range running 3 CPU's but now that I've gone back to 4 CPU usage the Wu's are taking 3-4 Min's longer ... :/ ... Will change back to 3 CPU's if the times don't come down soon & see what happens ...

This also solves Steve's problem :)

[STE\/E]

Exactly true. The amount of CPU usage BOINC reports from GPU projects is only what it's told to report by the app, not what it's actually using. The app setting this number low though (such as 0.05 CPUs) causes BOINC to reserve a lower amount of your cores, freeing them up for other tasks. The issue comes in where the GPU app suddenly needs more than 0.05 of your CPU, but it's busy with other things since BOINC didn't reserve the cores. This slows down your crunching.



This also solves Steve's problem :)

No not really ...

[John P. Myers]

Sure it does. You said yourself you were completing GPU WUs faster when you limited BOINC to 3 cores and slower when you had BOINC use all 4 cores.

[coronicus]

I would love to know what your running to get away without using a core per AMD/ATI GPU... anytime i try to lower the amount of cpu's my gpu usage become speratic and wont stay steady. Starting to sound like something between the drivers, opencl and boinc... :confused:

[spingadus]

I'm going to run a test for a day or two and see if my credit goes down by not reserving any threads.

Currently I have a pretty steady avg of 350k/day at Moo! with 1 reserved thread (cpu setting at 87.5%) on my i7 920 with AMD 6970.

[STE\/E]

Sure it does. You said yourself you were completing GPU WUs faster when you limited BOINC to 3 cores and slower when you had BOINC use all 4 cores.

Apparently You didn't read everything I said: the key phrase


changing the CPU Usage to 3 & then back to 4 again


Don't know what happened but the problem 5970 that had been running the MOO Wu's in the 14-15 Min Range for the last 2+ Days now has gone back to the 18-19 Min Range again. Probably must be something to do with changing the CPU Usage to 3 & then back to 4 again

I changed the usage from 4 CPU's to 3 & then back to 4 again, the Wu's were running as fast using 4 CPU's as it was using only 3 so I changed back to 4, that's when the slow down began. I have another Box using a 5970 & it's running the Wu'[s under 15 Min's with no CPU Core reserved for MOO ...

[DrPop]

@SteVe - well, that is a little confusing! :cool: I will add to this nutty project...on the rig with the 5870, I don't have to drop any cores - I can crunch BOINC on all 6 cores AND get 99% utilization of the 5870 on Moo!, plus 99% utilization of the two GTX 460s on either DiRT or PG.

Now, on Kat's rig with the two crossfired 4870s, I have to reserve 2 cores! That means it needs a core just to feed each lowly 4870, where as the 5870 doesn't even take 1 core, and it spits out more than double credit than the two 4870s combined!
What's up with that?:confused:

@Sping- no reason to do a lengthy test. Just open up Afterburner and watch the % GPU utilization as you play with the amount of CPU cores you are allowing BOINC to crunch CPU project(s) on. ;)

[John P. Myers]

still ran the Wu's in the 14-15 Min Range running 3 CPU's but now that I've gone back to 4 CPU usage the Wu's are taking 3-4 Min's longer

apparently you didn't read everything you said yourself.

[spingadus]

@Sping- no reason to do a lengthy test. Just open up Afterburner and watch the % GPU utilization as you play with the amount of CPU cores you are allowing BOINC to crunch CPU project(s) on. ;)

Yeah, not sure what I was thinking. I check my 2600k with gtx 590 by looking at %gpu utilization, why wouldn't I do the same with my other machines. doh!

[DrPop]

apparently you didn't read everything you said yourself.

In my best MODERATOR TONE: Alright now guys, let's keep this civil please. It's a strange problem going on and these 5970s don't always seem to behave "normally"... ;)

EDIT: yes, I think I see what you're saying, JPM - he should try going back to 3 on the CPU and see if the times go down, right? That would be the most consistent with what people are getting. Am I understanding it right?

[spingadus]

Now now fellas, no need to get testy, lol.

Could the differences in time have anything to do with the cpu project that is run? Or the cumulative cpu usage including NCI stuff? Maybe the gpu usage is right on the line between getting enough and running into contention.

[STE\/E]

I was taken out of Context ... :-|

For the Record I have run that Box with no CPU Cores running & the Moo Wu's still didn't Speed up, now I have gone back to 3 Cores running with no speed up of the MOO Wu's. So I don't think how many CPU Cores are running on that Box has anything to do with the Speed of the MOO Wu's, it still run's the Wu's 4-5 slower that the other 5970 Box which has all 4 CPU Cores running ... Kapish ... =))

[John P. Myers]

yes, I think I see what you're saying, JPM - he should try going back to 3 on the CPU and see if the times go down, right? That would be the most consistent with what people are getting. Am I understanding it right?

I'm certain we had this discussion on the old forums where someone actually did some testing to show that GPU WUs completed faster when all of the CPU cores weren't being used by BOINC. Not sure if it's been discussed in these forums or not. Slicker stated he backs his down to 87.5% to leave a thread clear to handle it. And i also know from my own experimentation that leaving a thread/core unused will generally speed up GPU crunching times. By how much varies, of course. Depends on what GPU you're using (slower ones are sped up less, if it all), how many GPUs you have, and what project your CPU would be crunching if it weren't left unused. It can even matter at what point during the CPU WU the GPU needs something taken care of. Sometimes the CPU just isn't in a place where it's ready to do the GPU's bidding, so it has to wait. Quite a few variables involved, but as a general rule, GPU WUs will be faster to some degree if a thread/core is left unused by BOINC.

EDIT: Just to be clear, leaving a thread/core free won't speed up every GPU WU everytime from every project. Just in general. Law of averages. :p

[spingadus]

Maybe this has something to do with how Moo! auto selects which core to use. Perhaps reserving and unreserving cores affects this. Or you have gremlins in your box and it needs a nice exorcism reboot :P

[John P. Myers]

Another Edit: It's also possible that leaving a thread/core free won't reflect any speedup at all on the GPU WUs, but instead all CPU WUs will be sped up by not having to stop to see what the GPU wants.

[DrPop]

OK, wait a sec, JPM's post got me thinking...what CPU project were you running before this? Because now you're running WCG, right? WCG, at least the malaria sub-project seems to run my CPUs harder than DiRT, because I had to back off my O/C some, as I was getting error on WCG at speeds that were just fine on DiRT, Prima, etc.

I also just saw the part about you running Moo! only on the GPU with nothing on your CPU crunching at all, and you still didn't get good times. Weird. Maybe it is a GPU issue afterall, then?:confused:

[STE\/E]

I'm gonna take my Box & go home ...

[DrPop]

I'm gonna take my Box & go home ...

Lolololol!!!=)):((:)):-??

[Fire$torm]

I'm gonna take my Box & go home ...

Gawd, I hope that was a joke.

BTW: Guys, you need to also consider system RAM quantity and speed, HDD throughput, type/make of Northbridge and any other hardware that contributes or degrades data speed. Not all MB's are created equal. Lesser MBs sacrifice high data load handling for lower cost. BOINC is one of the few things in the world that can tax every major component/system of a computer as well as many of the sub-systems all at the same time.

[DrPop]

Gawd, I hope that was a joke

er...yeah, me too. I think he's probably as frustrated as any of us would be, trying to make a 300W+ piece of hardware pull its proper weight! ;)

[Philadelphia]

er...yeah, me too. I think he's probably as frustrated as any of us would be, trying to make a 300W+ piece of hardware pull its proper weight! ;)

Good one, that got a good chuckle out of me, and I need it 'till I figure out those #%$@^& 5870's, lol.

[Fire$torm]

Oh, I just remembered one that gets overlooked all the time, onboard NIC. All low-end and most mid-range MBs that have integrated NICs use the CPU as a controller. So a system with one of those has a lot of network traffic like short wu's on multiple projects, it will degrade CPU performance.

I see it all the time on my torrent seed-box/server/cruncher.

[John P. Myers]

Oh, I just remembered one that gets overlooked all the time, onboard NIC. All low-end and most mid-range MBs that have integrated NICs use the CPU as a controller. So a system with one of those has a lot of network traffic like short wu's on multiple projects, it will degrade CPU performance.

I see it all the time on my torrent seed-box/server/cruncher.

+1

A crappy NIC will cause the CPU to handle a large portion of the uploading/downloading of WUs (and whatever else you're doing) and will slow down crunch times. Same with mobos that don't have built in audio chips. I DO remember we used to say on the old forums that all real crunchers have their audio disabled :D Just another reason to leave a core/thread unavailable to BOINC :p

[DrPop]

...on that note...I just got rid of a lovely sound card because I was making more room for GPUs! :)):D;)

How good does the NIC have to be to not offload any onto the CPU? I use the ones built-in the mobos right now, but I do have one of these (http://www.newegg.com/Product/Product.aspx?Item=N82E16833156139) sitting in my back room. Would it be worth someone (or even me) using it, or is it just "cheap junk" too?

[John P. Myers]

well to be honest, a GPU in that slot would help more than a NIC :p the difference it would make is marginal at best. though if you don't have a GPU for that slot, such as this one, (http://www.newegg.com/Product/Product.aspx?Item=N82E16814500221) you might as well stick it in there :)

[Fire$torm]

...on that note...I just got rid of a lovely sound card because I was making more room for GPUs! :)):D;)

How good does the NIC have to be to not offload any onto the CPU? I use the ones built-in the mobos right now, but I do have one of these (http://www.newegg.com/Product/Product.aspx?Item=N82E16833156139) sitting in my back room. Would it be worth someone (or even me) using it, or is it just "cheap junk" too?


well to be honest, a GPU in that slot would help more than a NIC :p the difference it would make is marginal at best. though if you don't have a GPU for that slot, such as this one, (http://www.newegg.com/Product/Product.aspx?Item=N82E16814500221) you might as well stick it in there :)

All true but....

As I see it, on a good mid to high end MB, yeah not a really big deal. For anything less, if you want consistent max performance you need a NIC with a dedicated controller as low end NIC cards are no different then the integrated NICs on low end MBs.

On Newegg the lowest priced NIC card I could find with a listed dedicated controller is this HP/Broadcom unit ---> http://www.newegg.com/Product/Product.aspx?Item=N82E16833316142

[DrPop]

Ah, ok thanks guys. I guess I'll just use what's on the mobo then. BTW, that is sweet to FINALLY see a decent crunching card on the PCI slot actually make it to production! :)

[John P. Myers]

Ah, ok thanks guys. I guess I'll just use what's on the mobo then. BTW, that is sweet to FINALLY see a decent crunching card on the PCI slot actually make it to production! :)

Yes! And i love that it turned out to be cheaper than the 9500 GTs were.

[Fire$torm]

SCREAM next time you see that deal and I'll upgrade. I fell for the x2 BE and wanted to oc the heck out of it. Only gpu's could care less about an oc'd cpu. This only applies to Moo and PG I believe. I say make sure you have the room to crunch whatever gpu project you want.

Since this is not quite the same deal, I will only SHOUT!!!!

AMD Phenom II x4 840 - $59.99 (In Store Only) (Link (http://www.microcenter.com/single_product_results.phtml?product_id=0371961))

MicroCenter Store Locations (Link (http://www.microcenter.com/at_the_stores/index.html))

If you don't live near a MicroCenter find someone that does! Hint: I do... :-"