Boinc + Linux + GPU = pain

[dan]

OMG, I just spent about 6 hours figuring out how to get boinc GPU to work on my linux box. First there are about 50 ways to get the Nividia drivers install and only one of the 50 works. Then there is the problems with Boinc not detecting a usable GPU. Lots of ideas on this one. It comes down to Boinc being able to access the cuda drivers. The final fix was for me to set SELinux to permissive. Basically turning it off.

Dan

[Steve Bohlen]

Sorry you had so much trouble, Dan. Been afraid to take the plunge myself on a GPU for one of my Linux boxes in case it was a pain to configure. Now I know who to bug when I finally get a GPU.

[zombie67]

I tried one looooong night, to get get cuda crunching to work with linux. Complete fail.

[Maxwell]

I tried one looooong night, to get get cuda crunching to work with linux. Complete fail.

There's a really easy way - send trigggl a 4-pack of his favorite beer, and give him remote access to your machine for a bit.

[trigggl]

A few steps that should be done in a specific order for cuda.

Make sure the nvidia driver is installed and working
Make sure nvidia cuda toolkit is installed
Make sure boinc is in the group video
reboot

[dan]

There is too much advice on how to do this and most won't work in your setup.

I ended up finding how to do it with "yum install kmod_nividia????". I had to rblacklist the nouveau drivers in grub.conf. Then had to make sure boinc had enough time after boot up to recognize the drivers by adding "sleep 6" to /etc/init.d/boinc_client. Then had to make sure it had access to the display drivers, so added boinc to the video group and added "xhost +" to the boinc_client script (not sure this was necessary). Then SELinux kept giving me failures, and all the fixes I found didn't work. Then I discovered others that never go SELinux to work, so I then modified /etc/selinux/config to make SELinux run in passive mode and it all worked.

Do not start with the drivers provided by Nividia this I eventually figured out how to work, but yum is the faster way.

Dan

I did install the Nividia toolkit, another pain (not sure it was needed).

[YoDude9999]

I think provides a lesson for linux.....Don't bother with it.

Yo-

[trigggl]

There is too much advice on how to do this and most won't work in your setup...

That's an understatement.

I ended up finding how to do it with "yum install kmod_nividia????". I had to rblacklist the nouveau drivers in grub.conf. Then had to make sure boinc had enough time after boot up to recognize the drivers by adding "sleep 6" to /etc/init.d/boinc_client.

Assuming a yum based package management system. ('apt-get install' on Debian based, 'emerge nvidia-drivers' on Gentoo, etc...) I think most distros today have a good graphical package management system that is easier to work with to get the right drivers installed and the x-server properly set up.

Then had to make sure it had access to the display drivers, so added boinc to the video group

This must be done on any distro. You're not going to get GPU support without it.

Then SELinux kept giving me failures, and all the fixes I found didn't work. Then I discovered others that never go SELinux to work, so I then modified /etc/selinux/config to make SELinux run in passive mode and it all worked.

One thing I learned when trying to create a Gentoo/BOINC LiveCD, you have to break your security to GPU crunch. Don't run BOINC on your secure servers. Don't waste your time trying to run SELinux on a cruncher. Secure Linux and BOINC don't mix. You have to break one to use the other. It's probably why you had problems with nouveau drivers. Secure Linux uses that instead of nvidia for security purposes. I guess they assume you're not going to be using your server for 3D gaming.

Do not start with the drivers provided by Nividia this I eventually figured out how to work, but yum is the faster way.

Good advice! You should always use the package management nvidia driver first and if it doesn't work, UN-INSTALL before trying the nvidia driver outside of the package management system (last resort).

So, here are some steps that should get most people working.

1. Install nvidia proprietary driver through the package management system.
   (Will probably require a reboot)
   (Get this working before bothering with the rest!)
   
2. Install CUDA with the package management system.
   
3. Add user 'boinc' to group 'video'.
   (If you don't know how, ask.)
   
4. Reboot. (To be on the safe side)
   
5. Start boinc if not already started

I noticed on the AMD twins that X needed to be started for cuda to work, so it might be a good idea to have the window manager (gdm or kdm) start before boinc.

I like Maxwell's idea, though. Send me a pack of Old Rasputin Imperial Stout and I'll do it for you.

[Fire$torm]

There is too much advice on how to do this and most won't work in your setup.

I ended up finding how to do it with "yum install kmod_nividia????". I had to rblacklist the nouveau drivers in grub.conf. Then had to make sure boinc had enough time after boot up to recognize the drivers by adding "sleep 6" to /etc/init.d/boinc_client. Then had to make sure it had access to the display drivers, so added boinc to the video group and added "xhost +" to the boinc_client script (not sure this was necessary). Then SELinux kept giving me failures, and all the fixes I found didn't work. Then I discovered others that never go SELinux to work, so I then modified /etc/selinux/config to make SELinux run in passive mode and it all worked.

Do not start with the drivers provided by Nividia this I eventually figured out how to work, but yum is the faster way.

Dan

I did install the Nividia toolkit, another pain (not sure it was needed).

That's an understatement.



Assuming a yum based package management system. ('apt-get install' on Debian based, 'emerge nvidia-drivers' on Gentoo, etc...) I think most distros today have a good graphical package management system that is easier to work with to get the right drivers installed and the x-server properly set up.



This must be done on any distro. You're not going to get GPU support without it.



One thing I learned when trying to create a Gentoo/BOINC LiveCD, you have to break your security to GPU crunch. Don't run BOINC on your secure servers. Don't waste your time trying to run SELinux on a cruncher. Secure Linux and BOINC don't mix. You have to break one to use the other. It's probably why you had problems with nouveau drivers. Secure Linux uses that instead of nvidia for security purposes. I guess they assume you're not going to be using your server for 3D gaming.



Good advice! You should always use the package management nvidia driver first and if it doesn't work, UN-INSTALL before trying the nvidia driver outside of the package management system (last resort).

So, here are some steps that should get most people working.

1. Install nvidia proprietary driver through the package management system.
   (Will probably require a reboot)
   (Get this working before bothering with the rest!)
   
2. Install CUDA with the package management system.
   
3. Add user 'boinc' to group 'video'.
   (If you don't know how, ask.)
   
4. Reboot. (To be on the safe side)
   
5. Start boinc if not already started

I noticed on the AMD twins that X needed to be started for cuda to work, so it might be a good idea to have the window manager (gdm or kdm) start before boinc.

I like Maxwell's idea, though. Send me a pack of Old Rasputin Imperial Stout and I'll do it for you.

Great info guys. Is there any chance the two of you could do a "How-To" so it can added to the others on the forum?

[trigggl]

Great info guys. Is there any chance the two of you could do a "How-To" so it can added to the others on the forum?

I could do a Gentoo how-to.

Getting the nvidia driver working is the hard part and that's going to vary depending on the distro. The best place to go for that is the message board of the distro. For Debian based distros, I use 'aptitude', but not everyone knows how to use that.

I have no experience with running CUDA on anything other than Ubuntu or Gentoo. Before I started using Ubuntu, the nvidia driver had to be installed by getting it from nvidia and installing it yourself. Obviously, things have changed since then.

I've basically been running nothing, but Gentoo the past 2 years. Perhaps we could get a user of each distro to write a how-to.