With a little help from the admin i figured out why my times sucked so bad.

My XEON CPU's are too slow to keep the GPU feed with just 1 WU and reserving more CPU's does not help as it appears the GPU app is single threaded.

I am now running 3 WU's on my 1070 and 4WU's on a 1080 and it appears to be working correctly now.
each WU gets its own CPU thread.