Can I use GPU computing inside of a parfor loop?

I am in the process of upgrading my computer and I am wondering if there is a potential for upgrading my GPU. Right now I am only using parfor. However, within the parfor loop there are operations that would seem to benefit from using GPU computing. I get the impression from this entry ( https://www.mathworks.com/matlabcentral/answers/36235-parfor-on-gpu ) that for what I want to do, I would need to use multiple GPUs. Now, I don't know enough about GPUs to know what kind of questions to ask, but I'll start with these:

If I am running 8 workers with parfor, does this mean that I would need to allocate 8 GPUs? Does GPU = graphics card and if so, then I need to buy 8 graphics cards (impractical)? Or do graphics cards have multiple GPUs? If this is possible, how would I go about choosing the right graphics card for the job?

Thanks in advance -jz

Answer 1

[Edric Ellis]

You do not need 8 separate graphics cards even if you have 8 workers using gpuArrays simultaneously. (In fact, I'm not even sure that you can put 8 GPUs in a single system). Having multiple workers using a single GPU simultaneously is called "oversubscribing". Whether or not you see any performance benefit from this depends strongly on the details of your problem - and how much of that you can actually run on the GPU.

For example, consider a case where you have a problem that, when run in a single MATLAB session, only spends 1/4 of its time using the GPU. In that case, you might expect good performance running 4 workers accessing the same GPU (providing the arrays all still fit in memory). If you run that against 8 workers, you will see diminishing returns as the workers all contend for access to the GPU.

[Jay]

This makes sense. So MATLAB can referee the simultaneous requests for the GPU? Meaning the GPU will be used serially inside the parfor loop, potentially becoming a bottleneck?

[Edric Ellis]

Actually, MATLAB isn't involved in serialising the requests - CUDA / the GPU does this automatically.

[Jay]

Great! Thanks for the answer. If you don't mind me asking, when looking for a graphics card, what should I look for? Do I have to make sure the graphics card memory can support the array(s) of data I will be sending it? Does more GPU cores = faster processing?

[Edric Ellis]

You could take a look at the GPUBench results. Generally speaking, NVIDIA's 'Tesla' series of cards are aimed more at computation and usually have better double-precision performance, as well as features like error correction (ECC). The 'GTX' cards often have very good single-precision performance.

Answer 2

[Matt J]

I get the impression from this entry ( https://www.mathworks.com/matlabcentral/answers/36235-parfor-on-gpu ) that for what I want to do, I would need to use multiple GPUs.

That's not the impression I get. Edric said in that thread that 1 GPU (he uses "GPU" to mean graphics card) might be enough, depending on the computation that you're doing. Have you tried it on your current graphics card?

[Jay]

No, that is part of my problem, I don't have a graphics card at the moment to try any of this out on. But if what Edric said is true, how would I know what graphics card to buy that could support computations that I am doing?

[Matt J]

If you don't currently have a CUDA capable graphics card, what is your theory that it will benefit you based on? What does your computation look like?

[Jay]

I have been going through the tutorials for GPU computing and it seemed like I could benefit from it. And most of my processing is image processing on 4MP images.