Run Optimization Programs with GPUs?

I'm trying to determine if it's worth investing the money in GPU cores over CPU cores. My research requires CPU cores to run a COMSOL model, but the eventual feedback control scheme includes large optimization problems. I know that the Optimization Toolbox is not GPU-supported, but I was wondering if Matlab Programs such as lsqnonlin can be used with GPUs? Are optimization problems typically parallelizable? If the built-in functions like lsqnonlin are not GPU compatible, what is your opinion on me writing my own version to use only GPU-compatible functions. My instinct says this would not be effective.

Thanks, Matt

Answer 1

[Matt J]

I'm trying to determine if it's worth investing the money in GPU cores over CPU cores.

GPU cores and CPU cores do very different things. You can only rarely use the GPU to parallel-split a task in the same way you would on a CPU. The way you would use the GPU to accelerate an optimization algorithm is to accelerate the operations done within the objective function and gradient calculations, the parts supplied by you. A list of operations that are GPU-enhanceable in MATLAB can be found here,

http://www.mathworks.com/help/distcomp/run-built-in-functions-on-a-gpu.html

If your objective function or Jacobian calculation uses any of these operations, then naturally the GPU can be used to make that part of the computation go faster. And, since that part is usually the bottleneck of an optimization algorithm in large problems, you should normally see some benefit from this.