fftn on GPU array

In my code, somewhere, I generate a gpuArray matrix with a size of 331x331x331x32, (single float) and I want to get an fftn from the array in a for loop. I have a 24GB TITAN RTX GPU, and before getting each fftn, I have 10 GB free on the GPU. But, when I run the code, it can calculate it without problem, but for the second loop, it returns:

An invalid FFT plan was used on the GPU.

If I pause the code and rerun the fftn, it works fine, but again, I get the error for the next loop, and the same story goes for the next loops!

I also tried the reset(gpuDevice(1)) before each for loop and noticed when I have this pice before each iteration, I will get the error, and if I run it twice, the problem will be solved! Would you please explain to me why I have this problem?

Answer 1

[Matt J]

before getting each fftn, I have 10 GB free on the GPU

But wit the array dimensions you've mentioned the result will consume more than 17 GB in double floating point.

331^3*64*8/2^30

ans = 17.2924

You could try using single floats, but it will be a tight squeeze.

[Mojtaba Zarei]

Sorry I missed to mention that I have a single floats data! I already have single float data and get these errors!

[Matt J]

Are you clearing memory on subsequent iterations of the loop? Like I said, it's a tight squeeze. Even with single floats, you will only have enough memory to hold the output - we are not sure about what temporary memory space fftn requires.

[Mojtaba Zarei]

I use "gather" to store back the data on the CPU and do not clear anything after that. I assumed using "gather," I will clear the memory on the GPU? Should I remove all the gpu variables at the end of each loop?

I even used "reset(gpuArray(1))" before the fftn and I got the same errors!

[Matt J]

A statement like

A=gather(A)

should erase the gpuArray copy of A, but

B=gather(A)

will not. It will just create a copy of A on the CPU.

[Matt J]

Incidentally, you might find whosGPU useful

https://www.mathworks.com/matlabcentral/fileexchange/88024-whosgpu

for an approximate idea of how much memory is consumed by your gpuArray variables on the GPU.

[Mojtaba Zarei]

Thank you Matt, I figured out how to free the memory before the fftn function. Now, it works fine!