Forming an Impulse response from phase data

I am taking an impulse response, taking an fft, then using just the phase data only to create a new impulse.

When I follow this process using phase and mag data then the impulse that comes back looks good, when i do it with just the phase then half of the impulse gets shoved to the end of the buffer which is inconvenient as I need to window it down. I've tried fftshift but this leaves me with delay that I don't want.

As below, here I've scaled the 'phase only' array by 1000 to give good visibility on the plot (attached).

Excess_Y_complx= (abs(Y_Ex_win)).*exp(angle(Y_Ex_win)*sqrt(-1)); Excess_Y_complx2= 1000.*exp(angle(Y_Ex_win)*sqrt(-1));

Excess_LF_imp = (ifft(Excess_Y_complx, N_FFT,'symmetric')); Excess_LF_imp2 = (ifft(Excess_Y_complx2, N_FFT,'symmetric'));

figure; hold on; plot(Excess_LF_imp,'g'); plot(Excess_LF_imp2,'r');

Best regards,

mat.