Actually, you don’t have to swap the different quadrants, it’s needed only if you’re a human and want a more natural looking visualization of the FFT result (i.e. with the 0 frequency in the middle, negative frequencies left/bottom and positive frequencies up/right).
To invert the FFT, you need to pass the result of the forward transform “as is” (or after the frequency filtering you wanted) to the same dft() function, only adding the flag DFT_INVERSE. If you remember your math about FFT, the forward and backward transforms have very tight kinks in the formulation…
— EDIT —
What exactly doesn’t work ?
The following code does perform forward-then-backward FFT, and everything works just fine as expected.
// Load an image
cv::Mat inputImage = cv::imread(argv[argc-1], 0);
// Go float
cv::Mat fImage;
inputImage.convertTo(fImage, CV_32F);
// FFT
std::cout << "Direct transform...\n";
cv::Mat fourierTransform;
cv::dft(fImage, fourierTransform, cv::DFT_SCALE|cv::DFT_COMPLEX_OUTPUT);
// Some processing
doSomethingWithTheSpectrum();
// IFFT
std::cout << "Inverse transform...\n";
cv::Mat inverseTransform;
cv::dft(fourierTransform, inverseTransform, cv::DFT_INVERSE|cv::DFT_REAL_OUTPUT);
// Back to 8-bits
cv::Mat finalImage;
inverseTransform.convertTo(finalImage, CV_8U);