i tried initiating the switch~ on block boundaries by using [bang~], but that didn't seem to work,
so the only idea i have is to ramp your volume down to zero, with a ramp time of 5ms or so (using [vline~]), do the switch, and then ramp volume back up to 1.
you'd have to have the FFT stuff in a subpatch, and the volume ramp outside the subpatch, otherwise the ramp itself would also get messed up with the block switch glitch.
if you need to switch VERY rapidly between block sizes, and CPU is not too much concern, then you could always just turn your FFT process into an abstraction, and load different instances of it for each block size, and then crossfade between those.
admittedly, not very elegant, but you could even do stuff like switching the abstractions off (with a 0 message to [switch~]) once they have crossfaded to 0.