Well, if you load a large audio file while sound is playing, you can get dropouts then too.

That's a 97.3 MB file -- "audio I/O error." Only vanilla objects in the patch. Sending that read message does glitch the audio.
So this isn't "a Gem issue."
Since Pd is single-threaded and the control message layer runs within the audio loop, heavy activities in the control message layer can cause audio processing to be late. Reading a hundred MB of audio?
Opening a window and preparing it for graphics rendering? 
AFAIK the available solutions are: A/ structure your environment so that all heavy initialization takes place before you start performing -- i.e., start Gem before your show, not during. Or B/ Put Gem in a different Pd process (which doesn't have to be [pd~] -- you could also run a second, completely independent Pd instance and use OSC to communicate between them). (Smart-alecky suggestion: Or C/ Move your audio production over to SuperCollider, whose audio engine shunts heavy loading off to lower-priority threads and whose control layer runs in a separate process, so it doesn't suffer this kind of dropout quite so easily.)
"Except that dropout when you create and destroy. Wish that could be changed" -- sure, but the issue is baked into Pd's fundamental design, and would require a radical redesign (of Pd) to fix properly.
hjh