After much confusion I found a helpful old thread with this info to put in to the Terminal:
"/Applications/PureData/Pd.app/Contents/Resources/bin/pd -nogui"
But once I've opened PD without the GUI, how do I open my patch? If I just call my patch from the command line then I can't get it to open without the GUI.