I am in a similar position to you; I just finished up my first year of college and I'm trying to become more comfortable with PD and audio processing in general. I only have taken a semester of calculus, so the math needed for DSP (linear algebra, differential equations, etc.) is all stuff that I am trying to teach myself (it's not happening very quickly).
The thing that I will say, though, is that the most interesting results you will get in audio are probably from experimentation, though. Knowing the math will help you do some crazy, professional sounding things with patience, but as mod said you are going to learn a lot by just trying new things and exploring patches.
Some really great results can come out of fairly simple processes. For example, I am trying to process guitar arpeggios that I recorded in different ways. So far I have sampled the original recording at normal, half, and double speeds and layered them together, then convolved them with white noise (I used [partconv~] it sort of blurs the different frequencies together and can create a nice drone-type sound).
Finally I played this audio back in a new patch and filtered it with [vcf~] objects tuned to different notes in the original arpeggio and triggered on and off with a simple sequencer. With a little delay and reverb, I ended up building a sound I really like and can probably use as the basis for a track.
I guess my point is that I barely have any idea what I'm doing, but I am enjoying myself and learning new, potentially useful techniques. I think the fact that you are already working with PD and reading pretty advanced material shows that you have passion and are learning quickly.
Sorry if I'm not really helping, but just realize that if even you only understand half of the logic behind this stuff, someone else like me only understands a quarter. Keep reading, asking questions, and experimenting.
