I only figured out how to make it with the first four notes, idk how to go back down, descending notes if that makes sense

Connect to [% 4] a [sel 3] or [sel 2] and connect this to your up/down toggle.

Other improvements to your patch:
Learn [t b f] , you should use it in conjuction with your [2( and [/ 12] to define execution order towards [pow] .
You might have a look at [mtof] too and use it instead of your conversion.

If you don't want the envelope to retrigger by changing the numberboxes of attack and release, you can store them in an [f] for each and flush them with your [r metro] and [r attack]