Hi everyone.
I use the [struct] and [pointer] objects to store datas from a CSV text file (by using the [textfile] object).
I never had problems before, but I just discovered that it's not possible to store more 248 lines of datas.
Is there a way to increase this limit, or another way to store datas without limitation?
Thanks in advance if you can help.

What does "datas" refer to in this case?

It's basically numbers.
The content of my text files looks like this:

0 0 200,
-1 -1 0,
-1 -1 0,
-1 -1 0,
-1 -1 0,
5 2 8,

(it's a basic example, I have some text files with a way more columns)

@Octopoulpe What error are you getting?

struct_pointer_problem.zip

Here is an example.

When I put the content of the [textfile] in the object, I receive the message "stack overflow".
After, when I try to read too many lines (more than 248), I receive the message "ptrobj_next: no current pointer" (but it makes sense because not the lines have been written)

Looks like a stack overflow from sending the bang back into the inlet of [textfile].

If you instead use an iterative loop there with [until] you won't hit the overflow.

@Octopoulpe nice way to use the struct object. do you just sequence your data with "next"? or do you access the data with array and element? what kind of advantage has the struct object in your case in comparison to the text object?

struct_pointer_problem.pd ps.: heres the solution from jancsika

Awesome! Thanks a lot for spending time on my problem!
Now it works perfectly!
I still don't totally get it, why in my case it worked until the 248th line...
But anyway, thanks a lot!

The Stack Overflow error is triggered by feedback. Using [until] solves the problem iteratively, not recursively, so it won't trigger the Stack Overflow.

I've always wanted to know more about the Stack Overflow error and exactly what conditions trigger it. Maybe @jancsika can explain?

It's interesting, thanks.
I'm also curious about what triggers the Stack Overflow.

@Octopoulpe Digging the hole too deep........
It is such a pernicious problem that one of the main programmer forums is named after it.....
https://stackoverflow.com/questions/11777302/stackoverflow-too-many-recursive-calls-in-c
https://stackoverflow.com/questions/15688019/recursion-versus-iteration
David.

I'm also curious about what triggers the Stack Overflow.

A safety mechanism that puts an upper limit on how deep an immediate "chain reaction" can be in response to an event like a mouse click, key press, netreceive, or clock callback (.e.,g. a bang coming from [delay 1000]). When I say "chain reaction", I mean an initial message to an object triggers an outgoing message to another object, and so on until there are no more outgoing messages to process. When I say "immediate", I mean the messages get sent without any logical time in between.

Here's an example:

[bng]
|
[bng]
|
[bng]

If you clicked the object chain above, you get an immediate chain reaction 3 objects deep.

Now, if you extended that chain so that there were 1001 [bng] objects-- each with a single connection to the next-- you'd get a stack overflow error because it exceeds the limit Pd puts on how deeply a single message is allowed to flow in an immediate chain reaction.

The way Pd is coded, these simple examples actually end up using recursive function calls in C. Using recursion with too many levels can end up overflowing the stack and cause a crash, which is why you get that particular error message. (1000 may be too small a number nowadays, but recursion is complex and I'm not sure how one would come up with a better number.)

When you use recursion explicitly in a Pd diagram-- that is, hooking an object's outlet to an inlet that is back up the chain-- you make it way more likely to hit this particular error. You can imagine that your recursive diagram ends up "unrolling". That is, if you have a chain of 4 objects where the bottom one connects back into the top one and recursing 251 times, you're going to hit a stack limit and get the error.

The [until] object solves this "explicit" recursion problem-- it sends a bang to its outlet, and when the resulting immediate chain reaction has finished it sends another until the specified limit is reached.

So if you have this:

[10000(
|
[until]
|
[bng]
|
[bng]
|
[bng]

You go two objects deep for the bang coming out of [until], then three more for the first time through the chain of [bng] objects. So now we're 5 deep. But once the first iteration of the [bng] objects are done they unroll, so we're back to being just 2 levels deep. Then the [until] object starts its next iteration, which goes 3 more deep, then unrolls. It can do this indefinitely, which is why [until] is preferred over using recursive object chains in Pd.

This is very interesting. It totally get it now.
Thanks a lot for this long and clear explanation.

@jancsika

I've been thinking more about recursion and the "unrolling" problem that leads to memory spikes. Am I right in thinking that recursive patches (ie. patches where an outlet is attached to a hot inlet upstream) can be equally as efficient as using [until] if you get the trigger order right?

Here's an example:

In the first example, PD has to stack all 100 calculations before prints out any one of them. But in the second it can print and "forget" them one by one. Is the second version theoretically just as efficient as using [until]?

(Note that creating a line of numbers is not the problem I'm trying to solve here. I'm just using a simple example for illustration.)

@LiamG You will still get stack overflow if you choose a higher number for moses. I believe that is because pd puts the floats on a stack before actually printing and deleting them and tries to finish the computation first until the stack is full.

Yes, as Jonathan explained, PD tracks recursive operations and spits out the error message when a certain limit is reached, in order to avoid stack overflows. Obviously there are better ways to perform the operation above, but I'm still wondering if the second one is theoretically just as efficient as using [until].

@LiamG Yes, both of your versions work (the first one prints reverse order). But if you try it with [moses 1000], both will likely break. Whereas with [until] you can easily count up 1000 times:

That's right. What I want to know is: do the two options that you have described use memory in the same way, or is one more efficient than the other? In other words, I want to know if the "stack overflow" message is pointing to an actual spike in the memory.

@LiamG i don't know that for sure, but i would guess so. But the stack is so small that it should not be relevant in terms of computer ram. What i observed once is that the print function seems to be internally disconnected to the actual computation so that the printout on the console can occur with some actual delay.

@LiamG The "stack overflow" message results from a simple counter-- when you send to an outlet, a generic outlet handling function 1) increments the counter, 2) forwards the outgoing message to the objects that are connected to that outlet, and finally 3) decrements the counter.

In the first chain you've initially got three objects involved in the recursive call, each of which adds an iteration to the loop. When the incremented value finally hits the target, you print out the value in order from last iteration to the first.

In the second chain, you've got four objects, but one returns before the leftmost "f" of the trigger starts the next iteration of recursion. So in that case you print out starting from the smallest value to the greatest.

As for which is more efficient, I'm not sure.