Any tips on debugging techniques for out of memory?

gloomyandy

@DonStauffer In some situations RRF will need to allocate more memory to allow it to process moves. What does M122 report after starting a print and before it runs out of memory?

DonStauffer

@gloomyandy I could embed an M122 in the print file and find out. How far before the failure point, or do you want more than one? I can pinpoint where the failure happens in the file and insert the command before that.

DonStauffer

@gloomyandy Interesting development:

I noticed there's a whole part of the tree of one of my globals which only ever gets used in the job starting script. So at the end of that, I set the root of that branch to null, apparently thus freeing all of the 15 integers below it (and perhaps a bit of array overhead). This appears to have gotten me past the spot it rebooted before, and I've been through several tool changes since. Need some adjustment to the tool change - red plastic shows well on black! But it's running.

And the LEDs are performing basically as intended. Also probably a little fine tuning needed there, but actually, pretty decent.

This project has taken about 15 months of full time work, actually. I really, really wanted this to work.

DonStauffer

@gloomyandy

The LEDs did a nice job of graphing heating & cooling of tools & bed, leveling & probing the bed (with error values), and showing percent progress (2 digits!) through the print. Everything to do with tool 0 was green; tool 1 was blue. They had 3 intensities, for off, standby and active. The bed is yellow. Probing has its own colors.

The LEDs handled the transition from printing to tool change and back again well. The worst thing about them is a slight lag, maybe about a second or so. So you can look at the web interface and see 60%, but the LEDs still show 58%. Acceptable.

In the process I rewrote all my routines, so they still need fine tuning as far as retraction & priming, and of course, tool change ooze. I'll get there. I did it once before.

I think I know the object model by heart by now!

dc42

@DonStauffer how about getting your macro to also echo the amount of free memory at various points? It's available in the object model.

DonStauffer

@dc42 OK, about my "knowing the object model by heart" ... except that!

boards[0].freeRam?

dc42

@DonStauffer yes that's the one. It's the same as Never Used RAM in the M122 report.

DonStauffer

@dc42 Thanks. I'll look at it, though my immediate problem seems to be solved. But undoubtedly I'll run out of memory again at some point. But I'm pretty happy now.

dc42

@DonStauffer I'd be interested to know whether you have less or more free memory after running a print with 3.6 alpha compared to 3.5.2.

DonStauffer

@dc42 I'm a little afraid to install an alpha though. Is reverting easy?

dc42

@DonStauffer assuming you are running in standalone mode not SBC mode, reverting is very easy. To make it even easier you can install just the 3.6 Duet2CombinedFirmware.bin file and continue running DWC 3.5.2. You can ignore the warning about incompatible software versions in this case.

gloomyandy

@DonStauffer Just running M122 in the console should provide an idea of what the memory situation is before the print starts and after your startup code has run.

DonStauffer

@gloomyandy Echoing boards[0].freeRam at various places throughout a small, 15 minute, 8-layer print job with tool changes, it looks like this (no units given; bytes?):

Machine turn-on: 8544
Partway through job startup: 8520
Before purge in job startup: 4000
After purge: 920
Beginning of Layer 1: 920
Layer 2: 848
Layer 3: 848
Layer 4: 848
Layer 5: 848
Layer 5 before tool change: 464
Layer 6: 72
Layer 6 before tool change: 72
Layer 7: 72
Layer 7 before tool change: 72
Layer 8: 72
Layer 8 before tool change: 72
After job finished: 72

It seems like I'm grazing the treetops. Most of my global data is created from config.g, so I'm not really sure why my startup script uses so much memory compared to the job proper.

DonStauffer

@dc42 OK, I'll do that. I have the details on free memory echoing throughout a job and I recorded them. So I'll have something to compare it with.

DonStauffer

@dc42 Same GCode file as the free RAM I posted for RRF 3.5.2, but after installing 3.6 alpha 2:

After upgrade: 22624
I couldn't believe that so I did an M999 and looked again: 22600 (where are my 24 bytes?)
Partway through job startup: 22600
Before purge in job startup: 16840
After purge: 14472
Beginning of Layer 1: 14472
Layer 2: 14472
Layer 3: 14304
Layer 4: 14280
Layer 5: 14280
Layer 5 before tool change: 13512
Layer 6: 13096
Layer 6 before tool change: 13096
Layer 7: 13096
Layer 7 before tool change: 13096
Layer 8: 13096
Layer 8 before tool change: 13096
After job finished: 13096

What wizardry is this?!

oliof

@DonStauffer probably the work noted at https://github.com/Duet3D/RepRapFirmware/blob/bd8d9119d49ba425864f2280b005875b368746c0/src/Todo-next.txt#L25