Reboots/crashes - RRF ≤3.5.0-rc1

Exerqtor

@NeoDue said in Reboots/crashes - RRF 3.5.0-rc1:

@Exerqtor Weird. I had connection issues with the previous versions with connections dropping sometimes and especially the Wifi module frequently only connecting after 10+ minutes (could be partially cured by turning wifi in config.pro on at the end and after an additional 10s G4 delay in front of the command). b6 has solved these for me.
One hint: if such an error happens, you might get brief error messages on the PanelDue which will not show up on DWC. But that is a different topic...

I can live with loosing connection over it crashing though thats for sure.

But i've managed to get three instances connected now (2x chrome, 1x orca slicer) and it's standing there idle. So lets see if anything happens.

Exerqtor

@dc42 It's crashing on 3.5.0-rc2 & 2.1b6 as well:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.2 (2023-12-15 13:33:26) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
Used output buffers: 13 of 40 (40 max)
=== RTOS ===
Static ram: 102812
Dynamic ram: 122028 of which 12 recycled
Never used RAM 13708, free system stack 180 words
Tasks: NETWORK(2,nWait,24.5%,230) HEAT(3,nWait,0.0%,336) Move(4,nWait,0.0%,351) CanReceiv(6,nWait,0.0%,776) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,67.7%,670) IDLE(0,ready,6.2%,29) AIN(4,delaying,0.8%,264), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 01:43:42 ago, cause: software
Last software reset at 2023-12-16 19:41, reason: HardFault invState, Platform spinning, available RAM 12548, slot 1
Software reset code 0x4060 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20011f88 Task NETW Freestk 482 ok
Stack: 000001af 00000002 200014e8 00000000 20032e5b 0009edb1 00000000 600f0000 00000000 00000000 00000000 00000000 20031c74 00000800 20035968 2002c0e0 20018660 2002bf7d 20018660 2001e868 0002ffc3 00000000 00000000 00000000 20012038 00000014 ffffffff
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 6223000, completed 6223000, timed out 0, errs 0
MCU temperature: min 35.3, current 35.7, max 35.7
Supply voltage: min 24.0, current 24.1, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/22, heap memory allocated/used/recyclable 2048/1908/1608, gc cycles 11636
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 0, ifcnt 59, reads 187, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 4, read errors 0, write errors 0, ifcnt 59, reads 187, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 2, read errors 0, write errors 0, ifcnt 198, reads 187, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 199, reads 187, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 197, reads 187, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2023-12-16 21:24:46
Cache data hit count 4294967295
Slowest loop: 7.71ms; fastest: 0.18ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 3.9ms, write time 4.1ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 0
Extruder 0 sensor: no filament
=== CAN ===
Messages queued 31, received 71, lost 0, errs 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 3, free buffers 26 (min 26), ts 17/17/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 21.20ms; fastest: 0.06ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 4 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1beta6
MAC address c4:5b:be:ce:91:93
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42812
WiFi IP address 192.168.10.x
Signal strength -46dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

NeoDue

@Exerqtor damn. Would have been nice if that would have solved itself with that firmware! Then it seems I cannot help... my Duet still stubbornly refuses to repeat the crashes I had.

Exerqtor

@NeoDue Well lets hope the issues are gone for you then Fingers crossed!

Still keeps acting up here at least 🤣

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.2 (2023-12-15 13:33:26) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
Used output buffers: 1 of 40 (40 max)
=== RTOS ===
Static ram: 102812
Dynamic ram: 121992 of which 0 recycled
Never used RAM 13756, free system stack 182 words
Tasks: NETWORK(1,ready,22.6%,228) HEAT(3,nWait,0.0%,353) Move(4,nWait,0.0%,357) CanReceiv(6,nWait,0.0%,799) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,70.1%,670) IDLE(0,ready,5.7%,29) AIN(4,delaying,0.8%,264), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:15:24 ago, cause: software
Last software reset at 2023-12-17 09:04, reason: HardFault invState, Gcodes spinning, available RAM 13756, slot 1
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20011f88 Task NETW Freestk 482 ok
Stack: 000001ae 00000002 200014e8 00000000 20032e3a 0009edb1 00000000 600f0000 00000000 00000000 00000000 00000000 20031c5c 00000800 2002c0e0 2002c0e0 00000001 2002bf7d 20018660 2001e868 0002ffc3 00000000 00000000 00000000 20012038 00000014 b5dd8a35
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 925347, completed 925347, timed out 0, errs 0
MCU temperature: min 35.9, current 36.3, max 39.2
Supply voltage: min 23.2, current 24.1, max 25.9, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/22, heap memory allocated/used/recyclable 2048/1300/1000, gc cycles 1724
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 101, reads 48658, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 4, read errors 0, write errors 1, ifcnt 101, reads 48658, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 240, reads 48658, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 241, reads 48658, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 239, reads 48658, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2023-12-17 09:20:19
Cache data hit count 1593565448
Slowest loop: 10.70ms; fastest: 0.13ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 7.9ms, write time 3.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 0
Extruder 0 sensor: no filament
=== CAN ===
Messages queued 8334, received 19000, lost 0, errs 1, boc 0
Longest wait 2ms for reply type 6053, peak Tx sync delay 268, free buffers 26 (min 25), ts 4625/4624/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 6.32ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 4 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1beta6
MAC address c4:5b:be:ce:91:93
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 36512
WiFi IP address 192.168.10.x
Signal strength -44dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

And another one:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.2 (2023-12-15 13:33:26) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
Used output buffers: 1 of 40 (40 max)
=== RTOS ===
Static ram: 102812
Dynamic ram: 122016 of which 0 recycled
Never used RAM 13732, free system stack 182 words
Tasks: NETWORK(1,ready,23.2%,209) HEAT(3,nWait,0.0%,353) Move(4,nWait,0.0%,357) CanReceiv(6,nWait,0.0%,799) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,67.5%,670) IDLE(0,ready,7.7%,29) AIN(4,delaying,0.8%,264), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:07:56 ago, cause: software
Last software reset at 2023-12-17 13:31, reason: HardFault invState, Gcodes spinning, available RAM 13756, slot 0
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x00487803 BFAR 0xe000ed38 SP 0x20011f88 Task NETW Freestk 482 ok
Stack: 000001b0 00000002 200014e8 00000000 ffffffff 0009edb1 00000000 600f0000 00000000 00000000 00000000 00000000 20031c5c 00000800 20035950 2002c0e0 20018660 2002bf7d 20018660 2001e868 0002ffc3 00000000 00000000 00000000 20012038 00000014 08935091
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 477040, completed 477038, timed out 0, errs 0
MCU temperature: min 35.7, current 35.9, max 36.2
Supply voltage: min 24.0, current 24.1, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/23, heap memory allocated/used/recyclable 2048/856/544, gc cycles 868
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 0, ifcnt 129, reads 1286, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 4, read errors 0, write errors 0, ifcnt 129, reads 1286, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 2, read errors 0, write errors 0, ifcnt 12, reads 1287, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 1286, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 11, reads 1286, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2023-12-17 13:39:19
Cache data hit count 830112743
Slowest loop: 12.71ms; fastest: 0.14ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 4.4ms, write time 3.9ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 0
Extruder 0 sensor: no filament
=== CAN ===
Messages queued 219, received 502, lost 0, errs 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 6, free buffers 26 (min 26), ts 122/122/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 20.59ms; fastest: 0.06ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 4 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1beta6
MAC address c4:5b:be:ce:91:93
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 43160
WiFi IP address 192.168.10.x
Signal strength -47dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

dc42

@Exerqtor unfortunately we had to release rc2 before we found a fix for the issue that you are experiencing. I will return to it soon.

Exerqtor

@dc42
No problem, I fully understand that this one little issue can't postpone a whole release.

The reversing extruder issue I'm having is more of a burning matter (for me) however.

Exerqtor

Been running 3.5.0-b4 without any crash issues since last post.

However this morning as i walked by the printer i saw it had restarted. Checked the debug log and it had restarted once in the middle of the night while idle.

Now I've only had Orca Slicer open since that's what seemed most stable.

I have no idea what value M122 report(s) will have but here they are at least:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-beta.4 (2023-06-08 23:40:14) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
Used output buffers: 1 of 40 (40 max)
=== RTOS ===
Static ram: 102996
Dynamic ram: 126916 of which 12 recycled
Never used RAM 9012, free system stack 138 words
Tasks: NETWORK(1,ready,149.0%,233) HEAT(3,nWait,0.2%,329) Move(4,nWait,0.0%,293) CanReceiv(6,nWait,0.4%,557) CanSender(5,nWait,0.0%,337) CanClock(7,delaying,0.1%,351) TMC(4,nWait,5.2%,80) MAIN(1,running,157.9%,704) IDLE(0,ready,6.7%,29) AIN(4,delaying,6.4%,266), total 325.9%
Owned mutexes:
=== Platform ===
Last reset 05:28:34 ago, cause: software
Last software reset at 2023-12-31 04:31, reason: HardFault invState, FilamentSensors spinning, available RAM 2712, slot 2
Software reset code 0x406d HFSR 0x40000000 CFSR 0x00020000 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20011fa8 Task NETW Freestk 482 ok
Stack: 000001a8 00000002 200014ec 00000000 ffffffff 0009d43d 00000000 600f0000 00000000 00000000 00000000 00000000 20031b54 00000800 20036060 2002c558 20018670 2002c3f5 20018670 2001e920 0002fe6f 00000000 00000000 00000000 20012058 00000014 b5dd8a35
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 19715096, completed 19715095, timed out 0, errs 0
MCU temperature: min 34.4, current 37.8, max 38.2
Supply voltage: min 3.9, current 24.1, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/1680/1248, gc cycles 4141
Events: 1 queued, 1 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 29, reads 54540, writes 29, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 28, reads 54541, writes 28, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 82, reads 54550, writes 18, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 78, reads 54550, writes 17, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 17, reads 54552, writes 17, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2023-12-31 10:00:19
Cache data hit count 4294967295
Slowest loop: 23.66ms; fastest: 0.12ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 6.8ms, write time 4.6ms, max retries 0
=== Move ===
DMs created 83, segments created 11, maxWait 19423330ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
=== DDARing 0 ===
Scheduled moves 19, completed 19, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is doing "G4 P250                                                                      " in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000807
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
Extruder 0 sensor: no filament
=== CAN ===
Messages queued 177457, received 404187, lost 0, boc 0
Longest wait 3ms for reply type 6053, peak Tx sync delay 5719, free buffers 18 (min 17), ts 98575/98574/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 24.79ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
= WiFi =
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
Firmware version 2.1beta6
MAC address c4:5b:be:ce:91:93
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42968
WiFi IP address 192.168.10.x
Signal strength -50dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

M122 B121
Diagnostics for board 121:
Duet TOOL1LC rev 1.1 or later firmware version 3.5.0-beta.4 (2023-06-08 16:22:30)
Bootloader ID: SAMC21 bootloader version 2.4 (2021-12-10)
All averaging filters OK
Never used RAM 1752, free system stack 142 words
Tasks: Move(3,nWait,0.0%,175) HEAT(2,nWait,3.5%,79) CanAsync(5,nWait,0.0%,54) CanRecv(3,nWait,0.1%,77) CanClock(5,nWait,0.1%,66) ACCEL(3,nWait,0.0%,53) TMC(2,nWait,23.2%,57) MAIN(1,running,35.3%,344) IDLE(0,ready,0.0%,27) AIN(2,delaying,37.7%,142), total 100.0%
Last reset 05:28:52 ago, cause: power up
Last software reset data not available
Driver 0: pos 0, 568.8 steps/mm, standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 35978, writes 12, timeouts 7, DMA errors 0, CC errors 0, failedOp 0x6a, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0, ebfmin 0.00, ebfmax 0.00
Peak sync jitter -3/10, peak Rx sync delay 217, resyncs 0/0, no timer interrupt scheduled
VIN voltage: min 24.2, current 24.3, max 24.3
MCU temperature: min 39.3C, current 39.8C, max 40.2C
Last sensors broadcast 0x00000012 found 2 139 ticks ago, 0 ordering errs, loop time 1
CAN messages queued 404551, send timeouts 0, received 177623, lost 0, free buffers 18, min 18, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: LIS3DH, status: 00
I2C bus errors 0, naks 3, other errors 0
=== Filament sensors ===
Interrupt 5726621 to 0us, poll 4 to 1618us
Driver 0: ok

Exerqtor

@dc42 I guess we can pick this up here for sake of continuity.

@dc42 said in 1LC - extruder randomly stopping/reversing - ≤3.5.0-rc1:

@Exerqtor thanks, I'm glad to get to the bottom of this. The problem was caused when the GCode called for a very short move that would take less than 2 microseconds to execute. When the move time was truncated to integer to send over CAN, the move ended up as taking zero time yet having nonzero extrusion. This resulted in a floating point divide-by-zero error. I changed the floating point library on the 1LC to a faster but less IEEE-conformant one between 3.4beta4 and 3.4rc1, and I think this caused a change in behaviour after the divide by zero occurred.

Now that we have solved this one, I'd like to revisit the issue with WiFi connection causing crashes. I note that you are doing a lot of things with an LED string connected to the Duet 3 Mini dedicated LED output. Please try changing enough of that LED macro code so that M150 commands to that output don't happen; then try running with DWC connected over WiFi. I am wondering whether the DMA to that port when you use M150 could be interfering with the WiFi DMA.

I'm quite sure I've disabled the whole LED control a couple times to try if that made a difference, and it didn't (if my memory serves me right).

I've also put in a check for the state of the LEDs, so that it only sends M150 commands when the requested LED state differs from the active one. And the led state don't change when the printer is idle, or during a print job (unless in the start phase and if it gets paused etc.).

- I just added echo's after both the M150 commands in sb_leds.g so that i can track every time they get sent, and they only get called when they are supposed to. And it's not right before the crashes happened.

Exerqtor

I changed it to be M118 L1 messages instead so that it would be written in the log, and it don't send M150 other than when i change the string in sb_leds (as intended).

Anything else you want me to test out specificaly @dc42?

Exerqtor

Happy easter everyone!

@dc42
I haven't really had any issues with crashes in a while now, since "we" finally got the printer working again after patching out the extrusion bug I've had the printer going more or less non-stop and only with one instance of OrcaSlicer open and checking in on my phone (android & chrome browser) every now and then to not provoce any crashes.

However when I got to the printer this morning i saw it had rebooted during the night for some reason!

Here is the last entries to the eventlog (had to switch to event logging since debug it putting out SO MUCH DATA since a little while back):

2024-03-29 13:56:58 [warn] Started printing file 0:/gcodes/O*********r_0.2mm_ABS_0.4n_17m37s.gcode
2024-03-29 14:01:54 [warn] M150 sent for logo : yellow
2024-03-29 14:01:54 [warn] M150 sent for nozzle : yellow
2024-03-29 14:02:14 [warn] M150 sent for logo : teal
2024-03-29 14:02:14 [warn] M150 sent for nozzle : bright_white
2024-03-29 14:02:28 [warn] M150 sent for logo : green
2024-03-29 14:02:28 [warn] M150 sent for nozzle : bright_white
2024-03-29 14:02:42 [warn] 9 points probed, min error -0.014, max error 0.005, mean -0.005, deviation 0.005
Height map saved to file 0:/sys/heightmap.csv
2024-03-29 14:02:43 [warn] M150 sent for logo : dim_gray
2024-03-29 14:02:43 [warn] M150 sent for nozzle : dim_red
2024-03-29 14:02:44 [warn] M150 sent for logo : yellow
2024-03-29 14:02:44 [warn] M150 sent for nozzle : yellow
2024-03-29 14:03:09 [warn] M150 sent for logo : light_blue
2024-03-29 14:03:09 [warn] M150 sent for nozzle : bright_white
2024-03-29 14:03:20 [warn] M150 sent for logo : red
2024-03-29 14:03:20 [warn] M150 sent for nozzle : bright_white
2024-03-29 14:24:21 [warn] Finished printing file 0:/gcodes/O*********r_0.2mm_ABS_0.4n_17m37s.gcode, print time was 0h 27m
2024-03-29 14:24:26 [warn] M150 sent for logo : dim_gray
2024-03-29 14:24:26 [warn] M150 sent for nozzle : dim_red
2024-03-29 20:05:42 [warn] HTTP client 192.168.10.xx disconnected
2024-03-30 09:31:09 [warn] HTTP client 192.168.10.xxx login succeeded (session key 2606324820)
2024-03-30 09:44:34 [warn] HTTP client 192.168.10.xx login succeeded (session key 2283911350)
2024-03-30 17:13:44 [warn] HTTP client 192.168.10.xx login succeeded (session key 702406583)
power up + 00:00:03 [info] Event logging started at level warn
power up + 00:00:03 [info] Running: Duet 3 Mini5plus WiFi: 3.5.0-rc.3+5 (2024-03-27 11:34:04)
power up + 00:00:04 [warn] WiFi module started
power up + 00:00:04 [warn] M150 sent for logo : n/a
power up + 00:00:04 [warn] M150 sent for nozzle : n/a
power up + 00:00:05 [warn] M150 sent for logo : n/a
power up + 00:00:05 [warn] M150 sent for nozzle : n/a
power up + 00:00:09 [warn] WiFi module is connected to access point RV32-IOT2G, IP address 192.168.10.x
power up + 00:00:10 [warn] HTTP client 192.168.10.xx login succeeded (session key 1857804272)
2024-03-31 03:37:09 [warn] Date and time set at power up + 00:00:10

And some reports:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.3+5 (2024-03-27 11:34:04) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ
Used output buffers: 18 of 40 (31 max)
=== RTOS ===
Static ram: 103264
Dynamic ram: 124000 of which 12 recycled
Never used RAM 11284, free system stack 180 words
Tasks: NETWORK(2,nWait 7,245.0%,231) HEAT(3,nWait 6,0.4%,326) Move(4,nWait 6,0.0%,341) CanReceiv(6,nWait 1,0.9%,771) CanSender(5,nWait 7,0.0%,337) CanClock(7,delaying,0.1%,349) TMC(4,nWait 6,12.7%,102) MAIN(1,running,113.1%,667) IDLE(0,ready,13.9%,30) AIN(4,delaying,13.1%,260), total 399.3%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 08:29:06 ago, cause: software
Last software reset at 2024-03-31 02:36, reason: HardFault invState, Platform spinning, available RAM 6444, slot 2
Software reset code 0x4060 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20012020 Task NETW Freestk 495 ok
Stack: 2002c5f8 20030ba8 200014e8 00000000 ffffffff 0002ffb1 00000000 60010000 00000000 00000000 00000000 00000000 200120a0 00000014 b5dd8a35 00000002 e8610050 640aa8c0 080001a8 00000001 000347f9 2002c3c0 2002c3c0 00000007 0002f72d 20011744 00000000
Error status: 0x00
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 30546654, completed 30546654, timed out 0, errs 0
MCU temperature: min 33.8, current 35.5, max 37.6
Supply voltage: min 3.8, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/32, heap memory allocated/used/recyclable 2048/664/248, gc cycles 6669
Events: 1 queued, 1 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 34825, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 34825, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 34825, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 34823, writes 13, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x51
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 34824, writes 13, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x6c
Driver 5: not present
Driver 6: not present
Date/time: 2024-03-31 12:06:09
Cache data hit count 4294967295
Slowest loop: 27.00ms; fastest: 0.12ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 7.8ms, write time 45.9ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.4
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is doing "G4 P250" in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 0
Extruder 0 sensor: no filament
=== CAN ===
Messages queued 274931, received 626212, lost 0, errs 369, boc 0
Longest wait 2ms for reply type 6053, peak Tx sync delay 8721, free buffers 26 (min 25), ts 152733/152732/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 10.10ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1beta7
MAC address c4:5b:be:ce:91:93
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 36388
WiFi IP address 192.168.10.x
Signal strength -46dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0
M122 B121
Diagnostics for board 121:
Duet TOOL1LC rev 1.1 or later firmware version 3.5.0-rc.3+5 (2024-03-27 11:29:09)
Bootloader ID: SAMC21 bootloader version 2.8 (2023-07-25)
All averaging filters OK
Never used RAM 3960, free system stack 136 words
Tasks: Move(3,nWait 7,0.0%,127) HEAT(2,nWait 6,6.0%,107) CanAsync(5,nWait 4,0.0%,55) CanRecv(3,nWait 1,0.2%,71) CanClock(5,nWait 1,0.3%,59) ACCEL(3,nWait 6,0.0%,53) TMC(2,nWait 6,53.8%,53) MAIN(1,running,255.9%,315) IDLE(0,ready,0.0%,27) AIN(2,delaying,76.9%,112), total 393.2%
Owned mutexes:
Last reset 08:29:46 ago, cause: power up
Last software reset at 2024-03-12 16:55, reason: StackOverflow, available RAM 2968, slot 0
Software reset code 0x0100 ICSR 0x0042600e SP 0x20007f34 Task Move Freestk 3342 ok
Stack: 20004a80 20004ab4 0001cf33 20004c98 20004938 00000000 0001c011 20003320 fffffffd a5a5a5a5 00000000 20007f8c 00000000 20007f8c 0001cc97 00000000 200017c4 20001748 0001c4d7 20001748 200017c4 00000032 454c4449 00022700 0001ac77 200018e0 200018e0
Driver 0: pos 0, 568.8 steps/mm, standstill, SG min 0, read errors 0, write errors 0, ifcnt 11, reads 23207, writes 11, timeouts 2, DMA errors 0, CC errors 0, failedOp 0x6a, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, segs 0, step errors 0, maxLate 0 maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0, ebfmin 0.00 max 0.00
Peak sync jitter -4/10, peak Rx sync delay 219, resyncs 0/0, no timer interrupt scheduled
VIN voltage: min 24.5, current 24.7, max 24.8
MCU temperature: min 40.5C, current 44.4C, max 44.5C
Last sensors broadcast 0x00000012 found 2 113 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 627040, send timeouts 0, received 275294, lost 0, errs 0, boc 0, free buffers 18, min 18, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: LIS3DH, status: 00
I2C bus errors 0, naks 3, contentions 0, other errors 0
=== Filament sensors ===
Interrupt 5726621 to 0us, poll 2 to 1300us
Driver 0: ok

dc42

I will be looking into this issue again in the near future. Please post M122 reports following any instances of Hard Fault resets on Duet 3 Mini WiFi running firmware 3.5.3.

dc42

All, please see https://forum.duet3d.com/topic/36689/random-hard-fault-resets-on-duet-3-mini-wifi and provide any additional reports there. I am locking this thread.