Duet 3 mini5+ nightly restarts 3.5.2

dc42

@Chriss this issue only affects Duet 3 Mini WiFi. I would need to see a M122 report after the axes have unexpectedly become not homed before I can give an opinion on whether it is the same issue.

Chriss

@dc42 here you are:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 6 of 40 (40 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 122336 of which 0 recycled
Never used RAM 12992, free system stack 202 words
Tasks: NETWORK(1,ready,10.6%,204) HEAT(3,nWait 6,0.0%,350) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,799) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,1.4%,102) MAIN(1,running,85.3%,665) IDLE(0,ready,1.7%,30) AIN(4,delaying,0.9%,260), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:00:15 ago, cause: software
Last software reset at 2024-06-01 05:35, reason: HardFault imprec, Gcodes spinning, available RAM 12968, slot 2
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00000400 ICSR 0x00487803 BFAR 0xe000ed38 SP 0x20012008 Task NETW Freestk 489 ok
Stack: 2002c5e0 20031f80 200014e8 00000000 2003311d 0002ff99 0002feb0 610f6000 20035c80 2002c5c8 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 b5ddcef1 00000002 a31a0050 0801a8c0 08000161 00000003 000347e1
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 15819, completed 15819, timed out 0, errs 0
MCU temperature: min 31.1, current 31.8, max 31.8
Supply voltage: min 24.3, current 24.3, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/11, heap memory allocated/used/recyclable 2048/332/160, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 126, reads 1351, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 176, read errors 0, write errors 1, ifcnt 16, reads 1348, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 48, read errors 0, write errors 1, ifcnt 13, reads 1348, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 1353, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 126, reads 1351, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 1, ifcnt 126, reads 1351, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 126, reads 1350, writes 13, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-06-01 05:36:04
Cache data hit count 28879827
Slowest loop: 9.23ms; fastest: 0.17ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 3.3ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 156, received 338, lost 0, errs 1, boc 0
Longest wait 3ms for reply type 6053, peak Tx sync delay 177, free buffers 26 (min 25), ts 77/76/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 7.01ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(2)
HTTP sessions: 3 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 40772
WiFi IP address 192.168.1.69
Signal strength -59dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 3 0 0 0 0 0

I'm more than happy to start a new thread if we see here a other problem with my Mini5+ if they are unrelated. (It seems to me that the board "rebooted" during the night again.)

Cheers, Chriss

ProteanReverie

Had another restart this morning

M122 (with P500 S0 enabled)

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: 65LYX-8Q6KL-K65J0-409N0-NP02Z-ZFK3T
Used output buffers: 3 of 40 (18 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 119804 of which 12 recycled
Never used RAM 15512, free system stack 156 words
Tasks: NETWORK(2,nWait 7,98.0%,218) HEAT(3,nWait 6,0.1%,372) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,817) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.1%,348) TMC(4,nWait 6,5.8%,102) MAIN(1,running,191.6%,951) IDLE(0,ready,0.1%,30) AIN(4,delaying,6.1%,260), total 301.7%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 05:33:21 ago, cause: software
Last software reset at 2024-06-03 04:20, reason: HardFault bfarValid precise, Gcodes spinning, available RAM 15092, slot 0
Software reset code 0x0063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00000803 BFAR 0x0000001c SP 0x20011ff0 Task NETW Freestk 483 ok
Stack: 2002c5e0 20031578 200014e8 00000000 20032707 0002ff99 0002feac 610f0000 0002ff85 00000000 00000000 00000000 20031584 00000800 20035278 2002c5c8 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 20468121
Error status: 0x00
MCU revision 3, ADC conversions started 20001182, completed 20001182, timed out 0, errs 0
MCU temperature: min 37.4, current 40.2, max 40.8
Supply voltage: min 24.0, current 24.1, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 22, reads 4091, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 22, reads 4091, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 22, reads 4090, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 22, reads 4090, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 22, reads 4091, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-06-03 09:54:05
Cache data hit count 4294967295
Slowest loop: 7.00ms; fastest: 0.16ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.7ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000000
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 100006, received 160012, lost 0, errs 1, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 360, free buffers 26 (min 26), ts 100006/100005/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 6.33ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:d0
Module reset reason: Power up, Vcc 3.39, flash size 2097152, free heap 42784
WiFi IP address 10.1.10.46
Signal strength -49dBm, channel 1, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

log from the full weekend

2024-06-01 09:38:40 [warn] Error: WiFi module reported: Lost connection, auto reconnecting
2024-06-01 09:38:44 [warn] Error: WiFi module reported: Auto reconnect succeeded
2024-06-02 01:37:59 [warn] Error: WiFi module reported: Lost connection, auto reconnecting
2024-06-02 01:38:02 [warn] Error: WiFi module reported: Auto reconnect succeeded
2024-06-02 04:14:01 [warn] HTTP client 10.1.10.9 login succeeded (session key 3101634077)
2024-06-02 17:36:37 [warn] Error: WiFi module reported: Lost connection, auto reconnecting
2024-06-02 17:36:40 [warn] Error: WiFi module reported: Auto reconnect succeeded
2024-06-03 00:25:59 [warn] HTTP client 10.1.10.9 login succeeded (session key 1063118671)

Chriss

Hi @dc42, I have almost every day a restart of my Mini 5+

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 2 of 40 (40 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 122360 of which 0 recycled
Never used RAM 12968, free system stack 188 words
Tasks: NETWORK(1,ready,113.0%,216) HEAT(3,nWait 6,0.2%,326) Move(4,nWait 6,0.0%,341) CanReceiv(6,nWait 1,0.4%,774) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.1%,348) TMC(4,nWait 6,10.1%,102) MAIN(1,running,90.7%,665) IDLE(0,ready,0.2%,30) AIN(4,delaying,6.0%,260), total 220.6%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 09:16:21 ago, cause: software
Last software reset at 2024-06-05 19:51, reason: HardFault imprec, Gcodes spinning, available RAM 12968, slot 1
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00000400 ICSR 0x00489803 BFAR 0xe000ed38 SP 0x20012008 Task NETW Freestk 489 ok
Stack: 2002c610 20031f98 200014e8 00000000 2003312d 0002ff99 0002feb0 610f6000 200352c8 2002c5f8 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 b5ddcef1 00000102 e52e0050 0801a8c0 08000161 00000003 000347e1
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 33382335, completed 33382335, timed out 0, errs 0
MCU temperature: min 31.1, current 31.3, max 32.2
Supply voltage: min 24.3, current 24.3, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/11, heap memory allocated/used/recyclable 2048/1420/1248, gc cycles 92
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 20, reads 20006, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 176, read errors 0, write errors 1, ifcnt 253, reads 20002, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 48, read errors 0, write errors 1, ifcnt 250, reads 20001, writes 16, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x51
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 117, reads 20007, writes 10, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x51
Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 20, reads 20005, writes 13, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x6f
Driver 5: standstill, SG min 0, read errors 0, write errors 1, ifcnt 20, reads 20005, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 20, reads 20005, writes 13, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-06-06 05:07:51
Cache data hit count 4294967295
Slowest loop: 15.11ms; fastest: 0.17ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 8.3ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 300454, received 667687, lost 0, errs 0, boc 0
Longest wait 2ms for reply type 6053, peak Tx sync delay 273, free buffers 26 (min 25), ts 166909/166908/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 6.99ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(2)
HTTP sessions: 3 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 37328
WiFi IP address 192.168.1.69
Signal strength -61dBm, channel 6, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 3 0 0 0 0 0 0

Chriss

Hi @dc42 ,

I have the next reset in the middle of a print:

Connected to 192.168.1.69.
Escape character is '^]'.
M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 2 of 40 (40 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 123888 of which 0 recycled
Never used RAM 11440, free system stack 198 words
Tasks: NETWORK(1,ready,14.1%,237) HEAT(3,nWait 6,0.0%,365) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,799) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,1.4%,111) MAIN(1,running,82.7%,665) IDLE(0,ready,0.8%,30) AIN(4,delaying,0.9%,256), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:00:33 ago, cause: software
Last software reset at 2024-06-06 16:43, reason: HardFault imprec, Platform spinning, available RAM 9916, slot 0
Software reset code 0x4060 HFSR 0x40000000 CFSR 0x00000400 ICSR 0x00446803 BFAR 0xe000ed38 SP 0x20012008 Task NETW Freestk 489 ok
Stack: 2002c648 20031fc8 200014e8 00000000 2003316d 0002ff99 0002feb0 610f6000 20034928 2002c630 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 b5ddcef1 00000202 bd180050 0801a8c0 08000161 00000007 000347e1
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 33839, completed 33838, timed out 0, errs 0
MCU temperature: min 46.6, current 47.3, max 47.8
Supply voltage: min 24.3, current 24.4, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/15, heap memory allocated/used/recyclable 2048/1396/1184, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 132, reads 3019, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 14, read errors 0, write errors 1, ifcnt 9, reads 3016, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 14, read errors 0, write errors 1, ifcnt 7, reads 3016, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 65, reads 3022, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 133, reads 3019, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 2, read errors 0, write errors 1, ifcnt 134, reads 3019, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 132, reads 3019, writes 13, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-06-06 16:44:29
Cache data hit count 63736334
Slowest loop: 11.00ms; fastest: 0.17ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 4.8ms, write time 1.3ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 319, received 711, lost 0, errs 1, boc 0
Longest wait 2ms for reply type 6013, peak Tx sync delay 162, free buffers 26 (min 25), ts 168/167/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 6.66ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(2)
HTTP sessions: 3 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.37, flash size 2097152, free heap 37840
WiFi IP address 192.168.1.69
Signal strength -63dBm, channel 1, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 3 0 5 0 0 0 0 0

I think that I will go back to a older version. It is a bit frustrating to loose a print after a print.

Cheers, Chriss

droftarts

@Chriss @ProteanReverie Thanks for your continued reports.

@Chriss said in Duet 3 mini5+ nightly restarts:

I think that I will go back to a older version. It is a bit frustrating to loose a print after a print.

That's fair enough. Which is the last version that works correctly for you? I have bumped this thread to @dc42. Unfortunately your Hardfaults are 'imprecise', so there's not much to go on to track down this fault.

To make Hardfault 'precise', send M122 P500 S0 at startup to disable the write buffer. Disabling the write buffer reduces performance, but can help when debugging if the processor resets and the M122 reset data indicates that the cause was an imprecise exception. Disabling the write buffer will usually make the exception precise in future. Which makes it more likely that the cause can be tracked down.

Ian

Chriss

@droftarts

I guess that it was the very last RC before 3.5.1 which worked as far as I can tell. What I can do is keeping the printer up for one or two days to disable the write buffer if that helps.

My observation so far is that this problem pops up at least one per 24h. But I have to admit that I do not think that it is related to timing directly. The current print stop after ruffly 4 hours and 115layers. The print before this print finished, hat was a other part so I can not compare it 1:1. But the print before that was the same one and that stopped at ruffly 95% at ruffly 70layers. And the printer did not performed a reset for some hours.

Just to point that out: I did a reset to the board before I started the last print, I had the hope that the board will last long enough to finish the print.

I understand that it is hard for you to find the issue when you can not reproduce it. I can offer two things here:
1: I have a Mini5+ here at my desk, this has nothing connected, just for config development etc. I can flash 3.5.1 to it and upload the current config. Just to verify that it is not a hardware fault.
2: I can hand you over my entire config and you can test with it. We may come up that it is a strange race condition with the config I use. (And you can make some fun out of my poor config, that could be entertaining for you.)

Let me know if you want me to do one of these, I will restart the board now and disable the write buffer in the meantime.

Edit: Or do you think that it is worth to try 3.5.2rc1?

Cheers, Chriss

droftarts

@Chriss I think any of those tests are worthwhile, and might narrow down what’s causing it. So yes, please run 3.5.1 a couple more days on your production machine if possible, and on the bench test board, and please zip up your config and macros (or whatever is needed at boot time) tell me the firmware and hardware versions you’re using (let’s stick with 3.5.1 for now, as I don’t think there’s any direct fix in 3.5.2-rc1), and share them with me, and I’ll run the bench test too.

Ian

Chriss

@droftarts Good... So I will prepare my bench board than. Is there a way to share the config directly to you? I'm not sure whether I can PM you or not.

Chriss

@droftarts

Here a reboot again:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 2 of 40 (40 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 122360 of which 0 recycled
Never used RAM 12968, free system stack 182 words
Tasks: NETWORK(1,ready,15.4%,216) HEAT(3,nWait 6,0.0%,326) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.0%,797) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,1.4%,102) MAIN(1,running,82.2%,665) IDLE(0,ready,0.0%,30) AIN(4,delaying,0.9%,256), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:28:35 ago, cause: software
Last software reset at 2024-06-06 19:14, reason: HardFault bfarValid precise, Gcodes spinning, available RAM 12480, slot 2
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00489803 BFAR 0x0000001c SP 0x20011ff0 Task NETW Freestk 483 ok
Stack: 2002c5e0 20031f98 200014e8 00000000 2003312d 0002ff99 0002feac 610f0000 0002ff85 00000000 00000000 00000000 20031fa4 00000800 20035c98 2002c5c8 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 b5ddcef1
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 1715656, completed 1715654, timed out 0, errs 0
MCU temperature: min 31.5, current 31.9, max 34.4
Supply voltage: min 24.3, current 24.3, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/11, heap memory allocated/used/recyclable 2048/1680/1508, gc cycles 4
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 160, reads 24832, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 52, read errors 0, write errors 1, ifcnt 77, reads 24829, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 48, read errors 0, write errors 1, ifcnt 76, reads 24829, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 87, reads 24835, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 161, reads 24832, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 2, read errors 0, write errors 1, ifcnt 162, reads 24832, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 160, reads 24832, writes 13, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-06-06 19:43:04
Cache data hit count 3232259166
Slowest loop: 18.31ms; fastest: 0.17ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 6.5ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 15455, received 34346, lost 0, errs 1, boc 0
Longest wait 2ms for reply type 6013, peak Tx sync delay 267, free buffers 26 (min 25), ts 8577/8576/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 11.27ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(2)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 37352
WiFi IP address 192.168.1.69
Signal strength -64dBm, channel 1, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 3 0 0 0 0 0 0

Chriss

@droftarts The bench board is running now some trouble with the upgrade of the wifi module. I can not home the axes, obviously, so I have to check the board manually from time to time and will paste the output of M112 if that board will reboot too.

Cheers, Chriss

droftarts

@Chriss you can send G92 X0 Y0 Z0 to set the axes as homed, then visually you’ll know from looking at DWC if it has reset.

Ian

Chriss

@droftarts Thanks... you are a walking G-Code dictionary...

Edit: One more comment: The bench board does not have toolboard connected. I will attach one if the problem will not pop up in the next 48 hours.

Chriss

@droftarts
The board in the printer seems to be stable now after almost 24h since the last reset. (According to M122). I have the feeling that the problem occurs more often when the printer is printing. I will leave it now how it is and wait for the next reset. Nest test will be a print without filament to get a better understanding whether it happens sooner if the printer is working or not. Let me know if you think that this does not make sense from your point of view.

I can replace the board in the printer with my bench board if we see that the board on the bench is stable. If we agree that this makes sense. And let me know if you want me to parcel the board to you guys.

I have two more printers with a mini5+ in similar setups, one with 2 toolboards, one without one. None of them have 3.5.1, I can update one of them and test with them too if we think that this will bring us forward.

Cheers, Chriss

Chriss

Hi @droftarts
Update: Both boards did not reboot after 90 and 80 hours of uptime.
I replaced the SD card od my bench board today to get rid od some strange problems with the the DWC (saving of files failed).

The real printer will get a print job now without any filament to understand whether the printer failed more often when it is printing.

Cheers, Chriss

droftarts

@Chriss I've had your config running on mine all weekend with axes homed, and it hasn't skipped a beat either. Can't run a print, it'll break my machine! I'm going to revert to my regular config.g, and with your results, I'm going to assume it doesn't happen when idle.

Ian

Chriss

@droftarts

Yes and no Ian, I saw the issue when the printer was not printing over night. And I saw that issue more than than once in 48 hours, all without printing. That is the reason for my confusion that it did not happen in the last 90 hours. And we should not forget that my printer and ProteanReveries printer has a other "board" conencted via CAN but my bench board not.

Anyway, I used SuperSlicer to generate new g-code with an object which is about 155mm high, that should produce the error. The only change is that I lowered the temperatures a lot and have no filament in the extruder. I do not want to waist more plastic.

Let's hope that we see the problem again. The bench board will lay here for some more days with the fresh SD card, so the counter is at 0h for now. I will keep you updated.

Cheers, Chriss

droftarts

@Chriss I have a 1LC CAN board on the Mini 5+ I was testing on.

Ian

Chriss

@droftarts
It is mega strange, remember that I spoke about the 2nd failed print which made me thinking about going back to a older release? I used this SLT today again and it completed successfully toady. I restarted the print with the same gcode. The uptime of the printer is 126h now.
OK, the bed is off and the hotend is at 80°C, but this should not make any difference. Electrostatic discharge came in my mind but that should not be the problem because I did see the incident when the printer was not moving at all (over night)

I'm a bit helpless to be honest.

@ProteanReverie : Did you see the problem again in the last 5-6 days?

Cheers, Chriss

Chriss

@droftarts I'm constantly printing the same since 2-3 days and had no reboot since the last planed reboot. :?
I think that I will do more tests during the weekend, this time with filament. Maybe somehow related to the heaters....

Cheers, Chriss