Duet 3 mini5+ nightly restarts 3.5.2

ProteanReverie

@Phaedrux said in Duet 3 mini5+ nightly restarts:

Were you able to test with 3.5.2?

I had missed the release of 3.5.2. I will get that updated now and continue to observe.

I was out of the office last week, and there appear to have been a couple of occurrences that I missed thus do not have M122 reports for. Latest M122 and log from while I was out are below.

M122:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: 65LYX-8Q6KL-K65J0-409N0-NP02Z-ZFK3T
Used output buffers: 1 of 40 (26 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 119940 of which 24 recycled
Never used RAM 15364, free system stack 150 words
Tasks: NETWORK(1,ready,22.3%,218) HEAT(3,nWait 6,0.0%,356) Move(4,nWait 6,0.0%,341) CanReceiv(6,nWait 1,0.0%,808) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.8%,102) MAIN(1,running,75.9%,569) IDLE(0,ready,0.1%,30) AIN(4,delaying,0.9%,260), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 136:15:22 ago, cause: software
Last software reset at 2024-07-09 18:41, reason: HardFault bfarValid precise, Gcodes spinning, available RAM 15388, slot 1
Software reset code 0x0063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00000803 BFAR 0x00000004 SP 0x20011ff0 Task NETW Freestk 483 ok
Stack: 00000160 00000002 200014e8 0000015f ffffffff 000a07ef 0002ff8e 810f0000 0002ff85 00000000 00000000 00000000 20031614 00000800 2002c5c8 2002c5c8 00000001 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 ffffffff
Error status: 0x00
MCU revision 3, ADC conversions started 490523144, completed 490523144, timed out 0, errs 0
MCU temperature: min 40.2, current 40.8, max 41.2
Supply voltage: min 24.0, current 24.1, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 87, reads 6413, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 87, reads 6413, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 87, reads 6413, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 87, reads 6412, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 87, reads 6413, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-15 10:56:39
Cache data hit count 4294967295
Slowest loop: 706.08ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.6ms, write time 1.7ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000000
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 609, received 976, lost 0, errs 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 7, free buffers 26 (min 26), ts 609/609/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 201.48ms; fastest: 0.07ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 8, noresp 1
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:d0
Module reset reason: Power up, Vcc 3.39, flash size 2097152, free heap 42776
WiFi IP address 10.1.10.46
Signal strength -57dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Log:

2024-07-04 02:48:55 [warn] HTTP client 10.1.10.9 login succeeded (session key 9483890)
2024-07-04 02:49:06 [warn] HTTP client 10.1.10.9 login succeeded (session key 930818349)
2024-07-05 02:52:38 [warn] HTTP client 10.1.10.9 login succeeded (session key 4040799154)
2024-07-05 02:52:47 [warn] HTTP client 10.1.10.9 login succeeded (session key 2855566171)
2024-07-05 02:52:58 [warn] HTTP client 10.1.10.9 login succeeded (session key 2244811411)
2024-07-05 15:41:41 [warn] HTTP client 10.1.10.9 login succeeded (session key 1092597384)
2024-07-05 15:41:59 [warn] HTTP client 10.1.10.9 login succeeded (session key 792491443)
2024-07-06 02:19:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 1617452959)
2024-07-06 02:20:05 [warn] HTTP client 10.1.10.9 login succeeded (session key 3536857478)
2024-07-06 02:20:19 [warn] HTTP client 10.1.10.9 login succeeded (session key 3441961249)
2024-07-06 02:20:30 [warn] HTTP client 10.1.10.9 login succeeded (session key 3419764404)
2024-07-06 03:38:37 [warn] HTTP client 10.1.10.9 login succeeded (session key 3275204183)
2024-07-06 03:38:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 1076010950)
2024-07-06 06:38:04 [warn] HTTP client 10.1.10.9 login succeeded (session key 99326213)
2024-07-06 09:37:53 [warn] HTTP client 10.1.10.9 login succeeded (session key 4281600970)
2024-07-06 17:14:58 [warn] HTTP client 10.1.10.9 login succeeded (session key 244420871)
2024-07-07 03:57:29 [warn] HTTP client 10.1.10.9 login succeeded (session key 1942473528)
2024-07-07 03:57:51 [warn] HTTP client 10.1.10.9 login succeeded (session key 902420159)
2024-07-07 21:37:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 3336424230)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.1 (2024-04-19 14:41:25)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.46
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 1810534383)
2024-07-08 01:43:36 [warn] Date and time set at power up + 00:00:08
2024-07-08 04:01:44 [warn] HTTP client 10.1.10.9 login succeeded (session key 2605107193)
2024-07-08 04:01:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 1991834332)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.1 (2024-04-19 14:41:25)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:00 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.46
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 592097959)
2024-07-09 04:20:33 [warn] Date and time set at power up + 00:00:08
2024-07-09 07:25:16 [warn] HTTP client 10.1.10.9 login succeeded (session key 4205354191)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.1 (2024-04-19 14:41:25)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.46
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 3903592583)
2024-07-09 18:41:26 [warn] Date and time set at power up + 00:00:08
2024-07-15 09:14:13 [warn] HTTP client 10.1.10.9 login succeeded (session key 1545598652)
2024-07-15 10:54:38 [debug] === Diagnostics ===

ProteanReverie

I have begun testing with 3.5.2. I also have now assembled this setup into a machine, moving it from the previous setup of just boards on my desk.

I had a crash occur this morning while I was watching the interface, which happened just after I got an M122 response as I was checking to see if it had any issues overnight. This setup has completed a couple of prints, but nothing had been run since last power-up yesterday afternoon. Interestingly, that M122 response does not appear in the log file, though I did get the response in DWC. I have also included that M122 report below as the last item on this post, just in-case it is useful.

Additionally, I noticed that on startup I got an error (which can also be seen in the M122) stating that none of the motor drivers on the CAN connected board exist. The axis assigned to these drivers are shown and can be interacted with, but they do not respond to commands. Other items on that CAN board, such as the toolhead heaters and thermistors do work. I am guessing this is just due to CAN timing being off from the restart and reconnecting part-way through the config.g, as the motor driver commands are some of the first things in config.g, but wanted to provide this additional information. I can confirm that power cycling the boards/machine resolves this issue.

M122 post-crash:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 17 of 40 (18 max)
Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
=== RTOS ===
Static ram: 103368
Dynamic ram: 122124 of which 0 recycled
Never used RAM 13068, free system stack 198 words
Tasks: NETWORK(2,nWait 7,14.1%,217) HEAT(3,nWait 6,0.0%,350) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,796) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.8%,101) MAIN(1,running,83.9%,665) IDLE(0,ready,0.1%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:04:32 ago, cause: software
Last software reset at 2024-07-26 08:21, reason: StuckInSpinLoop, none spinning, available RAM 15700, slot 0
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0000080f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 035a4c16 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 273048, completed 273048, timed out 0, errs 0
MCU temperature: min 36.5, current 37.3, max 37.6
Supply voltage: min 23.9, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/140/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 14333, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 14333, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 14333, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 14333, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 14333, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-26 08:26:41
Cache data hit count 522766768
Slowest loop: 6.01ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.6ms, write time 2.4ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 2577952
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 2473, received 5486, lost 0, errs 1, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 210, free buffers 26 (min 25), ts 1364/1363/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 10.96ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42940
WiFi IP address 10.1.10.21
Signal strength -55dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Log:

power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:08 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:10 [warn] HTTP client 10.1.10.9 login succeeded (session key 2372260450)
2024-07-25 16:44:19 [warn] Date and time set at power up + 00:00:10
2024-07-26 02:59:08 [warn] Error: WiFi module reported: Lost connection, auto reconnecting
2024-07-26 02:59:12 [warn] Error: WiFi module reported: Auto reconnect succeeded
2024-07-26 04:50:09 [warn] Error: WiFi module reported: Lost connection, auto reconnecting
2024-07-26 04:50:12 [warn] Error: WiFi module reported: Auto reconnect succeeded
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:08 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:09 [warn] HTTP client 10.1.10.9 login succeeded (session key 199109116)
2024-07-26 08:22:09 [warn] Date and time set at power up + 00:00:09
2024-07-26 08:26:41 [debug] === Diagnostics ===
2024-07-26 08:26:41 [debug] RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
2024-07-26 08:26:41 [debug] Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
2024-07-26 08:26:41 [debug] Used output buffers: 17 of 40 (18 max)
2024-07-26 08:26:41 [debug] Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
2024-07-26 08:26:41 [debug] === RTOS ===

M122 just prior to crash, not appearing in log file:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (18 max)
=== RTOS ===
Static ram: 103368
Dynamic ram: 122348 of which 0 recycled
Never used RAM 15700, free system stack 186 words
Tasks: NETWORK(1,ready,13.3%,217) HEAT(3,nWait 6,0.0%,325) Move(4,nWait 6,0.0%,341) CanReceiv(6,nWait 1,0.1%,773) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.8%,101) MAIN(1,running,84.8%,665) IDLE(0,ready,0.0%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 15:37:29 ago, cause: power up
Last software reset at 2024-07-24 10:53, reason: User, Gcodes spinning, available RAM 11900, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
MCU revision 3, ADC conversions started 56249400, completed 56249399, timed out 0, errs 0
MCU temperature: min 24.9, current 37.1, max 37.8
Supply voltage: min 23.9, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 11346, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 11346, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 11344, writes 10, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x01
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 11345, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 11346, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-26 08:21:37
Cache data hit count 4294967295
Slowest loop: 115.60ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 10.2ms, write time 111.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 536792218
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 506267, received 1125011, lost 0, errs 14, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 2156, free buffers 26 (min 25), ts 281247/281246/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 112.10ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0

ProteanReverie

Unsure of the relevancy to this thread, but I had another strange occurrence with this board just now that could potentially be related and provide more info as it related to a network action apparently disconnecting the SD card.

Following the power cycling in my last post to get all the motor drivers working again, I went to upload a print. This failed a little bit into the progress, which is not too uncommon in my experience here on this network when working with Duet 2 boards. See https://forum.duet3d.com/topic/35979/duet-2-wifi-network-slowdowns-and-board-restarts for more info on that. However, this time it came with a new message
"Error: Failed to create folder 0:/gcodes in path 0:/gcodes/PETG-0.8-0.5layers-adaptive0.2-adaptvar0.02_benderHead_fix3.gcode.part"
on top of the usual
"Failed to upload PETG-0.8-0.5layers-adaptive0.2-adaptvar0.02_benderHead_fix3.gcode
Network error"

It appears the SD card became unmounted or unresponsive at this time. Following that attempt to upload, I could still see the contents of directories such as Macros and Gcodes, but I would get errors about the directory not being present. Attempting to run a macro would ask if I wanted to run it, but then nothing would happen. I used a USB attached serial console to change over to host mode, which worked. However, trying to connect to DWC would result in the page stating that the page could not be loaded, and to check that the SD card was inserted and the www files were present. I was able to use the USB serial console to switch the board back over to client mode, and both of my already open DWC instances were able to reconnect but still not interact with files. However, new attempts to connect to DWC would result in the page stating to check the SD card.

Power cycling was able to clear this condition. The log file does not show any of this, which makes sense since the SD card appeared to be disconnected. Screenshot of the DWC console provided due to this. Unfortunately I did not get an M122 before power cycling, but I will try to if this occurs again. I am continuing with the print for now but intend to remove the SD card and check it for issues in windows later today.

ProteanReverie

@ProteanReverie said in Duet 3 mini5+ nightly restarts:

intend to remove the SD card and check it for issues in windows later today.

I pulled the SD card and tested with H2testw per https://docs.duet3d.com/en/User_manual/RepRapFirmware/SD_card#troubleshooting-sd-card-issues
This found no issues

I then re-installed the SD card to the board and the board to the machine and ran M122 P104 S10.

Following this, the board appeared to be back in the state of not recognizing the SD card - unable to locate directories, check SD page when trying to load a new DWC session.

M122 output following this occurring:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (36 max)
=== RTOS ===
Static ram: 103368
Dynamic ram: 122324 of which 0 recycled
Never used RAM 15724, free system stack 202 words
Tasks: NETWORK(1,ready,15.7%,203) HEAT(3,nWait 6,0.0%,371) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,798) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.8%,101) MAIN(1,running,79.8%,665) IDLE(0,ready,2.6%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:01:44 ago, cause: power up
Last software reset at 2024-07-26 08:21, reason: StuckInSpinLoop, none spinning, available RAM 15700, slot 0
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0000080f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 035a4c16 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 105047, completed 105047, timed out 0, errs 0
MCU temperature: min 28.9, current 30.2, max 30.4
Supply voltage: min 23.9, current 24.0, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 3591, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 3591, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 3591, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 3591, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 3591, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-26 14:56:49
Cache data hit count 219758010
Slowest loop: 4080.50ms; fastest: 0.16ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 101.1ms, write time 116.9ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 893256
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 613, received 1360, lost 0, errs 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 6, free buffers 26 (min 26), ts 341/341/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 2271.55ms; fastest: 0.06ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.39, flash size 2097152, free heap 42784
WiFi IP address 10.1.10.21
Signal strength -56dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Here are some additional tests I ran while it was in that state:

I power cycled the machine after this, and re-ran M39 which gave the same response, and M20 which did properly list gcode files this time. However, trying to run M122 P104 S10 resulted in the same condition occurring.

After power cycling again, I was able to get results for M122 P104 S10, which I was able to successfully run 10 times consecutively. First run results:
SD write speed for 10.0MByte file was 1.36MBytes/sec
SD read speed for 10.0MByte file was 1.28MBytes/sec

Across the first 8 tests I got values in the ranges of:
write = 1.36-1.37MB/s
read = 1.26-1.3MB/s

However, read speed for attempt 9 fell down to 0.94, and was 0.95 for test 10. The write speeds stayed within previous range on those tests. I ran a couple more tests after waiting 5, and then 15 minutes, and then tried M122, M39, M20, and then repeated the speed test again. Speeds on those tests remained the same as in tests 9&10, and the condition was not incurred again.

ProteanReverie

Had another incident just now where I sent an M122 to see if a restart had happened while I was out of the office, which resulted in DWC reporting the M122 info and then the board crashing and restarting, while failing to recognize the divers on the CAN connected board upon restart. Below is the output from M122 before, and after, as well as a section of the log file. Once again, the log file does not contain any of the M122 reported prior to the restart. I am going to swap out the main board with an unused spare I have on hand and try testing with that. I will then be able to do any tests on the current board from a benchtop setup.

M122 before restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (37 max)
=== RTOS ===
Static ram: 103368
Dynamic ram: 122436 of which 0 recycled
Never used RAM 12396, free system stack 132 words
Tasks: NETWORK(2,nWait 7,12.9%,195) HEAT(3,nWait 6,0.0%,325) Move(4,nWait 6,0.0%,238) CanReceiv(6,nWait 1,0.1%,773) CanSender(5,nWait 7,0.0%,323) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.9%,67) MAIN(1,running,85.2%,665) IDLE(0,ready,0.0%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 64:52:05 ago, cause: power up
Last software reset at 2024-07-26 08:21, reason: StuckInSpinLoop, none spinning, available RAM 15700, slot 0
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0000080f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 035a4c16 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 233525360, completed 233525360, timed out 0, errs 0
MCU temperature: min 36.2, current 38.4, max 43.8
Supply voltage: min 23.7, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/120/120, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 19, reads 54184, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 19, reads 54184, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 54193, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 54192, writes 0, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x41
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 54194, writes 0, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-29 08:04:42
Cache data hit count 4294967295
Slowest loop: 241.79ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 5.4ms, write time 121.6ms, max retries 0
=== Move ===
DMs created 83, segments created 15, maxWait 2466643ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 18, on retry 0, too short 0, wrong shape 6, maybepossible 0
=== DDARing 0 ===
Scheduled moves 80, completed 80, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 21], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.2
Heater 1 is on, I-accum = 0.2
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x000080f
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 2233026076
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 2082665, received 4627819, lost 0, errs 0, boc 0
Longest wait 1ms for reply type 6013, peak Tx sync delay 296, free buffers 26 (min 24), ts 1156944/1156944/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 95.60ms; fastest: 0.00ms
Responder states:
MQTT(0) HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0

M122 after restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (18 max)
Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
=== RTOS ===
Static ram: 103368
Dynamic ram: 122124 of which 0 recycled
Never used RAM 13068, free system stack 202 words
Tasks: NETWORK(1,ready,12.5%,217) HEAT(3,nWait 6,0.0%,362) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,798) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,delaying,0.8%,101) MAIN(1,running,84.9%,651) IDLE(0,ready,0.9%,29) AIN(4,delaying,0.9%,268), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:00:33 ago, cause: software
Last software reset at 2024-07-29 08:04, reason: StuckInSpinLoop, none spinning, available RAM 12396, slot 1
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044f80f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 0deb5022 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 33534, completed 33534, timed out 0, errs 0
MCU temperature: min 37.3, current 38.4, max 38.4
Supply voltage: min 23.9, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/140/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 30, reads 1738, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 30, reads 1738, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1738, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1737, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1738, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-29 08:05:40
Cache data hit count 64135214
Slowest loop: 6.20ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 1.5ms, write time 4.8ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 319150
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 319, received 696, lost 0, errs 1, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 288, free buffers 26 (min 25), ts 168/167/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 10.93ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 36232
WiFi IP address 10.1.10.21
Signal strength -67dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Log:

2024-07-28 13:21:39 [warn] Error: WiFi module reported: Auto reconnect succeeded
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 3150589034)
2024-07-29 08:05:15 [warn] Date and time set at power up + 00:00:08
2024-07-29 08:05:40 [debug] === Diagnostics ===
2024-07-29 08:05:40 [debug] RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
2024-07-29 08:05:40 [debug] Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
2024-07-29 08:05:40 [debug] Used output buffers: 1 of 40 (18 max)
2024-07-29 08:05:40 [debug] Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
2024-07-29 08:05:40 [debug] === RTOS ===

ProteanReverie

@ProteanReverie said in Duet 3 mini5+ nightly restarts:

I am going to swap out the main board with an unused spare I have on hand and try testing with that. I will then be able to do any tests on the current board from a benchtop setup.

I did not make it this far. I did some pre-install setup tasks and testing on the spare unused board, and wound up finding the fresh board and SD card seem to have the same SD disconnecting issue. Not always, but usually if I send an M122 P104 S10, the SD card will become unavailable. I had copied the files from the SD card of the board I originally found the issue on to the fresh board's fresh SD card via windows prior to this testing.

I have additionally tested older firmware versions on this fresh board I opened this morning. I have tried 3.5.1 and 3.4.6, both still having the SD card usually become unresponsive after the command is sent.

I have found that if it does not fail the first time I run the command after a power-up, it will likely not fail on subsequent tests during that power-cycling. However, power-cycling it and testing again can result in the failure. Worth noting perhaps that I had run the test over 10 times on a single power-up on Friday which then crashed this morning in my last post.

Additionally, in regards to a simple M122 for the general diagnostics causing crashes/restarts, that could maybe be related as I do have logging enabled and thus M122 would also be writing to the SD card for the log file.

This is now 2 of my 3 boards this SD issue is confirmed occurring on, and I see there are other posts on this forum indicating that at some point there were hardware/soldering issues with the SD slots on the Duet 3 Mini 5+, though that seems to have been resolved a couple of years ago. For reference the three boards I am working on are all v1.02 purchased from Filastruder on 5/10/24. The serial numbers of the two confirmed to have the SD issue are WD11865 and WD11866. My remaining board that is not yet tested for this issue is WD11864.

ProteanReverie

I spent some time today testing some things related to the SD card dismounting issue

This appears it may be related to the SD cards themselves, though not certain. I ran a series of tests of power cycling the board, running M122 P104 S10, noting result, and then starting that process over. If I encountered the issue where the SD card becomes unavailable at the start of writing, I stopped testing that setup. Otherwise, I initially considered a 10-success streak as passing. Until the last test, these were all using the SD cards provided with the boards. Here are my results:

Expansion board (previously untested), expansion card (previously untested) - passed 10 tests (may be invalid pass, see next section)
Bench board (new board from last post, tested bad before), expansion card - passed 10 tests (may be invalid pass, see next section)
Bench board, bench card (previously tested bad) - failed on test 3
Bench board, bench card, no wifi or logging - failed on test 5
Bench board, bench card, no logging and no debugging - failed on test 2
bench board, bench card, files from expansion board - failed test 3
bench board, bench card, reformatted with smallest allocation size - passed 10 tests, though very slow speeds
bench board, bench card, reformatted with largest allocation size - failed on test 8

At this point, I wanted to sanity check my first two tests and got these results:
bench board, expansion card - failed on test 6
expansion board, expansion card - failed on test 4

so, either something happened to that card after the initial tests, or the initial tests may be invalid...

Following that I tested with a new 16GB Sandisk micoSDHC UHS-I card I had which was removed from the package just before testing. I did not format, load files onto, or do anything with it between opening the package and inserting it into the Duet:
Bench board, new 16GB card - passed 20 tests in a row

Phaedrux

Interesting. Are you able to do a surface format of one of the potentially bad cards and try the test again?

ProteanReverie

@Phaedrux said in Duet 3 mini5+ nightly restarts 3.5.2:

Interesting. Are you able to do a surface format of one of the potentially bad cards and try the test again?

I am unfamiliar with the terminology "surface format", and a quick google isn't clearing that up. What I have done is used the official SD card formatting tool linked to here https://docs.duet3d.com/en/User_manual/RepRapFirmware/SD_card . I used the option of "Overwrite format" without "CHS format size adjustment" checked. This did merge everything into one partition thus resulting in one partition of ~8GB as opposed to how it arrived or how windows formatting left it, which had 2 partitions and a large chunk of unallocated space, with only a 2GB partition visible. This appears to have produced a promising result as it has survived 20 consecutive trials without error. I will proceed with adding files and testing, then trying with the other SD cards and boards and testing those, should no issues arise.

ProteanReverie

I have completed some additional testing. Following an official SD tool formatting, both the main board and the bench board SD cards worked just fine and were able to pass 20 tests without issue. However, the expansion board card still has an issue even after the official SD tool formatting, and initially failed on the second test. I also tested that card in a Duet 2 Wi-Fi running 3.5.1, which passed 20 tests without issue. I then tested it in the bench board where it failed on the first test.

I have replaced the faulty card with the new 16GB card from yesterday and re-built those boards into my machine to resume my testing. The bench board is also set up to idle away to see if it randomly restarts again but could also be used for other testing if any more data would be of use.

Phaedrux

@ProteanReverie said in Duet 3 mini5+ nightly restarts 3.5.2:

surface format

The opposite of a "quick" format. It zeros the entire drive area rather than just the file table. If there are bad cells they would get written to and potentially corrected or remapped.

@ProteanReverie said in Duet 3 mini5+ nightly restarts 3.5.2:

I used the option of "Overwrite format" without "CHS format size adjustment" checked.

This is what I war referring to.

@ProteanReverie said in Duet 3 mini5+ nightly restarts 3.5.2:

which had 2 partitions and a large chunk of unallocated space

that's the way the card comes imaged from the factory. One partition for standalone mode and another for the SBC.

ProteanReverie

@Phaedrux said in Duet 3 mini5+ nightly restarts 3.5.2:

The opposite of a "quick" format. It zeros the entire drive area rather than just the file table. If there are bad cells they would get written to and potentially corrected or remapped.

Ok, that is what I had assumed, just never heard it with that term. Thank you for clarifying.

Just for clarity of data, when I formatted yesterday using windows format I did not have "quick format" selected.

ProteanReverie

Another crash just now. This is on the board installed in a machine. Once again happened shortly after sending and receiving an M122 via DWC after being on overnight. Once again, the M122 prior to restart does not appear in the log file.

I also just noticed on this that the M122 response prior to restart ends part-way through the WiFi section. I double checked my previous posts where M122 resulted in a restart, and that appears to be the case on those as well. It appears when this happens it is just after the "failed messages" line and just before the wifi firmware version line.

M122 after restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (18 max)
Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
=== RTOS ===
Static ram: 103368
Dynamic ram: 122124 of which 0 recycled
Never used RAM 13068, free system stack 202 words
Tasks: NETWORK(1,ready,11.3%,217) HEAT(3,nWait 6,0.0%,364) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,798) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait 6,0.8%,101) MAIN(1,running,85.7%,665) IDLE(0,ready,1.2%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:00:21 ago, cause: software
Last software reset at 2024-08-01 08:11, reason: StuckInSpinLoop, none spinning, available RAM 12068, slot 2
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0000080f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 03552189 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 21418, completed 21418, timed out 0, errs 0
MCU temperature: min 38.4, current 39.1, max 39.2
Supply voltage: min 23.8, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/140/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 26, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 26, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1101, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 24, reads 1101, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-08-01 08:12:11
Cache data hit count 41086803
Slowest loop: 5.66ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.5ms, write time 2.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 206056
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 210, received 456, lost 0, errs 1, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 21, free buffers 26 (min 25), ts 107/106/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 8.77ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42944
WiFi IP address 10.1.10.21
Signal strength -65dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

M122 prior to restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (34 max)
=== RTOS ===
Static ram: 103368
Dynamic ram: 122500 of which 0 recycled
Never used RAM 12068, free system stack 128 words
Tasks: NETWORK(2,nWait 7,13.4%,203) HEAT(3,nWait 6,0.0%,325) Move(4,nWait 6,0.3%,241) CanReceiv(6,nWait 1,0.1%,773) CanSender(5,nWait 7,0.0%,327) CanClock(7,delaying,0.0%,348) TMC(4,delaying,0.9%,67) MAIN(1,running,84.5%,665) IDLE(0,ready,0.0%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 15:31:50 ago, cause: power up
Last software reset at 2024-07-29 08:04, reason: StuckInSpinLoop, none spinning, available RAM 12396, slot 1
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044f80f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 0deb5022 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 55911215, completed 55911214, timed out 0, errs 0
MCU temperature: min 38.4, current 39.2, max 56.2
Supply voltage: min 23.7, current 23.9, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/72/72, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 15, reads 58948, writes 15, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x01
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 15, reads 58949, writes 15, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 58953, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 58950, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 58954, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-08-01 08:11:29
Cache data hit count 4294967295
Slowest loop: 111.27ms; fastest: 0.15ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 10.2ms, write time 16.5ms, max retries 0
=== Move ===
DMs created 83, segments created 26, maxWait 71928ms, bed compensation in use: mesh, height map offset 0.068, max steps late 1, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 1.00
no step interrupt scheduled
Moves shaped first try 771, on retry 2134, too short 16058, wrong shape 44923, maybepossible 2196
=== DDARing 0 ===
Scheduled moves 69464, completed 69464, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.4
Heater 1 is on, I-accum = 0.3
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 60938590 clear 466545122
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 534665, received 1118265, lost 0, errs 0, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 337, free buffers 26 (min 25), ts 279554/279553/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 15.51ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0

Log file:

2024-07-31 18:09:14 [warn] HTTP client 10.1.10.161 login succeeded (session key 2966924725)
2024-07-31 18:37:06 [debug] Done printing file
2024-07-31 18:37:06 [debug] Done printing file
2024-07-31 18:37:06 [warn] Finished printing file 0:/gcodes/3.6A2_testing/PETG-0.4_Ghosting_test.gcode, print time was 1h 57m
2024-07-31 23:08:38 [warn] HTTP client 10.1.10.9 login succeeded (session key 1279128385)
2024-07-31 23:08:40 [warn] HTTP client 10.1.10.9 login succeeded (session key 270777459)
2024-07-31 23:09:09 [warn] HTTP client 10.1.10.9 login succeeded (session key 2746186916)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 562008642)
2024-08-01 08:11:59 [warn] Date and time set at power up + 00:00:08
2024-08-01 08:12:11 [debug] === Diagnostics ===
2024-08-01 08:12:11 [debug] RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
2024-08-01 08:12:11 [debug] Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW

Chriss

I had a restart during printing after a long time again.

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 1 of 40 (40 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 123888 of which 0 recycled
Never used RAM 11440, free system stack 192 words
Tasks: NETWORK(1,ready,14.5%,220) HEAT(3,nWait 6,0.0%,332) Move(4,nWait 6,0.0%,341) CanReceiv(6,nWait 1,0.0%,797) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,1.4%,102) MAIN(1,running,83.1%,665) IDLE(0,ready,0.1%,30) AIN(4,delaying,0.9%,260), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:12:15 ago, cause: software
Last software reset at 2024-08-18 17:50, reason: HardFault imprec, Expansion spinning, available RAM 9284, slot 1
Software reset code 0x4072 HFSR 0x40000000 CFSR 0x00000400 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20012008 Task NETW Freestk 489 ok
Stack: 2002c5e0 200307c8 200014e8 00000000 200331ad 0002ff99 0002feb0 610f6000 2002c5c8 2002c5c8 00000001 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 00000000 00000002 ea400050 0801a8c0 08000161 00000001 000347e1
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 735652, completed 735650, timed out 0, errs 0
MCU temperature: min 39.1, current 39.3, max 53.2
Supply voltage: min 24.3, current 24.4, max 24.5, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/15, heap memory allocated/used/recyclable 2048/1408/1196, gc cycles 2
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 35, reads 1289, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 150, read errors 0, write errors 1, ifcnt 241, reads 1286, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 138, read errors 0, write errors 1, ifcnt 231, reads 1286, writes 16, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 32, reads 1292, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 35, reads 1289, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 1, ifcnt 35, reads 1289, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 35, reads 1289, writes 13, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-08-18 18:02:53
Cache data hit count 1378274719
Slowest loop: 11.12ms; fastest: 0.17ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 5.7ms, write time 1.8ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 6636, received 14746, lost 0, errs 1, boc 0
Longest wait 2ms for reply type 6031, peak Tx sync delay 263, free buffers 26 (min 25), ts 3678/3677/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 11.55ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(1)
HTTP sessions: 3 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 40612
WiFi IP address 192.168.1.69
Signal strength -63dBm, channel 1, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 3 0 0 0 0 0 0 0

Btw: My bench board has no problem at all and is on since 8+ weeks. :?

Chriss

One more thing... The reboot happened again at 85-95% of the print. As it always does when it happens to my printer. Am I the only one with this observation?

oliof

@Chriss does it also happen when simulating a print?

Chriss

@oliof No idea to be honest. I saw it only when the printer was actually printing. (I grounded my hotend and my e-stepper some month ago for a other problem) I have only one printer with this firmware and I need it at the moment to finish some projects. And the problem is not easy to reproduce. It seems to me like it was wore at the beginning, like every morning a rebooted printer, but this is not the case anymore. I guess that this was a false observation but is not that I experiance a reboot at every print or every 2nd print.

I have the uneducated feeling that the failure happens only on higher prints too. I printed many pars which were below 100mm in Z. And two higher parts and the 2nd failed.

Not saying that any of them are causing the issue, I'm listing my observations only.

@droftarts Is there any kind of debug firmware which we can run to log the data to the serial interface? I would be more than happy to test more and capture as much as I can? This topic is getting older and older and I have the impression that we make no progress at all.
I have 4 of this boards currently, 1 or two of them should be from the first batch, the others are a bit younger. I'm not sure which one is in which printer. Would it make sense to check that? (I guess that my bench board is the youngest one.) I can place the bench board into the working printer if this make any sene? Just to see whether it happens in a real printer with the bench board not. The bench board looks like at so far:

=== Platform ===
Last reset 1667:38:29 ago, cause: power up
Last software reset details not available
Error status: 0x00

I have plenty of steppers and hotends and all the stuff in my lab too. I can build a "mock printer" with this parts and the board which runs into the issue if needed. Just ask for it.

droftarts

@Chriss I asked @dc42 if he'd made any progress on this issue. He said:

My previous investigations suggested that the crashing was caused by a hardware error in the MCU when a DMA transfer finished and the SPI peripheral was disabled. So not a software bug but a hardware defect that I need to find a workaround for.
It's possible that the RRF 3.6 builds are not affected by this issue, so worth getting the affected users to try 3.6.0-alpha.4 at https://www.dropbox.com/scl/fo/cckwiq91gn16hvl1zdjnp/AF0SMEtkVfiArSPeYaBDGPY?rlkey=kqkknk9q1kiq684u4s55ce8d4&dl=0

Could you try the 3.6.0-alpha.4 build?

Ian

Chriss

@droftarts said in Duet 3 mini5+ nightly restarts 3.5.2:

Could you try the 3.6.0-alpha.4 build?

Your wish is my command, but not before mid of the week. I'm installing a new hardware feature at the moment. I will do the upgrade asap. (Any new traps you want to tell me?)

Cheers, Chriss

droftarts

@Chriss There's a few things not fully working, check the release notes: https://github.com/Duet3D/RepRapFirmware/wiki/Changelog-RRF-3.x-Beta

Ian