Solved Duet 2 Wifi rebooting/crashing during print
-
Hi!
I've had my duet for several months without issue - recently it has been rebooting randomly during prints, roughly anywhere from 10 - 60 mins into the print. The only thing I changed recently was adding a berd-air max kit, which is wired directly onto a fan header.
I'm on FW 3.1.1, but I've tried swapping to the 3.2 beta and the issue persists. Suspecting the SD Card, I did try swapping SD cards and that mitigated the issue for a little bit, but it started happening again not long after.
Here's my M122 report for last night's crash:
=== Diagnostics === RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP Used output buffers: 3 of 24 (21 max) === RTOS === Static ram: 27980 Dynamic ram: 93236 of which 60 recycled Exception stack ram used: 312 Never used ram: 9484 Tasks: NETWORK(blocked,384) HEAT(blocked,1224) MAIN(running,1880) IDLE(ready,80) Owned mutexes: WiFi(NETWORK) === Platform === Last reset 07:56:02 ago, cause: software Last software reset at 2020-11-05 03:02, reason: Hard fault, spinning module PrintMonitor, available RAM 8996 bytes (slot 3) Software reset code 0x4069 HFSR 0x40000000 CFSR 0x00010000 ICSR 0x0441f803 BFAR 0xe000ed38 SP 0x200053fc Task MAIN Stack: 0042ea7f 0042b6f2 a5038400 00000000 00000000 00000000 00000000 20005480 00000001 10000000 e000ed04 Error status: 0 MCU temperature: min 23.3, current 28.5, max 28.8 Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes Driver 0: standstill, SG min/max not available Driver 1: standstill, SG min/max not available Driver 2: standstill, SG min/max not available Driver 3: standstill, SG min/max not available Driver 4: standstill, SG min/max not available Date/time: 2020-11-05 10:58:06 Cache data hit count 4294967295 Slowest loop: 5.47ms; fastest: 0.14ms I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0 === Storage === Free file entries: 10 SD card 0 detected, interface speed: 20.0MBytes/sec SD card longest read time 2.0ms, write time 0.0ms, max retries 0 === Move === Hiccups: 0(0), FreeDm: 169, MinFreeDm: 169, MaxWait: 0ms Bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === AuxDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === Heat === Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 === GCodes === Segments left: 0 Movement lock held by null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 Daemon is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 200.90ms; fastest: 0.09ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions HTTP sessions: 1 of 8 - WiFi - Network state is active WiFi module is connected to access point Failed messages: pending 0, notready 0, noresp 1 WiFi firmware version 1.23 WiFi MAC address dc:4f:22:6e:6c:ca WiFi Vcc 3.43, reset reason Unknown WiFi flash size 4194304, free heap 18416 WiFi IP address 192.168.86.26 WiFi signal strength -49dBm, reconnections 0, sleep mode modem Socket states: 0 0 0 0 0 0 0 0Thanks!
-
How is the berd air wired in exactly? I seem to recall seeing that pumps and such might require a fly back diode or some such.
Do the crashes persist with the berd air pump disconnected?
Can you post your config.g as well?
-
@Phaedrux
The berd air is wired without a flyback, directly to the fan0 header - as far as i could tell, a flyback diode isn't required for newer revisions of the duet, as each fan header has a flyback diode built in.I'll try a print without the berd air wired in and get back to you.
config.g is here:
; Configuration file for Duet WiFi (firmware version 3) ; executed by the firmware on start-up ; ; generated by RepRapFirmware Configuration Tool v3.1.4 on Wed Sep 09 2020 23:24:59 GMT-0400 (Eastern Daylight Time) ; General preferences G90 ; send absolute coordinates... M83 ; ...but relative extruder moves M550 P"Casey" ; set printer name M665 R140.3 L288 B120 H530 ; Set delta radius, diagonal rod length, printable radius and homed height M666 X0 Y0 Z0 ; put your endstop adjustments here, or let auto calibration find them ; Network M552 S1 ; enable network M586 P0 S1 ; enable HTTP M586 P1 S0 ; disable FTP M586 P2 S0 ; disable Telnet ; Drives M569 P0 S1 ; physical drive 0 goes forwards M569 P1 S1 ; physical drive 1 goes forwards M569 P2 S1 ; physical drive 2 goes forwards M569 P3 S1 ; physical drive 3 goes forwards M584 X0 Y1 Z2 E3 ; set drive mapping M350 X16 Y16 Z16 E16 I1 ; configure microstepping with interpolation M92 X80.00 Y80.00 Z80.00 E397.50 ; set steps per mm M566 X1200.00 Y1200.00 Z1200.00 E1200.00 ; set maximum instantaneous speed changes (mm/min) M203 X18000.00 Y18000.00 Z18000.00 E1200.00 ; set maximum speeds (mm/min) M201 X1000.00 Y1000.00 Z1000.00 E1000.00 ; set accelerations (mm/s^2) M906 X1200 Y1200 Z1200 E1000 I30 ; set motor currents (mA) and motor idle factor in per cent M84 S30 ; Set idle timeout ; Axis Limits M208 Z0 S1 ; set minimum Z ; Endstops M574 X2 S1 P"xstop" ; configure active-high endstop for high end on X via pin xstop M574 Y2 S1 P"ystop" ; configure active-high endstop for high end on Y via pin ystop M574 Z2 S1 P"zstop" ; configure active-high endstop for high end on Z via pin zstop ; Z-Probe M558 P5 R0.4 C"zprobe.in+zprobe.mod" H5 F1200 T6000 ; set Z probe type to effector and the dive height + speeds G31 P100 X0 Y0 Z-0.24 ; set Z probe trigger value, offset and trigger height M557 R90 S20 ; define mesh grid ; Heaters M308 S0 P"bedtemp" Y"thermistor" T100000 B4138 ; configure sensor 0 as thermistor on pin bedtemp M950 H0 C"bedheat" T0 ; create bed heater output on bedheat and map it to sensor 0 M307 H0 B1 S1.00 ; enable bang-bang mode for the bed heater and set PWM limit M140 H0 ; map heated bed to heater 0 M143 H0 S120 ; set temperature limit for heater 0 to 120C M308 S1 P"e0temp" Y"thermistor" T100000 B4138 ; configure sensor 1 as thermistor on pin e0temp M950 H1 C"e0heat" T1 ; create nozzle heater output on e0heat and map it to sensor 1 M307 H1 B0 S1.00 ; disable bang-bang mode for heater and set PWM limit ;pressure advance M572 D0 S0.25 ; Fans ; PART FAN BerdAir M950 F0 C"fan0" Q25500 ; create fan 1 on pin fan0 and set its frequency M106 P0 H-1 C"Print Fan" F25500 ; set fan 1 value. Thermostatic control is turned off ; BOARD FAN M950 F1 C"fan1" Q50 ; create fan 0 on pin fan1 and set its frequency M106 P1 S1 H0 T45 C"Board Fan" ; set fan 0 value. Thermostatic control is turned on ; HOTEND FAN M950 F2 C"fan2" Q25000 ; create fan 2 on pin fan2 and set its frequency M106 P2 S0.5 H1 T45 C"Hotend Fan" ; set fan 2 value. Thermostatic control is turned on ; Tools M563 P0 D0 H1 F0 ; define tool 0 G10 P0 X0 Y0 Z0 ; set tool 0 axis offsets G10 P0 R0 S0 ; set initial tool 0 active and standby temperatures to 0C ; Custom settings are not defined ; Miscellaneous M575 P1 S1 B57600 ; enable support for PanelDue M501 -
I also just remembered that the other change I did around the time the crashes started was swapping to a mosquito hotend.
I should also probably post what my machine is:
Custom Delta
Duet 2 Wifi
duet smart effector
24v 350w meanwell PSU
berd air max
mosquito mounted with the 713maker smart effector adapter
24v heated bed, wired directly into the duet
e3d hemera, mounted as a flying extruder
paneldue 7i -
Neither the bird air nor the osquito seem likely to cause the fault.
Will have to wait to see what @dc42 thinks of the error message.
In the meantime, please continue to gather M122 reports if it crashes.
Perhaps share a gcode file that produces the crash.
-
@Phaedrux
Another crash from this morning (berd air still connected)The gcode file is too large to attach here -- is there a preferred hosting site for me to upload the file?
M122 === Diagnostics === RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP Used output buffers: 3 of 24 (11 max) === RTOS === Static ram: 27980 Dynamic ram: 93236 of which 60 recycled Exception stack ram used: 480 Never used ram: 9316 Tasks: NETWORK(ready,368) HEAT(blocked,1224) MAIN(running,1848) IDLE(ready,80) Owned mutexes: WiFi(NETWORK) === Platform === Last reset 00:12:58 ago, cause: software Last software reset at 2020-11-05 11:55, reason: Hard fault, spinning module PrintMonitor, available RAM 8996 bytes (slot 1) Software reset code 0x4069 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x0441f803 BFAR 0xe000ed38 SP 0x20005414 Task MAIN Stack: 0042eaf1 20003d50 60030000 00000001 00000009 00000014 0044eaab 0044eaae 21030000 3f000000 41492f1a Error status: 0 MCU temperature: min 24.1, current 28.4, max 28.7 Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes Driver 0: standstill, SG min/max 0/158 Driver 1: standstill, SG min/max 5/150 Driver 2: standstill, SG min/max 0/166 Driver 3: standstill, SG min/max not available Driver 4: standstill, SG min/max not available Date/time: 2020-11-05 12:08:02 Cache data hit count 1269324432 Slowest loop: 5.60ms; fastest: 0.14ms I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0 === Storage === Free file entries: 10 SD card 0 detected, interface speed: 20.0MBytes/sec SD card longest read time 1.2ms, write time 0.0ms, max retries 0 === Move === Hiccups: 0(0), FreeDm: 169, MinFreeDm: 163, MaxWait: 19944ms Bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves: 5, completed moves: 5, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === AuxDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === Heat === Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 === GCodes === Segments left: 0 Movement lock held by null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 Daemon is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 15.91ms; fastest: 0.00ms Responder states: HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions HTTP sessions: 1 of 8 - WiFi - Network state is active WiFi module is connected to access point Failed messages: pending 0, notready 0, noresp 0 WiFi firmware version 1.23 WiFi MAC address dc:4f:22:6e:6c:ca WiFi Vcc 3.43, reset reason Unknown WiFi flash size 4194304, free heap 21840 WiFi IP address 192.168.86.26 WiFi signal strength -53dBm, reconnections 0, sleep mode modem Socket states: 4 0 0 0 0 0 0 0 -
@bwnance said in Duet 2 Wifi rebooting/crashing during print:
is there a preferred hosting site for me to upload the file?
You may be able to zip it and change the file extension to .txt and upload it here if it's within 4Mb (i think) otherwise dropbox or google drive.
-
@Phaedrux
Good call! Attached is the zipped gcode file, renamed to .txt: -
What's in the macro here:
M98 P"0:/macros/start.g"
-
@Phaedrux
I've got a little bit of a macro tree:
start.g:G28 M98 P"0:/macros/brushclean.g" M98 P"0:/macros/bottomhome.g" G29 S1 ; load height mapbrushclean.g:
G0 Z25 F12000 M564 S0 ; disable limits G0 X0 Y0 Z12 F3000 ; middle of bed, z=5 G0 X0 Y-130 ; middle of brush G0 Z8 ; drop nozzle slightly G0 X15 ; G0 Y-128 X-15 G0 Y-125 X15 G0 Y-125 X0 G0 Z12 G0 X0 Y0 M564 S1 ; enable limits againbottomhome.g:
M106 P0 S0 G0 Z25 F12000 G30 G0 Z15 F12000 -
@Phaedrux @dc42 Hey y'all, just bumping this again -- here's another crash report from today, along with the offending gcode:
tree_new.txt=== Diagnostics === RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP Used output buffers: 3 of 24 (11 max) === RTOS === Static ram: 27980 Dynamic ram: 93236 of which 60 recycled Exception stack ram used: 304 Never used ram: 9492 Tasks: NETWORK(ready,368) HEAT(blocked,1224) MAIN(running,1880) IDLE(ready,80) Owned mutexes: WiFi(NETWORK) === Platform === Last reset 00:29:46 ago, cause: software Last software reset at 2020-12-14 23:54, reason: Hard fault, spinning module GCodes, available RAM 8940 bytes (slot 2) Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x04427003 BFAR 0xa5a5a5a5 SP 0x2001ffbc Task MAIN Stack: 0040f451 0040f452 0100001f 200052f0 200052ef 20008c2c 20003d50 a5a5a5a5 ffffffed 00000000 00f00000 Error status: 0 MCU temperature: min 23.4, current 28.8, max 29.5 Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes Driver 0: standstill, SG min/max not available Driver 1: standstill, SG min/max not available Driver 2: standstill, SG min/max not available Driver 3: standstill, SG min/max not available Driver 4: standstill, SG min/max not available Date/time: 2020-12-15 00:24:45 Cache data hit count 2932587391 Slowest loop: 3.44ms; fastest: 0.14ms I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0 === Storage === Free file entries: 10 SD card 0 detected, interface speed: 20.0MBytes/sec SD card longest read time 0.9ms, write time 0.0ms, max retries 0 === Move === Hiccups: 0(0), FreeDm: 169, MinFreeDm: 169, MaxWait: 0ms Bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === AuxDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === Heat === Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 === GCodes === Segments left: 0 Movement lock held by null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 Daemon is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 15.73ms; fastest: 0.00ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions HTTP sessions: 1 of 8 - WiFi - Network state is active WiFi module is connected to access point Failed messages: pending 0, notready 0, noresp 0 WiFi firmware version 1.23 WiFi MAC address dc:4f:22:6e:6c:ca WiFi Vcc 3.43, reset reason Unknown WiFi flash size 4194304, free heap 21056 WiFi IP address 192.168.86.34 WiFi signal strength -57dBm, reconnections 0, sleep mode modem Socket states: 0 0 0 0 0 0 0 0 -
and another one just now, the more the merrier, i guess
=== Diagnostics === RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP Used output buffers: 3 of 24 (11 max) === RTOS === Static ram: 27980 Dynamic ram: 93236 of which 60 recycled Exception stack ram used: 472 Never used ram: 9324 Tasks: NETWORK(ready,408) HEAT(blocked,1224) MAIN(running,1840) IDLE(ready,80) Owned mutexes: WiFi(NETWORK) === Platform === Last reset 00:00:37 ago, cause: software Last software reset at 2020-12-15 02:33, reason: Hard fault, spinning module none, available RAM 8892 bytes (slot 3) Software reset code 0x4071 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x0441f003 BFAR 0x91200200 SP 0x2001ff8c Task NETW Stack: 00454e15 00454e20 6100000e 40090000 0041d011 00000000 20002860 10000000 2000b98c 00000000 200034f0 Error status: 0 MCU temperature: min 24.2, current 24.8, max 25.0 Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes Driver 0: ok, SG min/max not available Driver 1: ok, SG min/max not available Driver 2: standstill, SG min/max not available Driver 3: standstill, SG min/max not available Driver 4: standstill, SG min/max not available Date/time: 2020-12-15 02:34:35 Cache data hit count 58687792 Slowest loop: 3.66ms; fastest: 0.14ms I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0 === Storage === Free file entries: 8 SD card 0 detected, interface speed: 20.0MBytes/sec SD card longest read time 1.0ms, write time 0.0ms, max retries 0 === Move === Hiccups: 0(0), FreeDm: 166, MinFreeDm: 166, MaxWait: 18624ms Bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves: 1, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: 3 === AuxDDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 CDDA state: -1 === Heat === Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 === GCodes === Segments left: 0 Movement lock held by HTTP HTTP is idle in state(s) 1 0 8, running macro Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 Daemon is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 15.75ms; fastest: 0.00ms Responder states: HTTP(2) HTTP(4) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions HTTP sessions: 1 of 8 - WiFi - Network state is active WiFi module is connected to access point Failed messages: pending 0, notready 0, noresp 0 WiFi firmware version 1.23 WiFi MAC address dc:4f:22:6e:6c:ca WiFi Vcc 3.43, reset reason Unknown WiFi flash size 4194304, free heap 21408 WiFi IP address 192.168.86.34 WiFi signal strength -55dBm, reconnections 0, sleep mode modem Socket states: 4 2 0 0 0 0 0 0 -
Since no one has jumped in here, my usual opinion when I see something like this is that (assuming the board is ok) it's either a power supply issue or a voltage noise issue.
How much power does your heated bed use? Is it enough to impact the power supply?
How is the 24V to the pump connected? Could noise on the 24V line be caused by the pump pwm?
-
@markz
Resistance on my heated bed (measured at connector on board) is 3.3ohms, it's a 24v bed, so that would be a ~175watt bed, correct? I'm using a Mean Well 24v 350w PSU and my hotend is 50w, so i should have plenty of extra room, wattage wise, right?24v to the pump is over a fan header on the board (AFAIK newer duets have the flyback diode on all fan inputs internally), I'm not sure how to diagnose if i have noise on the 24v line.
-
@bwnance thanks.
The pump has two inputs (kind of) power and control. I know the control line goes to the fan pins but where do you pull off the 24V supply for the pump - is that from the power supply or the duet board somewhere? Are you getting the 24v from the fan header?
-
@markz I'm not using "EZ-Switch" board that was packaged with the berd-air, so both lines are hooked up to the fan header -- as best as i could tell before installing, this was fine, since the fan headers have flybacks wired into them
-
Can you try 3.2 Beta 4? It's believed that a fix for this crash may already be present there.
-
@Phaedrux just upgraded, will do a few test prints! We shall see...
-
@bwnance Just realized there is an odd ordering problem on that page that doesn't list the most recent at the top.
https://github.com/Duet3D/RepRapFirmware/releases/tag/3.2beta4.1
Beta 4.1 can be found here.
-
@Phaedrux ah, ty! gonna let this print finish on beta 4, then will update to 4.1
