Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect
-
@Phaedrux I have installed the new Network Module. This second board has only failed 3 times that I know of making it very hard to diagnose. I did take photos of the solder joints of the header to board on both the old and new module as well as the top and bottom of the main board. I also looked a the surface mounted components on the module .. none of the joints look like they were cold or had large amounts of inclusions.. The photos are here.
Photos of Solder joints magnified
Now the waiting .. -
@Phaedrux Connection dropped once after replacement.. does not look like that fixed it.. Workaround using usb to disable/enable networking did work. I give up!
Will just live with it unless it starts happening much more frequently. Can't trouble shoot if it can not be reproduced on demand.. -
@Phaedrux Connection dropped again (2 times today) This time I check the monitor all was well, pulled out the print bead, Knocked off the print job with the spatula, looked at the monitor no issue. Went to fit the part, then back to the PC in the other room and now we are disconnected. Logon to octoprint and disable/enable network via usb and back in business. Seem like new module is worse than the original or it has nothing to do with the module. Well keep trying to find a way to make it fail on demand. Any of those solder joints look like an issue to you? (see previous post)
-
I didn't see any glaring issues on the photos of the soldering.
If it's happening with the new module as well I'm not sure what else to say. We've replaced the entire board and the module itself. Logically it would seem to be an issue inherent to the local network or the printer setup. Did you happen to break a mirror in the last 7 years?
-
@trobison said in Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect:
I open up a new Enterprise HP Switch as a test. There are only two ports in use on this 24 port switch. While working on tpre0.g the network dropped and cleared all contents in the file.
Does the switch have some kind of power saving that puts it in to standby until it sees traffic??
Or, if the only other thing connected is your computer, does that have power saving enabled on the Ethernet port?
I know some machines do shut down ports when idle and the wake-up time can cause lost packets - which in theory are automatically re-sent, but that relies on a normal network driver stack..
If you are only using two ports on the switch, can't you use a direct cable & omit the switch, at least for testing?
Also, you say you are "pulling the bed out" - does that cause any motor(s) to rotate??
If it does, that could be affecting or even damaging the Duet board, as the back EMF from an externally turned motor [acting as a generator] feeds back in via the driver electronics.Robert J.
Printers: Overlord pro, Kossel XL+ with Duet 6HC and "Frankentron", TronXY X5SA Pro converted to E3D toolchange with Duet 6HC and 1LC toolboards.
-
@rjenkinsgb Wow moving the motors damage the board??? That was never an issue with the 8bit Rambo.. why in gods name would 10 year newer tech not be able to do what the old did?
This has happened with the old board in the middle of a print job. That board failed more often. Also wanted to mention again, this board and the first board were only a couple numbers different in serial number assuming that is the order they were built on the line. Frist one was w09221 and the replacement is w09219 (actually 2 units prior to the first one if they are built in sequential order)The switch is a dumb switch, not much I can do. The original board failed plugged directly into the router.
I am not sure what to say about the moving of the bed, I do that very often and have done so since 2018 and will be very difficult to stop doing. If I had known I can not do this, I don't think I would have changed boards.
I will change the end script to move the Y all thay out so I do not have to move the bed to see if that eliminates the network drop.
-
@airscapes said in Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect:
I am not sure what to say about the moving of the bed, I do that very often and have done so since 2018 and will be very difficult to stop doing. If I had known I can not do this, I don't think I would have changed boards.
You'd have to move the bed quite quickly by hand to cause damage, and it would likely damage the driver. Sliding it out gently to remove the part shouldn't be an issue. Changing the end gcode to present the part for removal would be the best way to go, though I don't think it's the issue here.
-
@phaedrux said in Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect:
@airscapes said in Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect:
I am not sure what to say about the moving of the bed, I do that very often and have done so since 2018 and will be very difficult to stop doing. If I had known I can not do this, I don't think I would have changed boards.
You'd have to move the bed quite quickly by hand to cause damage, and it would likely damage the driver. Sliding it out gently to remove the part shouldn't be an issue. Changing the end gcode to present the part for removal would be the best way to go, though I don't think it's the issue here.
Oh that is good to know! It will be interesting to see if the keep happening. Have only done 4 prints jobs today with no issue.. and yes bed as been moved but am updating the end script as I go forward. Should have done that years ago, but back then did not know how..
-
@rjenkinsgb I tried several switches and a direct cable. I have then moved to a different PC when the fault happens and still have this issue. There is no power saving on any nics here. The only fix is cycling the Duet in my case. I hope to replace the Network port on the duet this weekend.
-
@trobison
I replaced the Ethernet module, and the issue returns. I upgraded to 3.4 hoping this could help, but the issue returns randomly. Cycling the Duet and I am good. The i7 screen is the only way to interact with the printer at this time. I replace the cable with a new cable during the fault. That did not fix it. Different cable and switches, I am not sure what to do next. -
@trobison said in Duet 2 Ethernet WC 3.3.0 crashes, have to reset to reconnect:
I am not sure what to do next.
I guess at this point a wireshark capture may help, but if the duet isn't even reachable by ping I'm not sure that would be fruitful.
-
@phaedrux
Ok, I have left the browser open to the Duet web page when the printer is in an idle state. It seems ok. I seem to lose connection during prints. After this print job completed, network connectivity returns.
While in a faulted state (I was losing connectivity during a print) I connected directly to a laptop (no switch). I had the same issue. After the print, connection returns.
If you are changing a file on the duet during a network dropout, it removes all content in the file, 0 bytes. That does bite when that happens. -
It sounds like maybe an interference problem? Do you have the frame, motors, etc grounded?
-
@phaedrux I believe everything is grounded. It's an E3D ToolChanger - built to their spec in the instructions. The frame is grounded, the bed is grounded, X Y Z motors are grounded. I am not sure if the extruders are grounded. The extruders are held via printed parts. Do I need to ground these? I am not sure how I can do this with the ToolChanger.
I am getting fine prints now, but the disconnections are disconcerting.
Where would you start? Any suggestions?
Regards.
-
@trobison I grounded the tool heads and verified that they are grounded. All four tool tools checked with a multimeter and they are now grounded with an extra wire in the harness.
I still have the same issue.20/03/2022, 14:43:29 Connection established 20/03/2022, 14:43:28 Connection interrupted, attempting to reconnect... Operation failed (Reason: Service Unavailable) 20/03/2022, 14:42:45 Connection established 20/03/2022, 14:42:45 Connection interrupted, attempting to reconnect... Operation failed (Reason: Service Unavailable) 20/03/2022, 14:40:54 Connection established 20/03/2022, 14:40:54 Connection interrupted, attempting to reconnect... Operation failed (Reason: Service Unavailable) 20/03/2022, 14:39:41 Connection established 20/03/2022, 14:39:40 Connection interrupted, attempting to reconnect... Operation failed (Reason: Service Unavailable) 20/03/2022, 14:38:18 Connection establishedI can't see anything else to ground. What else could cause interference?
-
@trobison It was a long shot. I'm really not sure what the source of the connection drops could be at this point. There's no problem with the config and we've replaced the module. Reason would say it's a network issue, but where or how is hard to determine. It's not impossible that the new module couldn't have the same issue as the old one, but we're not sure what the mechanism of failure could be.
-
@phaedrux Is there something like top in the Duet 2? I have hooked a single network cable from printer to computer, so I don't think it's anything on the network or funky broadcasts on the network. If there was, I would see this issue while the printer was idle.
It prints well, and finishes nicely, I just have difficulty looking at the web page during an active print. I typically load up all the jobs and start them via the screen now. It's not the best solution but I can see the estimated remaining times or height of Z.
I will continue to look at the wiring. All the harnesses were supplied by E3D, and they look great.
Regards, -
@trobison
Are you running any DWC plugins?
Have you posted your config.g yet? -
@phaedrux The only plugin I have run is the G-Code Viewer.
I have replaced the SD Card with and SanDisk Extreme, but no change.
The next change will be swapping out the power supply. I have a spare on the bench and I will install that in a few days.Here is my config.g file
; Configuration file for Duet WiFi (firmware version 3.3) ; executed by the firmware on start-up ; ; generated by RepRapFirmware Configuration Tool v3.3.5 on Sat Nov 13 2021 18:20:50 GMT+1100 (Australian Eastern Daylight Time) ; General preferences M111 S0 ; Debugging off G21 ; Work in millimetres M575 P1 S1 B57600 ; enable support for PanelDue G90 ; send absolute coordinates... M83 ; ...but relative extruder moves M550 P"ToolChanger" ; set printer name M669 K1 ; select CoreXY mode ; Network M552 S1 P192.168.10.60 ; enable network and set IP address M553 P255.255.255.0 ; set netmask M554 P192.168.10.1 ; set gateway M586 P0 S1 ; Enable HTTP M586 P1 S0 ; Disable FTP M586 P2 S1 T0 ; Enable Telnet ; Drives M569 P0 S0 ; X physical drive 0 goes forwards M569 P1 S0 ; Y physical drive 1 goes forwards M569 P2 S1 ; Z physical drive 2 goes backwards M569 P3 S1 ; E0 physical drive 3 goes backwards M569 P4 S1 ; E1 physical drive 4 goes backwards M569 P5 S1 ; E2 physical drive 5 goes backwards M569 P6 S1 ; E3 physical drive 6 goes backwards M569 P7 S0 ; Coupler physical drive 7 goes forwards M569 P8 S0 ; Unused - physical drive 8 goes forwards M569 P9 S0 ; Unused - physical drive 9 goes forwards M584 X0 Y1 Z2 C7 E3:4:5:6 ; set drive mapping M350 X16 Y16 Z16 E16:16:16:16 I1 ; configure microstepping with interpolation M350 C16 I10 ; Configure microstepping without interpolation M92 X100 Y100 Z800 C91.022 E395:395:395:395 ; set steps per mm M566 X900 Y900 Z8 C2 E800:800:800:800 ; set maximum instantaneous speed changes (mm/min) - Jerk Settings M203 X35000 Y35000 Z1200 C5000 E5000:5000:5000:5000 ; set maximum speeds (mm/min) M201 X6000 Y6000 Z400 C500 E3000:3000:3000:3000 ; set accelerations (mm/s^2) M906 X1800 Y1800 Z1330 I30 ; Idle motion motors to 30% M906 E1000:1000:1000:1000 C500 I10 ; Idle extruder motors to 10% M84 S30 ; Set idle timeout ; Axis Limits M208 X-10 Y-7 Z0 S1 ; set axis minima M208 X350 Y275 Z300 S0 ; set axis maxima M208 C-45:360 ; Endstops M574 X1 Y1 S3 ; Set X / Y endstop stall detection M574 C0 Z0 ; No C Z endstop ; Z probe M558 P8 C"zstop" H3 F360 I0 T20000 ; Set Z probe type to switch, the axes for which it is used and the dive height + speeds G31 P200 X0 Y0 Z0 ; Set Z probe trigger value, offset and trigger height M557 X30:333 Y50:249 S40 ; Set probing points ; Heaters M308 S0 P"bedtemp" Y"thermistor" A"Bed" T100000 B4138 ; configure sensor 0 as thermistor on pin bedtemp M950 H0 C"bedheat" T0 q10 ; create bed heater output on bedheat and map it to sensor 0 M307 H0 B0 S1.00 ; disable bang-bang mode for the bed heater and set PWM limit M140 H0 ; map heated bed to heater 0 M143 H0 S200 ; set temperature limit for heater 0 to 200C M308 S1 P"e0temp" Y"thermistor" A"T0" T100000 B4725 C7.06e-8 ; configure sensor 1 as thermistor on pin e0temp M950 H1 C"e0heat" T1 ; create nozzle heater output on e0heat and map it to sensor 1 M307 H1 B0 S1.00 ; disable bang-bang mode for heater and set PWM limit M143 H1 S300 ; set temperature limit for heater 1 to 300C M308 S2 P"e1temp" Y"thermistor" A"T1" T100000 B4725 C7.06e-8 ; configure sensor 2 as thermistor on pin e1temp M950 H2 C"e1heat" T2 ; create nozzle heater output on e1heat and map it to sensor 2 M307 H2 B0 S1.00 ; disable bang-bang mode for heater and set PWM limit M143 H2 S300 ; set temperature limit for heater 2 to 300C M308 S3 P"duex.e2temp" Y"thermistor" A"T2" T100000 B4725 C7.06e-8 ; configure sensor 3 as thermistor on pin duex.e2temp M950 H3 C"duex.e2heat" T3 ; create nozzle heater output on duex.e2heat and map it to sensor 3 M307 H3 B0 S1.00 ; disable bang-bang mode for heater and set PWM limit M143 H3 S300 ; set temperature limit for heater 3 to 300C M308 S4 P"duex.e3temp" Y"thermistor" A"T3" T100000 B4725 C7.06e-8 ; configure sensor 4 as thermistor on pin duex.e3temp M950 H4 C"duex.e3heat" T4 ; create nozzle heater output on duex.e3heat and map it to sensor 4 M307 H4 B0 S1.00 ; disable bang-bang mode for heater and set PWM limit M143 H4 S300 ; set temperature limit for heater 4 to 300C ; Fans M950 F0 C"fan0" Q0 ; create fan 0 on pin fan0 and set its frequency - Fan Not used M106 P0 S0 H-1 ; set fan 0 value. Thermostatic control is turned off - Fan not used M950 F1 C"fan1" Q500 ; create fan 1 on pin fan1 and set its frequency M106 P1 S1 H1 T70 ; set fan 1 value. Thermostatic control is turned on M950 F2 C"fan2" Q0 ; create fan 2 on pin fan2 and set its frequency M106 P2 S0 H-1 ; set fan 2 value. Thermostatic control is turned off M950 F3 C"duex.fan3" Q500 ; create fan 3 on pin duex.fan3 and set its frequency M106 P3 S1 H2 T70 ; set fan 3 value. Thermostatic control is turned on M950 F4 C"duex.fan4" Q0 ; create fan 4 on pin duex.fan4 and set its frequency M106 P4 S0 H-1 ; set fan 4 value. Thermostatic control is turned off M950 F5 C"duex.fan5" Q500 ; create fan 5 on pin duex.fan5 and set its frequency M106 P5 S1 H3 T70 ; set fan 5 value. Thermostatic control is turned on M950 F6 C"duex.fan6" Q0 ; create fan 6 on pin duex.fan6 and set its frequency M106 P6 S0 H-1 ; set fan 6 value. Thermostatic control is turned off M950 F7 C"duex.fan7" Q500 ; create fan 7 on pin duex.fan7 and set its frequency M106 P7 S1 H4 T70 ; set fan 7 value. Thermostatic control is turned on M950 F8 C"duex.fan8" Q0 ; create fan 8 on pin duex.fan8 and set its frequency M106 P8 S0 H-1 ; set fan 8 value. Thermostatic control is turned off ; Tools M563 P0 S"T0" D0 H1 F2 ; define tool 0 G10 P0 X0 Y0 Z0 ; set tool 0 axis offsets G10 P0 R0 S0 ; set initial tool 0 active and standby temperatures to 0C M563 P1 S"T1" D1 H2 F4 ; define tool 1 G10 P1 X0 Y0 Z0 ; set tool 1 axis offsets G10 P1 R0 S0 ; set initial tool 1 active and standby temperatures to 0C M563 P2 S"T2" D2 H3 F6 ; define tool 2 G10 P2 X0 Y0 Z0 ; set tool 2 axis offsets G10 P2 R0 S0 ; set initial tool 2 active and standby temperatures to 0C M563 P3 S"T3" D3 H4 F8 ; define tool 3 G10 P3 X0 Y0 Z0 ; set tool 3 axis offsets G10 P3 R0 S0 ; set initial tool 3 active and standby temperatures to 0C ;tool offsets ; !ESTIMATED! offsets for: ; Hemera-tool: X20 Y43.5 Z-6 ; G10 P0 X-0.10 Y-1.0 Z-7 - Use this as initial starting point for leveling nozzle heights to avoid crashing nozzles into bed ; Tool Offsets G10 P0 X-0.00 Y0.00 Z-5.31 ; Tool 0 G10 P1 X-0.32 Y-0.76 Z-5.40 ; Tool 1 G10 P2 X-0.20 Y-0.98 Z-5.42 ; Tool 2 G10 P3 X-0.32 Y-0.96 Z-5.53 ; Tool 3 ; Pressure advance was turned off by default. M572 D0 S0.020 ; pressure advance T0 Hemera M572 D1 S0.020 ; pressure advance T1 M572 D2 S0.020 ; pressure advance T2 M572 D3 S0.020 ; pressure advance T3 ; Custom settings - Switch in tool grabber ;Safty Switches M950 J0 C"!^e0stop" ; NC Microswitch - In use on Tool Grabber ;M950 J1 C"!^e1stop" ; NO Microswitch Attached yet (Free) ;Duet 2 WiFi and Ethernet: Use this command to tell RRF about the accelerometer: ;M955 P0 C"spi.cs4+spi.cs3" M593 F42.2 ; cancel ringing at 42.2Hz (https://forum.e3d-online.com/threads/accelerometer-and-resonance-measurements-of-the-motion-system.3445/) M593 P"daa" F42.2 ; use DAA to cancel ringing at 42.5Hz ; Pebble Wiper Config M950 S0 C"duex.pwm5" ; Miscellaneous M501 ; load saved parameters from non-volatile memory -
@trobison I replaced the power supply, this has not fixed the problem.
Replaced the SD card.
I put the whole system on a 3000 VA UPS to check if there is noisy power causing an issue. This has not fixed the issue.
I have disconnected and reconnected all plugs on the board as well. Same issue.
The issue seems to be getting worse. I have gone through the wiring a few times and can not see any issues. The printer can set for yours without an issue. When I send a print, disconnects occur.
Is there anything else I can check?
25/03/2022, 16:34:55 Connection established 25/03/2022, 16:34:54 Connection interrupted, attempting to reconnect... Operation failed (Reason: Service Unavailable) 25/03/2022, 16:34:42 File 0:/gcodes/Benchy_N2_PetG.gcode selected for printing 25/03/2022, 16:34:41 M32 "0:/gcodes/Benchy_N2_PetG.gcode" File 0:/gcodes/Benchy_N2_PetG.gcode selected for printing 25/03/2022, 16:09:49 Connection establishedThe space the printer is located is 20 degrees. This is a shot of the main board. Not sure it this helps. The network daughter board is not hot ~35 C. There is no back cover on the ToolChanger yet, so air flow should not be an issue.
Regards
