Duet 2 Ethernet weird connection issue
-
My toolchanger has a Duet 2 Ethernet. Haven't really used the printer, trying to get it up again (it sat for a few years).
Running rr3.4.5, fresh 8GB SD card install.
I'm trying to connect to it from my Mac. If my mac is connected to wired ethernet, I can connect to DWC just fine in Firefox, Chrome, or Safari. Ping is <1ms.
If I disconnect the ethernet cable and go wifi, I can connect to DWC just fine with Safari or Chrome. Firefox (unfortunately my main browser) has issues. It will get into a loop of "Connection interrupted, attempting to reconnect","Connection Established", about a 1 second cycle. It'll do that for a minute or two and eventually connect. I can see more and more of the DWC info (temperatures, etc.) showing up as it cycles.
The printer is idle during all of this.
Weirdly, if I have Safari open to DWC and then try to connect with Firefox, Safari will start losing connection and cycling as well.
If I go into Developer Tools/Network on Firefox and 'throttle' the connection to "Regular 4G/LTE" or slower, it can connect just fine, not even a hiccup...
It seems like Firefox might be flooding the board with requests for a bunch of frames at once(?) and the board can't keep up? Chrome/Safari make requests in a different way? Just speculating here. Doesn't explain why it's fine wired, though...
I have solid wifi (500 Mb/s out to the world, should be better, 4 ft from wifi router, one hop from there to router duet is connected to, 3ms ping).
It seems to be getting a bunch of these errors (from the Firefox network tab console):
Error: Network error ExtendableBuiltin app.d0ad7b20.js:42 _createSuperInternal Babel NetworkError errors.js:16 onerror PollConnector.js:168 PollConnector.js:584:11 Error: Network error ExtendableBuiltin app.d0ad7b20.js:42 _createSuperInternal Babel NetworkError errors.js:16 onerror BaseConnector.js:59 index.js:68:13 Error: Network errorI'm going to keep debugging, but just curious if anybody has had similar issues.
-
Some new info: I see the issue in Chrome as well. I uploaded a few macro files through DWC and that kicked Chrome into the "connection lost" loop for 30 seconds. I'm starting to think my ethernet daughter-board is bad or unreliable.
I definitely don't want to invest more into an old Duet2. No way am spending >$200 for a duet3. I absolutely need the DWC connection to be rock solid so I can control prints. I'm exploring flashing Klipper on there as an alternative.
That will solve my other problem: I hate editing config files through the DWC interface, I want emacs.
That was what I was doing before things went bad: working on scripts to synch all the sys/ and macro/ files between my mac and the duet so I could edit locally. I suspect (without much data) that all those fast file transfers pushed my ethernet daughter-board over the edge.
Hopefully the USB-Serial connection is fine and I can just ssh into the pi to edit my klipper configs.
-
@pmeyer At first glance I'd say it is somehow related to your WiFi since the problem doesn't occur on a wired connection. If you see "Network error" without a following message, that typically means that something interrupts your HTTP connections.
Could you create a Wireshark dump when on WiFi so I could have a look at the traffic? I've worked with DWC and Firefox for ages and I have not seen such a problem before.
-
@chrishamm Thanks for replying!
Wifi definitely seems to exacerbate the problem, but I have no issues with the connectivity elsewhere. My mac is 4 ft from ASUS ZenWiFi AX6600, ~2ms ping. I now have the duet wired to the same router.
I'll think about debug steps. Possibilities:
Software:
- wireshark on my mac
- try it on a Pi4 or ubuntu over wifi
- try a different wifi router on a standalone network, fixed IPs
- see if I can reproduce it with wired connection
- multiple windows talking to the duet seems to make it worse, maybe can leverage that
- try it on Win11 over wifi, different browsers
Hardware:
-
disconnect all the printer wiring from the board, isolate it from fan noise
-
try it with a 24v bench supply rather than the toolchanger PSU
-
last ditch: replace ethernet module (it's not soldered, right?)
I'll see what I can do, might take a few days.
What it feels like is that in certain traffic situations the webserver gets lost. Backpressure on the network? request flood? Overheat?
As a quick test I'm uploading a 512MB gcode file from my win11 machine (wired). While that is ongoing, another tab on the same machine, or another browser on the same machine can connect just fine. Slightly slower, but the duet seems to juggle the work ok.
However, when that upload is ongoing, no browser from the mac can connect. Safari gets closest, and actually displays a horribly formatted page (as if the css didn't get downloaded but the main content did).
Actually, after uploading that file I have difficulty connecting from any browser. Safari consistently shows the corrupted version above. Chrome now reconnects 5-6 times before finally getting all the files and dropping into refresh mode where it just updates the data and seems stable.
Debug continues.
-
@pmeyer Note that the W5500 Ethernet chip of the Duet 2 Ethernet only supports up to 8 sockets, and some of them are typically allocated for other purposes than HTTP. You can send
M122to the Duet to see the actual allocation and socket states. If you want to dive deeper into the network internals, you could also enable debugging for the network and webserver parts in RRF usingM111, but then you'd have to hook up a USB cable and serial terminal to the board, too. I hope a Wireshark trace will be sufficient to track this down, though. -
@chrishamm I already have a pi set up for usb-serial connect, so I'll enable debugging.
I'm wondering if somehow the browser is using multiple sockets (establishing multiple connections for faster loading...)
-
@pmeyer That's likely. AFAIR there was another bug fix in a 3.5 beta allowing HTTP connections to remain alive, you may want to check if it works better with v3.5-b3, too.
Duet software engineer
-
@chrishamm Ok, I've got some debug dumps.
the "*.log" ascii files are serial port dumps.
firefox_wired.log has m115 and m122 dumps, along with m111 (network/webserver) debug of a successful page load (wired)
The other two *.log have failed attempts (wifi, chrome and firefox).
There are also two wireshark dumps. Wireshark was running on my mac, capturing all traffic on the wired or wireless main connection (filter "host 10.200.20.20" => the duet fixed ip).
One dump for wired pass, one dump for wireless fail.
Let me know if you can make anything of it. What I'm seeing is some packet loss and the combination of chrome/firefox, the web app, and the server not being able to correctly retransmit.
-
@chrishamm What is the best way to upgrade to 3.5b3?
Is it this?
- Download these two files:
Duet2CombinedFirmware.bin DuetWebControl-SD.zip- go to the dwc interface, system tab
- upload the bin file, then the webcontrol zip?
-
@pmeyer Yes, or vice versa. It doesn't really matter.
-
@pmeyer Thanks, that should be useful! I'll have a look at those logs/dumps tomorrow.
Duet software engineer
-
@chrishamm 3.5b3 didn't change the behavior.
If repeating the debug traces or wireshark on 3.5b3 will help, or running with different filters/flags would help, let me know.
I'm assuming right now it is some amount of packet loss and the web-app/server not able to overcome it.
To see if my wifi is especially bad/lossy, I also did a wireshark dump of a web connection to my local pfsense box. Traced a full session paging around through a bunch of options, no dropped packets.
My (tentative) conclusion is that something about the connection from my mac -> wifi -> router -> cable -> duet is making life difficult for the ethernet board and packets are being dropped...
Is there any way to do lower level debug of the ethernet daughterboard?
-
Ok, I think I figured it out. Yesterday before all this started, I was trying to figure out how to manage my config files (working to set up a tool changer and wanted a real system with git to manage the files). I was exploring how to essentially rsynch the files from my mac to the duet.
As part of this I enabled FTP.
I then started doing some test scripts to synch the files, including copying back and forth a bunch of tiles.
I then did a 'clean' install of 3.4.5, but of course it had an empty /sys directory and was complaining, so I copied back in my config. That config had a line enabling ftp. So, not quite the "clean install" I thought I had.
With ftp disabled, I don't see issues. I do sometimes see a single "connection lost retrying" cycle before the full page comes up, but it appears to be able to recover from it. Normally I don't see any retry, just a clean load.
Summary:
If you are having issues with Duet Web Control having difficulty connecting on a Duet 2 Ethernet (or likely Duet 2 Wifi), make sure you have ftp disabled. For some reason, the resources allocated to ftp (even if they aren't currently in use) make the web server much less resilient to any network noise.
