Duet 2 Ethernet weird connection issue

pmeyer

Some new info: I see the issue in Chrome as well. I uploaded a few macro files through DWC and that kicked Chrome into the "connection lost" loop for 30 seconds. I'm starting to think my ethernet daughter-board is bad or unreliable.

I definitely don't want to invest more into an old Duet2. No way am spending >$200 for a duet3. I absolutely need the DWC connection to be rock solid so I can control prints. I'm exploring flashing Klipper on there as an alternative.

That will solve my other problem: I hate editing config files through the DWC interface, I want emacs.

That was what I was doing before things went bad: working on scripts to synch all the sys/ and macro/ files between my mac and the duet so I could edit locally. I suspect (without much data) that all those fast file transfers pushed my ethernet daughter-board over the edge.

Hopefully the USB-Serial connection is fine and I can just ssh into the pi to edit my klipper configs.

chrishamm

@pmeyer At first glance I'd say it is somehow related to your WiFi since the problem doesn't occur on a wired connection. If you see "Network error" without a following message, that typically means that something interrupts your HTTP connections.

Could you create a Wireshark dump when on WiFi so I could have a look at the traffic? I've worked with DWC and Firefox for ages and I have not seen such a problem before.

pmeyer

@chrishamm Thanks for replying!

Wifi definitely seems to exacerbate the problem, but I have no issues with the connectivity elsewhere. My mac is 4 ft from ASUS ZenWiFi AX6600, ~2ms ping. I now have the duet wired to the same router.

I'll think about debug steps. Possibilities:

Software:

wireshark on my mac
try it on a Pi4 or ubuntu over wifi
try a different wifi router on a standalone network, fixed IPs
see if I can reproduce it with wired connection
- multiple windows talking to the duet seems to make it worse, maybe can leverage that
try it on Win11 over wifi, different browsers

Hardware:

disconnect all the printer wiring from the board, isolate it from fan noise
try it with a 24v bench supply rather than the toolchanger PSU
last ditch: replace ethernet module (it's not soldered, right?)

I'll see what I can do, might take a few days.

What it feels like is that in certain traffic situations the webserver gets lost. Backpressure on the network? request flood? Overheat?

As a quick test I'm uploading a 512MB gcode file from my win11 machine (wired). While that is ongoing, another tab on the same machine, or another browser on the same machine can connect just fine. Slightly slower, but the duet seems to juggle the work ok.

However, when that upload is ongoing, no browser from the mac can connect. Safari gets closest, and actually displays a horribly formatted page (as if the css didn't get downloaded but the main content did).

safari corrupted screenshot

Actually, after uploading that file I have difficulty connecting from any browser. Safari consistently shows the corrupted version above. Chrome now reconnects 5-6 times before finally getting all the files and dropping into refresh mode where it just updates the data and seems stable.

Debug continues.

chrishamm

@pmeyer Note that the W5500 Ethernet chip of the Duet 2 Ethernet only supports up to 8 sockets, and some of them are typically allocated for other purposes than HTTP. You can send M122 to the Duet to see the actual allocation and socket states. If you want to dive deeper into the network internals, you could also enable debugging for the network and webserver parts in RRF using M111, but then you'd have to hook up a USB cable and serial terminal to the board, too. I hope a Wireshark trace will be sufficient to track this down, though.

pmeyer

@chrishamm I already have a pi set up for usb-serial connect, so I'll enable debugging.

I'm wondering if somehow the browser is using multiple sockets (establishing multiple connections for faster loading...)

chrishamm

@pmeyer That's likely. AFAIR there was another bug fix in a 3.5 beta allowing HTTP connections to remain alive, you may want to check if it works better with v3.5-b3, too.

pmeyer

@chrishamm Ok, I've got some debug dumps.

google drive link

the "*.log" ascii files are serial port dumps.

firefox_wired.log has m115 and m122 dumps, along with m111 (network/webserver) debug of a successful page load (wired)

The other two *.log have failed attempts (wifi, chrome and firefox).

There are also two wireshark dumps. Wireshark was running on my mac, capturing all traffic on the wired or wireless main connection (filter "host 10.200.20.20" => the duet fixed ip).

One dump for wired pass, one dump for wireless fail.

Let me know if you can make anything of it. What I'm seeing is some packet loss and the combination of chrome/firefox, the web app, and the server not being able to correctly retransmit.

pmeyer

@chrishamm What is the best way to upgrade to 3.5b3?

Is it this?

Download these two files:

Duet2CombinedFirmware.bin
DuetWebControl-SD.zip

go to the dwc interface, system tab
upload the bin file, then the webcontrol zip?

chrishamm

@pmeyer Yes, or vice versa. It doesn't really matter.

chrishamm

@pmeyer Thanks, that should be useful! I'll have a look at those logs/dumps tomorrow.

pmeyer

@chrishamm 3.5b3 didn't change the behavior.

If repeating the debug traces or wireshark on 3.5b3 will help, or running with different filters/flags would help, let me know.

I'm assuming right now it is some amount of packet loss and the web-app/server not able to overcome it.

To see if my wifi is especially bad/lossy, I also did a wireshark dump of a web connection to my local pfsense box. Traced a full session paging around through a bunch of options, no dropped packets.

My (tentative) conclusion is that something about the connection from my mac -> wifi -> router -> cable -> duet is making life difficult for the ethernet board and packets are being dropped...

Is there any way to do lower level debug of the ethernet daughterboard?

pmeyer

@chrishamm

Ok, I think I figured it out. Yesterday before all this started, I was trying to figure out how to manage my config files (working to set up a tool changer and wanted a real system with git to manage the files). I was exploring how to essentially rsynch the files from my mac to the duet.

As part of this I enabled FTP.

I then started doing some test scripts to synch the files, including copying back and forth a bunch of tiles.

I then did a 'clean' install of 3.4.5, but of course it had an empty /sys directory and was complaining, so I copied back in my config. That config had a line enabling ftp. So, not quite the "clean install" I thought I had.

With ftp disabled, I don't see issues. I do sometimes see a single "connection lost retrying" cycle before the full page comes up, but it appears to be able to recover from it. Normally I don't see any retry, just a clean load.

Summary:
If you are having issues with Duet Web Control having difficulty connecting on a Duet 2 Ethernet (or likely Duet 2 Wifi), make sure you have ftp disabled. For some reason, the resources allocated to ftp (even if they aren't currently in use) make the web server much less resilient to any network noise.