Web interface rejected my connection attempts

misan

My browser (Chrome) shows the message:

ERR_CONNECTION_REFUSED

A Wireshark network capture shows that the Duet 2 Wifi board rejects the connection (RST answers for each SYN attempt from the browser).

I was hesitant if poor wifi communication could be the problem, so I kept a ping command ongoing that started ten seconds before the first connection attempt.

PING 192.168.1.7 (192.168.1.7) 56(84) bytes of data.
64 bytes from 192.168.1.7: icmp_seq=1 ttl=255 time=37.9 ms
64 bytes from 192.168.1.7: icmp_seq=2 ttl=255 time=2.27 ms
64 bytes from 192.168.1.7: icmp_seq=3 ttl=255 time=3.55 ms
64 bytes from 192.168.1.7: icmp_seq=4 ttl=255 time=1.70 ms
64 bytes from 192.168.1.7: icmp_seq=5 ttl=255 time=4.97 ms
64 bytes from 192.168.1.7: icmp_seq=6 ttl=255 time=20.7 ms
64 bytes from 192.168.1.7: icmp_seq=7 ttl=255 time=1.48 ms
64 bytes from 192.168.1.7: icmp_seq=8 ttl=255 time=6.14 ms
64 bytes from 192.168.1.7: icmp_seq=9 ttl=255 time=1.54 ms
64 bytes from 192.168.1.7: icmp_seq=10 ttl=255 time=1.94 ms
64 bytes from 192.168.1.7: icmp_seq=11 ttl=255 time=1.88 ms
64 bytes from 192.168.1.7: icmp_seq=12 ttl=255 time=60887 ms
64 bytes from 192.168.1.7: icmp_seq=13 ttl=255 time=59879 ms
64 bytes from 192.168.1.7: icmp_seq=14 ttl=255 time=58857 ms
64 bytes from 192.168.1.7: icmp_seq=38 ttl=255 time=34287 ms
64 bytes from 192.168.1.7: icmp_seq=39 ttl=255 time=33264 ms
64 bytes from 192.168.1.7: icmp_seq=40 ttl=255 time=32244 ms
64 bytes from 192.168.1.7: icmp_seq=41 ttl=255 time=31221 ms
...

That was mostly to be sure wifi was working ok before and even during the failed connection attempt (though something delayed some of the ICMP responses awfully).
I am attaching, too, the response to the M122 command. Meanwhile, the CNC machine is machining a gcode file that will last more than one hour.
M112.txt

The machine is a 3-axis CNC working at relatively high speeds (F12000).

Not being able to connect through the web interface is a major problem. It forces us to power cycle the machine after each finished project to regain web access.

Is there anything we could do to avoid this behaviour?

jay_s_uk

@misan i would first suggest upgrading the firmware to 3.4.5 and also upgrading the wifi firmware from 1.23 to 1.27 and then see how you get on

misan

@jay_s_uk Thanks a lot. Doing it as soon as the current job finishes. I will report back.

misan

I have upgraded the board to 3.4.5 and the webserver to 1.27, but I am seeing disconnects like the one captured in the log picture below:

I was capturing the network traffic meanwhile in case that could help, and I am enclosing a Wireshark capture too. duet-disconnect.pcapng.bin

(J had to add the .bin extension to keep the editor happy, feel free to remove the .bin fro then end of the filename after downloading). One detail about the traffic capture is that the machine is not in my home network but on a workshop I am tunneling my computer's port 7777 to the machine's port 80.

So the bottom line is the software upgrade seems to have done nothing to fix the issue (which has me puzzled as I cannot rule out possible interference from VFD).

jay_s_uk

@misan can you grab an output of M122?

misan

@jay_s_uk Sure.

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.4.5 (2022-11-30 19:36:12) running on Duet WiFi 1.02 or later
Board ID: 08DLM-996RU-N85T0-6JTD6-3S86R-KASBN
Used output buffers: 1 of 26 (26 max)
=== RTOS ===
Static ram: 23836
Dynamic ram: 74000 of which 136 recycled
Never used RAM 13412, free system stack 120 words
Tasks: NETWORK(ready,32.9%,195) HEAT(notifyWait,0.0%,388) Move(notifyWait,0.5%,296) MAIN(running,66.5%,543) IDLE(ready,0.1%,30), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 02:24:52 ago, cause: software
Last software reset at 2022-12-09 07:38, reason: User, GCodes spinning, available RAM 13180, slot 0
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x14
Step timer max interval 0
MCU temperature: min 20.9, current 22.5, max 22.8
Supply voltage: min 11.6, current 11.7, max 11.9, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min n/a
Driver 1: standstill, SG min n/a
Driver 2: standstill, SG min n/a
Driver 3: standstill, SG min n/a
Driver 4: standstill, SG min n/a
Driver 5: 
Driver 6: 
Driver 7: 
Driver 8: 
Driver 9: 
Driver 10: 
Driver 11: 
Date/time: 2023-01-05 16:38:05
Cache data hit count 4294967295
Slowest loop: 557.46ms; fastest: 0.17ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 6.5ms, write time 224.4ms, max retries 0
=== Move ===
DMs created 83, segments created 29, maxWait 7712398ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 11246, completed 11207, hiccups 0, stepErrors 0, LaErrors 0, Underruns [8, 0, 2], CDDA state 3
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Segments left: 1
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X796.96 Y333.85 Z15.70" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== Network ===
Slowest loop: 1465.65ms; fastest: 0.07ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 2 of 8
= WiFi =
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 2
WiFi firmware version 1.27
WiFi MAC address 84:0d:8e:b2:ef:91
WiFi Vcc 3.39, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 25408
WiFi IP address 192.168.1.7
WiFi signal strength -49dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

jay_s_uk

@misan said in Web interface rejected my connection attempts:

WiFi signal strength -49dBm, mode 802.11n, reconnections 0, sleep mode modem

thats a reasonable wifi signal so its not that. have to wait for one of the duet guys to respond.
the other option is try 3.5b1 (although get the latest version from dc42 on the forum) and the corresponding wifi server 2.1b2 as thats built using a newer version of the SDK

misan

@jay_s_uk It is a production machine, I rather use a stable version. But mostly, I am not sure where or what to look for. The workshop has several CNC working, but they only seem to have problems once the rest of the office starts working (which does not use the wifi network nor run power equipment).

jay_s_uk

@misan best to wait then

misan

@jay_s_uk A bit more testing shows that during periods of unresponsiveness from the board (192.168.1.8), the computer (192.168.1.133) makes many connection attempts that go unanswered. The problem starts at 14.95 seconds into the capture capture3-filtered.pcapng.bin
That is a Wireshark packet capture over the wifi network. At no point does the wifi network stop working or show a degraded performance, so it does not seem to be related to interference.

jay_s_uk

@misan one for @Phaedrux

T3P3Tony

@misan Although its not obvious from the M122 report you posted before, one thing to try is a new SD card in case that is the issue. You can copy your existing SD card structure onto a new card and see if that resolves the issue.

misan

@T3P3Tony One additional detail to the issue is this other capture, where pings can be seen responded with a huge time difference, suggesting the board receives the messages but respond to them abnormally late (once the issue has passed). Have a look at ICMP request issued at 130 and responded at 2020 (almost 50 seconds later).
capture5.pcapng.bin

misan

@T3P3Tony A recent M122 output:

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.3 (2021-06-15 21:44:54) running on Duet WiFi 1.02 or later
Board ID: 0JD0M-9K662-MGPSS-6J1FG-3S46Q-KVUGY
Used output buffers: 1 of 24 (18 max)
=== RTOS ===
Static ram: 23876
Dynamic ram: 74152 of which 0 recycled
Never used RAM 16388, free system stack 126 words
Tasks: NETWORK(ready,647.7%,237) HEAT(delaying,0.3%,386) Move(notifyWait,52.2%,307) MAIN(running,544.4%,546) IDLE(ready,14.3%,29), total 1258.9%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 05:12:00 ago, cause: power up
Last software reset at 2022-11-08 16:55, reason: User, GCodes spinning, available RAM 16388, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Step timer max interval 0
MCU temperature: min 18.8, current 24.2, max 24.8
Supply voltage: min 23.6, current 23.8, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Driver 0: position 75056, standstill, SG min/max not available
Driver 1: position 42138, standstill, SG min/max not available
Driver 2: position 6103, standstill, SG min/max not available
Driver 3: position 0, standstill, SG min/max not available
Driver 4: position 0, standstill, SG min/max not available
Driver 5: position 0
Driver 6: position 0
Driver 7: position 0
Driver 8: position 0
Driver 9: position 0
Driver 10: position 0
Driver 11: position 0
Date/time: 2023-01-16 12:09:06
Cache data hit count 4294967295
Slowest loop: 217.84ms; fastest: 0.11ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 4.4ms, write time 42.8ms, max retries 0
=== Move ===
DMs created 83, maxWait 366651ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 164159, completed moves 164120, hiccups 5, stepErrors 0, LaErrors 0, Underruns [27567, 0, 0], CDDA state 3
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X1172.75 Y718.41 Z42.40" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 948.00ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(2), 1 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.25
WiFi MAC address 48:3f:da:48:b2:81
WiFi Vcc 3.43, reset reason Power up
WiFi flash size 4194304, free heap 19872
WiFi IP address 192.168.1.8
WiFi signal strength -56dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 2 0 0 0 0 0 0 0

T3P3Tony

@misan said in Web interface rejected my connection attempts:

Slowest loop: 948.00ms; fastest: 0.00ms

That looks long. not sure if it could be waiting for an SD card read or write which is why i suggested that test.

misan

@T3P3Tony I am going to test the SD replacement, but my order is in the mail

Anyway, what seems to be the cause is Dropbox's operation. It looks like some of the network traffic it broadcasts might be setting the ESP8266 into a weird state.

We know that the problem does not appear if we stop Dropbbox on our workshop computers from synchronizing.

I have a longer traffic capture, with the problem starting at 15.2 seconds (approx) but the file is too long for your system, so it can be downloaded from here (for a limited time) https://we.tl/t-m0dDrqfM2K

T3P3Tony

@misan ahh that's interesting, is dropbox doing some broadcast synchronisation on the LAN between your local computers? Maybe it has a setting that allows you to set the machines specifically and not broadcast?

misan

@T3P3Tony I have set the systems not to use LAN synchronization for a while to see if that helps or fixes the problem entirely. I will know more at the end of the day.

Please note I added a link to a more comprehensive network capture in the previous email I edited.

dc42

@misan you could try updating the wifi firmware to 2.1beta2 from https://github.com/Duet3D/RepRapFirmware/releases/tag/3.5.0beta1.

misan

@dc42 Sure. I will do it and report back.

Oops, I just saw I cannot do that remotely (wifi parameters must be re-entered). That means I have to go to the workshop. This will then take longer.