Wifi 2.1beta6 from 3.5.0-rc.2/3 still disconnecting

Chriss

@gloomyandy I wounder how that works than. What I see on linux is that the board behaves like any other serial device. Like a Arduino, a USB2Serial adapter etc. And what I do is connection with a terminal program like minicom to that port. So the "getty" (to stay here in the linux term) on the other side (on the board) must use the same speed as the terminal does. Both sides can not decode the signals via the serial interface otherwise.
That is at least how I understand serial connections, that is the first time in more than 30 years that I have heard that the baud rated does not matter. Do not get me wrong, I believe you, I'm a bit surprised only and I would like to understand it because that is against 80% of my understanding how serial connections work. (Or the duet board has some magic to detect the baud rate from the terminal and use that speed.)

Cheers, Chriss

Chriss

I had a look at my WiFi access point. The AP told me that the "problematic board" is connected, the mac and the IP are correct. My workstation is connected to the AP via wire.
Don't know whether that helps or not, but I think that I should share every observation.

Cheers, Chriss

gloomyandy

@Chriss As I said there is no actual serial port hardware involved in the connection between your linux box and the RRF board. There is a software driver on the linux side that sits on top of the USB endpoint that basically makes the USB connection look like a serial port, the USB system transfers the bytes to the RRF board and on the RRF side there is again software that allow RRF to simply view the data in and out as a simple byte stream. That entire process is defined by the USB CDC standard I mentioned above. This standard does allow for the communication via USB of the baudrate and other settings but in the case of a software only stack as used by RRF this information is not needed and is ignored.

That same standard is used to provide USB to UART devices. These present the same USB interface to your linux system, but they have an actual hardware UART interface that can then talk to other hardware UART devices. Some devices (like the original Arduinos) do not have built in USB hardware so instead they use a USB to UART converter (like the various FTDI chips or the CH340 and CP2102) and this in turn connects via an actual UART serial protocol to the UART built into the Arduino. In this case you do need to ensure that the baud rate and other settings match.

But more advanced micro-controllers like the those used by the Duet boards or many of the the other 32bit chips like the rPi Pico and STM32 chips provide hardware that supports USB directly and in this case the extra complexity (and cost) of the the USB to UART converter is not needed.

gloomyandy

@Chriss When it is in this state (showing up on the access point as being connected), but DWC not working. What happens if you force a refresh of the DWC window? Does it then connect?

What URL are you using to connect to your board? Are you using the IP address or are you using the .local name of your printer?

Chriss

@gloomyandy Hahaha.... we can start some levels deeper, my host does not get back the mac address. I see the arp requests (one collision domain) but I do never get an answer from the printer to inform my host which MAC access should be reached.

So you can forget DNS and all of the other stuff.

Cheers, Chriss

Chriss

@Chriss

Here the screenshot from wireshark while I try to ping the printer:

Cheers, Chriss

gloomyandy

@Chriss Can you confirm what version of the WiFi firmware you are running (It should be displayed as part of the M122 output, which we would also really like to see).

Chriss

@gloomyandy You will get that tomorrow noonish. I need the long USB cable first. You will get the M122 as soon as Amazon delivered +10Minutes.

oliof

@Chriss do you have docker running on your host by chance? Depending on how that's set up traffic may be routed into the docker network and then you won't be able to get to the wifi connected printer.

rechrtb

@Chriss Thanks for the reply.

I'm not 100% sure, so please do not nail me down to it. But I think that it is 3.5.0RC2 I had before on it. But I saw that behaviour with the reconnects in 3.4 something already. It never bother me because the printer did always reconnect.

That's ok. So you suspect it was 3.5.0RC2 previously, do you happen to remember also the WiFi firmware version?

My two other printers and my "on my desk test board" are on 3.4.6. And I have found a other printer which I did not upgraded to RC2, it is still 3.5.0rc2 and this one does not have the problem. And this is very much in use at the moment.

In the second sentence, do you mean to say "not upgraded to RC3, it is still 3.5.0rc2" or "not upgraded to RC2, it is still 3.5.0rc1 (or maybe 3.4.x)". Also, can you also confirm the WiFi firmware version of these boards?

Just to be clear here about the status and for unblocking my confusion:
1: RC3 with that problem, with PanelDue and printing (My Voron 2.4)
2: RC3 without problem, without PenalDue and not printing (VCast)
3: RC2 without problem, without PenalDue and printing (V0)
4: 3.4.7 without problem, without PenalDue and not printing (Workbench, VCore, Creality)

Can you also confirm the wifi firmware version for these boards?

You'll notice that I'm constantly asking for the WiFi firmware version, since flashing a certain RRF version does not guarantee a certain WiFi firmware version. That is, you might have upgraded the RRF to 3.5.0rc2 or rc3, but the wifi firmware version can still be on an older version (or the opposite can also be true - you might have new WiFi firmware + old RRF).

Chriss

@oliof said in Wifi 2.1beta6 from 3.5.0-rc.2/3 still disconnecting:

@Chriss do you have docker running on your host by chance? Depending on how that's set up traffic may be routed into the docker network and then you won't be able to get to the wifi connected printer.

For god sake... no Docker in my household. Seriously: I have a bridge device for other reasons. (And other hosts can not reach the board too)

Cheers, Chriss

Chriss

@gloomyandy @rechrtb

The cable just arrived, here is the status of my Voron 2.4 with 3.5.0rc3 and the corresponding Wifi version:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.3 (2024-01-24 17:56:48) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: V9NWJ-R296U-D65J0-40KM6-4113Z-HM83B
Used output buffers: 1 of 40 (40 max)
Error in macro line 21 while starting up: in file macro line 21 column 33: M558: unknown variable 'probe_settingsH'
=== RTOS ===
Static ram: 103200
Dynamic ram: 126032 of which 12 recycled
Never used RAM 8500, free system stack 122 words
Tasks: NETWORK(1,ready,103.0%,156) HEAT(3,nWait 6,0.9%,326) Move(4,nWait 6,29.9%,243) CanReceiv(6,nWait 1,1.5%,771) CanSender(5,nWait 7,0.8%,328) CanClock(7,delaying,0.2%,349) TMC(4,nWait 6,42.2%,102) MAIN(1,running,44.2%,500) IDLE(0,ready,6.2%,30) AIN(4,delaying,25.1%,260), total 254.2%
Owned mutexes: USB(MAIN)
=== Platform ===
Last reset 29:39:53 ago, cause: power up
Last software reset at 2024-02-04 02:34, reason: User, Gcodes spinning, available RAM 8420, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00487000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x04
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 106795063, completed 106795063, timed out 0, errs 0
MCU temperature: min 31.3, current 42.6, max 50.6
Supply voltage: min 24.2, current 24.4, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/48, heap memory allocated/used/recyclable 2048/868/104, gc cycles 471
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 83, reads 7572, writes 83, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 66, reads 7589, writes 66, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 64, reads 7591, writes 64, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 7644, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 79, reads 7576, writes 79, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 0, ifcnt 79, reads 7576, writes 79, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 0, ifcnt 78, reads 7577, writes 78, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2024-02-06 12:38:38
Cache data hit count 4294967295
Slowest loop: 527.32ms; fastest: 0.10ms
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 11.5ms, write time 122.4ms, max retries 0
=== Move ===
DMs created 83, segments created 34, maxWait 4268975ms, bed compensation in use: mesh, height map offset 0.000, max steps late 1, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 913360, completed 913360, hiccups 0, stepErrors 0, LaErrors 0, Underruns [438, 0, 8], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is ready with "M122" in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000807
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 1964276, received 2196723, lost 0, errs 0, boc 0
Longest wait 7ms for reply type 6029, peak Tx sync delay 441, free buffers 26 (min 24), ts 533967/533966/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 522.93ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 0 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1beta6
MAC address f0:08:d1:02:e6:75
Module reset reason: Power up, Vcc 3.40, flash size 2097152, free heap 33592
WiFi IP address 192.168.1.69
Signal strength -64dBm, channel 6, mode 802.11n, reconnections 1
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

(The IP is correct, the MAC has a reservation in my DHCP server for that IP)

More to come soon....

Cheers, Chriss

Chriss

@rechrtb said in Wifi 2.1beta6 from 3.5.0-rc.2/3 still disconnecting:

I'm not 100% sure, so please do not nail me down to it. But I think that it is 3.5.0RC2 I had before on it. But I saw that behaviour with the reconnects in 3.4 something already. It never bother me because the printer did always reconnect.

That's ok. So you suspect it was 3.5.0RC2 previously, do you happen to remember also the WiFi firmware version?

I suspect that it was the version which comes with 3.5.0RC2. I'm a lazy dude here. I download all files from the releases page and upload all of them to the printer. I have the confidence that the board does picks the correct files.

My two other printers and my "on my desk test board" are on 3.4.6. And I have found a other printer which I did not upgraded to RC2, it is still 3.5.0rc2 and this one does not have the problem. And this is very much in use at the moment.

In the second sentence, do you mean to say "not upgraded to RC3, it is still 3.5.0rc2" or "not upgraded to RC2, it is still 3.5.0rc1 (or maybe 3.4.x)". Also, can you also confirm the WiFi firmware version of these boards?

Just to be clear here about the status and for unblocking my confusion:
1: RC3 with that problem, with PanelDue and printing (My Voron 2.4)
2: RC3 without problem, without PenalDue and not printing (VCast)
3: RC2 without problem, without PenalDue and printing (V0)
4: 3.4.7 without problem, without PenalDue and not printing (Workbench, VCore, Creality)

Can you also confirm the wifi firmware version for these boards?

I just saw a typo... sorry, my bad! No4 is not 3.4.7 they are on 3.4.6, is think that there is no 3.4.7 as far as I know. Sorry, typo.

The RC[2|3] have the versions from the releases in github. So RC2 and RC3 are on 2.1beta6.

You'll notice that I'm constantly asking for the WiFi firmware version, since flashing a certain RRF version does not guarantee a certain WiFi firmware version. That is, you might have upgraded the RRF to 3.5.0rc2 or rc3, but the wifi firmware version can still be on an older version (or the opposite can also be true - you might have new WiFi firmware + old RRF).

Got you point. Well, the situation is a bit strange, isn't it? We see rc RC2 board with 2.1beta6 working, but the RC3 board is failing. At least the one who was printing.
Correct me if I'm wrong but the DuetFW seams to work fine. The printer was printing, the PanelDue worked so that is cool. We see from my last post that the Formware sees a connected Wifi module etc, so the WiFi module reported (or is reporting) the current connection status.

My Access told me that the WiFi module is connected. But the module itself does not react on any IP traffic anymore.

Is there any M-Command to ping a ip from the Mini5+? (Like M4711 T192.168.1.1) I have a feature request if not.

If would be nice to see if it would help send some traffic from the WiFi to module too the AP and the wired network behind the AP.

I will walk the dogs for about an hour and will restart the disconnecting printer than. I hope that I bring him back to the failing mode than. Let me know if you need more info from the board in its current (disconnected) state.

Cheers, Chriss

gloomyandy

@moth4017 It seems like your problem is different to that being seen by @Chriss and he is also running Duet hardware, so perhaps we should continue looking into your problem on another thread. I've created one here: https://forum.duet3d.com/topic/34897/wifi-disconnects-but-connects-after-refresh

rechrtb

@Chriss Ok, I may finally have hit the same issue you're experiencing after running my setup for a few days.

The logs you would have should tell if it's the same issue. Please turn on debug outputs for network and wifi via M111 P1 S1 and M111 P14 S1 and also messages logging M929 P"eventlog.txt" S3.
Keep the serial terminal open and also log serial terminal to a file - I think most serial terminals support this while waiting for disconnect.

Once in the disconnected state:

See wifi module if LED is on.
Send M122 command and take note of the output (if the serial terminal is logging to file, the output should be in the file automatically).
Verify from access point if module is connected (the reason I ask is that you mentioned previously that the access point reports the module is connected, however in my case it's not - I was wondering if you perhaps mistook the DHCP lease list as the connected stations list?).

After the above, you can try reviving the module with M552 S-1 and M552 S1. Please post the serial terminal log and the eventlog.txt here afterwards.

Chriss

@rechrtb Thanks god, I'm not the only one! I was a bit scared that it is a race condition with my home AP.

Anyway, I executed the commands you mentioned. I was about to perform 3 "tiny" prints in a row since I have the usb cable. I need to leave now, more updates tomorrow.

Chriss

@rechrtb said in Wifi 2.1beta6 from 3.5.0-rc.2/3 still disconnecting:

I was wondering if you perhaps mistook the DHCP lease list as the connected stations list?).

Nope that is not the case, the DHCP server and my AP are two different devices.

(Still waiting for the disconnect)

Cheers, Chriss

Chriss

Hi @gloomyandy Hi @rechrtb

The board was again in the failed state when I came back to my office. The printer was not printing and not pingable anymore. So I went to the terminal session and did the requested "M122", and here is the outcome:

That is strange, I was able to send the M122 to the board yesterday when it was in the failed state before I connected the USB. Could the power via the USB have any influence here?

The board is still not reachable via the network. I have the feeling that the reset did not perform a "reset" on the wifi module. That would match my obeservations from the past. I had to power cycle the printer to bring it back online.

Cheers, Chriss

Chriss

Here are the requested log files. I was unable to upload them as txt or as bz2. So here is a tar file which I names .bin. Extract the tar file and you will find a bz2 which is the output from the serial since yesterday. And the eventlog.txt from yesterday till the it failed.

logs.bin

The :
M552 s-1
M552 s1

Brought back the Wifi access if this is what you want to read.

And one more word about my console logfile. You will see there at the end that the board did the reset when I did the M122. The 2nd M122 was successfully. (See my other post)

I hope that this help you guys to find the issue.

Do you want me to downgrade the board to RC2 and the older WiFI firmware now? Or Should I keep RC3 and downgrade WiFi only?

I will be off for 6 days from Friday onwards and can not test anything during this period.

Cheers, Chriss

droftarts

@Chriss From your log file, you sent M112 (which is emergency stop), not M122! After M112, you need to send M999 to reset. Did you notice if the WiFi LED was still on?

I changed the file ending of the log file from .bin to .txt to look at it. Usually the log files are in text format - did you change the ending to .bin? There's a lot of garbage text at the end of the log file. I don't know what that is, or if it is useful; one for @rechrtb to have a look at.

Ian