GCODE upload issue after upgrade

casperen

Working on upgrading a Voron with a duet 3 mini WIFI(has a 1LC board as well) to version 3.4.5. Upgraded to the latest version and now I can't upload GCODE to the printer. It's starts out at 400ish KB/s then immediately jumps down to 10B/s which will eventually fail.

Looking over the forums I thought it might be a SD card issue. I purchased an additional SANDISK 8gig SD card. Formatted it to 64kb unit size FAT 32 per the guidance of dc42 on a few of the posts.

I have looked through everything related the to WIFI side of the problem and I'm not seeing an issue.

Here is the output of M122:

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.4.5 (2022-11-30 19:41:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: Z5KV7-9096U-D65J0-40KM8-KM03Z-RSJ95
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 103652
Dynamic ram: 110532 of which 0 recycled
Never used RAM 24672, free system stack 190 words
Tasks: NETWORK(notifyWait,26.2%,225) HEAT(notifyWait,0.0%,342) Move(notifyWait,0.0%,363) CanReceiv(notifyWait,0.0%,774) CanSender(notifyWait,0.0%,336) CanClock(delaying,0.0%,339) TMC(notifyWait,1.2%,106) MAIN(running,71.7%,425) IDLE(ready,0.0%,30) AIN(delaying,0.8%,263), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:27:57 ago, cause: software
Last software reset at 2023-05-27 09:46, reason: User, GCodes spinning, available RAM 24600, slot 0
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 1677821, completed 1677821, timed out 0, errs 0
Step timer max interval 1474
MCU temperature: min 38.8, current 38.9, max 40.1
Supply voltage: min 23.7, current 23.8, max 23.9, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/66/42, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 29, reads 21421, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 2, read errors 0, write errors 1, ifcnt 29, reads 21421, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 29, reads 21421, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 29, reads 21420, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 19, reads 21424, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 1, ifcnt 29, reads 21421, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 1, ifcnt 29, reads 21421, writes 12, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2023-05-27 10:14:00
Cache data hit count 3016532539
Slowest loop: 14.23ms; fastest: 0.13ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 2.1ms, write time 12.8ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== CAN ===
Messages queued 15122, received 33568, lost 0, boc 0
Longest wait 2ms for reply type 6026, peak Tx sync delay 165, free buffers 18 (min 17), ts 8389/8388/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 15.40ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
= WiFi =
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.27
WiFi MAC address f0:08:d1:02:ed:fe
WiFi Vcc 3.39, reset reason Power up
WiFi flash size 2097152, free heap 26448
WiFi IP address 10.150.10.40
WiFi signal strength -52dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

I am not sure what else to try.

dc42

@casperen which version of RRF were you previously running, when you had a good upload speed?

casperen

@dc42 I was running 3.4.3. I read thru the upgrade notes between the various versions and didn't see anything that I need to change with the configuration.

Phaedrux

Not much difference between 3.4.3 and 3.4.5. You could roll back just to test the theory that it was the upgrade that caused it.

casperen

@Phaedrux hmmm, reverted back to 3.4.3 and the upload issue is there as well.

Does these mean that something went bad in the board during the attempt to bring it to version 3.4.5?

Phaedrux

What wifi server version does M122 show now with the 3.4.3 firmware?

casperen

@Phaedrux I used a M997 to update the firmware since I am unable to upload anything to the board anymore.

Here is the output of the M122:

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.4.3 (2022-10-05 09:05:15) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: Z5KV7-9096U-D65J0-40KM8-KM03Z-RSJ95
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 103684
Dynamic ram: 110532 of which 0 recycled
Never used RAM 24640, free system stack 200 words
Tasks: NETWORK(notifyWait,13.0%,255) HEAT(notifyWait,0.0%,342) Move(notifyWait,0.0%,363) CanReceiv(notifyWait,0.0%,774) CanSender(notifyWait,0.0%,336) CanClock(delaying,0.0%,348) TMC(notifyWait,1.2%,115) MAIN(running,84.1%,425) IDLE(ready,0.7%,30) AIN(delaying,0.8%,263), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:00:32 ago, cause: power up
Last software reset at 2023-05-28 13:09, reason: User, GCodes spinning, available RAM 24600, slot 1
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 32504, completed 32503, timed out 0, errs 0
Step timer max interval 1031
MCU temperature: min 34.6, current 37.7, max 37.7
Supply voltage: min 23.7, current 23.8, max 23.8, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/66/42, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 2902, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 2899, writes 12, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2023-05-28 20:58:16
Cache data hit count 62258653
Slowest loop: 3.99ms; fastest: 0.13ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 1.5ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== CAN ===
Messages queued 299, received 638, lost 0, boc 0
Longest wait 2ms for reply type 6026, peak Tx sync delay 6, free buffers 18 (min 17), ts 163/162/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 12.99ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
= WiFi =
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.27
WiFi MAC address f0:08:d1:02:ed:fe
WiFi Vcc 3.39, reset reason Power up
WiFi flash size 2097152, free heap 26680
WiFi IP address 10.150.10.41
WiFi signal strength -42dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

Phaedrux

Try placing the wifi server bin file from 3.4.3 in your firmwares folder on the SD card and then send M997 S1 to reflash.

https://github.com/Duet3D/RepRapFirmware/releases/download/3.4.3/DuetWiFiServer.bin

Then send M122 again to check the wifi server version. I think it should show 1.26

casperen

@Phaedrux The wifi firmware is not changing it appears. I re-downloaded the DuetWIFIServer.bin from the link you provided and ran the M122 S1 command. The PanelDue showed that the firmware was being updated but the WIFI version is still 1.27.

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.4.3 (2022-10-05 09:05:15) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: Z5KV7-9096U-D65J0-40KM8-KM03Z-RSJ95
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 103684
Dynamic ram: 110532 of which 0 recycled
Never used RAM 24640, free system stack 196 words
Tasks: NETWORK(notifyWait,20.1%,131) HEAT(notifyWait,0.0%,346) Move(notifyWait,0.0%,363) CanReceiv(notifyWait,0.0%,797) CanSender(notifyWait,0.0%,336) CanClock(delaying,0.0%,348) TMC(notifyWait,1.2%,106) MAIN(running,77.4%,425) IDLE(ready,0.5%,30) AIN(delaying,0.8%,263), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:02:13 ago, cause: power up
Last software reset at 2023-05-29 08:24, reason: User, GCodes spinning, available RAM 24640, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Aux0 errors 0,0,0
MCU revision 3, ADC conversions started 133974, completed 133974, timed out 0, errs 0
Step timer max interval 1459
MCU temperature: min 25.6, current 32.9, max 33.3
Supply voltage: min 23.6, current 23.8, max 23.8, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/66/42, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 12128, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 5: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 6: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 12125, writes 12, timeouts 0, DMA errors 0, CC errors 0
Date/time: 2023-05-29 08:50:15
Cache data hit count 329879654
Slowest loop: 6.04ms; fastest: 0.11ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 2.0ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== CAN ===
Messages queued 2096, received 3552, lost 0, boc 0
Longest wait 3ms for reply type 6013, peak Tx sync delay 388, free buffers 18 (min 17), ts 670/669/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 1628.71ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
= WiFi =
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.27
WiFi MAC address f0:08:d1:02:ed:fe
WiFi Vcc 3.40, reset reason Power up
WiFi flash size 2097152, free heap 23416
WiFi IP address 10.150.10.40
WiFi signal strength -48dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

Phaedrux

@casperen said in GCODE upload issue after upgrade:

M122 S1 command

Did you mean M997 S1?

casperen

@Phaedrux Shoot, yes, that is what I meant. Apologies.

Phaedrux

Alright, so we've determined it's not related to the firmware.

Has anything else changed on your network recently?

Normally I'd say SD card, but you've already changed that.

If you can reproduce this consistently it could help to gather a wireshark capture for us to examine.

Capturing your traffic with Wireshark
After starting Wireshark, do the following:
Select Capture | Interfaces
Select the interface on which packets need to be captured. This will usually be the interface where the Packet/s column is constantly changing, which would indicate the presence of live traffic). If you have multiple network interface cards (i.e. LAN card and Wi-Fi adapter) you may need to check with your IT administrator to determine the right interface.
Click the Start button to start the capture.
Recreate the problem. The capture dialog should show the number of packets increasing. Try to avoid running any other internet applications while capturing, closing other browsers, Instant messengers etc.
Once the problem which is to be analyzed has been reproduced, click on Stop. It may take a few seconds for Wireshark to display the packets captured.
Save the packet trace in the default format. Click on the File menu option and select Save As. By default Wireshark will save the packet trace in libpcap format. This is a filename with a.pcap extension.

casperen

@Phaedrux Here is the Wireshark capture. I scrubbed the results to make it easier to see what is going on.

Yeah, everything I am doing I try on both SD cards before posting anything.

IPs:
Desktop: 10.150.10.76
Duet: 10.150.10.40

Looking at the results, I am dropping a lot of packets between devices. The router is in the same room as the printer. I have not changed any of the network configuration before or after the software update.

The forum will not allow me to upload a .pcap file type. I might be able to message it to you directly?

I added a CSV version for now.Wireshark output.csv

Phaedrux

@casperen said in GCODE upload issue after upgrade:

The forum will not allow me to upload a .pcap file type. I might be able to message it to you directly?

You could add a .txt extension to it, or if it's too large, use a cloud storage service.

casperen

@Phaedrux you have ran into this issue before.

Here it is: Duet Mini Output - filtered.pcap.txt

dc42

@casperen do you know what RRF and WiFiServer versions you were running when the upload speed was good?

I have a few suggestions:

Can you try rolling back RRF to version 3.4.1? We have one other user who reports issues with uploading files since 3.4.2. Keep the WiFi server version at 1.27.
Check what channel your router or access point is using. We have seen issues before with low channel numbers, in particular channel 1. If your router doesn't say which channel it is using, you can use a smartphone app such as WiFi Analyser to find out.
You could try installing wifi firmware 2.1beta3 from https://github.com/Duet3D/WiFiSocketServerRTOS/releases/tag/2.1beta3.

casperen

@dc42 I was using 3.4.3 and 1.27 when I updated to the latest version.

On a side note. I have been updating two different SD cards during this process to ensure that I test both of those as well(they are different manufacturers formetted fat32, 64K unit size). A few after inserting the card and restarting the print it would hang on the connecting screen and never load. I would have to reformat it once I added it to the computer to check the files. Does this happen often?

Went through the suggestions:

Reverted the board back to version 3.4.1 but left the WIFI server the same:

This did not work. Same network issue as before on both SD cards.

My router will allow me to move the channels (I typically move it around to ensure I stay away from what my neighbors are using). It was set to channel 11, move to channel 8 and retested. Receiving the same network issues error and the upload fails. Again tried it on both SD cards with the same result.

3.Updated just the WIFI server to the new version 2.1 beta3:

This fixed the issue. It seems to upload just fine now...but why?

I went back to version 3.4.5 for the main firmware but left the WIFI server version the same. The upload speed is still working great.

Downgraded the WIFI server version back to 1.27, and the slow upload speed returned.

This doesn’t make sense though as it was working before. Maybe my internet provider updated the firmware of my router and that changed something on that side? I am not sure as I haven't changed anything on my end.

Are there any other tests I can do to help in bug fixing the firmware?

casperen

Just to be thorough, I also went back to 3.4.1 and WIFI server 1.26. Still have an issue with the upload speed. Must be something on the ISP/router side that is creating an issues. You might see some more issues if this is the case.

Looks like upgrading to WIFI server 21.beta 3 fixes it though.

dc42

@casperen I'm glad it's fixed. One possibility is that your router installed an automatic update that introduced an incompatibility with the older firmware for the Duet WiFi module.