Duet3 6CH + 3CH expansion board - Missing steps.

dc42

Thanks for the additoinal data.

I've just run your print up to the 93% finished mark. This is the data:

m122b1
Diagnostics for board 1:
Duet EXP3HC firmware version 3.3beta (2021-01-16 19:11:54)
Bootloader ID: SAME5x bootloader version 2.2 (2021-01-16b1)
Never used RAM 154492, free system stack 142 words
HEAT 92 CanAsync 92 CanRecv 79 TMC 30 MAIN 274 AIN 259
Last reset 03:06:56 ago, cause: power up
Last software reset data not available
Driver 0: position 78458, 160.0 steps/mm,  standstill, reads 11912, writes 2 timeouts 0, SG min/max 0/72, steps req 49298753 done 49298753
Driver 1: position 82701, 160.0 steps/mm,  standstill, reads 11912, writes 2 timeouts 0, SG min/max 0/72, steps req 50413393 done 50413393
Driver 2: position 719839, 172.0 steps/mm,  standstill, reads 11913, writes 2 timeouts 0, SG min/max 0/67, steps req 1793879 done 1791393
Moves scheduled 210097, completed 210097, in progress 0, hiccups 15372, step errors 0
Peak sync jitter 24, peak Rx sync delay 49110, resyncs 200, no step interrupt scheduled
VIN: 24.4V, V12: 12.2V
MCU temperature: min 44.1C, current 44.2C, max 44.4C
Ticks since heat task active 40, ADC conversions started 11216032, completed 11216031, timed out 0
Last sensors broadcast 0x00000000 found 0 44 ticks ago, loop time 0
CAN messages queued 27, send timeouts 0, received 245625, lost 2226, free buffers 36, error reg 0

So I am still not seeing any loss of XY done vs. requested steps, only a loss of 0.14% of E steps. This is with pressure advance set to zero. OTOH there have been 2226 lost CAN messages, presumably mostly movement messages, so moves will have been lost.

My pause.g and resume.g are currently empty. You've provided your pause.g. Please provide your resume.g file too, then I will try again using your files, in case it makes a difference.

I will work on fixing the lost CAN messages tomorrow or Monday.

evomotors

@dc42 said in Duet3 6CH + 3CH expansion board - Missing steps.:

Thanks for the additoinal data.

I've just run your print up to the 93% finished mark. This is the data:
m122b1
Diagnostics for board 1:
Duet EXP3HC firmware version 3.3beta (2021-01-16 19:11:54)
Bootloader ID: SAME5x bootloader version 2.2 (2021-01-16b1)
Never used RAM 154492, free system stack 142 words
HEAT 92 CanAsync 92 CanRecv 79 TMC 30 MAIN 274 AIN 259
Last reset 03:06:56 ago, cause: power up
Last software reset data not available
Driver 0: position 78458, 160.0 steps/mm,  standstill, reads 11912, writes 2 timeouts 0, SG min/max 0/72, steps req 49298753 done 49298753
Driver 1: position 82701, 160.0 steps/mm,  standstill, reads 11912, writes 2 timeouts 0, SG min/max 0/72, steps req 50413393 done 50413393
Driver 2: position 719839, 172.0 steps/mm,  standstill, reads 11913, writes 2 timeouts 0, SG min/max 0/67, steps req 1793879 done 1791393
Moves scheduled 210097, completed 210097, in progress 0, hiccups 15372, step errors 0
Peak sync jitter 24, peak Rx sync delay 49110, resyncs 200, no step interrupt scheduled
VIN: 24.4V, V12: 12.2V
MCU temperature: min 44.1C, current 44.2C, max 44.4C
Ticks since heat task active 40, ADC conversions started 11216032, completed 11216031, timed out 0
Last sensors broadcast 0x00000000 found 0 44 ticks ago, loop time 0
CAN messages queued 27, send timeouts 0, received 245625, lost 2226, free buffers 36, error reg 0
So I am still not seeing any loss of XY done vs. requested steps, only a loss of 0.14% of E steps. This is with pressure advance set to zero. OTOH there have been 2226 lost CAN messages, presumably mostly movement messages, so moves will have been lost.

My pause.g and resume.g are currently empty. You've provided your pause.g. Please provide your resume.g file too, then I will try again using your files, in case it makes a difference.

I will work on fixing the lost CAN messages tomorrow or Monday.

So why it's not working for me? I'm not even talking about diagnostics. It is not printing when motors connected to expansion board. But works flawlessly when connected to Duet3.

All configs are attached. Compressed and renamed to txt
[0_1610838265664_RRF_3.3_Bakcup.zip](Uploading 100%) RRF_3.3_Bakcup.zip.txt

dc42

@evomotors said in Duet3 6CH + 3CH expansion board - Missing steps.:

It is not printing when motors connected to expansion board.

Please explain what you mean by this. I thought it was printing, but the print quality was poor.

evomotors

@dc42 said in Duet3 6CH + 3CH expansion board - Missing steps.:

@evomotors said in Duet3 6CH + 3CH expansion board - Missing steps.:

It is not printing when motors connected to expansion board.

Please explain what you mean by this. I thought it was printing, but the print quality was poor.

I'm taking about stuff that's happening on my videos. I cannot call this printing. Did you actually see the videos?

evomotors

@dc42 There are 2 videos I posted earlier in this thread. One showing motors connected to Duet3 and another then motors connected to expansion board. Please watch

dc42

I was confusing you with the other guy having a problem with print quality when the extruder is on an expansion board. I searched through the 128 posts in this thread and eventually found your video. It looks like it's printing correctly most of the time, but occasionally there is a big jolt which results in a shift in X and/or Y. That ties in with the lost CAN messages that are reported by M122 B1.

evomotors

@dc42 said in Duet3 6CH + 3CH expansion board - Missing steps.:

I was confusing you with the other guy having a problem with print quality when the extruder is on an expansion board. I searched through the 128 posts in this thread and eventually found your video. It looks like it's printing correctly most of the time, but occasionally there is a big jolt which results in a shift in X and/or Y. That ties in with the lost CAN messages that are reported by M122 B1.

The bottom line is when I connect XY motors to Duet3 board that problem disappears, but E steps still missing or not synchronized with XY movement and that produces bad quality print. Unfortunately I cannot connect extruder to duet3 as well as Duet3 has 6 stepper drivers, but I need 7.

jbarros

@dc42 said in Duet3 6CH + 3CH expansion board - Missing steps.:

I was confusing you with the other guy having a problem with print quality when the extruder is on an expansion board.

If you're refering to a ToolBoard then I'm the guy and this is my thread: https://forum.duet3d.com/topic/20899/duet3d-6hc-toolboard-missing-filament-on-curves

dc42

Yes, it figures that extrusion will be affected as well. Most of the moves in a print involve X, Y and E movement. So even if you move X and Y to the main board, there is still a CAN packet sent for the E movement.

I think the reason we haven't seen this before is that your print uses sequences of tiny movements, smaller than our other tests use. For example:

G0 F12000 X273.288 Y248.054
G1 F4800 X273.303 Y248.082 E0.00106
G1 X273.539 Y248.591 E0.01866
G1 X273.542 Y248.6 E0.00032
G1 X273.603 Y248.759 E0.00566
G1 X273.682 Y248.966 E0.00737
G1 X273.759 Y249.171 E0.00728
G1 X273.763 Y249.184 E0.00045

Here are those moves imported into a spreadsheet. The columns are X, Y, change in X, change in Y, move distance, and move time.

There are really small moves. The fourth one is less than 10 microns long and it will execute in just 120 microseconds. The longest one takes only 7 milliseconds and all the rest are below 3 milliseconds. So the system is having to process a very high rate of moves. Still, I don't see any evidence that the CAN bus is being overloaded, so I expect to resolve the issues with this print by improving the efficiency of processing the incoming move commands on the expansion board.

evomotors

@dc42 Could my issue exists due to faulty Duet board ? I'm dead in the water. I cannot print at all for several weeks already.

Luke'sLaboratory

@evomotors

@dc42 said:

So I am still not seeing any loss of XY done vs. requested steps, only a loss of 0.14% of E steps. This is with pressure advance set to zero. OTOH there have been 2226 lost CAN messages, presumably mostly movement messages, so moves will have been lost.

evomotors

@dc42 do you have any time estimates? I’m wondering when I’ll be able to print.

evomotors

Can I have some updates? Please...

Phaedrux

Still being worked on. Thanks for your patience.

Multisaft

I would like to tell u that ur not alone with this issue. My colleague and myself have exactly the same problem. Connecting xy to the expansion board and we had what u have shown on ur Youtube Video. Switching back to the Mainboard and everything is fine.

Unfortunately we need also 7 motors driven to print. Now we have the extruder on the expansion board and have the issue there.

It's nice to see that dc is working on it now especially because I have this problem since the board was officially released... It's some time now.

Fingers crossed that there is a solution soon.

Best regards.
Multisaft

dc42

@Multisaft, it's likely that the changes already made in 3.3beta will fix the issue for you. The issue I am pinning down now occurs when all of XYE are on the same expansion board and there is a very high rate of very short moves sent to the board. Even if 3.3beta doesn't fix the issue, the improved diagnostics will reveal what is going on.

Multisaft

@dc42 ty, we will try it.

dc42

Good progress today. I had been assuming that the remaining problem seen by @evomotors was caused by the expansion board firmware being too slow, so over the past few days I have been making it more efficient. But additional diagnostics reveal that it's a problem with the way the main board firmware is scheduling CAN moves. With this particular file it's sending the expansion board some moves which are impossible to execute in time. Main board firmware is easier to debug than expansion board firmware, so tracking down why shouldn't take long.

dc42

I've located the problem and tested a provisional fix. It affects systems using CAN expansion boards that have DAA enabled.

evomotors

@dc42 said in Duet3 6CH + 3CH expansion board - Missing steps.:

I've located the problem and tested a provisional fix. It affects systems using CAN expansion boards that have DAA enabled.

When and how can I test?