Missing Steps - Cant Print SpreadCycle StealthChop tuning help

Phaedrux

@carcamerarig M122 gives a dump of info on the current state. And there is additional logging you can enable but I'm not sure it will capture anything useful in this case.

https://duet3d.dozuki.com/Wiki/Logging
and
https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

fulg

Just another data point: I am having a similar problem on a CoreXY setup, Mini5+ and 0.9 motors. The printer worked perfectly with a Duet2 but will randomly (and loudly!) skip steps with a Mini5+.

I have raised the motor amps and lowered jerk a lot (M566), reduced travel & print speed in the slicer, forced spreadCycle mode all the time on XY but still I get skipping, exactly like @carcamerarig describes (suddenly a loud "thump" and then there is a major layer shift). It is random, sometimes I can finish a small print but usually not.

I see this in the M122 dump, could it be related? Write errors, timeouts, failedOps...

Driver 0: position 99200, standstill, SG min/max 0/54, read errors 0, write errors 1, ifcnt 59, reads 53605, writes 19, timeouts 184, DMA errors 0, failedOp 0x41
Driver 1: position 0, standstill, SG min/max 0/52, read errors 0, write errors 1, ifcnt 59, reads 53758, writes 19, timeouts 31, DMA errors 0, failedOp 0x72
Driver 2: position 5755, standstill, SG min/max 0/56, read errors 0, write errors 1, ifcnt 58, reads 53596, writes 18, timeouts 194, DMA errors 0, failedOp 0x6f
Driver 3: position 0, standstill, SG min/max 0/56, read errors 0, write errors 1, ifcnt 58, reads 53650, writes 19, timeouts 138, DMA errors 0, failedOp 0x6f
Driver 4: position 0, standstill, SG min/max 0/62, read errors 0, write errors 1, ifcnt 41, reads 53792, writes 13, timeouts 3, DMA errors 0, failedOp 0x01
Driver 5: position 0, standstill, SG min/max 0/510, read errors 0, write errors 1, ifcnt 55, reads 53598, writes 18, timeouts 192, DMA errors 0, failedOp 0x72
Driver 6: position 0, standstill, SG min/max 0/496, read errors 0, write errors 1, ifcnt 55, reads 53787, writes 18, timeouts 3, DMA errors 0, failedOp 0x41

I tried upgrading from 3.2.2 to 3.3b2 and that made no difference, skipping is still random.

SputnikOC3d

@fulg - what motors are you running - brand and model # ?

Thank you for posting ...

carcamerarig

What are @fulg 's write errors? I have seen those in my M122 reports in the past.

fulg

@sputnikoc3d Motors are 17HM19-2004S bought directly from OMC StepperOnline.

It seems not everyone with a Mini5+ and these motors is affected, I am talking with another VORON user who has the same Mini5+ setup with the same motors and he has no issues at any speed, with the "default" M569 parameters (i.e. no need to increase current, force spreadCycle mode or anything special). I believe he has a 0.5 board though (mine is 1.0).

He also sees similar errors reported from the drivers in M122 but it works for him so it's probably unrelated. He is also running 3.3b2.

Note that I am not using any of the sensorless features, I have real endstops on my machine and do not care about tuning hybrid mode (it was unusable on Maestro, I had to disable it over there too on a different printer).

I really wish we can get to the bottom of it because I am contemplating going back to Duet2, this Mini5+ is completely unusable.

carcamerarig

@fulg said in Missing Steps - Cant Print SpreadCycle StealthChop tuning help:

@sputnikoc3d Motors are 17HM19-2004S bought directly from OMC StepperOnline.

It seems not everyone with a Mini5+ and these motors is affected, I am talking with another VORON user who has the same Mini5+ setup with the same motors and he has no issues at any speed, with the "default" M569 parameters (i.e. no need to increase current, force spreadCycle mode or anything special). I believe he has a 0.5 board though (mine is 1.0).

He also sees similar errors reported from the drivers in M122 but it works for him so it's probably unrelated. He is also running 3.3b2.

Note that I am not using any of the sensorless features, I have real endstops on my machine and do not care about tuning hybrid mode (it was unusable on Maestro, I had to disable it over there too on a different printer).

I really wish we can get to the bottom of it because I am contemplating going back to Duet2, this Mini5+ is completely unusable.

Who is it?

gloomyandy

@fulg It is normal to get a single write error when RRF first talks to a TMC driver (after power on/reset etc). Basically the two sides need to get in sync to be able to communicate correctly, as a result of this process the first write from RRF will fail (and be recorded as a write error). Not sure about the timeouts though.

fulg

@carcamerarig very well then, I will drag him into this thread.

Hello @oc_geek, you have been summoned! I don't know if you can share more details that haven't been shared already. You seem to have found the magic recipe that eludes many of us.

I know we have the same printer, same motors, same LCD, same firmware version, so perhaps you can help...

For now I am trying another workaround, which is to force stealthChop all the time via M569 ... V1. It is a bit too early to call but so far I am further into a print that has always failed before (I need to slowly remove all my other attempts at working around this issue!). The motors are quite a bit noisier in stealthChop (I would have expected the opposite) but they seem not to stall anymore.

EDIT: this turned out to be false and does not help, I don't want to mislead anyone reading this later.

SputnikOC3d

@fulg - Thanks so much!

Im in process of building a V2.4 and have the pi - expansion board needed and the Duet 3 Mini 5+ in hand ... I have OMC stepper online motors in a cart waiting to pull trigger and / or the LDO's. Both of those options seem questionable atm.

We too have an associate on the full duet3 main board and V2.4 using stepper online [ omc 1.8* motors and No missing steps issues - yet ]

Baffling to say the least ...

droftarts

Thanks all for the extra information, please do keep it coming. Can everyone remind me if they are using their Duet 3 Mini 5+ with SBC (and which one, if so) or not? If you are using SBC, if you try printing without SBC, do you still get the thumping and move errors?

Ian

fulg

@droftarts I knew I should have waited before posting. Forcing stealthChop with M569 V1 does not help, I just had that same random stall, it just progressed quite a bit further than before.

I am running in stand-alone mode.

carcamerarig

@droftarts said in Missing Steps - Cant Print SpreadCycle StealthChop tuning help:

Thanks all for the extra information, please do keep it coming. Can everyone remind me if they are using their Duet 3 Mini 5+ with SBC (and which one, if so) or not? If you are using SBC, if you try printing without SBC, do you still get the thumping and move errors?

Ian

Im try to setup standalone now if I can get past this damn wifi issue...

droftarts

@carcamerarig in config.g, I comment these lines:

;M550 P"Duet3"													; Set machine name
;M552 S1														; Enable network
;M586 P0 S1														; Enable HTTP
;M586 P1 S0														; Disable FTP
;M586 P2 S0

If you haven’t added the WiFi SSID, connect via USB and follow https://duet3d.dozuki.com/Guide/1.)+Getting+Connected+to+your+Duet/7#s9

Ian

dc42

I am trying to catch up with this thread. From what I have read, the main issue is that the motor is making banging/thumping noises when switching between low and high speeds, even when the motor is supposedly running in spreadCycle mode.

Some questions for those affected:

Does the problem still occur if you have never changed the M569 settings from the defaults? If you are using stall-detect homing, then to test this you will needs to "pretend home" using G92 instead of homing using stall detection.
If you pause the print, then re-send the M569 commands to select spreadCycle, confirm the setting using M122, and then resume the print, does the problem continue?
If #2 doesn't help, what happens if you pause the print, send M569 P# V4000 for each axis driver #, and then resume the print: does the problem continue?

One possibility that I am considering is that after switching to stealthChop, switching back to spreadCycle isn't working; although in the test that I have just done, it does work.

fulg

@dc42 said in Missing Steps - Cant Print SpreadCycle StealthChop tuning help:

the motor is making banging/thumping noises when switching between low and high speeds, even when the motor is supposedly running in spreadCycle mode.

This is not completely exact. It seems to work fine most of the time (normal moves are well over the default threshold of 2.3mm/s and usually don't have an issue), the problem is that randomly during a print you will hear this "thump" and then realize you now have a layer shift in your print. It behaves exactly like the head has hit something mid-print, yet there is no evidence of any obstruction. It does seem to correspond to a travel move. The behavior is random, restarting the same print may fail at a different point along the way (or not at all).

Does the problem still occur if you have never changed the M569 settings from the defaults? If you are using stall-detect homing, then to test this you will needs to "pretend home" using G92 instead of homing using stall detection.

Yes, the problem happens with default M569 values, i.e. M569 P5 S1. I am not using sensorless homing. The problem also occurs by forcing the drivers to stay in spreadCycle mode (M569 D2 ...) or stealthChop mode (M569 V1 ...). I guess I should try M569 V4000 ... too, perhaps just D2 is not enough.

As for your questions #2 and #3, after the problem happens the printer has essentially lost its XY reference due to the skip, so it is hard to continue a print. If I am not next to the printer to stop it, it keeps going fine (but shifted!). I have not let it continue up to a second shift during the same print.

Also note that this happens even with increased motor current, reduced printing & travel speeds and significantly lower instant movement thresholds (everything much slower than my Duet2 settings on the same machine).

So far I've only had the problem happen in one direction on my CoreXY. I don't know if it matters but so far it is always the X motor that skips, as confirmed by G1 H2 X10 moving in the same diagonal. Could be luck of the draw too...

Is there a way to dump out the TMC2209 registers easily? I should build my own version of the firmware to help out.

ctilley79

@dc42 Not exactly. The issue is that it thumps in all modes even when explicitly setting the driver to spreadcycle using for example M569 P0 D2 after homing. The thumps seem to be slightly less abrupt in spreadcycle, but still enough to cause layer shifts in the OP’s case.

The thumps occur in stealthchop with no changeover speed, Spreadcycle entirely, and everywhere in between.

Argo

@fulg said in Missing Steps - Cant Print SpreadCycle StealthChop tuning help:

For now I am trying another workaround, which is to force stealthChop all the time via M569 ... V1. It is a bit too early to call but so far I am further into a print that has always failed before (I need to slowly remove all my other attempts at working around this issue!). The motors are quite a bit noisier in stealthChop (I would have expected the opposite) but they seem not to stall anymore.

Did you try to set the V parameter to "0"? This should disable hybrid mode. With Marlin I'm also having bangs in hybrid mode with TMC2209s.

SputnikOC3d

@dc42 - et al ... I have also given the motor #s of the users involved here to my contact at LDO motors for him to give to his engineering team to review and to compare their motors to their competitors that are both working and not working as well - to see if they have some insight that might help us all resolve.

Not sure if that task will bear fruit - and not holding my breath - but hopeful.

carcamerarig

@ctilley79 said in Missing Steps - Cant Print SpreadCycle StealthChop tuning help:

@dc42 Not exactly. The issue is that it thumps in all modes even when explicitly setting the driver to spreadcycle using for example M569 P0 D2 after homing. The thumps seem to be slightly less abrupt in spreadcycle, but still enough to cause layer shifts in the OP’s case.

The thumps occur in stealthchop with no changeover speed, Spreadcycle entirely, and everywhere in between.

@dc42 I’ve just finished a test print in standalone with M569 S0 V0 at very tame speeds and jerk and the issue persists. I’m happy to bring this setup down to you for you to see for yourself??

fulg

@argo I can confirm (same as @carcamerarig), the issue persists with M569 ... V0.

This time there was one interesting variation, instead of a "thump" one of the two motors decided to stop working altogether, so I was suddenly printing diagonally. Then the console in DWC was flooded with "short to ground" errors:

Interestingly this was a different motor than the one that is always skipping (in my build drivers 5 and 6 are the A and B of CoreXY). Perhaps I could move A and B to another set of drivers.

I grabbed an M122 dump when this happened in case it's useful:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2 running on Duet 3 Mini5plus Ethernet (standalone mode)
Board ID: NY57W-W096U-D65J0-40KMU-NP03Z-ZKK2Z
Used output buffers: 3 of 40 (14 max)
=== RTOS ===
Static ram: 99720
Dynamic ram: 108484 of which 176 recycled
Never used RAM 32468, free system stack 119 words
Tasks: NETWORK(ready,207) ETHERNET(notifyWait,550) HEAT(delaying,293) CanReceiv(notifyWait,943) CanSender(notifyWait,358) CanClock(delaying,340) TMC(notifyWait,100) MAIN(running,314) IDLE(ready,20) AIN(delaying,260)
Owned mutexes:
=== Platform ===
Last reset 01:37:52 ago, cause: software
Last software reset at 2021-04-07 16:20, reason: User, GCodes spinning, available RAM 35540, slot 0
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Aux0 errors 0,0,0
Aux1 errors 0,0,0
Supply voltage: min 0.0, current 23.9, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/82/0, gc cycles 0
Driver 0: position 99200, standstill, SG min/max 0/72, read errors 0, write errors 1, ifcnt 40, reads 15989, writes 18, timeouts 274, DMA errors 0, failedOp 0x41
Driver 1: position 0, standstill, SG min/max 0/100, read errors 0, write errors 1, ifcnt 41, reads 16253, writes 19, timeouts 9, DMA errors 0, failedOp 0x71
Driver 2: position 5114, standstill, SG min/max 0/50, read errors 0, write errors 1, ifcnt 41, reads 15966, writes 19, timeouts 296, DMA errors 0, failedOp 0x71
Driver 3: position 0, standstill, SG min/max 0/60, read errors 0, write errors 1, ifcnt 41, reads 16198, writes 19, timeouts 64, DMA errors 0, failedOp 0x6f
Driver 4: position 0, standstill, SG min/max 0/48, read errors 0, write errors 1, ifcnt 35, reads 16267, writes 13, timeouts 1, DMA errors 0, failedOp 0x06
Driver 5: position 0, standstill, SG min/max 0/510, read errors 0, write errors 1, ifcnt 43, reads 16203, writes 19, timeouts 59, DMA errors 0, failedOp 0x6a
Driver 6: position 0, short-to-ground, standstill, SG min/max 0/510, read errors 0, write errors 1, ifcnt 42, reads 16258, writes 19, timeouts 4, DMA errors 0, failedOp 0x01
Date/time: 2021-04-07 17:58:10
Cache data hit count 4294967295
Slowest loop: 125.41ms; fastest: 0.08ms
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 100.7ms, write time 30.0ms, max retries 0
=== Move ===
DMs created 83, maxWait 4423638ms, bed compensation in use: mesh, comp offset 0.000
=== MainDDARing ===
Scheduled moves 25385, completed moves 25385, hiccups 0, stepErrors 0, LaErrors 0, Underruns [3, 1369, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.3
Heater 1 is on, I-accum = 0.4
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X136.015 Y157.355 E0.04029" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 52892, send timeouts 52889, received 0, lost 0, longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 17 (min 17)
Last cancelled message type 30 dest 127
=== Network ===
Slowest loop: 179.42ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- Ethernet -
State: active
Error counts: 0 0 0 0 0
Socket states: 5 2 2 2 2 2 0 0

When I restarted the Mini5+ (via Emergency Stop in DWC) the errors stopped and I restarted the print again, this time the usual skipping happened during the very first travel move of the print so I did not have to wait long.

I have yet to find a particular mode that works. My board is being exchanged via warranty due to an unrelated issue (SD card problems) so I guess we'll see if switching to another board helps; I doubt it will but it's one more data point.

Cheers,
Ben.