Extruder motors skip steps with pressure advance enabled

Edgars Batna

@wilriker Just a heads-up: I just reuploaded 2.02a and removed the second extruder from the config.g entirely and the behavior is the same.

Will try disabling volumetric extrusion.

Edgars Batna

Just for reference, these are the places with absolute worst behavior (Top+Lid+2.gcode in original post) from that video:

0_1548429374400_2f87729c-9521-4913-ab99-bbef2288c97d-grafik.png

Excerpt from gcode:
G1 X307.783 Y59.405 E0.01269
G1 X307.955 Y59.360 E0.01269
G1 X309.040 Y59.360 E0.07748
G1 X309.211 Y59.405 E0.01269
G1 X309.383 Y59.360 E0.01269
G1 X309.925 Y59.360 E0.03873
G1 X309.925 Y55.870 E0.24921
G1 X310.033 Y55.542 E0.02466
G1 X310.092 Y55.447 E0.00801
G1 X310.472 Y55.094 E0.03701
G1 X310.814 Y54.981 E0.02576
G1 X311.896 Y54.981 E0.07726
G1 X312.068 Y55.027 E0.01269
G1 X312.240 Y54.981 E0.01269
G1 X313.325 Y54.981 E0.07748
G1 X313.496 Y55.027 E0.01269
G1 X313.668 Y54.981 E0.01269
G1 X314.753 Y54.981 E0.07748
G1 X314.925 Y55.027 E0.01269
G1 X315.096 Y54.981 E0.01269
G1 X316.181 Y54.981 E0.07748
G1 X316.353 Y55.027 E0.01269
G1 X316.525 Y54.981 E0.01269
G1 X317.610 Y54.981 E0.07748
G1 X317.781 Y55.027 E0.01269
G1 X317.953 Y54.981 E0.01269
G1 X319.038 Y54.981 E0.07748
G1 X319.210 Y55.027 E0.01269
G1 X319.381 Y54.981 E0.01269
G1 X320.466 Y54.981 E0.07748
G1 X320.638 Y55.027 E0.01269
G1 X320.810 Y54.981 E0.01269
G1 X321.895 Y54.981 E0.07748
G1 X322.066 Y55.027 E0.01269
G1 X322.238 Y54.981 E0.01269
G1 X323.323 Y54.981 E0.07748
G1 X323.495 Y55.027 E0.01269
G1 X323.666 Y54.981 E0.01269
G1 X324.751 Y54.981 E0.07748
G1 X324.923 Y55.027 E0.01269
G1 X325.095 Y54.981 E0.01269
G1 X326.180 Y54.981 E0.07748
G1 X326.351 Y55.027 E0.01269
G1 X326.523 Y54.981 E0.01269
G1 X327.608 Y54.981 E0.07748
G1 X327.780 Y55.027 E0.01269
G1 X327.951 Y54.981 E0.01269
G1 X329.036 Y54.981 E0.07748
G1 X329.208 Y55.027 E0.01269
G1 X329.380 Y54.981 E0.01269
G1 X330.464 Y54.981 E0.07748
G1 X330.636 Y55.027 E0.01269
G1 X330.808 Y54.981 E0.01269
G1 X331.893 Y54.981 E0.07748
G1 X332.064 Y55.027 E0.01269
G1 X332.236 Y54.981 E0.01269
G1 X333.321 Y54.981 E0.07748
G1 X333.493 Y55.027 E0.01269
G1 X333.664 Y54.981 E0.01269
G1 X334.749 Y54.981 E0.07748
G1 X334.921 Y55.027 E0.01269
G1 X335.093 Y54.981 E0.01269
G1 X336.178 Y54.981 E0.07748
G1 X336.349 Y55.027 E0.01269
G1 X336.521 Y54.981 E0.01269
G1 X337.606 Y54.981 E0.07748
G1 X337.778 Y55.027 E0.01269
G1 X337.949 Y54.981 E0.01269
G1 X339.034 Y54.981 E0.07748
G1 X339.206 Y55.027 E0.01269
G1 X339.378 Y54.981 E0.01269
G1 X340.463 Y54.981 E0.07748
G1 X340.634 Y55.027 E0.01269
G1 X340.806 Y54.981 E0.01269
G1 X341.886 Y54.981 E0.07710
G1 X342.082 Y55.020 E0.01429
G1 X342.234 Y54.981 E0.01121
G1 X343.319 Y54.981 E0.07752
G1 X343.491 Y55.027 E0.01269
G1 X343.663 Y54.981 E0.01269
G1 X344.748 Y54.981 E0.07748
G1 X344.919 Y55.027 E0.01269
G1 X345.091 Y54.981 E0.01269
G1 X346.176 Y54.981 E0.07748
G1 X346.348 Y55.027 E0.01269
G1 X346.519 Y54.981 E0.01269
G1 X347.598 Y54.981 E0.07704
G1 X347.744 Y55.012 E0.01061
G1 X347.999 Y55.132 E0.02016
G1 X348.339 Y55.472 E0.03437
G1 X348.461 Y55.730 E0.02032
G1 X348.487 Y55.841 E0.00813
G1 X348.490 Y59.300 E0.24703

The places are obvious as they sort of oscillate. Maybe underflow somewhere?

Oh my, it's 2019 and we still haven't figured out how to use double precision or at least exponential notation:

G1 X387.722 Y86.349 E0.00507
G1 X387.540 Y86.167 E0.01838
G1 X387.358 Y86.349 E0.01838
G1 X387.292 Y86.376 E0.00507
G1 X386.359 Y86.376 E0.06664
G1 X386.294 Y86.349 E0.00507
G1 X386.112 Y86.167 E0.01838
G1 X385.930 Y86.349 E0.01838
G1 X385.864 Y86.376 E0.00507
G1 X384.931 Y86.376 E0.06664
G1 X384.865 Y86.349 E0.00507
G1 X384.683 Y86.167 E0.01838
G1 X384.501 Y86.349 E0.01838
G1 X384.436 Y86.376 E0.00507
G1 X383.503 Y86.376 E0.06664
G1 X383.437 Y86.349 E0.00507
G1 X383.298 Y86.210 E0.01400
G1 X383.150 Y86.358 E0.01493
G1 X382.645 Y86.358 E0.03608
G1 X382.497 Y86.210 E0.01493
G1 X382.359 Y86.349 E0.01400
G1 X382.293 Y86.376 E0.00507
G1 X381.360 Y86.376 E0.06664
G1 X381.294 Y86.349 E0.00507
G1 X381.113 Y86.167 E0.01838
G1 X380.931 Y86.349 E0.01838
G1 X380.865 Y86.376 E0.00507
G1 X379.932 Y86.376 E0.06664
G1 X379.866 Y86.349 E0.00507
G1 X379.684 Y86.167 E0.01838
G1 X379.502 Y86.349 E0.01838
G1 X379.437 Y86.376 E0.00507
G1 X378.503 Y86.376 E0.06664
G1 X378.438 Y86.349 E0.00507
G1 X378.256 Y86.167 E0.01838
G1 X378.074 Y86.349 E0.01838
G1 X378.008 Y86.376 E0.00507

Maybe disabling volumetric extrusion could cause it to get worse as the values would be lower. I'll try...

Without cough volumetric extrusion cough cough:
G1 X165.838 Y66.956 E0.00211
G1 X165.656 Y66.774 E0.00764
G1 X165.474 Y66.956 E0.00764
G1 X165.408 Y66.983 E0.00211
G1 X164.475 Y66.983 E0.02771
G1 X164.410 Y66.956 E0.00211
G1 X164.228 Y66.774 E0.00764
G1 X164.046 Y66.956 E0.00764
G1 X163.980 Y66.983 E0.00211
G1 X163.047 Y66.983 E0.02771
G1 X162.981 Y66.956 E0.00211
G1 X162.799 Y66.774 E0.00764
G1 X162.617 Y66.956 E0.00764

Maybe I need absolute E values?

wilriker

@edgars-batna Absolute E values will be converted internally to relative ones as soon as that line of GCode is parsed so there will probably be no change in switching to this.

Still I'd like to know what happens if you print this file with regular relative extrusion. It still might be that there is a bug in the combination of PA and volumetric extrusion. Even though I did not find anything suspicious in the source code. But everything regarding movement is very complicated and distributed so it is easy to miss some aspects if you are not very familiar with the code.

Edgars Batna

@wilriker @deckingman I'm now running the EccentricGearTest in non-volumetric mode and the print speed feels like 150% than what was in volumetric mode and I'm sure it extrudes same amount of filament. The extruder motors still produce knocking sounds as if losing steps, but this time it's independent of E microstepping - same thing happens on 16 and 256.

0_1548436488558_EccentricGearTest 0 0 non-volumetric.gcode
0_1548436494441_EccentricGearTest 0 0 volumetric.gcode

Layer width is still inconsistent and I can hear the steps are lost.

EDIT: added files

Edgars Batna

@wilriker I'm in the code after all. First place I think needs some clarifying:

dev branch
GCodes.cpp:2333
volumetricExtrusionFactors seems to be used squared when only one E parameter is present, why?

		if (eMoveCount != 0)
		{
			// Set the drive values for this tool
			float eMovement[MaxExtruders];
			size_t mc = eMoveCount;
			gb.GetFloatArray(eMovement, mc, false);

			if (mc == 1)
			{
				// There may be multiple extruders present but only one value has been specified, so use mixing
				const float moveArg = eMovement[0] * distanceScale;
				float requestedExtrusionAmount;
				if (gb.MachineState().drivesRelative)
				{
					requestedExtrusionAmount = moveArg;
				}
				else
				{
					requestedExtrusionAmount = moveArg - virtualExtruderPosition;
					virtualExtruderPosition = moveArg;
				}

				// rawExtruderTotal is used to calculate print progress, so it must be based on the requested extrusion before accounting for mixing,
				// otherwise IDEX ditto printing and similar gives strange results
				if (moveBuffer.moveType == 0 && !doingToolChange)
				{
					rawExtruderTotal += requestedExtrusionAmount;
				}

				for (size_t eDrive = 0; eDrive < eMoveCount; eDrive++)
				{
					const int drive = tool->Drive(eDrive);
					float extrusionAmount = requestedExtrusionAmount * tool->GetMix()[eDrive];
					if (extrusionAmount != 0.0)
					{
						if (gb.MachineState().volumetricExtrusion)
						{
							extrusionAmount *= volumetricExtrusionFactors[drive];
						}
						rawExtruderTotalByDrive[drive] += extrusionAmount;

						// Don't count extrusion done in filament loading or tool change macros towards total filament consumed, it distorts the print progress
						moveBuffer.coords[drive + numTotalAxes] = extrusionAmount * extrusionFactors[drive];
#if HAS_SMART_DRIVERS
						if (moveBuffer.moveType == 1)
						{
							SetBit(moveBuffer.endStopsToCheck, drive + numTotalAxes);
						}
#endif
					}
				}
			}
			else
			{
				// Individual extrusion amounts have been provided. This is supported in relative extrusion mode only.
				if (gb.MachineState().drivesRelative)
				{
					for (size_t eDrive = 0; eDrive < eMoveCount; eDrive++)
					{
						const int drive = tool->Drive(eDrive);
						float extrusionAmount = eMovement[eDrive] * distanceScale;
						if (extrusionAmount != 0.0)
						{
							if (gb.MachineState().volumetricExtrusion)
							{
								extrusionAmount *= volumetricExtrusionFactors[drive];
							}
							rawExtruderTotalByDrive[drive] += extrusionAmount;

							// Don't count extrusion done in filament loading or tool change macros towards total filament consumed, it distorts the print progress
							if (moveBuffer.moveType == 0 && !doingToolChange)
							{
								rawExtruderTotal += extrusionAmount;
							}
							moveBuffer.coords[drive + numTotalAxes] = extrusionAmount * extrusionFactors[drive] * volumetricExtrusionFactors[drive];
#if HAS_SMART_DRIVERS
							if (moveBuffer.moveType == 1)
							{
								SetBit(moveBuffer.endStopsToCheck, drive + numTotalAxes);
							}
#endif
						}
					}
				}
				else
				{
					platform.Message(ErrorMessage, "Multiple E parameters in G1 commands are not supported in absolute extrusion mode\n");
				}
			}
		}

DriveMovement.cpp:241

		mp.cart.accelStopStep = (uint32_t)((dda.accelDistance + accelCompensationDistance) * effectiveStepsPerMm) + 1;

DriveMovement.cpp:249

		mp.cart.accelStopStep = (uint32_t)(dda.accelDistance * effectiveStepsPerMm) + 1;

and some others where a +1 is added. How do I interpret this? Does this mean acceleration will occur for a full microstep? Shouldn't this be rounded instead of forced to +1?

I'm fresh to the code and I can't wait to fix my printer, so any help is much appreciated... Need to figure out how to build this so I can break my machine for good and move on...

wilriker

@edgars-batna I couldn't help and had to get my machine back working to try to print the EccentricGearTest non-volumetric. Only a couple of layers though because I only sloppily set it back to working condition and it was unable to cope with the high jerk for Y. Anyway, until I stopped it there were no observable skipped steps for the extruder. running with PA at 0.1s. Also no issues reported in M122 (which I would not expect also on your side).

Ref double application of volumetricExtrusionFactors: a couple of lines above that it will multiply requestedExtrusionAmount with the current extruder's mixing-ratio. In your case at 0.5 this will half the extrusion amount for this extruder. And this value will then be multiplied to convert it back from volumetric to "regular" extrusion. So I don't see anything wrong there.

About this +1 I would have to read the code more in depth to answer that. Maybe @dc42 can step in here?

Edgars Batna

@wilriker said in Extruder motors skip steps with pressure advance enabled:

@edgars-batna I couldn't help and had to get my machine back working to try to print the EccentricGearTest non-volumetric. Only a couple of layers though because I only sloppily set it back to working condition and it was unable to cope with the high jerk for Y. Anyway, until I stopped it there were no observable skipped steps for the extruder. running with PA at 0.1s. Also no issues reported in M122 (which I would not expect also on your side).

M122 never reported any errors for me.

There are places that take the speed changes and extrusion amount into consideration in the code and I'm trying to find anything obvious there, but it's a long shot as I'm sure nothing obvious was left there by the mighty firmware devs. With enough jerk and with low enough segment size these parameters are nearing zero. My theory is that there could be a condition when no pressure advance is actually required, but I'm in well over my head now. My other theory is that my family will print a gun and kill me and the printer once it's is up and running first thing in the morning.

wilriker

@edgars-batna said in Extruder motors skip steps with pressure advance enabled:

M122 never reported any errors for me.

Meh! That's what I wanted to say that I do not expect any errors in M122 in your side either. This happens when you get lost on negations.

There are places that take the speed changes and extrusion amount into consideration in the code and I'm trying to find anything obvious there, but it's a long shot as I'm sure nothing obvious was left there by the mighty firmware devs.

I think finding obvious errors is rather unlikely. Simply because those would most probably have affected others as well.

With enough jerk and with low enough segment size these parameters are nearing zero. My theory is that there could be a condition when no pressure advance is actually required, but I'm in well over my head now.

You could try to only print very tight arcs using G2|G3. This should result in segments being a short as 0.2mm. if you also change direction as often as you can it should lead to PA not being applied.

My other theory is that my family will print a gun and kill me and the printer once it's is up and running first thing in the morning.

I suggest a knife. This is less likely to hurt the person using it.

Edgars Batna

@wilriker I've got more findings. Could you please try something for me:

Remove filament
If possible, remove or disconnect axes so that motors can run at high jerk. We need to achieve high jerk, or the problem does not occur.
Try to get to at least 1500 jerk, ideally 2000 for XY and around 1000 for E.
Accelerations can be set to same values as jerk. Top speeds are unimportant, but at least 30mm/s on XY and, I dunno, 75mm/s for extruder. We're running without filament and, potentially, the axes.
If running with axes, make sure you've got space for at least 200mm for X and 100mm for Y movement.
Set E microstepping to 16, volumetric or non-volumetric doesn't matter, I suggest trying both. Obviously this only works for low stepsPerMM extruder setups, so for geared extruders the values might need to be way higher.
Set PA to 0.6 for the test: M572 D0:1 S0.6
Enable cold extrudes: M302 P1
Run this macro: 0_1548503110156_PA tiny segment test 4.gcode

I've found that unless I increase XY jerk to above 1500 skipping barely happens on my printer.
The skipping is caused by these small segments:
G1 X172.798 Y66.774 E0.00764
G1 X172.616 Y66.956 E0.00764
G1 X172.550 Y66.983 E0.00211
G1 X171.617 Y66.983 E0.02771
G1 X171.551 Y66.956 E0.00211
G1 X171.369 Y66.774 E0.00764
G1 X171.187 Y66.956 E0.00764
G1 X171.122 Y66.983 E0.00211
G1 X170.189 Y66.983 E0.02771
G1 X170.123 Y66.956 E0.00211
G1 X169.941 Y66.774 E0.00764
G1 X169.759 Y66.956 E0.00764
G1 X169.694 Y66.983 E0.00211
G1 X168.760 Y66.983 E0.02771

And there is a semi-sudden change in behavior depending on printer configuration where skipping appears and is rampant. It must be related to startSpeed == endSpeed == topSpeed condition, but I'm still just building up theory and repro cases.

deckingman

@edgars-batna Looks like could be homing in on something. I had a notification from GitHub that you had made a comment regarding the issue I raised with Slic3R generating "random" segment lengths for arcs. This was once thought to be the root of the problem that I was having with PA. AFAIK, the slicer issue was never fixed but eventually something changed in firmware and the problems I was having went away. Sooooo.....

If I get chance, I'll run your gcode file on my machine - I can run at those accels and jerk settings without probs (although my machine doesn't print well with those values). For test purposes, without filament, I can set steps per mm the same as yours (and mixing ratios too). Give me some time though - I'm in the middle of quite a large print run........

Edgars Batna

@deckingman Interesting coincidence. For reference, here's the slic3r PE issue: https://github.com/prusa3d/Slic3r/issues/1000

The thing about pressure is that in the gcode I posted above it alternates between 0.0297 and 0.0296 and I don't see any obvious means to take such errors into account and prevent PA from happening. This low-precision stuff is something that should have been left to rot in the past...

There are a few things I'd like to try in the code using the process of elimination to see how the behavior changes, but I'm afraid to brick my Duet (yeah, I'll try it anyway).

irp53 created this issue in prusa3d/Slic3r

open Problem with uneven segment sizes for arcs and circles #1000

deckingman

@edgars-batna Yes, that's the one I started - I'm irp53 on Github - (my initials and year of birth). Guess you must be mdealer.

You could try a different slicer - see if it makes any difference? - Just a thought.......

Edgars Batna

@deckingman said in Extruder motors skip steps with pressure advance enabled:

@edgars-batna Yes, that's the one I started - I'm irp53 on Github - (my initials and year of birth). Guess you must be mdealer.

You could try a different slicer - see if it makes any difference? - Just a thought.......

Yes, mdealer - that's me.

So far I haven't seen any free slicers that work as expected. Slic3r came close, but is nearly abandoned now. Slic3r PE is now the best one out there as far as I can tell. It has bugs, but it's actively moving forward. They've done a good job so far, even considering the bugs. They've worked around so many other issues already (e.g. Perl).

Btw, I'm modifying and testing the firmware as we speak. Just by commenting out extrusionPending (the fraction from previous move, which was added in 2.02) extruders no longer skip as hard. They still skip, tho, which sort of confirms that this can be an old issue. Tightly balancing between 3-4 potential problems in some corner cases here and my printer has "corner case" etched all over it.

deckingman

@edgars-batna Yes I agree with you about slicers. Tried them all - free and paid. My problem is multi-colour with 5 extruders - none of them do that well (if at all) apart from Slic3R. PE version of slic3r is now too tightly aligned with PE firmware for MMU style multi colour printing so I have to turn off all that stuff and use my own macros and post processing scripts. I'm actually starting to hate it with a vengeance and reverted back to non PE Slic3R. There has been a bit of movement on non PE Slic3R - there are some dev versions around which aren't bad.

Brae of you to modify and test the firmware - I wouldn't know where to start with any of that stuff.

mrehorstdmd

Late to the party here....

My printer has a single 40mm NEMA-17 motor running at <1.5A and it pushes the filament much harder than necessary to extrude for printing even at 200 mm/sec print speed. A recent operator error jammed the hot-end and the extruder pushed the filament so hard that it pushed out the Bowden tube- I lowered the current so the motor would skip if that happens again. It runs smoothly and quietly and the motor doesn't get hot enough to need a heatsink, even in a 50C chamber.

Forgive me if my analysis is too simple, but I noticed you've got two NEMA-23 motors running at 2.4A each, pushing on a single piece of filament, and they are covered with heat sinks. Are they getting that hot? The config files you posted at the top of the thread set motor current to 2.5A (all motors) and the spec sheet you posted a little later says the motor is rated for 2.1A. A more recent post has the XYZ motors down to 2.1A, but the extruders are at 2.4A, the limit of the firmware/board.

In my limited experience, running steppers above their rated current leads to rough motion and excessive heat. Have you tried lowering the motor currents? Is there a fan blowing on the controller board? It's probably getting pretty hot. Hot driver chips might cause rough motion and strange behavior, too.

Edgars Batna

@mrehorstdmd I'm aware of the problems regarding overheating. With cooling fans and heatsinks the motors stay acceptable to the touch, even at higher amps. The board is cooled from both sides. I've actually lowered XY and Z current to 2.1 and 2.0 respectively since then. I've tried lowering E current to 1.5 and it had no impact and, since I'm diagnosing lost steps on E, I'd rather have them in overdrive to rule out torque problems. Right now I've got a Nema 23 and Nema 17 in push-pull arrangement respectively. The Nema 23 is rated at 3A and is getting hotter than the 2.1A Nema 17, which indicates inefficient operation like lost steps etc. If the motors are faulty, I'd rather have them blow up immediately.

I've actually narrowed down a few things in code since yesterday and there seems to be multiple "issues" at play here:

The accumulated rounding error correction causes a snowball effect on very tiny segments when jerk is high.
Pressure advance is applied to these tiny segments when the pressure is not actually changing or is changing just marginally (less or equal than 0.0001 mm/s), causing lost steps.

Now, I'm totally new to programming 3d printers, so pardon if I'm getting some things wrong. As soon as I've got something tangible in the code, I'll make pull request on github or a patch, or I'll disassemble my printer and eat it.

wilriker

@edgars-batna OK, I was trying the short Gcode segment you posted together with the settings you listed and I can now reproduce skipped steps on my machine with 100 steps/mm@x16. Even if I reduce the speed factor to just 1% the behavior will stay the same.

@dc42: the instructions and code snippet in this post can reproduce this error also for me. So this seems to be either a firmware bug or a physical limitation with low-steps/mm-extruders.

Edgars Batna

@wilriker It'll be a while until I get anything meaningful as there are a ton of computations and I don't know how to debug this CPU over USB or even if it's possible. I assume it's possible as there are drivers in the github repo, but what tools are used, I've got no idea... except that it must be a flavor of gdb and I might need Atmel Studio.

Any pointers from anyone?

dc42

@wilriker said in Extruder motors skip steps with pressure advance enabled:

@edgars-batna OK, I was trying the short Gcode segment you posted together with the settings you listed and I can now reproduce skipped steps on my machine with 100 steps/mm@x16. Even if I reduce the speed factor to just 1% the behavior will stay the same.

@dc42: the instructions and code snippet in this post can reproduce this error also for me. So this seems to be either a firmware bug or a physical limitation with low-steps/mm-extruders.

Thanks for reproducing it. I have added this to my list of pending investigations. As you say, it may be an issue triggered by low steps/mm extruders and short segments.

Edgars Batna

@wilriker @dc42 As discussed, I've been experimenting with source code:

https://github.com/mdealer/RepRapFirmware/commits/pressure-advance-skips

Expect blatant errors... it would be interesting to hear any thoughts. Nevertheless, the implementation works way better for my printer, but it obviously prints slower now. On the bright side, no longer any lost layers or extruder skips on any print I've run. Lots of retractions are also not a problem anymore. It now obeys extruder InstantDv more.

The longest test so far was 8 hours on my CoreXY. I'd upload a binary, but I do not want any unsuspecting souls trashing their printer without the devs looking at this first. The logic by which it slows down on high extrusion compensation is a dirty hack somewhere between DriveMovement and DDA and that rounding error bug check override right there is sketchy at best. Oh, and it uses a bit more memory and CPU...

Anyways, maybe this helps anyone. Is it alright if I post a binary of this?