Hi, I've experienced in software testing in the Automotive Industries before.
Upon looking at the source code on Github, I've not seen any information or anything related about testing on Duet firmware.

Recently (v3.4+), I've experienced somewhat huge amount of software issues that weren't there in the previous betas, but surprisingly introduced in the newer beta/RC versions. That is why I'm starting to question about the quality of the firmware itself with regards to the implementation of the new features.

Duet firmware is to me seems like a reasonably sized project with demand in perfectly executed software features and execution order. Imagine people use this software on their CNC mill, or plasma cutter and suddenly the software behave unexpectedly and led to a crash on CNC or fire hazard on the plasma cutter. However I've seen none of the basic set of software tests perform on the project itself.

I'm just trying to hint out that the longer this thing released untested with increasing number of new features, the more messy and problematic and unreliable it will become in the future. Do not count on real world user to test software and report bug to you. People will tend to move on and find alternative that ensure the expected quality.

From what I've observed on the Github commit activities, the firmware is mostly a one man show who does almost all the implementations (1 man for firmware, 1 man for DWC, 1 man for PanelDue, 1 man for DSF, etc. From my experience, this has led to several project downfall and abandonware in the Automotive industries. OEM that experienced this kind of problem will mostly find a suppliers/subcontractors to overtake this deep pile of shit from them, and let the cheaper labor ensure the software quality for them instead. Recall is very costly in the industry, and I've seen a recall from a software module that was tested once and everybody thinks that retest in unnecessary because the module was untouched between software releases. Bugs begin to show once this module in integrated with another module in the system.

Each software feature/functionality deserves, at least, its own unit tests, that must be executed prior to release. I've not even move up to the Integration tests or even System tests that in usually perform sequentially to ensure quality in the proper software release cycle.

You have written up well enough System/Architecture requirements in Dozuki, but lack of traceability and the verification step to ensure the requirements are fully met. I think starting with well written requirements and a test project on Github that could be linked to the intended requirements will be a great starting point.

Hello,

M291 works in RC1 but stop working in RC2.
The babystepping buttons in M291 permanently greyed out.

It is implemented in the custom macro.

M291 P"Please move the nozzle down to 0.10mm above the bed (use thickness gauge), then press OK to perform calibration." R"Nozzle Distance Calibration" S3 Z1 ; display message to jog nozzle down
M292 P0 ; continue operation

@oliof @dc42
Thank you very much for both of your responses. I'm just personally doubting about the quality and the test coverage of the more recently software releases. I know that a whole team of devs/testers is needed with enough funds to feed into the development process. I'm not blaming anyone here and also not trying to say who is not competent enough to develop good software.

V3.4 is still under development cycle and a release candidate was just recently released. Release candidate is from my point of view, should be the most recent stable version with zero new features allowed to be introduced to the software cycle. Only intensive testing and bug fixing should be focused at the current state onwards. Maybe you guys have already done enough testing that wasn't documented on Github. I don't mind If the current development process already fits your team workflow. It's really hard for a small team to develop really stable software, It's also hard for one man to keep track bugs that was reported, while also bringing new features to the software at the same time.

Dev hates testers for how annoying and picky they can be.
Testers also hate it when dev made errors on a simple task. No human is perfect even though you thought so.
If the dev has to test his own software, maybe they will find a few bugs here and there, but not much when a 2nd/3rd person looking at it against the written requirements.

Automotive ECUs are fairly flexible as well, with homologation laws and multiple variants of car configuration possible. The same ECU will also go into several car brands and models in its parent group. Some control units from Audi will also land in VW, Porsche, Lamborghini, Bentley, Skoda, etc. Very flexible indeed. They were still able to ensure great software quality in their system as well.

System test are very costly and time consuming to implement and run. But skipping the earlier tests in the V-chain is not gonna help uncover the bug in an isolated environment. At least some tests are better than no tests at all.

I like Duet ecosystem a lot and own genuine board. All the printers that I've built were designed to run Duet from the very start. Never plan to made a switch to Klipper or anything else similar in the near future. I'm glad to hear that further testing are planned in the near future.

Hi, I've experienced in software testing in the Automotive Industries before.
Upon looking at the source code on Github, I've not seen any information or anything related about testing on Duet firmware.

Recently (v3.4+), I've experienced somewhat huge amount of software issues that weren't there in the previous betas, but surprisingly introduced in the newer beta/RC versions. That is why I'm starting to question about the quality of the firmware itself with regards to the implementation of the new features.

Duet firmware is to me seems like a reasonably sized project with demand in perfectly executed software features and execution order. Imagine people use this software on their CNC mill, or plasma cutter and suddenly the software behave unexpectedly and led to a crash on CNC or fire hazard on the plasma cutter. However I've seen none of the basic set of software tests perform on the project itself.

I'm just trying to hint out that the longer this thing released untested with increasing number of new features, the more messy and problematic and unreliable it will become in the future. Do not count on real world user to test software and report bug to you. People will tend to move on and find alternative that ensure the expected quality.

From what I've observed on the Github commit activities, the firmware is mostly a one man show who does almost all the implementations (1 man for firmware, 1 man for DWC, 1 man for PanelDue, 1 man for DSF, etc. From my experience, this has led to several project downfall and abandonware in the Automotive industries. OEM that experienced this kind of problem will mostly find a suppliers/subcontractors to overtake this deep pile of shit from them, and let the cheaper labor ensure the software quality for them instead. Recall is very costly in the industry, and I've seen a recall from a software module that was tested once and everybody thinks that retest in unnecessary because the module was untouched between software releases. Bugs begin to show once this module in integrated with another module in the system.

Each software feature/functionality deserves, at least, its own unit tests, that must be executed prior to release. I've not even move up to the Integration tests or even System tests that in usually perform sequentially to ensure quality in the proper software release cycle.

You have written up well enough System/Architecture requirements in Dozuki, but lack of traceability and the verification step to ensure the requirements are fully met. I think starting with well written requirements and a test project on Github that could be linked to the intended requirements will be a great starting point.

@dc42 Hi, It's hard for me to make the issue reproducible as well.

What you could try is to set the M203 extruder max speed to some really high value (50,000mm/min) or some value that the extruder will definitely stalled upon rapid movement. Also up the jerk/acceleration values to allow the extruder to stall on unwanted rapid movement as well.

Make the retraction length really long (10-20mm) but only set M207 retract at normal speed (35-60mm/s).

Slice and run the retraction tower or anything similar that will perform G10/G11 a number of times. The problem is that the issue is inconsistent, with the same Gcode file, extruders will sometime randomly do rapid extrusions/retractions. The behavior sometime happen with just a manual G1 E** input in the console as well.

** I ended up rolled back to v3.3 to get the normal behavior back.

@diamondback The issue is still there, but I have to find the way to make the machine work reliable again around this issue. With the rapid retracts/extrusions, sometime I get extruder jam on softer filaments (HIPS/ABS/Some PLA variants).

@diamondback Hi, do you mind sharing your jerk/acceleration/max speed for extruder here as well?

The problem seems to go away when I reduced the value of jerk/acceleration of extruder. I would like to know how your machine is setup for these values.

The problem is definitely there, I'm just setup the config to avoid this specific firmware issue.

@dc42 I was be able to make the problem go away by changing acceleration/jerk/max. speed of the extruder. The same print file running with old config.g and new config.g show no sign of unwanted rapid extrusion.

The test ran using 3.4b7+7. This might help narrow down the issue.

config-old.g
config-new.g

The jerk was reduced from 1000 to 300mm/min (5mm/s)
Max speed was reduced from 9000 to 3600mm/min (60mm/s)
Acceleration was reduced from 10000 to 5000 mm/s2
Since the above parameter was reduced, I've reduced the extruder motor current from 800mA to 600mA as well.

@dc42 Unfortunately, the issue is still persist with the 3.4b7+3 firmware.

@dc42 I'll test with the attached firmware and let you know how it is

@dc42 I have found something interesting. Since the firmware retraction behave unexpectedly, I've switch to slicer retraction and ran some test prints. The retract/unretract or let say any commands that involve extruder movement. Sometime the extruder move at a very high speed as well (Command that involve pure extruder move e.g. G1 E5).

This show that the issue is not purely related to retractions.

I think it will be easier to debug the issue by simply setup a gcode with a bunch of G1 E** commands and observe the extruder speed.

Hope it help for further investigation.