How many decimal places of precision does RRF use in its calculations?
-
I've noticed, for example, that when I input a delta rod length of 439.246 mm, the returned value when I send m665 is 439.25. Is the duet actually ignoring the last decimal place for calculations, or simply rounding for display?
-
It's rounding for display. The floating point calculations are done in single precision floating point, so 24 bits of precision. The integer calculations used in step generation are done mostly using 32 bits, with 64 bits where necessary.
Let me know if you find any figures that need to be displayed with more decimal digits.
-
Perfect, thanks for the clarification!
I don't think it's essential to have m665 return more precise values, but it might be a good feature to have so it isn't confusing for some people. It was slightly confusing for me, and I think I did end up inputting the rounded up value into config.g rather than .246 (I mean, 4 microns is not much :P)
