So, I finally solved my issue. The culprits where all these idlers:
These are the typical cheap ones, that had a fair amount of play between the inner 3mm bore and the m3 bolt which is of course less a bit less that 3mm. I did a quick test by replacing them with pairs of F623ZZ (which i was able to tighten all the way without blocking their rotation...) and had a huge quality improvement, so i went ahead and redesigned all the idlers to use F625ZZ to go from 3mm to 5mm shaft, and placed an order for shoulder bolts. That should be considerably stiffer.
That of course as far as the mechanical part is concerned. The slicer related issues might be compensated a fair amount by tweaking the settings, but not entirely disappear.
Thanks for all the ideas!
Edit: forgot to add some samples, not anything complicated but that only prints i have at the moment