I made a lot of more tests and for me the problem seems to be related to the speed difference between infill and walls or between inner outer wall(s).
When printing the object with constant speed, even high one (e.g. 80 mm/s), for all features the quality is better than with different infill/wall speed.
Also printing more walls is not helpful but identical speed for all walls will improve the quality as well.
In summary my printer, hotend (E3D V6) or settings can not handle speed changes well with respect of constant extrusion.
As far I understand pressure advance should be a solution for such issues however not for me (tried various settings).
My solution for now: infill at 80 mm/s, three walls at 40 mm/s.