Oh i'm blushing so hard to see my design here!
the idea of using block magnets like that on the other side too (instead of balls) is honestly clever too no idea why i didn't think of that it will save space and have more "suction". tho I worry it'll cause some alignment issues :?
btw for a direct drive tool I suggest a middle ring magnet.
the ball/block design I had did handle the weight but it'd fly off at high speeds. but with an additional magnet it'd be very very very secure! but if something hits the tool there's always the possibility it'd get lose, for that I can suggest you to add a mechanical switch to detect an attached tool and stop and warn you!