I use the Quick Layers palette a lot for this kind of thing. Just select the problem layer, hide it, do your magic wand bit then undo the hide. This is probably one of those divisive issues where you can't please everyone, I know I have taken advantage of overhead lines to hatch areas in the past.
Apple iMac macOS Ventura / AC26UKI (most recent builds)