The fill handle is a standard one metre long when 'link to fill origin' is selected. I suppose one metre is a convenient length that would be visible at most zoom levels, rather that 1mm or 1cm which wouldn't be, but I agree with the initial poster that it is a bit too long when working on details and complex profiles.
When you change to 'distorted fill' it allows you to stretch the handles and change the scale by typing directly into the tracker, as it interprets that distance as the scaling factor; 0.5m halves the scale in that particular direction, 2.0m doubles the scale.
There is a button on the 'On-Screen Display Options' toolbar which allows you to toggle their display on and off. As the name suggests, these handles are on-screen display options only and do not print whatever the setting.
Hope that clears it up!