The only way I have ever made Align View work with absolute perfection is as described in "Illustration in Archicad," my very first book. In placing a piece of furniture into a rendering of the surrounding room, it works if you zoom in enough to place your anchor/reference points to absolute pixel accuracy. Make a rendering of the surround room and use that image as the reference for the new rendering. It will correctly place the camera. Yay. Congratulations GS for making a tool work right, even though, like adding salt to a recipe, they do say that you must tweak the results to suit. Weasels.
Things in the real world that detract from emulating this perfection include:
1: as Karl says, any pincushion or barrel distortion - this rules out any zoom lens on an idiot camera, say. Me likem Hasselblad SWC, eh? No bulging there.
2: lack of plumbness - consult a mason - crooked pictures make it difficult to line things up.
3: supremely wide angle pictures are harder to match.
4: overlapping panoramas are impossible to match. I once had a guy send me an assembled panorama for a manual alignment without warning me that it was nonsense. Grr.
5: And more. Let me think.
The secret to aligning a photo is the ability to visualize in 3D. I can't understand why they'd make an Align View tool except that some users can't see for spit. You still need enough navigating practice to intuitively "walk" the model into place.
When i do an alignment on a blank site, I stake the site corners and mark a level. In the model, I make a matching slab. That is the easy way. Make the corners of the slab touch the marks and you are done. Except that the ground is often sloped.... can't rely on that.
Another way to do this - one that i write about in the Artlantis Attitude is to actually draw guidelines on the background photo to then drive your model into position.
Realistically, all you can hope for is plausibility. In my book "LightWorks in Archicad" there's a long article on making a model match a photo where we have both the model and professional images of the final building. Much to my horror, in the architectural photography of the actual building, the upper floors were greatly vertically stretched relative to the Archicad model - either through a crop of a wide angle picture or a lens shift. I say this to indicate that freedom from distortion is almost impossible to avoid in the real world of commercial photography. Matching those two images required use of Photoshop's warp tool, and not because we needed to meet up with the Romulans on the edge of the Neutral Zone to sign the Treaty of Algernon, who also likes flowers.
This will be an ongoing problem imposing orthographics on a spherical universe. There is no such thing as white, an a straight line.
Dwight Atkinson