This problem occurs on Mac computers with mainly AMD (formerly ATI) video cards. The video card on my iMac is ATI Radeon HD 2600 PRO. According to the article, this card's drivers contain bugs, so antialiasing was turned off by ArchiCAD. I downloaded recommended Pref Setter application and turned on antialiasing on AC17. The objects’ edges in the 3D view are shown perfectly now… at the cost of loosing some of the 3D window's feedback guides.