It seems to me that it is actually AC16 that is doing it correctly since it really triangulates the terrain based on the new geometry, whereas in the video there are tetragons left in the Mesh.
This is probably because the program code has been changed between these versions.
However, you are referring to sawtooth.
Do you mean by that that the vertical heights of the new generated mesh point along to edge of the cut vary drastically in height? That would be another issue.
Can you clarify that, possibly with a 3D screen shot?
Loving Archicad since 1995 - Find Archicad Tips at x.com/laszlonagy
AMD Ryzen9 5900X CPU, 64 GB RAM 3600 MHz, Nvidia GTX 1060 6GB, 500 GB NVMe SSD
2x28" (2560x1440), Windows 10 PRO ENG, Ac20-Ac28