ArchiCAD Benchmarks

[Anonymous]

It would be helpful if the members of the forum start a benchmark that compares how different computer configurations handle ArchiCAD. There are lots of non-ArchiCAD benchmarks comparing equipment but none that specifically addresses how equipment performs when running ArchiCAD. By establishing an ArchiCAD benchmark we could begin to paint a picture of what hardware makes ArchiCAD run better/faster.

We would need a common ArchiCAD file to use for the benchmark. A first step would be to find out if there is already a benchmark file in existence. If none exists one could be created quickly that has a good sampling of ArchiCAD's native library parts and elements. The file should contain enough elements or polygons to actually tax system performance. This might include a meshed terrain or other object with a good many polygons. The test file could be made available here on the forum as a downloadable file.

The next step would be to establish the parameters for testing. The following are some suggestions:
1) Computers should be restarted before testing to clear ram and other programs that might effect performance.
2) Record time to open the file.
3) Record time to manipulate the file in some manner that tests each system's performance speed. THis might be something simple like opening a preestablished section view in the project.
4) Record the time to open a 3D window of the project with a commonly established set of 3D Window settings.
5) Record the time to produce a PhotoRender of the project with a common set of PhotoRendering settings.
These four benchmarks would then be recorded here on the forum along with each system's configuration.

This sort of user benchmark testing has been done successfully on other software forums. It would help us, the users, become more informed about what equipment configurations allows us to use ArchiCAD at it's fullest.

Thank you,
John

[Djordje]

johncassel wrote:
It would be helpful if the members of the forum start a benchmark that compares how different computer configurations handle ArchiCAD.

http://archicad.saalmann.de/archispeed.html

[Karl Ottenstein]

johncassel wrote:
It would be helpful if the members of the forum start a benchmark that compares how different computer configurations handle ArchiCAD.

Hi John,

All good thoughts! ...and pretty much what used to be done. Ole Saalmann has been the keeper of the benchmark data at:
http://archicad.saalmann.de/archispeed.html

and David Morris used to keep us on track to follow the specifications for performing the benchmark in an apples to apples fashion. But, I haven't heard much from Ole or David on ArchiCAD-Talk since we moved to this new web format.

You might email Ole at the address on the page above (not copied here to limit his spam) and see what's up.

IMHO, expressed on the old list, the benchmark needs to be written by GS themselves as an API add-on so that the entire process is automated and guaranteed to be identically executed on every computer. (A 3rd party developer such as myself cannot write this benchmark since the current API kit does not provide all of the necessary features.)

I'll paste the "old" (as used in Ole's table) benchmark below - as defined and posted by David Morris. Note that the FORE TI file isn't part of the AC distribution anymore (didn't use new features). One of the two example files (Orchard and Residential) now included could be used ... except neither of them properly use all of the 8.1 features either... and now that we have linking to PLN from LBK, a benchmark should include a PM update via BG AC too...

Regards,
Karl


----- Original Message -----
From: "David Morris"
To: archicad-talk
Sent: Monday, July 23, 2001 4:43 AM
Subject: Re: Speed


> Here it is:
>
>
> ARCHICAD BENCHMARK PROCEDURE
>
>
> 1. Suggest performing system maintenance prior to
> benchmark (defrag, etc.). Close all other programs.
>
> 2. Open the file FORE TI 3D.PLA, located in ArchiCAD
> Examples folder.
>
> 3. Benchmark times consists of three components: time
> to generate a Hidden Line 3D Window, time to generate
> a Shaded 3D Window, and time to generate a PhotoRender
> Projection 3D Window from a Hidden Line.
>
> 4. Under Image/PhotoRendering Settings/Effects, make
> sure ArchiCAD Rendering engine is selected, and Light
> sources = sun (only) and Shadow casting = sun (only).
>
> 5. Under Image/PhotoRendering Settings/Size &
> Background, change Resulting Image to 640 x 400.
>
> 6. Under Options/Preferences/Imaging and Calculations,
> check 3D Settings changes bring 3D Window to front.
>
> 7. Generate Hidden Line 3D window, open
> Windows/Report, add the Elapsed times to get total,
> close window.
>
> 8. Generate Shaded 3D window, open Windows/Report, add
> the Elapsed times to get total, close window.
>
> 9. Highlight the Wireframe and then generate a
> PhotoRender Projection, open Windows/Report, add the
> Elapsed times to get the total, close window.
>
> 10. Add times of three components to get total
> benchmark time.
>
> This procedure takes less than 15 minutes.
>
> Post the whole report so that settings can be
> verified.

[Anonymous]

erm,
just wondering where can we download the main file for the benchmark?

[Karl Ottenstein]

L. wrote:
just wondering where can we download the main file for the benchmark?

Nowhere that I know. It was part of the distribution with earlier versions of AC. The license agreement prohibits any of us from posting it to a public place.

Karl

[Djordje]

L. wrote:
erm,
just wondering where can we download the main file for the benchmark?

It was included in all ArchiCAD installations until 8; so find a 7 CD!

[Anonymous]

This is my result after the test.
Version: Archicad 8.1 r1 and Archicad 7.0 r3
Platform: Single P4 3.06, Asus 9800XT, 1GB ddr333 ram.

Version: Archicad 7.0 r3

Wire frame
Document name : FORE TI 3D
Processed : 1176 walls 2335 objects 0 columns 0 ncBeams
264 slabs 0 meshes 30 pitch roofs
Start time : 3/05/2004 11:05:37 PM
End time : 3/05/2004 11:05:39 PM
Elapsed time : 2 seconds
----------------------------------------------------------------------

Hidden line

Projection : Custom axonometry
Camera azimuth : 262.457
Start time : 3/05/2004 11:05:39 PM
End time : 3/05/2004 11:05:39 PM
Elapsed time : 0 seconds
----------------------------------------------------------------------
Hidden line removal (analytic)
Start time : 3/05/2004 11:05:51 PM
End time : 3/05/2004 11:05:56 PM
Elapsed time : 5 seconds


Shading 3D

Polygon painting
Sun direction : vertical 35 horizontal 240
Start time : 3/05/2004 11:06:05 PM
End time : 3/05/2004 11:06:13 PM
Elapsed time : 8 seconds

Total time: 15 seconds------------------------------------------------


Version: Archicad 8.1 r1

Wire frame

Document name : FORE TI 3D
Processed : 1 walls 2355 objects 0 columns 0 ncBeams
Start time : 3/05/2004 11:11:47 PM
End time : 3/05/2004 11:11:48 PM
Elapsed time : 1 seconds
----------------------------------------------------------------------

Hidden Line

Projection : Custom axonometry
Camera azimuth : 255.609
Start time : 3/05/2004 11:11:48 PM
End time : 3/05/2004 11:11:48 PM
Elapsed time : 0 seconds
----------------------------------------------------------------------
Hidden line removal (analytic)
Start time : 3/05/2004 11:12:02 PM
End time : 3/05/2004 11:12:04 PM
Elapsed time : 2 seconds
----------------------------------------------------------------------

Shading 3D

Polygon painting
Sun direction : vertical 35 horizontal 240
Start time : 3/05/2004 11:12:13 PM
End time : 3/05/2004 11:12:24 PM
Elapsed time : 11 seconds

Total time: 14 seconds

This is interesting:
I added another test, same setting with the Archicad 8.1 version,
except changing from Internal 3D engine to openGL.


Wire frame
Document name : FORE TI 3D
Processed : 1 walls 2355 objects 0 columns 0 ncBeams
Start time : 3/05/2004 11:19:27 PM
End time : 3/05/2004 11:19:28 PM
Elapsed time : 1 seconds
----------------------------------------------------------------------
Projection : Custom axonometry
Camera azimuth : 255.609
Start time : 3/05/2004 11:19:28 PM
End time : 3/05/2004 11:19:28 PM
Elapsed time : 0 seconds
----------------------------------------------------------------------
----------------------------------------------------------------------

Shading 3D
Document name : FORE TI 3D
Processed : 1 walls 2355 objects 0 columns 0 ncBeams
Start time : 3/05/2004 11:20:21 PM
End time : 3/05/2004 11:20:22 PM
Elapsed time : 1 seconds
----------------------------------------------------------------------
Projection : Custom axonometry
Camera azimuth : 255.609
Start time : 3/05/2004 11:20:22 PM
End time : 3/05/2004 11:20:22 PM
Elapsed time : 0 seconds

Total time: 2 seconds

[Anonymous]

Hi everyone,

It would be great if we can have some result of Dual G5 in this test.
So that we can discover the myth between both platforms in Archicad.
And it would be very helpful for those who are going to buy new workstations or computers.

Thanks.

[Anonymous]

Karl wrote:

Hi John,
IMHO, expressed on the old list, the benchmark needs to be written by GS themselves as an API add-on so that the entire process is automated and guaranteed to be identically executed on every computer. (A 3rd party developer such as myself cannot write this benchmark since the current API kit does not provide all of the necessary features.
Karl

Karl,
I wrote to Ole and he encouraged the further development of the benchmark criteria and the benchmark test file by other people. He thought that it would be good to get a group of users to collaborate on a new benchmark file and and updated testing criteria.
I take it the prohibition to posting the file is due to the previous file having been the property of GS. In lieu of GS creating the benchmark test file do you foresee any problems or resistance to the forum members creating and posting our own benchmark file?
It wouldn't be as slick or accurate as the API add-on that you mention but I do think that it would be a handy reference point in guiding users in their hardware decisions.

Are there other users who would be interested in collaborating on a new benchmark test?
Thanks,
John

[Karl Ottenstein]

johncassel wrote:
I wrote to Ole and he encouraged the further development of the benchmark criteria and the benchmark test file by other people. He thought that it would be good to get a group of users to collaborate on a new benchmark file and and updated testing criteria.
I take it the prohibition to posting the file is due to the previous file having been the property of GS. In lieu of GS creating the benchmark test file do you foresee any problems or resistance to the forum members creating and posting our own benchmark file?
It wouldn't be as slick or accurate as the API add-on that you mention but I do think that it would be a handy reference point in guiding users in their hardware decisions.

Are there other users who would be interested in collaborating on a new benchmark test?

Thanks, John! Bummed that Ole has been too busy to work on this more.

Yes, the prohibition to posting the old file is that it is GS property ... but as Djordje pointed out, anyone with a 7.0 or earlier CD has a copy of it.

I'd really like to concurrently encourage GS to produce an automated benchmark suite, but agree, a group here should develope a new variant of the old manual method in the meantime. The problem with the old method is that is is impossible to verify that anyone follows the instructions exactly for all of the settings, and all too often the instructions weren't followed properly. Perhaps this time if the instructions are painfully detailed as far as what the settings needed to be for each test, the results would be more comparable.

The old benchmark instructions were pre-publisher. It seems like we could take the Orchard project and set up sufficient linked sections and 3D views and renderings so that time-to-publish could give an interesting measure of overall performance.

Hope that others jump on the bandwagon with you here!

Karl