Previously, we discussed the meaning behind the various index scores reported by the Cadalyst Systems Benchmark. Next, we talked about operating the benchmark. In this, part 3 of our blog, we finish discussing the operation of the Cadalyst Systems Benchmark.
The final choice of the benchmark’s initial dialog box enables the compare option, which lets you save and compare times for different test runs. This is a powerful tool that we added to C2008 v5.1 to help us develop new tests for the benchmark itself. You can use it to easily see the effects (if any) for alternate configurations of your workstation. This option is disabled by default.
Compare Options Menu
You have six choices here. The first three concern the operation of the compare function: Save Current Test Times for Later Comparison, then EXIT; Compare Current Test Times to Previous Test Times; and Save Current Test Times and Compare to Previous Test Times.
The compare function lets you compare the times from two different test runs, creating a new set of relative index numbers. It calculates the index numbers based on the ratio of the test times from the first selected file compared to the test times of the second selected file. If nothing has changed and the two different test times are virtually identical, the new calculated index is approximately 1.00. Where something has changed, the new index number clearly shows the relative improvement. For example, an index number of 1.50 indicates that performance has improved by 50%. This is a handy method for directly quantifying the benefits of, say, using a RAID 0 configuration for your hard drives.
The fourth choice, Just Exit C2012, simply aborts the compare option. The last two choices deal with help dialog boxes: Enable Informational Alert Boxes and Disable Informational Alert Boxes—the default setting.
Informational Alert Boxes
If enabled, informational alert dialog boxes popup (using AutoLISP’s alert message function) to provide contextual help when saving and comparing different test results. These dialogs, one for each of the three compare options, guide you through the process of using the compare function until you are ready to disable them.
One More Option
There is a hidden option (disabled by default) that appends the actual times for each individual test, in seconds, to the end of the C2012_data.dta file. To enable this option, you must edit the C2012_indx.lsp file. You can edit this text file using Windows’ WordPad utility. To enable the option, just remove the two semicolons at the beginning of the line located near the end of the file, which reads: ;;(load “times”).
(Note: The AutoLISP interpreter ignores any code on a line after a semicolon.)
This wraps up our blog on the Cadalyst Systems Benchmark. We hope it helps you to evaluate and compare the performance of different workstations running AutoCAD.
Author: Art Liddle
Previously, we discussed the meaning behind the various index scores reported by the Cadalyst Systems Benchmark. Here, we will talk about operating the benchmark. For our discussion, we will use the latest version of the Cadalyst Systems Benchmark, C2012 v5.4, which was recently released.
The Readme_C2012.txt file (included in the Zip file available for download) gives instructions for installing the Cadalyst Systems Benchmark. They are straightforward, so we will not repeat them here.
The initial dialog box for the C2012 benchmark offers several radio-button options for customizing the test, as well as edit boxes for recording selected information about your workstation configuration. We will discuss each item in order, working our way down from the top of the dialog box.
Record Current System Configuration
This collection of six edit boxes prompts you for key information about your system’s configuration. We strongly suggest that you take full advantage of this section; record as much data as you can at the time of the test—up to a maximum of 132 characters per edit box. This information will make your life easier when you are pouring over the test data later. The benchmark stores your responses in the C2012_data.dta file, using them as defaults the next time you run the test. (If it hasn’t changed, you don’t need to retype information for each test.) The last edit box (Remarks) lets you record general notes to yourself. The benchmark automatically determines and records the following information: AutoCAD version, graphics window size in pixels, and the current date and time.
This is where you choose which type of test to run: 3D/2D/Other Functions (the only option that includes the Total Index score), 2D/Other Functions, or 3D Functions Only. Depending upon the type of work you do with AutoCAD, you can save yourself a lot of time by choosing either of the latter two options. In addition, if you do not have at least a midrange 3D graphics card, skip the 3D test—it can take several hours to run with a low-end card.
Number of Test Loops
For our reviews, we typically choose to run three loops of the test; the benchmark automatically calculates and reports average scores and times. For the record, we have found that the scores from a single run of the test closely match the average scores from three iterations. New to the C2012 version: We added a Battery Rundown Test option, which runs 99 loops. As the name indicates, we use this option for measuring the battery performance of mobile workstations.
Next time, in part 3 of our series, we will discuss the Compare Option. Originally added to the Cadalyst Systems Benchmark strictly for our internal use, it proved to be so handy, we enabled it for everyone.
Author: Art Liddle
SPEC’s Application Performance Characterization (SPECapc) group has released the long-awaited new benchmark test for evaluating the performance of systems running Autodesk 3ds Max 2011. Launched last month, the updated test is available in professional ($495) and personal ($20) versions.
The SPECapc for 3ds Max 2011 benchmark is valuable for IT professionals, CAD managers and users, or others who want to evaluate the performance of new desktop or mobile computers running Autodesk 3ds Max 2011, gauge the impact of a hardware upgrade, or compare the performance of one system to another.
New features in SPECapc for 3ds Max 2011 include:
- updated tests based on new functionality in 3ds Max 2011;
- an improved user interface that makes it easier to configure and run tests;
- increased level of testing for shading and rendering in the pro version, including use of the Autodesk Quicksilver engine for accelerated CPU and GPU rendering; and
- automated benchmark results compilation in the pro version.
Merriam-Webster’s online dictionary lists one definition for benchmark as: a standardized problem or test that serves as a basis for evaluation or comparison.
The Cadalyst Systems Benchmark is designed to help you evaluate and compare the performance of workstations running AutoCAD. Comparing the performance scores between workstations (or different configurations) will help you make intelligent choices when purchasing a new computer or upgrading an existing one.
The Cadalyst Systems Benchmark reports a total index score and four component index scores keyed to specific performance areas, as well as individual numbers for each subroutine of the test. Note: the index numbers are simply a ratio of the base time for an operation compared to the current test time for an operation. Larger index numbers indicate better performance.
Total Index: This is an average of the four component indexes: It gives a quick look at the overall performance for a workstation. The Total Index score is the number we focus on for our reviews, but you can dig a little deeper for additional performance information relevant to your specific requirements.
3D Graphics Index: This is closely tied to a workstation’s graphics card. Depending on the rendering mode you typically use when working with 3D models in AutoCAD, you may want to focus on just one of the 3D graphics subcategories: Wireframe, Hidden, Conceptual, and Realistic. If you don’t work with 3D models, you can safely ignore this index all together.
2D Graphics Index: This measures more than just 2D graphics performance: It effectively measures all onscreen performance that does not involve rendering a 3D model. This component of the test creates, copies, blocks, moves, arrays, changes layers, changes colors, explodes, and erases three different types of objects: orthogonal lines, radial polylines, and text. Zoom and Pan commands are sprinkled throughout each test. Don’t pay any attention to the individual subcategories for this one, what matters is the total score.
Disk Index: This measures a workstation’s performance for reading and writing files to the hard drive. Most of the drives on the market today provide similar performance. What will make a big difference is having a pair of drives in a RAID 0 configuration. For RAID 0, file operations are simultaneously split between the two drives, nearly doubling performance.
CPU Index: This gauges the performance of the central processing unit at the heart of each workstation. It has proven to be an accurate measure of relative performance, especially with the turbo-mode of the new generation CPUs.
Author: Art Liddle
In this series, we’ve talked about why it’s so hard to keep your graphics card driver updated and what to do about it. In this post, I’ll give you my rules of thumb for updating.
The rule of thumb I use is to try out most updated certified driver from the list of apps that you are using. So if AutoCAD 2011 is certified for v8.773, but ANSYS 13 is only is certified to v8.43, try using v8.773. If you encounter an issue in ANSYS, I would recommend dropping back to the last working certified driver or the lowest common denominator certified driver.
What I Want to See from AMD and Nvidia
In the ideal world what I would like to see from AMD and Nvidia is for the Catalyst Control Center software (or Nvidia equivalent) to let me set a weekly or monthly update schedule to automatically check for updates.
It should also offer the option to check manually for when I don’t have a continuous Internet connection or am regularly behind a firewall. If the software finds a new driver version to download, it lets me know (as well as clearly letting me know what performance enhancements or bug fixes are included in the driver). Most importantly, it would let me scan my system for applications that are ISV certified and indicates if they are or are not certified for this new version. So at a glance I could see which of my critical apps are certified for the driver update. Then I can decide whether or not to update.
Hopefully I will see this kind of feature in a future Control Center update. Until then, I will keep checking the CAD website for driver update news. I also have an Outlook calendar alert set to remind me to check the AMD site every 3 months. I look forward to the next update and performance boost!
CAD/CAE ISVs Links for the Latest Certified Drivers
AutoCAD or any other Autodesk products
Creo Elements/Pro (formally called Pro/ENGINEER—at V5 they rebranded it to this marketing name) & Creo Element/Direct (formally Cocreate)
(V7900 and V5900 are certified and will be posted here soon)
NX, Teamcenter, Technomatix, Solid Edge:
Side Effects Houdini
Know of other good resources? Add them to the comments below!
Author: Tony DeYoung
My last post talked about why it is so hard to keep graphics card drivers updated. This post will help you figure out what you should do about it.
One option is to check the ISVs website to see what they post as the latest driver to use with their product. I will post several links to common ISV pages at the end of this blog series. However two caveats regarding checking the ISV’s website:
- Not every ISV posts the actual driver that they certify. They might just list the latest certified driver version and then you have to go to the AMD site to actually download it.
- The ISV may have certified a new driver, but be slow to actually list or post it, lagging behind by a few weeks or more.
What About Automatic Checking in the Application?
Autodesk AutoCAD, Inventor and Revit all support automatic checking. They have an XML that checks what the user is using and if it doesn’t match what Autodesk certified / tested then the user is notified. Also, these applications will notify the user if there is an updated driver that was tested/certified. This is a great per-application feature. Unfortunately this is not common practice for other software vendors. ( I encourage you to write to your favorite CAD vendor and get them to implement this!) So if you want to know what the latest driver is for your particular software, then you need to check the AMD website or the particular software vendor’s site.
Where to Get the Latest Drivers?
For me the simplest thing is to go to the AMD website:
This page is updated every other week with the latest information on which drivers are certified for which versions of software using a particular graphics card. So if it is on the AMD site, the ISV has certified that version of the driver for accuracy and performance.
If you own a Quadro you can do the same thing from their list of certified drivers (although their page interface is a bit more confusing, listing every software version with every driver and card at once).
For the last post in this series, I’ll give you some rules of thumb for updating.
Author: Tony DeYoung
We have a poll going over on Cadalyst.com, asking the question: What type of computer do you use primarily for CAD-related work? So far, desktop PCs and desktop workstations are neck-and-neck, but notebook PCs and even Macs are making a good showing. Head on over to cast your own vote and view results! Don’t forget to bookmark the Results page so you can come back later to see the latest stats.
I am always excited when AMD releases a new driver update. I’ve got a FirePro V8800 and with each update I typically I experience a hefty performance boost along with any bug fixes. My one frustration is that I only find out about this new release by reading about it on CAD news sites. That can often be weeks or months after the release. I kept wishing for some kind of automatic solution.
Auto-Updates are in My Browser Already
Automatic updates are one of the things I appreciate about Google Chrome. I don’t have to do any checking if I have the latest version. The browser silently updates so I always have the latest features, performance enhancements, and security fixes. Nice!
When Auto-Updating is Not Good
While I appreciate auto-updating in Chrome, I don’t want it other applications. Auto-updates in Firefox could break my many add-ons. Auto-updates in Adobe CS5 apps would seriously interrupt and delay my workflow. For my GIS client, I work behind a Firewall so any applications that try to automatically “call home” give errors. Every now and then, even in Chrome, auto-updates get annoying because I don’t like being the guinea pig for only partially tested “beta” features. And when my system starts slowing down, the first thing I start searching out and disabling are uncontrolled background processes. So auto-updates are not always such a great thing.
There is a New Version of Your Graphics Card Driver. Would You Like to Update?
But what about auto-updates for my FirePro graphics card? Wouldn’t this solve my driver “currency” issue? My FirePro V8800 would just automatically update every time there was a performance update. Or at least that it would pop up a notification: “Hey there is a new version of your graphics card driver. Would you like to update now?”
ISV Certified Drivers for CAD
It sounded like a great idea until I started thinking about all of the consequences. Aside from the reasons I mentioned above, there is another more critical reason not to have auto updates. I regularly use several different CAD and DCC software, and I depend on drivers that are ISV certified for optimal performance and stability. The problem is that not every application that I use is certified for the same driver version. It depends on the ISV (and the specific FirePro or Quadro card). AutoCAD 2011 might be certified for driver v8.773, but my copy of ANSYS is only certified for v8.43. With scores of CAD/CAE/DCC ISVs and hundreds of possible application combinations that can be installed on any one workstation, it is not likely that each application will be certified for the latest or even the same version of FirePro V8800 driver.
So if my FirePro driver automatically updated like Chrome, I could potentially get screwed with different CAD/CAE applications no longer working perfectly. In reality, with AMD at least, the situation is not that dire. From my mid July 2011 check, about 85-90% or so of the latest ISV certified drivers are all for a common single driver (this is more true for the newest cards). But still there are variations, particularly if you use applications across domains (e.g., CAD vs. Visualization vs. CFD) or have an older graphics card.
So what should you do to ensure your graphics card updates will play nicely with your software applications? I’ll cover that in my next post.
Author: Tony DeYoung