Click HERE to download a .ZIP file with the WORD97 and PowerPoint97 files

NT Benchmarking

Dr. Bernard Domanski

The City University of New York/College of Staten Island

Email: {domanski [at] mail [dot] csi [dot] cuny [dot] edu } Phone: 732-303-1500

Overview

Our objective here is simply to identify the current state-of-the-art in benchmarking the speed of processors capable of running Window NT^®. We will describe some of the standard processor ratings and benchmarks that currently exist in the industry, and, where possible, we will offer ratings of various processors. Our research indicates that a fair amount of work has been done in developing different benchmark suites that correspond to different workloads, each yielding different processor ratings. Finally, pointers to World Wide Web sites will be provided for the reader to access more information on a particular subject.

Existing Benchmarks -

AIM's Server Benchmark For The Windows NT Operating System

AIM Technology's Server Benchmark for Windows NT is a system level WIN32-compliant Benchmark for Windows NT. This benchmark utilizes a load-mix benchmarking technology and is designed to test overall system performance of standard Windows NT Server configurations on Alpha, Intel, and PowerPC platforms.

AIM Technology uses load-mix benchmarking to test how well multi-user systems perform under different application loads. The role of load-mix benchmarking is to apply any type of load to a system running NT. The benchmark includes a pre-defined set of application mixes to model the most general uses of server systems. The two initial application mixes for the Server Benchmark are the Domain Server Mix and File Server Mix. AIM plans to develop additional application mixes in future months.

Standard Mixes

Domain Server Mix/Windows NT

The AIM Domain Server Mix/Windows NT is composed of 50 different tests from what appears to be all subsystem categories. The Domain Server Mix represents a balanced usage of subsystems that are configured as a reportedly typical enterprise shared servur. (Aside - what is really typical? ) The major tasks performed by this typical domain server include email, shared applications such as spreadsheets and word processing, light file transfers, network routing and packet forwarding, and network maintenance.

File Server Mix/Windows NT

The AIM File Server Mix/Windows NT is composed of 37 different tests from nearly all major subsystem categories. The File Server Mix represents a balanced usage of subsystems for a gateway file server. The major tasks performed include file transfers of various sizes (both synchronous and asynchronous), network routing and packet forwarding, system security and access permission checkinG, heavy memory usage and IPC calls.

Custom User Mix

If an application load is thought of as unique, the benchmark is designed to allow customization so that it comes closer`to modeling the actual environment using what is called an application mix template. This provides more control over how to the mix of application loads, giving a more in-depth picture of how systems will run in the unique environment. The application mix can be customized by:

• Varying the standard mix to more closely model the applications in your environment;
• Adding C-language tests;
• Adding command scripts.

Certified Server Reports

AIM uses these benchmark results to generate third-party independent Certified Reports for Windows NT. AIM produces a Certified Report for Windows NT by combining the results from Server Benchmark, the Workstation Benchmark and the AIM Subsystem Benchmark.

These reports detail the performance results for the selected system running a specific application mix. The Server Report for NT also includes WinNT Peak Performance in application jobs per minute (the system's highest performance level when a significant amount of CPU, RAM, and disk caching is used) and the WinNT Sustained Performance (a measurement taken at maximum system capacity).

Aim Server Tests For Windows NT

The following table lists 59 different subsystem tests for the AIM Server Benchmark for Windows NT. They test the configuration in the following categories: Floating Point Performance, Integer Math Performance, Disk Performance, InterProcess Communication, Directory Routines, System and Memory Performance, Run Time Support, and Algorithmic Operations.

NT Benchmark Results - Certified Server Report

Below we've reproduced a small portion of an AIM Certified Server Report, so you can get a better idea of what they provide. Performance ratings are noted by the Peak and Sustained values listed, and they are defined as:

• The Peak Performance rating reflects the highest system throughput at the point in which the system is able to process the most jobs per minute.
• The Sustained Performance metric indicates the multitasking load where the system's performance may become unacceptable (ie less than 1 job/minute/AIM load). This value relates to the maximum system capacity.
• The Thread Variation Index (TVI) on Windows NT is a measurement of the difference in completion time for concurrent jobs. This number reveals the ability of a system to run processes simultaneously.

AIM's Subsystem Benchmark For the Windows NT Operating System

The AIM Subsystem Benchmark for Windows NT exercises and measures each component of a computer system running NT. The benchmark uses 73 subtests to generate absolute processing rates, in operations per second, for subsystems, I/O transfers, function calls, and system calls. Test results can be used to compare different machines on a test-by-test basis, or to measure the success or failure of system tuning and configuration changes on a single system. This benchmark yields specific results on a per-test basis.

The individual values shown with the Subsystem benchmark represent how well the subsystem components operate. These values can show how changes in the components, OS, or compiler options affect each subsystem. For example, differences in disk speed are very apparent with a glance at the disk tests. ThE amount of RAM effects the memory tests. A 20 minute test can show the results of adding ram, changing disks, or changing CPUs.

AIM Technology’s Workstation Benchmark for Windows NT^® is a system level WIN32-compliant Benchmark for the Microsoft Windows NT operating system. It is designed to test overall system performance of standard Windows NT Workstation configurations on Alpha and Intel platforms.

AIM Technology uses Load/Mix Modeling to test how well multithreading systems perform in different application environments. The role of Load/Mix modeling is to allow AIM to apply a set load to any workstation running the Windows NT operating system. The benchmark includes a pre-defined application mix to model the most general uses of workstation systemsn The Workstation Benchmark includes both 2-D and 3-D graphical tests, as well as the ability to run across multiple processors. The primary application mix for the Workstation Benchmark is the General Workstation Mix, which represents a balanced usage of subsystems that are configured as a typical corporate desktop workstation. The major tasks performed by the typical workstation include applications such as spreadsheets and word processors, graphical applications, Internet access, peer-to-peer connections, and email.

For more information on AIM: (800)848-8649 and www.aim.com or email: benchinfo@aim.com . AIM Technology is a wholly-owned subsidiary of Network General Corporation.

SYSmark/NT 4.0 is designed for five basic market segments: corporations that are migrating to Windows NT and need a tool to evaluate computer performance; end users who make purchasing decisions for computers running Windows NT; testing labs and publications whose benchmarking results influence purchasing decisions; systems developers who need to analyze and tune products under development; and VARs/resellers.

Benchmarking workloads within SYSmark/NT 4.0 represent word processing (WP), spreadsheet (SS), project management (PM), computer-aided design (CAD), and presentation graphics programs (Pres.).

.

"The demand for SYSmark/NT, our first NT-based benchmark, exceeded our expectations," says John Sammons, BAPCo's president. "We expect to have a high demand for this product from both those upgrading from our first release and from other NT users seeking performance benchmarks based on real-world business applications."

Workloads for SYSmark/NT 4.0 were developed based on BAPCo's standardized practice of surveying users to determine how they exercise popular applications in day-to-day work. The applications selected for testing had to be able to run across Intel and Alpha architectures. SYSmark/NT 4.0 can generate performance metrics as a composite of all the different applications or for a specific application, such as word processing or spreadsheets.

Workloads based on these applications for Intel and Alpha platforms are included in SYSmark/NT 4.0:

SYSmark/NT 4.0 is available for those who want to do their own benchmarking at a nominal fee. The benchmark was developed and is fully supported by the current BAPCo membership, which includes: AER Energy Resources, Amdahl, Apricot Computers, AT&T Global Information Solutions, Client/Server Labs LLC, Compaq, Dell, Digital Equipment Corp., Duracell, EMAP Computing Labs, Gateway2000, Hewlett-Packard, IBM, InfoWorld, Intel, Lotus, Microsoft, Motorola, NEC Technologies, Texas Instruments, Unisys and Ziff-Davis Labs.

The following summary of the top 20 NT performers using the Sysmark32 benchmark comes from http://www.ideasinternational.com/benchmark/bapco/sysnt4.html --

For more information, contact BAPCo, 2200 Mission College Blvd., RN2-02, Santa Clara, CA 95052; phone: 408-988-7654; fax: 408-765-4920; Internet: http://www.bapco.com.

ByteMark

This data is based on a usenet post to comp.sys.ibm.pc.hardware.chips by Eric Mintz. Byte Magazine's "Byte CPU DOS 32" is used for testing. It's popular because it's relatively easy to run and report, it's easy to get and free, and they publish the source code. But it really is a processor-only type of test.

#     CPU           Int    FP    NS    SS   Bit   FPem  Four  Assg  Idea  Huff  NNet  LUde
3.  i486DX2-66      0.41  0.25  0.43  0.27  0.43  0.50  0.28  0.38  0.40  0.49  0.20  0.30
6.  i486DX2-66      0.42  0.26  0.42  0.36  0.43  0.50  0.28  0.38  0.40  0.48  0.20  0.31
2.  Am486DX2/80     0.51  0.31  0.52  0.44  0.52  0.60  0.34  0.46  0.48  0.57  0.25  0.34
11. AM5x86-133      0.58  0.39  0.80  0.89  0.87  0.50  0.28  0.37  0.40  0.48  0.42  0.50
15. AMD486dx4-100   0.60  0.38  0.65  0.74  0.41  0.55  0.60  0.71  0.28  0.39  0.59  0.36
12. AM5x86-133      0.61  0.25  0.80  0.89  0.87  0.81  0.28  0.37  0.40  0.47  0.21  0.26
5.  Am486DX4/100    0.61  0.37  0.63  0.41  0.65  0.74  0.42  0.56  0.60  0.72  0.29  0.42
1.  iPOD-83         0.79  0.85  0.81  0.47  0.89  0.85  0.89  0.86  0.91  0.86  0.85  0.82
10. iP5-75          0.84  0.96  0.88  0.84  0.83  0.84  0.85  0.88  0.84  0.77  0.91  1.17
9.  AMD5x86-133     0.86  0.49  0.79  0.89  0.87  1.01  0.57  0.74  0.80  0.95  0.42  0.50
4.  Cx5x86-120      0.97  0.69  0.86  1.46  0.94  1.02  0.81  0.79  0.89  0.93  0.55  0.73
13. AM5x86-160      1.01  0.58  0.87  1.07  1.03  1.21  0.67  0.84  0.95  1.14  0.51  0.57
14. AM5x86-160      1.02  0.58  0.91  1.07  1.04  1.21  0.68  0.87  0.95  1.14  0.51  0.58
17. AM5x86-160      1.03  0.59  0.94  1.07  1.04  1.21  0.68  0.88  0.95  1.15  0.51  0.59
7.  iP5-100         1.10  1.21  1.12  1.08  1.10  1.12  1.12  1.16  1.11  1.01  1.16  1.35
16. P100            1.17  1.12  1.11  1.13  1.13  1.17  1.12  1.02  1.20  1.53  1.12  1.27
8.  iP5-120         1.30  1.38  1.27  1.28  1.33  1.35  1.35  1.36  1.33  1.21  1.35  1.45
18. iP5-166         1.80  1.74  1.82  1.73  1.84  1.87  1.86  1.87  1.84  1.68  1.89  1.48
Int = Overall Integer
FP = Overall Floating Point
NS = Numeric Sort
SS = String Sort
Bit = Bitfield
FPem = Floating Point Emulation

Four = Fourier
Assg = Assignment
Idea = Idea
Huff = Huffman
NNet = Neural Net
LUde = LU Decomposition

SPEC

What is SPEC

SPEC, the Standard Performance Evaluation Corporation, is a non-profit corporation formed to "establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers" (from SPEC's bylaws). SPEC's founders believe that the user community will benefit greatly from an objective series of applications-oriented tests, that can serve as common reference points and be considered during the evaluation process. While no one benchmark can fully characterize overall system performance, the results of a variety of realistic benchmarks can give valuable insight into expected real performance. The list of current members of SPEC includes: Open Systems Group (OSG): Amdahl, AT&T, Auspex Systems, Bull, Compaq Computer, Cray Research, Dansk Data Elektronik, Data General, Digital Equipment, Electronic Data Systems, FirePower Systems, Fujitsu, HAL Computer Systems, Hewlett-Packard, Hitachi, IBM, Intel, Intergraph, International Computer (ICL), Locus Computing, Motorola, Network Appliance, Nikkei Datapro, Novell, Olivetti, Pyramid Technology, Ross Technology, Siemens Nixdorf Informationssysteme, Silicon Graphics, Sun Microsystems, Tricord Systems, Unisys, and ZIFF Davis Publishing.

The Open Systems Group associates includes: Center for Scientific Computing (Finland), Defense Logistics Agency / Systems DeSign Center, Leibniz-Rechenzentrum (Germany), NASA AMES Research Center, National Taiwan University (Taiwan), Oregon Graduate Institute, OSF Research Institute, Princeton University, Technische Universitaet Chemnitz-Zwickau (Germany), University of Aizu (Japan), University of California at Berkeley. (2) High Performance Group (HPG): Convex Computers, Cray Research, Digital Equipment, Electronic Data Systems, Fujitsu America, Hewlett-Packard, International Supercomputing Technology Institute (ISTI, France), Kuck & Associates, NEC/HNSX Supercomputer, Silicon Graphics, and Sun Microsystems.

Legally, SPEC is a non-profit corporation registered in California. SPEC basically performs two functions:

• SPEC develops suites of benchmarks thay measure computer performance. These suites are packaged with source code and tools and are extensively tested for portability before release. They are available to the public for a nominal fee covering development and administrative costs. The license agreement specifies that SPEC members and customers agree to run and report results as defined in each benchmark suite's documentation.
• SPEC publishes a quarterly report of SPEC news and benchmark results: the SPEC Newsletter. This provides a single, central source of information for SPEC benchmark results. Both SPEC members and non-SPEC members may publish in the SPEC Newsletter, though there is a fee for non-members. (Note that results may be published elsewhere as long as the format specified in the SPEC Run Rules and Reporting Rules is followed.)

The SPEC organization really comprises two groups, each with their own benchmarks: - Open Systems Group (OSG) which covers benchmarks in a UNIX, NT, or VMS environment. The High Performance Computing Group (HPCG) covers benchmarking in a numeric computing environment, with emphasis on high-performance numeric computing. Many people think about the OSG benchmarks only (often: the CPU benchmarks only) when they hear "SPEC", as these benchmarks are indeed the best known ones of SPEC's benchmarks.

How to Contact SPEC

SPEC (Standard Performance Evaluation Corporation)

10754 Ambassador Drive, Suite 201

Manassas, VA 22110, USA

Phone: (703) 331-0180 Fax: (703) 331-0181

E-Mail: info@spec.org

Ziff-Davis Benchmarks

Ziff-Davis provides 3 different benchmarks at no charge that test application servers (ServerBench), file servers (NetBench), and web servers (WebBench). These are summarized here in terms of what they are, what they measure, and what benchmarks tricks are possible to improve scores.

ServerBench 4.0 - http://www.zdnet.com/zdbop/svrbench/svrbench.html

What Is ServerBench? ServerBench measures the performance of application servers in a client/server environment. It provides an overall score for a server and individual scores for the clients, which are PCs running Windows® 95 or Windows NT®. Test runs are started and monitored from the controller, which is a PC running Windows 95 or Windows NT.

TPS: The Units Used To Report Scores Results are in TPS, or transactions per second. Each client measures how long each transaction takes and how many transactions ran. The client calculates its TPS score by dividing the total number of transactions by the total amount of time for completion. ServerBench combines the individual client TPS scores to calculate the overall server score.

ServerBench provides an overall measure of how a servers' performance stacks up against others if the the standard system test0suite was run. If one of ServerBench’s subsystem test suites is run, the results obtrained indicate how well that server subsystem is performing. The higher the score, the better your application server performed.

What Is A "Transaction" A transaction consists of the request a client sends to the server, the response it gets back, and the time it takes from the moment the client sends the request until it receives the reply from the server. ServerBench breaks down the time the transaction spends traveling along the network to and from the server, waiting in a queue on the server to receive a service, and receiving service.

How Performance Is Measured ServerBench uses a weighted harmonic to calculate the overall server performance score. By using a harmonic mean, the scores for different transactions can be combined to create a single representative score. Different transactions are weighted based on how often clients request each transaction in one iteration of a mix.

To calculate the overall server TPS score, the ServerBench benchmark:

· Tracks the amount of time each transaction takes to complete.

· Tallies the number of completed transactions. Incomplete transactions or transactions that began during the Ramp up or Ramp down periods at the start and end of a benchmark run are not counted.

· Creates a total TPS score for each transaction by adding together the TPS score for each client.

· Uses a weighted harmonic mean to turn the total TPS scores into a single overall server score.

Setting Test Parameters To Get The Best Results Adjusting the test parameters for different mixes can also influence results. Specifically, Ziff-Davis points out:

· Number Of Transaction Iterations. The higher the value, the greater the weight that transaction will have`in the test results. In addition, the more times a client requests a transaction during an iteration of a mix, the longer one iteration of the mix takes.

· Number Of Test Iterations. The more test iterations specified, the longer the transaction. lower TPS scores. Lower TPS scores than expected could be seen because each client measures the amount of time it takes the server to get a transaction and then send a reply.

· Number Of Iterations For The Processor Test. The more iterations specified as the Total Size value for the Processor test, the larger the CPU load that the test places on the system. By creating tests with different Processor test iterations, improvements are made in the way the benchmark stresses the processor subsystem on servers with different processor power.

· Data Segment Size. When the size is increased, the amount of file data accessed during a test is increased. This has a significant impact on the file system cache. As you add memory to the server, this parameter should be increased so as to stress the disk subsystem.

· Segment Access Ratio. This is an indication of how spread out the file data is across more physical disk space without increasing the amount of data being accessed.

Creating Test Mixes To Stress The Server If the goal is to stress the server to the point where TPS drops off, then these "hints" from Ziff-Davis will likely help:

• Reduce the Think time parameter.
• Increase the number of clients in the test.
• Use a large value for the data range parameters (data segment size, segment access ratio)
• Increase the total size of the disk tests.

Determining What To Test On Your System When studying the different tests, consider what would be the most useful information desired about a system. For example:

• What subsystems do the primary software applications use the most? For example, if applications that spend a lot of time accessing the disk are used, then you may want to spend more time running tests that use the benchmark disk tests.
• Where are the bottlenecks in the system? To find those bottlenecks, you may want to run three different test suites, where each test suite focuses on a different server subsystem.
• Have you set the test parameters to provide a reasonable test of the system? For example, if only one Ethernet is used for a test that uses 200 clients and no Think time, it will be difficult to get a valid score for the server. Instead, you’ll be measuring the speed of a heavily congested Ethernet.
• Are there things that could be done to optimize the server's performance?
• What affect does changing a parameter have on results? You should experiment with changing system parameters and comparing the benchmark results before and after the change.
• How do different servers perform? By running the same test suites with thE same settings on different servers, you can make fair, non-biased comparisons across these servers and then determine the server that best meets the business requirements at hand.

Factoring In Multitasking Multitasking is at the core of the ServerBench tests. Many tasks share processing time in a multitasking environment. Tasks are swapped in and out of memory by the processor. Many of the machines ServerBench and the other benchmarks run on use multiple processors. The following, though, is a single processor example.

Each mix contains a fixed number of clients. The mix tells the clients which specific transactions to send to the server. The clients begin the test and start sending transaction requests to the server. The server, instead of telling the clients to wait while it handles a single client's request, takes all the transactions from all the clients. Because a processor can only respond to one request at a time, the server gathers up all the other requests and puts them in a queue to wait for the processor time they need. If there are many clients sending requests, the queues can get unruly. Meanwhile, as the clients wait for the server to reply to their tests, they're running a clock and calculating how much time the server is taking.

By knowing how many clients are competing for processing time and how much time it takes for them to get service, the throughput for the tests can be calculated and the point at which throughput drops off can also be identified.

Figuring Out The Knees In The Curves A typical curve that plots a server's total TPS throughput against the number of active clients is shown below. The point at which the number of clients is increasing but the throughput starts to level off is called the knee of the curve.

A Results Curve Containing a Single Knee

As the number of clients sending requests to a server increases, the total throughput on the server goes up. This is because at low client counts, the server is not being utilized to its full potentiall. Some of the components of a computer system will remain idle.

With a server, as the number of client requests increases, the overhead of managing the requests also increases. The additional requests take more memory, there is more work for the scheduler, there is only a limited bandwidth for the network, and so on. Eventually, the throughput starts decreasing, indicating the server reached the knee of the curve. This is the point where the contention for resources and the overhead of managing the resources causes throughput to decrease.

A server's configuration can affect where the knee of the curve occurs. Peak throughput can often be improved by:

• Adding another CPU.
• Increasing the amount of server memory used for the disk cache.
• Using an intelligent network card or disk controller.
• Adding RAM.
• Changing your processor's scheduling.

There are three general types of curves that you will see with ServerBench as well .

• A curve with a single knee that shows a very steep drop in throughput as the client load is increased.
• A curve with a single knee that shows a gradual, sloping drop in throughput as the client load is increased.
• A curve with a double knee. In this type of curve, the throughput reaches its knee and then levels out, just as a plateau does, as the client load in increased. Then, after adding a number of clients, the throughput drops off sharply. A double knee often means there are physical bottlenecks that affect the server's performance. These include the amount of traffic the network can handle, the speed of the disk, the amount of main memory, and so on. Because of these bottlenecks, the server throughput quits going up, but the server maintains a fairly level throughput as it continues to add clients. Eventually, though, after enough clients are added, the throughput will begin a rapid decline. This is the second knee of this curve.

A Results Curve With A Double Knee

To get a valid measure of a server's performance, run the benchmark test until you see the knee in the server's performance curve. This means the client load should be increased until the TPS scores begins to level off. In the standard test suites, add clients incrementally as the tests run (i.e., start with 1 client, then run the test with 4 clients, next 8 clients, and so on).

If the ServerBench test suites are run and don't hit a knee, then the intensity of the tests needs to be intensified. Try:

• Making the tests bigger; for example, increasing the initial size of the test data range. Use the Segment Access Ratio to do this. When the range is larger, the server will break out of disk cache and hit the disk more often.
• Increasing the Request size parameter so that more data is read off the disk. This will also cause the disk to be hit more often. In general, the random read disk tests are more likely to make a server break out of cache than the sequential read tests.
• Increasing the Total size parameter also adds intensity to the disk subsystem. For example, if you have a 100 MB cache with 80 clients where each client reads 4 MB, then the server will probably break out of cache with each disk operation.
• Making the mix longer. This is done by adding more transactions to the mix or just simply adding more tests to the individual transactions.
• If you make the test bigger, it usually Makes it longer.
• Reducing the amount of Think time. In general, the smaller the value for Think time, the more the test will stress the server.
• Adding clients.

How To Evaluate Throughput In Terms Of Your Server As you examine at the throughput graphs for different servers, keep in mind that the server with the best peak throughput number may not be the best server for your requirements. Exactly how the server handles intense loads must be studied. ServerBench measures how much the throughput degrades as the load is increased on a server.

An example: Suppose two servers are being compared. Server A reaches a knee at 30 clients and then experiences a sharp drop in throughput. Server B, however, reaches a knee at 25 clients and then begins a very slow drop in throughput. If you only have 25 to 30 users, then Server A is probably the server you want. But, if you have a workload equivalent to 40 clients, Server B may be the better server for you. This is because, while the throughput on Server B drops off sooner than on Server A, it doesn't deteriorate as sharply. This says that probably more users will get reasonable service. Since Server A's throughput shows such a sharp drop, then if you have a workload equivalent to more that 30 clients, some of those clients will get really poor service.

System Throughput Versus Client Response When looking at results, you may notice that the system throughput as shown by the overall score is different from the throughput for the individual clients. In other words, system throughput may increase even as client response time decreases. Don't worry. This is normal.

To understand how this happens, consider the following example. Imagine you are at a bank where each transaction takes ten minutes. There are four tellers and you. From your point of view, the process took you ten minutes (i.e., that is your response time). From the bank's point of view, one transaction occurred in ten minutes; thus, the transaction throughput was one in ten minutes or .1 per minute. Now, go through the same process but this time you are one of four customers. You still have to wait ten minutes, but the throughput is now four in ten minutes or .4 per minute.

Consider this scenario one more time but with eight customers instead of four. The tellers are working harder (probably, they're not asking people how they are any more), so the transaction time is now eight minutes. This means that, if you were at the end of the line, you had to wait sixteen minutes. This response time seems bad to you; however, throughput from the bank's point of view is now eight in sixteen minutes or .5 per minute. In other words, the bank's throughput improved even though your response time increased.

Similarly, suppose you are running the benchmark on a four-processor totally symmetric, multiprocessing system and a test mix is run that only does processor tests. It takes the first client 150 milliseconds to run the test. It takes the same amount of time for the second client and also for the third client. Thus, from the client's point of view, nothing has changed ¾ the test took the same amount of time for each client. From the server's point of view, it has tripled its throughput. This is a case where the overall system throughput improves but the client response times stay the same. In evaluating a server, consider both the system throughput and the client response time. You want to find out how evenly the server handles the clients.

Processor Scaling With a multiprocessor system, consider, too, the effect that the number of processors has on the scores. Within a certain range, the morE processors a system has, the better its scores. A point will be reached where the overhead required to manage more processors outweighs the advantage of having them. Finding how much throughput improves in relation to the number of processors is called processor scaling.

The multiprocessor systems likely to be used with ServerBench and other NT benchmarks are tightly coupled processors. This means all the processors have an equal access to memory. While access to memory is an advantage, it also results in overhead. The system has to coordinate access to memory. This means the hardware and software spend more time making this processing system work.