To test the performance of your application, you
need to run it on some input data. Preparing good data for performance
testing can be a difficult task. For many server-type applications
this may involve setting up several machines, and creating an “artificial” load. Constructing the benchmark itself is probably
the single most time-consuming step in application tuning, but when
done correctly, it can be worth the effort.
Generally, you want the benchmark to resemble
as much as possible the real load that the application will face when
finally deployed.
There are several mistakes that are often made
at this stage. To avoid the most common ones, remember the following:
Make the benchmark repeatable; this is key to the
systematic approach to performance tuning.
Do not create unrealistic situations which could speed
up or slow down your application considerably. For example:
Do not direct all the traffic to just a few records
in the database which may cause exceptionally good performance because
of caching, or exceptionally bad performance because of lock contention.
Do not run all clients from one machine. This might
not exercise your application properly because of the performance
bottleneck on the client side.
Do not use too much invalid data; it may never get
to the components of the application that do the real work.
Do not use a different environment (like the JVM,
or options) for the performance testing than the one to be used for
the real deployment.
Size your benchmark appropriately:
If the benchmark runs only briefly, the profile data
will contain too much “startup noise” from the effects
caused by class loading, dynamic libraries loading, bytecode compilation
and optimization, and the application initializations (database connections,
threads, and data structures).
If the benchmark runs for too long, it will be very
time consuming to repeat the run, and you'll be tempted to make
several changes to your application between the consecutive profiling
runs, which is not recommended.
As your application grows in terms of performance,
you may need to scale up your benchmark as well.