|
|
The scope of this article is
limited to the need,
approach for requirement
collection and the steps
involved for Performance
Testing.
We all know and understand
that not “one-size-fits-all”.
The approach discussed in
this article predominantly
fits in to most scenarios
but is not limited to or the
only process to go about.
|
Introduction
Performance is a "must have"
feature. No matter how rich
your product is
functionally, if it fails to
meet the performance
expectations of your
customer the product will be
branded a failure.
Application architectural
design decisions may be
greatly influenced by the
importance placed by the
customer on one or more
specific requirements.
Incorrect design decisions,
made at the outset of a
project as a result of
invalid assumptions, may
become impossible to remedy
downstream. The
goal of performance testing
is not to find bugs, but to
eliminate bottlenecks and
establish a baseline for
future regression testing.
Where
do you start?
Identify your customer needs
It is
extremely important that you
fully understand your
customer's intentions and
requirements as early as
possible regarding software
performance i.e. the
operating environment (both
hardware and software) in
which your product will be
deployed and the manner in
which it will be used.
So,
how do you understand your
customer’s requirements?
Classify their needs
End-User Requirements:
The users of your
application don’t know or
care what the results of the
performance tests are, how
many seconds it takes
something to display on the
screen past their threshold
for "too long," or what the
throughput is. The only
thing application users know
is that they either notice
that the application seems
slow or not based on
what they have become
accustomed to.
In order
to quantify what the end
user needs,
- Determine
Application
Functionality and the
scenarios that they use
frequently for their
regular work.
- Verbalize and
Capture Performance
Requirements and Goals
- Quantify Captured
Performance Requirements
and Goals
- Record Performance
Requirements and Goals based
on their past and current
experience
- Summarize it back to
them reassure what you
have recorded and ask
for a formal sign off
Business Requirements
The Business
Requirements would look like
-
Transaction response
times < 3 seconds
-
Support 10000 users in
standard conditions
-
The
Amount of hardware they
have currently that
could be used to max
capabilit
Technical Requirements
Technical requirements are usually not directly
related to other categories
of requirement .These would
look something like
- CPU utilization rate
of over 80% is observed
for more than a few
seconds is a defect
For knowing these kind of requirements
ask the architects, or other
team members who may be
responsible for enforcing
and/or making decisions
regarding targets and
thresholds, to share those
decisions with you.
Standards, Compliance and Contracts
Determining performance
characteristics in this
category is conceptually
Straight-forward, but there
are often challenges in
obtaining and interpreting
the documents that spell out
the characteristics of
interest.
Often, contracts are viewed
as proprietary and are not
made available for review by
non-executives. It is
important to explain when
facing roadblocks to
reviewing contracts that the
specific language and
context of any statement
related to application
performance is critical to
determining compliance.
The bottom line is simply
this, don’t assume that you
are done capturing goals and
requirements until you have
checked to see if some
official or de-facto
standard that your
application is likely to be
compared against.
Determining all the above
information is particularly
important when a customer is
seeking your advice on
purchasing suitable hardware
and software specifications
in order to best deploy your
product or his product (in
case you are a service
provider). As there will
undoubtedly be lead times
for purchasing and
configuring such items, your
customer is likely to want
this information at the
earliest opportunity.
Now that you have identified
the requirements, the
following would be your
action items.
- Derive Transactions
mix (based on
Application/System
usage)
- Understand Data
volumes needed.
- Understand Maximum
allowable response
times.
- Understand Minimum
transaction throughput
rates.
- Choose a Performance
test tool which best
suites your product
needs
- Perform the tests
- Give updates on a
regular basis.
Each one of the above steps
listed is an exercise by
itself – I shall try and
cover in my upcoming
articles. Please leave your
feedback about this
article.
You
might also find our other
articles on
performance testing
interesting.
|
