After automating
the test case, how many times you will use/run the automation script
for testing, before dumping it forever or modifying/upgrading it for
future Releases?
It takes roughly 3 times the effort to automate a test case than to
manually execute it once.The more times you run/use the automation script
without modifying it, the more will be the ROI(Return on Investment).Atleast
you should be able to use it 5 times.So don't go for automation tools
if the product is a short term product which doesn't involve many rounds
of testing.
Is 'Time spent
running the automation script' + 'Analysing the test results' less than
'Time spent on manual execution'?
Usually a automated script runs faster than manual execution.There are
exceptions to this assumption. Also we first run the automation script
completely and then analyze the test results, which can take considerable
time, but when we manually execute, we don't need separate time to analyze
the results,as we know what actions caused a check point to fail. Where
as in the former case, we need to do some investigation as to what actions
resulted to failing a check point.
So the 'Time spent on running the automation script' + 'analysing the
test results' must be less than the 'Time spent on manual execution'
Ofcourse, if the automation scripts are well developed, don't require
constant monitoring and run without manual intervention, then you can
kick start them at night, come back and check the results in the morning.
This way you can cut the time on running the scripts
Is it resuable
for future releases of the product?If Yes how much effort will be required
to upgrade the automation script?
The ROI will be more, if with little change, you can completely reuse
the automation script.
Is the Product
stable or does it keep changing quite often because of which the automation
script needs to modified?
Never develop automation scripts on products that are already not stable.
An unstable product keeps changing quite often resulting in the change
of the automation script.
Are the test cases
fit to be automated?
Certain test cases are not fit to be automated. Identiy them and don't
automate them.The criteria on how to identify such cases change from
product to product.I will share an example from my own experience. We
automated a complex test case, which took long time to run. Also the
product is an highly integrated application, so the test case often
jumps from one application to another.The product's performance was
not consistent.Adding to this, if somebody changed some settings of
one application, then sometimes the test case came to a sudden stop.If
the automation script stopped running because of any of the above mentioned
issues, then because of the nature of the test case and the way automation
script
was developed, we have to start running again right from the beginning.
So the end result was, the time taken to run the automation script was
way more than the time taken to execute it manually.
Fit test cases when automated run from end to end without any manual
intervention and verify all the check points they intend to. It partly
depends on how the automation script is developed.