Software Testing Sucks - Learn Software Testing

Recommended System Testing Process

Lets start by defining the term 'process'. Process is nothing but a set a activities that represent the way work is to be performed , simply put its a specific way of doing certain thing, in our context it is developing a software product.

The following is the team structure needed to implement the process

Feature Owners: This team interacts directly with the customers. They gather requirements, group them into features and a single person in the team may 'own' one or more features. They take the initiative, interact with different teams and provide direction, in developing this feature.

UI Team: User Interface is very important for any product.No matter how many good features a product has, if the UI sucks, then the product is doomed.So we have separate UI team, that does a lot of research in UI and knows the difference between a good UI and bad one, they specialize in designing UI for any product.

This team will design the UI for our product or features of the product. Feature owners will sit with UI team to get the UI for their features.They basically will come up with some sort of 'Page designs' or some mockups'.These mockups contains all the UI elements that should be present in that page,along with the look and feel, the navigation between pages should also be working.

Development team: Develops the Product

Testing team: Testing team tests the product

Actual Process:
The process begins after the feature owners develop a High level design document for each feature.Apart from the document,the page designs or the UI Mockups should be released for reference.

Development team starts coding the features, where as testing team starts preparing the test cases.

First, before the actual test cases preparation,its always better to prepare a test outline.A test outline basically contains multiple test scenarios/flows at a high level including the information regd what should be checked/verified at what point in the flow.
Apart from the flows, the document can contain a matrix which maps the requirements(the HLD can give a uniqueid for each requirement) from the HLD to the test flows to ensure that all requirements have been thoroughly checked and there is no redundancy

Second,Each test flow/scenario is converted to a single test case.The test case should be very detailed.It should specify the exact navigation steps,exact data and what should be checked.This is especially helpful when the test case writers are different from test case executers.

Third and optional step is automating these test cases with automation tools.

By the time the development team finishes coding the features and optionally do some testing from their side, the testing team should be ready with test cases(in case of manual execution) or automation scripts(in case test execution with automated tools)

Once the testing cycle starts, testing team tests the product and logs the bugs where as development team fixes the bugs.

Its always better to maintain two different application instances.One instance is for testing team to test, other instance is for development team/bug fix team.Both should be at the same code
hlevel.When a bug is logged,it should first verified whether it is reproducible in the developer's instance, if it is then it should be assigned to appropriate developer for fixing.If it is not,then the dev should sit with the testing team to find out whether it is really bug which requires code fix or some kind of application setting issue(these issues most commonly when testing suite of products which are tightly integrated).This approach has the following advantages.

If the issue is not reproducible on dev instance,then most likely the issue is some sort of setup issue
After the bug is fixed,then first the codefix should be applied on dev instance, verified and then it should be applied to testing instance.

How to reduce the amount of regression issues?
The rule of thumb is the more times the application code is changed,then the number of regression issues shoot up.So don't patch the system frequently,the following is the recommended patching policy
In a multiple testing round scenario, patch all the bugs between test rounds with the exception of extremely critical bugs.Extremely critical bugs which is severly hampering testing should be patched/code fix should be applied as and when the bug is fixed, but rest of the bugs should be fixed and kept ready, but should be patched between testing rounds.Sanity testing by dev should be done on the application instance after the patching is done and it should be released for next round of testing.In this round of testing all the test cases(even those that have passed in previous rounds) should be run again.

When the testing should be stopped?
In a multiround scenario whether to go for a next round or stop there depends on the number of bugs logged in last round of testing. The following criteria can be used.
No new critical bugs/regression issues were found
Minor issues found are less(less is relative term which depends on the application being tested)

How to catch and resolve the misunderstandings of the product requirements?
Since both the testing team and development team start with HLD, it might happen that one(or very rarely both) teams might have misunderstood the requirements.This can be catched earlier by having the testing documents and code reviewed by feature owners before even the testing cycle starts.
But if these type of misunderstandings went into the testing cycle,then testers might log a bug which the dev thinks is the expected behaviour.In such cases,those bugs must be brought to the feature owners notice and let them decide what should be expected behaviour.

Home