Manual Testing:Fundamental Of Testing

1. What is Software Testing and Why is it Important?
Defect can be caused by a flaw in the application software or by a flaw in the application specification. For example, unexpected (incorrect) results can be from errors made during the construction phase, or from an algorithm incorrectly defined in the specification. Testing is commonly assumed to mean executing software and finding errors. This type of testing is known as dynamic testing, and while valid, it is not the most effective way of testing. Static testing, the review, inspection and validation of development requirements, is the most effective and cost efficient way of testing. A structured approach to testing should use both dynamic and static.
Testing techniques:
1.Subjective Testing
Subjective testing mainly depends on an expert's opinion. An expert is a person who is proficient and experienced in the system under test. Conducting the test involves test runs of the simulation by the expert and then the expert evaluates and validates the results based on some criteria.
One advantage of this approach over objective testing is that it can test those conditions which cannot be tested objectively. For example, an expert can determine whether the joystick handling of the flight feels "right".
One disadvantage is that the evaluation of the system is based on the "expert's" opinion, which may differ from expert to expert. Also, if the system is very large then it is bound to have many experts. Each expert may view it differently and can give conflicting opinions. This makes it difficult to determine the validity of the system. Despite all these disadvantages, subjective testing is necessary for testing systems with human interaction.
2.Objective Testing :
Objective testing is mainly used in systems where the data can be recorded while the simulation is running. This testing technique relies on the application of statistical and automated methods to the data collected.
Statistical methods are used to provide an insight into the accuracy of the simulation. These methods include hypothesis testing, data plots, principle component analysis and cluster analysis.
Automated testing requires a knowledge base of valid outcomes for various runs of simulation. This knowledge base is created by domain experts of the simulation system being tested. The data collected in various test runs is compared against this knowledge base to automatically validate the system under test. An advantage of this kind of testing is that the system can continually be regression tested as it is being developed.
2. Heuristics of Software Testing
(a):Testability
Software testability is how easily, completely and conveniently a computer program can be tested.
Software engineers design a computer product, system or program keeping in mind the product testability. Good programmers are willing to do things that will help the testing process and a checklist of possible design points, features and so on can be useful in negotiating with them.
Here are the two main heuristics of software testing.
1. Visibility
2. Control
Visibility is our ability to observe the states and outputs of the software under test. Features to improve the visibility are
• Access to Code
• Event logging
• Error detection mechanisms
• Resource Monitoring
Categories of Heuristics of software testing
• Operability
The better it works, the more efficiently it can be tested.The system should have few bugs, no bugs should block the execution of tests and the product should evolve in functional stages (simultaneous development and testing).
• Observability
What we see is what we test.
 Distinct output should be generated for each input
 Current and past system states and variables should be visible during testing
 All factors affecting the output should be visible.
 Incorrect output should be easily identified.
 Source code should be easily accessible.
• Controllability
The better we control the software, the more the testing process can be automated and optimized.
Check that
 All outputs can be generated and code can be executed through some combination of input.
 Software and hardware states can be controlled directly by the test engineer.
 Inputs and output formats are consistent and structured.
 Test can be conveniently, specified, automated and reproduced.
• Decomposability
By controlling the scope of testing, we can quickly isolate problems and perform effective and efficient testing.
• Simplicity
The less there is to test, the more quickly we can test it.
The points to consider in this regard are functional (e.g. minimum set of features), structural (e.g. architecture is modularized) and code (e.g. a coding standard is adopted) simplicity.
• Stability
The fewer the changes, the fewer are the disruptions to testing.The changes to software should be infrequent, controlled and not invalidating existing tests. The software should be able to recover well from failures.
• Understandability
The more information we will have, the smarter we will test.The testers should be able to understand well the design, changes to the design and the dependencies between internal, external and shared components.
Technical documentation should be instantly accessible, accurate, well organized, specific and detailed.
• Suitability
The more we know about the intended use of the software, the better we can organize our testing to find important bugs.

3. When Testing should occur?
Wrong Assumption
Testing is sometimes incorrectly thought as an after-the-fact activity; performed after programming is done for a product. Instead, testing should be performed at every development stage of the product .Test data sets must be derived and their correctness and consistency should be monitored throughout the development process. If we divide the lifecycle of software development into “Requirements Analysis”, “Design”, “Programming/Construction” and “Operation and Maintenance”, then testing should accompany each of the above phases. If testing is isolated as a single phase late in the cycle, errors in the problem statement or design may incur exorbitant costs. Not only must the original error be corrected, but the entire structure built upon it must also be changed. Therefore, testing should not be isolated as an inspection activity. Rather testing should be involved throughout the SDLC in order to bring out a quality product.

Testing Activities in Each Phase
The following testing activities should be performed during the phases
• Requirements Analysis - (1) Determine correctness (2) Generate functional test data.
• Design - (1) Determine correctness and consistency (2) Generate structural and functional test data.
• Programming/Construction - (1) Determine correctness and consistency (2) Generate structural and functional test data (3) Apply test data (4) Refine test data.
• Operation and Maintenance - (1) Retest.

Now we consider these in detail.
Requirements Analysis
The following test activities should be performed during this stage.
• Invest in analysis at the beginning of the project - Having a clear, concise and formal statement of the requirements facilitates programming, communication, error analysis an d test data generation.
The requirements statement should record the following information and decisions:
1. Program function - What the program must do?
2. The form, format, data types and units for input.
3. The form, format, data types and units for output.
4. How exceptions, errors and deviations are to be handled.
5. For scientific computations, the numerical method or at least the required accuracy of the solution.
6. The hardware/software environment required or assumed (e.g. the machine, the operating system, and the implementation language).

Deciding the above issues is one of the activities related to testing that should be performed during this stage.
• Start developing the test set at the requirements analysis phase - Data should be generated that can be used to determine whether the requirements have been met. To do this, the input domain should be partitioned into classes of values that the program will treat in a similar manner and for each class a representative element should be included in the test data. In addition, following should also be included in the data set: (1) boundary values (2) any non-extreme input values that would require special handling.
The output domain should be treated similarly.Invalid input requires the same analysis as valid input.
• The correctness, consistency and completeness of the requirements should also be analyzed - Consider whether the correct problem is being solved, check for conflicts and inconsistencies among the requirements and consider the possibility of missing cases.

Design
The design document aids in programming, communication, and error analysis and test data generation. The requirements statement and the design document should together give the problem and the organization of the solution i.e. what the program will do and how it will be done.
The design document should contain:
• Principal data structures.
• Functions, algorithms, heuristics or special techniques used for processing.
• The program organization, how it will be modularized and categorized into external and internal interfaces.
• Any additional information.

Here the testing activities should consist of:
• Analysis of design to check its completeness and consistency - the total process should be analyzed to determine that no steps or special cases have been overlooked. Internal interfaces, I/O handling and data structures should specially be checked for inconsistencies.

• Analysis of design to check whether it satisfies the requirements - check whether both requirements and design document contain the same form, format, units used for input and output and also that all functions listed in the requirement document have been included in the design document. Selected test data which is generated during the requirements analysis phase should be manually simulated to determine whether the design will yield the expected values.

• Generation of test data based on the design - The tests generated should cover the structure as well as the internal functions of the design like the data structures, algorithm, functions, heuristics and general program structure etc. Standard extreme and special values should be included and expected output should be recorded in the test data.
Programming/Construction
Here the main testing points are:
• Check the code for consistency with design - the areas to check include modular structure, module interfaces, data structures, functions, algorithms and I/O handling.
• Perform the Testing process in an organized and systematic manner with test runs dated, annotated and saved. A plan or schedule can be used as a checklist to help the programmer organize testing efforts.
• Use available tools - the programmer should be familiar with various compilers and interpreters available on the system for the implementation language being used because they differ in their error analysis and code generation capabilities.
• Apply Stress to the Program - Testing should exercise and stress the program structure, the data structures, the internal functions and the externally visible functions or functionality. Both valid and invalid data should be included in the test set.
• Test one at a time - Pieces of code, individual modules and small collections of modules should be exercised separately before they are integrated into the total program, one by one. Errors are easier to isolate when the no. of potential interactions should be kept small. Instrumentation-insertion of some code into the program solely to measure various program characteristics – can be useful here. A tester should perform array bound checks, check loop control variables, determine whether key data values are within permissible ranges, trace program execution, and count the no. of times a group of statements is executed.
Operations and maintenance
Corrections, modifications and extensions are bound to occur even for small programs and testing is required every time there is a change. Testing during maintenance is termed regression testing. The test set, the test plan, and the test results for the original program should exist. Modifications must be made to accommodate the program changes, and then all portions of the program affected by the modifications must be re-tested. After regression testing is complete, the program and test documentation must be updated to reflect the changes.