Testing

Software testing can be stated as the process of validating and verifying that a computer program/application/product:

• meets the requirements that guided its design and development,
• works as expected,
• can be implemented with the same characteristics,
• and satisfies the needs of stakeholders.

Verification:

Verification is the process to make sure the product satisfies the conditions imposed at the start of the development phase. In other words, to make sure the product behaves the way we want it to.

Validation:

Validation is the process to make sure the product satisfies the specified requirements at the end of the development phase. In other words, to make sure the product is built as per customer requirements.

Testing methods

Static vs. dynamic testing

Under Static Testing code is not executed. Rather it manually checks the code, requirement documents, and design documents to find errors. Hence, the name “static”.

Main objective of this testing is to improve the quality of software products by finding errors in early stages of the development cycle. This testing is also called as Non-execution technique or verification testing.

Static testing involves manual or automated reviews of the documents. This review, is  done during initial phase of testing to catch defect early in STLC. It examines work documents and provides review comments

Under Dynamic Testing code is executed. It checks for functional behavior of software system , memory/cpu usage and overall performance of the system. Hence the name “Dynamic”

Main objective of this testing is to confirm that the software product works in conformance with  the business requirements. This testing is also called as Execution technique or validation testing.

Dynamic testing executes the software and validates the output with the expected outcome. Dynamic testing is performed at all levels of testing and it can be either black or white box testing.

There are many approaches to software testing. Reviews, walkthroughs, or inspections are referred to as static testing, whereas actually executing programmed code with a given set of test cases is referred to as dynamic testing. Static testing can be omitted, and unfortunately in practice often is. Dynamic testing takes place when the program itself is used. Dynamic testing may begin before the program is 100% complete in order to test particular sections of code and are applied to discrete functions or modules. Typical techniques for this are either using stubs/drivers or execution from a debugger environment.

The box approach

Software testing methods are traditionally divided into white- and black-box testing. These two approaches are used to describe the point of view that a test engineer takes when designing test cases.

White-Box testing

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) tests internal structures or workings of a program, as opposed to the functionality exposed to the end-user. In white-box testing an internal perspective of the system, as well as programming skills, are used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs.

While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements.

Techniques used in white-box testing include :-

• API testing (application programming interface) – testing of the application using public and private APIs
• Code coverage – creating tests to satisfy some criteria of code coverage (e.g., the test designer can create tests to cause all statements in the program to be executed at least once)
• Fault Injection methods – intentionally introducing faults to gauge the efficacy of testing strategies
• Mutation testing methods
• Static testing methods

Black-box testing

Black-box testing treats the software as a “black box”, examining functionality without any knowledge of internal implementation. The tester is only aware of what the software is supposed to do, not how it does it.

Black-box testing methods include:-

a) Equivalence partitioning
b) Boundary value analysis
c) All-pairs testing
d) State transition tables
e) Decision table
f)  Fuzz testing
g) Model-based testing
h) Use case testing
i)  Exploratory testing
j)  Specification-based testing.

*Specification-based testing aims to test the functionality of software according to the applicable requirements.
This level of testing usually requires thorough test cases to be provided to the tester, who then can simply verify that for a given input, the output value (or behavior), either “is” or “is not” the same as the expected value specified in the test case. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and designs to derive test cases. These tests can be functional or non-functional, though usually functional.
Specification-based testing may be necessary to assure correct functionality, but it is insufficient to guard against complex or high-risk situations.

Black box testing is often used for validation and white box testing is often used for verification.

Grey-box testing

Grey-box testing involves having knowledge of internal data structures and algorithms for purposes of designing tests, while executing those tests at the user, or black-box level. The tester is not required to have full access to the software’s source code. Manipulating input data and formatting output do not qualify as grey-box, because the input and output are clearly outside of the “black box” that we are calling the system under test. This distinction is particularly important when conducting integration testing  between two modules of code written by two different developers, where only the interfaces are exposed for test. However, modifying a data repository does qualify as grey-box, as the user would not normally be able to change the data outside of the system under test.

Visual testing

The aim of visual testing is to provide developers with the ability to examine what was happening at the point of software failure by presenting the data in such a way that the developer can easily find the information he requires, and the information is expressed clearly.

Testing levels

Unit testing

Unit testing, also known as component testing, refers to tests that verify the functionality of a specific section of code, usually at the function level. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors.

These types of tests are usually written by developers as they work on code (white-box style), to ensure that the specific function is working as expected. One function might have multiple tests, to catch corner cases or other branches in the code. Unit testing alone cannot verify the functionality of a piece of software, but rather is used to assure that the building blocks the software uses work independently of each other.

Integration testing

Integration testing is any type of software testing that seeks to verify the interfaces between components against a software design. Software components may be integrated in an iterative way or all together.

or

Integration testing is a level of the software testing process where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units.

System testing

System testing is a level of the software testing process where a complete, integrated system/software is tested. The purpose of this test is to evaluate the system’s compliance with the specified requirements.

Acceptance testing

At last the system is delivered to the user for Acceptance testing.

****************************************************************************************

Smoke and sanity testing

sanity testing determines whether it is reasonable to proceed with further testing.

smoke testing is used to determine whether there are serious problems with a piece of software, for example as a build verification test.

Regression testing

Regression testing focuses on finding defects after a major code change has occurred.

The purpose of regression testing is to confirm that a  recent program or code change has not adversely affected existing features.

Regression testing is nothing but full or partial selection of already executed test cases which are re-executed to ensure existing functionalities work fine.

This testing is done to make sure that new code changes should not have side effects on the existing functionalities. It  ensures that old code still works once  the new code changes are done.

Acceptance testing

Acceptance testing can mean one of two things:-

A smoke test is used as an acceptance test prior to introducing a new build to the main testing process, i.e. before integration, regression.

Acceptance testing performed by the customer, often in their lab environment on their own hardware, is known as user acceptance testing (UAT) . Acceptance testing may be performed as part of the hand-off process between any two phases of development.

Alpha testing

Alpha testing is simulated or actual operational testing by potential users/customers or an independent test team at the developers’ site. Alpha testing is often employed for off-the-shelf software as a form of internal acceptance testing, before the software goes to beta testing.

Beta testing

Beta testing comes after alpha testing and can be considered a form of external user acceptance testing. Versions of the software, known as beta versions are released to a limited audience outside of the programming team.

Before its official release, a beta version ALWAYS undergoes a full cycle of internal testing, after which the application is sufficiently stable in the majority of computing environments.

A release notes file is supplied with each beta version. Release notes provide the following information:

• the exact version number,
• system and technical requirements for the equipment used for testing,
• the list of changes since the previous version, and,
• descriptions of known problems (if any) and other relevant information.

Sometimes, beta versions are made available to the open public to increase the Feedback field to a maximal number of future users.

Please note that a beta version is NOT the final version of the product and therefore the developer does not guarantee an absence of errors that may disrupt the computer’s operation and/or result in data loss.