Boundary Value Analysis & Equivalence Partitioning with Examples

Practically, due to time and budget considerations, it is not possible to perform exhausting testing for each set of test data, especially when there is a large pool of input combinations.

·      We need an easy way or special techniques that can select test cases intelligently from the pool of test-case, such that all test scenarios are covered.

·      We use two techniques - Equivalence Partitioning & Boundary Value Analysis testing techniques to achieve this.

Boundary Testing:

Boundary testing is the process of testing between extreme ends or boundaries between partitions of the input values.

• So these extreme ends like Start- End, Lower- Upper, Maximum-Minimum, Just Inside-Just Outside values are called boundary values and the testing is called "boundary testing".
• The basic idea in boundary value testing is to select input variable values at their:

1. Minimum
2. Just above the minimum
3. A nominal value
4. Just below the maximum
5. Maximum

  In Boundary Testing, Equivalence Class Partitioning plays a good role

  Boundary Testing comes after the Equivalence Class Partitioning.

Equivalent Class Partitioning:

Equivalent Class Partitioning is a black box technique (code is not visible to tester) which can be applied to all levels of testing like unit, integration, system, etc. In this technique, you divide the set of test condition into a partition that can be considered the same.

• It divides the input data of software into different equivalence data classes.
• You can apply this technique, where there is a range in input field.

Example 1: Equivalence and Boundary Value

• Let's consider the behavior of tickets in the Flight reservation application, while booking a new flight.
• Ticket values 1 to 10 are considered valid & ticket is booked. While value 11 to 99 are considered invalid for reservation and error message will appear, "Only ten tickets may be ordered at one time."

Here is the test condition

1. Any Number greater than 10 entered in the reservation column (let say 11) is considered invalid.
2. Any Number less than 1 that is 0 or below, then it is considered invalid.
3. Numbers 1 to 10 are considered valid
4. Any 3 Digit Number say -100 is invalid.

We cannot test all the possible values because if done, the number of test cases will be more than 100. To address this problem, we use equivalence partitioning hypothesis where we divide the possible values of tickets into groups or sets as shown below where the system behavior can be considered the same.

The divided sets are called Equivalence Partitions or Equivalence Classes. Then we pick only one value from each partition for testing. The hypothesis behind this technique is that if one condition/value in a partition passes all others will also pass. Likewise, if one condition in a partition fails, all other conditions in that partition will fail.

Boundary Value Analysis- in Boundary Value Analysis, you test boundaries between equivalence partitions

In our earlier example instead of checking, one value for each partition you will check the values at the partitions like 0, 1, 10, 11 and so on. As you may observe, you test values at both valid and invalid boundaries. Boundary Value Analysis is also called range checking.

Equivalence partitioning and boundary value analysis are closely related and can be used together at all levels of testing.

Example 2: Equivalence and Boundary Value

Suppose a password field accepts minimum 6 characters and maximum 10 characters

That means results for values in partitions 0-5, 6-10, 11-14 should be equivalent

Test Scenario #	Test Scenario Description	Expected Outcome
1	Enter 0 to 5 characters in password field	System should not accept
2	Enter 6 to 10 characters in password field	System should accept
3	Enter 11 to 14 character in password field	System should not accept

Examples 3: Input Box should accept the Number 1 to 10

Here we will see the Boundary Value Test Cases

Test Scenario Description	Expected Outcome
Boundary Value = 0	System should NOT accept
Boundary Value = 1	System should accept
Boundary Value = 2	System should accept
Boundary Value = 9	System should accept
Boundary Value = 10	System should accept
Boundary Value = 11	System should NOT accept

Why Equivalence & Boundary Analysis Testing

1. This testing is used to reduce very large number of test cases to manageable chunks.
2. Very clear guidelines on determining test cases without compromising on the effectiveness of testing.
3. Appropriate for calculation-intensive applications with large number of variables/inputs

Summary:

• Boundary Analysis testing is used when practically it is impossible to test large pool of test cases individually
• Two techniques - Equivalence Partitioning & Boundary Value Analysis testing techniques is used
• In Equivalence Partitioning, first you divide a set of test condition into a partition that can be considered.
• In Boundary Value Analysis you then test boundaries between equivalence partitions
• Appropriate for calculation-intensive applications with variables that represent physical quantities

Decision Table Testing:

Decision table testing is a testing technique used to test system behavior for different input combinations. This is a systematic approach where the different input combinations and their corresponding system behavior (Output) are captured in a tabular form. That is why it is also called as a Cause-Effect table where Cause and effects are captured for better test coverage.

A Decision Table is a tabular representation of inputs versus rules/cases/test conditions. Let's learn with an example.

Example 1: Decision Base Table for Login Screen

Let's create a decision table for a login screen.

The condition is simple if the user provides correct username and password the user will be redirected to the homepage. If any of the input is wrong, an error message will be displayed.

Conditions	Rule 1	Rule 2	Rule 3	Rule 4
Username (T/F)	F	T	F	T
Password (T/F)	F	F	T	T
Output (E/H)	E	E	E	H

Legend:

• T – Correct username/password
• F – Wrong username/password
• E – Error message is displayed
• H – Home screen is displayed

Interpretation:

• Case 1 – Username and password both were wrong. The user is shown an error message.
• Case 2 – Username was correct, but the password was wrong. The user is shown an error message.
• Case 3 – Username was wrong, but the password was correct. The user is shown an error message.
• Case 4 – Username and password both were correct, and the user navigated to homepage

While converting this to test case, we can create 2 scenarios ,

• Enter correct username and correct password and click on login, and the expected result will be the user should be navigated to homepage

And one from the below scenario

• Enter wrong username and wrong password and click on login, and the expected result will be the user should get an error message
• Enter correct username and wrong password and click on login, and the expected result will be the user should get an error message
• Enter wrong username and correct password and click on login, and the expected result will be the user should get an error message

As they essentially test the same rule.

Decision Table testing importance:

This testing technique becomes important when it is required to test different combination. It also helps in better test coverage for complex business logic.

Boundary value and equivalent partition are other similar techniques used to ensure better coverage. They are used if the system shows the same behavior for a large set of inputs. However, in a system where for each set of input values the system behavior is different, boundary value and equivalent partitioning technique are not effective in ensuring good test coverage.

In this case, decision table testing is a good option. This technique can make sure of good coverage, and the representation is simple so that it is easy to interpret and use.

This table can be used as the reference for the requirement and for the functionality development since it is easy to understand and cover all the combinations.

The significance of this technique becomes immediately clear as the number of inputs increases. Number of possible Combinations is given by 2 ^ n , where n is the number of Inputs. For n = 10, which is very common in the web based testing, having big input forms, the number of combinations will be 1024.Obviously, you cannot test all but you will choose a rich sub-set of the possible combinations using decision based testing technique

Advantages of decision table testing

When the system behavior is different for different input and not same for a range of inputs, both equivalent partitioning, and boundary value analysis won't help, but decision table can be used.

The representation is simple so that it can be easily interpreted and is used for development and business as well.

This table will help to make effective combinations and can ensure a better coverage for testing

Any complex business conditions can be easily turned into decision tables

In a case we are going for 100% coverage typically when the input combinations are low, this technique can ensure the coverage.

Disadvantages of decision table testing

The main disadvantage is that when the number of input increases the table will become more complex

Cause-effect Graphing Based Testing:

·         Cause and effect graph is a dynamic test case writing technique. Here causes are the input conditions and effects are the results of those input conditions.

·         Cause-Effect Graphing is a technique which starts with a set of requirements and determines the minimum possible test cases for maximum test coverage which reduces test execution time and ultimately cost.

·         The goal is to reduce the total number of test cases still achieving the desired application quality by covering the necessary test cases for maximum coverage.

·         But at the same time obviously, there are some downsides of using this test case writing technique. It takes time to model all your requirements into this cause-effect graph before writing test cases.

The Cause-Effect graph technique restates the requirements specification in terms of the logical relationship between the input and output conditions. Since it is logical, it is obvious to use Boolean operators like AND, OR and NOT.

Notations we are going to use:

Now let’s try to implement this technique with some example.

1. Draw a cause and effect graph based on a requirement/situation
2. Cause and Effect graph is given, draw a decision table based on it to draw the test case.

Let’s see both of them one by one.

Let’s draw a cause and effect graph based on a situation

Situation:

The “Print message” is software that read two characters and, depending on their values, messages must be printed.

• The first character must be an “A” or a “B”.
• The second character must be a digit.
• If the first character is an “A” or “B” and the second character is a digit, the file must be updated.
• If the first character is incorrect (not an “A” or “B”), the message X must be printed.
• If the second character is incorrect (not a digit), the message Y must be printed.

Solution:

The causes for this situation are:
C1 – First character is A
C2 – First character is B
C3 – the Second character is a digit

The effects (results) for this situation are
E1 – Update the file
E2 – Print message “X”
E3 – Print message “Y”

LET’S START!!

First, draw the causes and effects as shown below:

Key – Always go from effect to cause (left to right). That means, to get effect “E”, what causes should be true.

In this example, let’s start with Effect E1.

Effect E1 is to update the file. The file is updated when
–  The first character is “A” and the second character is a digit
–  The first character is “B” and the second character is a digit
–  The first character can either be “A” or “B” and cannot be both.

Now let’s put these 3 points in symbolic form:

For E1 to be true – following are the causes:
–  C1 and C3 should be true
–  C2 and C3 should be true
–  C1 and C2 cannot be true together. This means C1 and C2 are mutually exclusive.

Now let’s draw this:

So as per the above diagram, for E1 to be true the condition is
(C1  C2)  C3

The circle in the middle is just an interpretation of the middle point to make the graph less messy.
There is a third condition where C1 and C2 are mutually exclusive. So the final graph for effect E1 to be true is shown below:

Let’s move to Effect E2:
E2 states to print message “X”. Message X will be printed when the First character is neither A nor B.
Which means Effect E2 will hold true when either C1 OR C2 is invalid. So the graph for Effect E2 is shown as (In blue line)

For Effect E3.
E3 states to print message “Y”. Message Y will be printed when Second character is incorrect.
Which means Effect E3 will hold true when C3 is invalid. So the graph for Effect E3 is shown as (In Green line)

This completes the Cause and Effect graph for the above situation.

Now let’s move to draw the Decision table based on the above graph.

Writing Decision table based on Cause and Effect graph

First, write down the Causes and Effects in a single column shown below

Key is the same. Go from bottom to top which means traverse from effect to cause.

Start with Effect E1. For E1 to be true, the condition is (C1  C2)  C3.
Here we are representing True as 1 and False as 0

First, put Effect E1 as True in the next column as

Now for E1 to be “1” (true), we have the below two conditions –
C1 AND C3 will be true
C2 AND C3 will be true

For E2 to be True, either C1 or C2 has to be false shown as

For E3 to be true, C3 should be false.

So it’s done. Let’s complete the graph by adding 0 in the blank column and including the test case identifier.

Writing Test cases from the decision table

I am writing a sample test case for test case 1 (TC1) and Test Case 2 (TC2).

In a similar fashion, you can create other test cases.

(A test case contains many other attributes like preconditions, test data, severity, priority, build, version, release, environment etc. I assume all these attributes to be included when you write the test cases in actual situation)

Conclusion

Summarizing the steps once again:

1. Draw the circles for Causes and Graphs
2. Start from effects and move towards the cause.
3. Look for mutually exclusive causes.

This finishes the Cause and Effect graph dynamic test case writing technique. We have seen how to draw the graph and how to draw the decision table based on it. The final step of writing test cases based on decision table is comparatively easy.

Need for White Box Testing

•          Better to test anything from outside-in as well as inside-out

•          Most of the functional and performance issues arise due to bad coding

•          Ultimately the code matters – hence test the code left right and center

•          Black box tests are used to check cause and effect without getting into internals

•          White box tests do the same by getting into the internals of every program

•          Every developer is by default a white box tester

•          Medical field requires general physicians and surgeons

•          Certain problems can be identified and solved only by surgeons

•          A person may look healthy but internally the person may have high bp and sugar; they are not visible externally

•          So far, the importance of white box testing is not upto the mark

•          Product companies definitely need white box testing

•          If your application must scale to a very large extent, white box tests are inevitable

What is White Box Testing?

White box testing (WBT) is also called Structural or Glass box testing.

White box testing involves looking at the structure of the code. When you know the internal structure of a product, tests can be conducted to ensure that the internal operations performed according to the specification. And all internal components have been adequately exercised.

White Box Testing is coverage of the specification in the code.

Code coverage:

Segment coverage:

Ensure that each code statement is executed once.

Branch Coverage or Node Testing:  Coverage of each code branch in from all possible was.

Compound Condition Coverage: For multiple conditions test each condition with multiple paths and combination of the different path to reach that condition.

Basis Path Testing: Each independent path in the code is taken for testing.

Data Flow Testing (DFT): In this approach you track the specific variables through each possible calculation, thus defining the set of intermediate paths through the code.DFT tends to reflect dependencies but it is mainly through sequences of data manipulation. In short, each data variable is tracked and its use is verified. This approach tends to uncover bugs like variables used but not initialize, or declared but not used, and so on.

Path Testing:  Path testing is where all possible paths through the code are defined and covered. It’s a time-consuming task.

Loop Testing: These strategies relate to testing single loops, concatenated loops, and nested loops. Independent and dependent code loops and values are tested by this approach.

Why we do White Box Testing?

To ensure:

• That all independent paths within a module have been exercised at least once.
• All logical decisions verified on their true and false values.
• All loops executed at their boundaries and within their operational bounds internal data structures validity.

The need of White Box Testing?

To discover the following types of bugs:

• Logical error tend to creep into our work when we design and implement functions, conditions or controls that are out of the program
• The design errors due to difference between logical flow of the program and the actual implementation
• Typographical errors and syntax checking

Skills Required:  We need to write test cases that ensure the complete coverage of the program logic.
For this we need to know the program well i.e. We should know the specification and the code to be tested. Knowledge of programming languages and logic.

Limitations of WBT: Not possible for testing each and every path of the loops in the program. This means exhaustive testing is impossible for large systems. This does not mean that WBT is not effective. By selecting important logical paths and data structure for testing is practically possible and effective.

What is logic coverage Testing ?

Logic corresponds to the internal structure of the code and this testing is adopted for safety-critical applications such as softwares used in aviation industry. This Test verifies the subset of the total number of truth assignments to the expressions.

Logic Coverage Sources:

Logic coverage comes from any of the below mentioned sources:

·        Decisions in programs

·        Finite State Machines and Statecharts

·        Requirements

 

Basis Path Testing:

The basis path testing is same, but it is based on a White Box Testing method, that defines test cases based on the flows or logical path that can be taken through the program. Basis path testing involves execution of all possible blocks in a program and achieves maximum path coverage with least number of test cases. It is a hybrid of branch testing and path testing methods.

The objective behind basis path testing is that it defines the number of independent paths, thus the number of test cases needed can be defined explicitly (maximizes the coverage of each test case).

Here we will take a simple example, to get better idea what is basis path testing include

In the above example, we can see there are few conditional statements that is executed depending on what condition it suffice. Here there are 3 path or condition that need to be tested to get the output,

• Path 1: 1,2,3,5,6, 7
• Path 2: 1,2,4,5,6, 7
• Path 3: 1, 6, 7

Steps for Basis Path testing

The basic steps involved in basis path testing include

• Draw a control graph (to determine different program paths)
• Calculate Cyclomatic complexity (metrics to determine the number of independent paths)
• Find a basis set of paths
• Generate test cases to exercise each path

Benefits of basis path testing

• It helps to reduce the redundant tests
• It focuses attention on program logic
• It helps facilitates analytical versus arbitrary case design
• Test cases which exercise basis set will execute every statement in program at least once

Conclusion:

Basis path testing helps to determine all faults lying within a piece of code.

What is Loop Testing?

Loop Testing is the variant of testing that completely focuses on the validity of the loop constructs. It is one of the part of Control Structure Testing (path testing, data validation testing, condition testing).

Loop testing is a White box testing. This technique is used to test loops in the program.

Types of loop Tested

The types of loop tested are,

• Simple loop
• Nested loop
• Concatenated loop
• Unstructured loop

Why do Loop Testing?

Loop Testing is done for the following reasons

• Testing can fix the loop repetition issues
• Loops testing can reveal performance/capacity bottlenecks
• By testing loops, the uninitialized variables in the loop can be determined
• It helps to identify loops initialization problems.

Testing Strategy for loop Testing

While testing loop, it has to be checked at three different levels:

• When loop is entered
• During its execution and
• When the loop is left

The testing strategy for all these loops is as follow

Simple loop

A simple loop is tested in the following way:

1. Skip the entire loop
2. Make 1 passes through the loop
3. Make 2 passes through the loop
4. Make a passes through the loop where a<b, n is the maximum number of passes through the loop
5. Make b, b-1; b+1 passes through the loop where "b" is the maximum number of allowable passes through the loop.

Nested Loop

For nested loop, you need to follow the following steps.

1. Set all the other loops to minimum value and start at the innermost loop
2. For the innermost loop, perform a simple loop test and hold the outer loops at their minimum iteration parameter value
3. Perform test for the next loop and work outward.
4. Continue until the outermost loop has been tested.

Concatenated Loops

In the concatenated loops, if two loops are independent of each other then they are tested using simple loops or else test them as nested loops.

However if the loop counter for one loop is used as the initial value for the others, then it will not be considered as an independent loops.

Unstructured Loops

For unstructured loops, it requires restructuring of the design to reflect the use of the structured programming constructs.

Limitation in Loop testing

• Loop bugs show up mostly in low-level software
• The bugs identified during loop testing are not very subtle
• Many of the bugs might be detected by the operating system as such they will cause memory boundary violations, detectable pointer errors, etc.

Summary:

• Loop testing is a White Box Testing. This technique is used to test loops in the program.
• Loops testing can reveal performance/capacity bottlenecks
• Loop bugs show up mostly in low-level software

What is Data Flow Testing?

Data flow testing is a family of test strategies based on selecting paths through the program's control flow in order to explore sequences of events related to the status of variables or data objects. Dataflow Testing focuses on the points at which variables receive values and the points at which these values are used.

Advantages of Data Flow Testing:

Data Flow testing helps us to pinpoint any of the following issues:

·        A variable that is declared but never used within the program.

·        A variable that is used but never declared.

·        A variable that is defined multiple times before it is used.

·        Deallocating a variable before it is used.

 

What is Mutation Testing?

Mutation Testing is a type of software testing where we mutate (change) certain statements in the source code and check if the test cases are able to find the errors. It is a type of White Box Testing which is mainly used for Unit Testing. The changes in mutant program are kept extremely small, so it does not affect the overall objective of the program.

The goal of Mutation Testing is to assess the quality of the test cases which should be robust enough to fail mutant code. This method is also called as Fault based testing strategy as it involves creating fault in the program

Mutation was originally proposed in 1971 but lost fervor due to high costs involved. Now, again it has picked steam and is widely used for languages such as Java and XML.

Following are the steps to execute mutation testing:

Step 1: Faults are introduced into the source code of the program by creating many versions called mutants. Each mutant should contain a single fault, and the goal is to cause the mutant version to fail which demonstrates the effectiveness of the test cases.

Step 2: Test cases are applied to the original program and also to the mutant program. A Test Case should be adequate, and it is tweaked to detect faults in a program.

Step 3: Compare the results of original and mutant program.

Step 4: If the original program and mutant programs generate the same output, then that the mutant is killed by the test case. Hence the test case is good enough to detect the change between the original and the mutant program.

Step 5: If the original program and mutant program generate different output, Mutant is kept alive. In such cases, more effective test cases need to be created that kill all mutants.

How to Create Mutant Programs?

A mutation is nothing but a single syntactic change that is made to the program statement. Each mutant program should differ from the original program by one mutation.

Original Program	Mutant Program
If (x>y)      Print "Hello" Else     Print "Hi"	If(x<y)        Print "Hello" Else       Print "Hi"

What to change in a Mutant Program?

There are several techniques that could be used to generate mutant programs. Let's look at them

Operand replacement operators	Expression Modification Operators	Statement modification Operators
Replace the operand with another operand (x with y or y with x) or with the constant value.	Replace an operator or insertion of new operators in a program statement.	Programmatic statements are modified to create mutant programs.
Example- If(x>y) replace x and y values If(5>y) replace x by constant 5	Example- If(x==y) We can replace == into >= and have mutant program as If(x>=y) and inserting ++ in the statement If(x==++y)	Example- Delete the else part in an if-else statement Delete the entire if-else statement to check how program behaves Some of sample mutation operators: GOTO label replacement Return statement replacement Statement deletion Unary operator insertion (Like - and ++) Logical connector replacement Comparable array name replacement Removing of else part in the if-else statement Adding or replacement of operators Statement replacement by changing the data Data Modification for the variables Modification of data types in the program

Automation of Mutation Testing:

Mutation testing is extremely time consuming and complicated to execute manually. To speed up the process, it is advisable to go for automation tools. Automation tools reduce cost of testing as well.

Types of Mutation Testing

Mutation testing could be fundamentally categorized into 3 types– statement mutation, decision mutation, and value mutation

1. Statement Mutation - developer cut and pastes a part of code of which the outcome may be removal of some lines
2. Value Mutation- values of primary parameters are modified
3. Decision Mutation- control statements are to be changed

Advantages and Disadvantages of Mutation Testing:

Following are the advantages of Mutation Testing:

• It is a powerful approach to attain high coverage of the source program.
• This testing is capable comprehensively testing the mutant program.
• Mutation testing brings a good level of error detection to the software developer.
• This method uncovers ambiguities in the source code, and has the capacity to detect all the faults in the program.
• Customers are benefited from this testing by getting most reliable and stable system.

On the other side, following are the disadvantages of Mutant testing:

• Mutation testing is extremely costly and time consuming since there are many mutant programs that need to be generated.
• Since its time consuming, it's fair to say that this testing cannot be done without an automation tool.
• Each mutation will have the same number of test cases than that of the original program. So, a large number of mutant programs may need to be tested against the original test suite.
• As this method involves source code changes, it is not at all applicable for Black Box Testing.

Conclusion:

Do you want an exhaustive testing of your application? Answer is Mutation testing. It is the most comprehensive technique to test a program. This is the method which checks for the effectiveness and accuracy of a testing program to detect the faults or errors in the system