Whole test suite generation

Whole Test Suite Generation
ABSTRACT:
Not all bugs lead to program crashes, and not always is there a formal specification
to check the correctness of a software test’s outcome. A common scenario in
software testing is therefore that test data are generated, and a tester manually adds
test oracles. As this is a difficult task, it is important to produce small yet
representative test sets, and this representativeness is typically measured using
code coverage. There is, however, a fundamental problem with the common
approach of targeting one coverage goal at a time: Coverage goals are not
independent, not equally difficult, and sometimes infeasible—the result of test
generation is therefore dependent on the order of coverage goals and how many of
them are feasible. To overcome this problem, we propose a novel paradigm in
which whole test suites are evolved with the aim of covering all coverage goals at
the same time while keeping the total size as small as possible. This approach has
several advantages, as for example, its effectiveness is not affected by the number
of infeasible targets in the code. We have implemented this novel approach in the
EVOSUITE tool, and compared it to the common approach of addressing one goal
at a time. Evaluated on open source libraries and an industrial case study for a total
of 1,741 classes, we show that EVOSUITE achieved up to 188 times the branch

coverage of a traditional approach targeting single branches, with up to62 percent
smaller test suites.
EXISTING SYSTEM:
Software testing is an essential component of any successful software development
process. A software test consists of an input that executes the program and a
definition of the expected outcome. Many techniques to automatically produce
inputs have been proposed over the years, and today are able to produce test suites
with high code coverage. We produce test inputs, and then a human tester needs to
specify the oracle in terms of the expected outcome. To make this feasible, test
generation needs to aim not only at high code coverage, but also at small test suites
that make oracle generation as easy as possible. A common approach in the
literature is to generate a test case for each coverage goal (e.g., branches in branch
coverage), and then to combine them in a single test suite. However, the size of a
resulting test suite is difficult to predict as a test case generated for one goal may
implicitly also cover any number of further coverage goals.
DISADVANTAGES OF EXISTING SYSTEM:
The problem of the expected outcome persists and has become known as the
oracle problem. Sometimes, essential properties of programs are formally

specified or have to hold universally such that no explicit oracles need to be
defined (e.g., programs should normally not crash).
Some targets are more difficult to cover than others.
Coverage goals can be infeasible such that there exists no input that would
cover them Even if this particular infeasible branch may be easy to detect,
this is not true in general and thus targeting infeasible goals will, per
definition, fail and the effort would be wasted.
PROPOSED SYSTEM:
In this paper, we evaluate a novel approach for test data generation, which we call
whole test suite generation, which improves upon the current approach of targeting
one goal at a time. We use an evolutionary technique in which, instead of evolving
each test case individually, we evolve all the test cases in a test suite at the same
time, and the fitness function considers all the testing goals simultaneously. The
technique starts with an initial population of randomly generated test suites, and

then uses a Genetic Algorithm to optimize toward satisfying a chosen coverage
criterion, while using the test suite size as a secondary objective. At the end, the
best resulting test suite is minimized, giving us a test suite as shown in fig 2. Stack
example from fig 1. With such an approach, most of the complications and
downsides of the one target at a time approach either disappear or become
significantly reduced. The technique is implemented as part of our testing tool
EVOSUITE which is freely available online.
ADVANTAGES OF PROPOSED SYSTEM:
 Because to effectively address the problem of test suite generation we had to
develop specialized search operators, there would be no guarantee on the
convergence property of the resulting search algorithm. To cope with this
problem, we formally prove the convergence of our proposed technique.
 The EVOSUITE approach yields significantly better results (i.e., either
higher coverage or, if the same coverage, then smaller test suites) compared
to the traditional approach of targeting each testing goal independently.

 In some cases, EVOSUITE achieved up to 188 times higher coverage on
average, and test suites that were 62 percent smaller while maintaining the
same structural coverage.
 Furthermore, running EVOSUITE with a constrained budget (1 million
statement executions during the search, up to a maximum 10 minutes
timeout) resulted in an impressive83 percent of coverage on average on our
case study.

SYSTEM CONFIGURATION:-
HARDWARE CONFIGURATION:-
 Processor - Pentium –IV
 Speed - 1.1 Ghz
 RAM - 256 MB(min)
 Hard Disk - 20 GB
 Key Board - Standard Windows Keyboard

 Mouse - Two or Three Button Mouse
 Monitor - SVGA
SOFTWARE CONFIGURATION:-
 Operating System : Windows XP
 Programming Language : JAVA
 Java Version : JDK 1.6 & above.
REFERENCE:
Gordon Fraser, Member, IEEE, and Andrea Arcuri ―Whole Test Suite Generation‖
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 39, NO. 2,
FEBRUARY 2013.

Whole test suite generation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to Whole test suite generation

Similar to Whole test suite generation (20)

Recently uploaded

Recently uploaded (20)

Whole test suite generation