Logo

teaching

data structures and software dependability

computer science department

brandenburg university of technology cottbus - senftenberg

Software Testing

Exercise, Prof. Dr.-Ing. M. Heiner, WS 2018/2019
This course will be in English.

latest updated: February 05, 2019, at 03:56 PM


when and where:

mondays, VG 1c, R 2.01;

  • 5. block (15.30-17.00), according to table below

back2lecture


Preliminaries

There are three criteria to fulfil for successfully passing the exercises for the course "Software Testing":

  1. continuous and active participation in all exercises;
  2. presentation of one (or two) tools for software testing (depending on the number of participants);
  3. compiling a brief summary (data sheet) for the test tool presented;

Please follow this template for the summary in the style of a data sheet:

Date sheet test tool (MS Word, German)
Date sheet test tool (MS Word, english)
. . . you may also create your latex form.

Schedule

DateToolStudentpresentation/slides, Data sheet
29/10/2018------
05/11/2018------
12/11/2018------
19/11/2018------
26/11/2018FindBugsBahij Sayeghrepetition 28/01/2019
03/12/2018Burp Suite, DVWA - test targetVoronovslides, data sheet
17/12/2018lint/splintJohannes Kuhnslides - data sheet
07/01/2019american fuzzy lop (afl)Sebastian Eydamslides - data sheet
14/01/2019CPP unitStefan Kraatzslides - data sheet
21/01/2019Flaw FinderSiddique Reza Khanslides - data sheet
28/01/2019Selenium WebDriver - TestNG - testing of JenkinsHarpreet Kaur Oberoislides - data sheet
28/01/2019FindBugs (second try)Bahij Sayeghslides - data sheet

Tools

Category Index


Tool Index

A - abbot - afl - autohotkey - AutoIt - B - Badboy - bugzilla - C - checkstyle - cobertura - CppUnit - csunit - D - Dogtail - E - emma - expect - F - findbugs - Fitnesse - flyspray - fxcop - G - gcov - gprof - GraphWalker - J - Jacareto - Jameleon - javapathfinder - javaprof - jDiffChaser - jester - jfcunit - jlint - junit - L - lint/splint - M - marathon - mbunit - memcheck - O - OProfile - P - pmd - pounder - pywinauto - Q - qat - S - Sahi - specexplorer - Sourcery Analyzer - staf - T - TestMaster - V - valgrind - Z - zanebug


STATIC PROGRAM CHECKERS


lint / splint
http://www.splint.org

Splint is a tool for statically checking C programs for security vulnerabilities and coding mistakes. With minimal effort, Splint can be used as a better lint. If additional effort is invested adding annotations to programs, Splint can perform stronger checking than can be done by any standard lint.


checkstyle
http://checkstyle.sourceforge.net

Checkstyle can check many aspects of your java source code. Historically it's main functionality has been to check code layout issues, but since the internal architecture was changed in version 3, more and more checks for other purposes have been added. Now Checkstyle provides checks that find class design problems, duplicate code, or bug patterns like double checked locking.


jlint
http://artho.com/jlint/

A Java Programm checker. Jlint will check your Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.


findbugs
http://findbugs.sourceforge.net/

FindBugs is a program to find bugs in Java programs. It looks for instances of "bug patterns" --- code instances that are likely to be errors.


PMD
https://pmd.github.io

is a Java source code analyzer. It finds unused variables, empty catch blocks, unnecessary object creation, and so forth. PMD scans Java source code and looks for potential problems. It has plugins for JEdit, JBuilder, Netbeans/Forte/Sun ONE, IntelliJ IDEA, Maven, Ant, Eclipse, Gel, and Emacs.


FxCop
https://msdn.microsoft.com/en-us/library/bb429476%28v=vs.80%29.aspx

FxCop is a code analysis tool that checks .NET managed code assemblies for conformance to the Microsoft .NET Framework Design Guidelines. It uses reflection, MSIL parsing, and callgraph analysis to inspect assemblies for more than 200 defects in the following areas:: Library design, Localization, Naming conventions, Performance, Security. FxCop includes both GUI and command line versions of the tool.


CODE COVERAGE


gcov coverage testing tool
http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html

gcov is a test coverage program. Use it in concert with GNU CC to analyze your programs to help create more efficient, faster running code. You can use gcov as a profiling tool to help discover where your optimization efforts will best affect your code. Material: Buch: Zeller/Krinke, Programmierwekzeuge S.229 ? 235


cobertura
http://cobertura.sourceforge.net/

Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.


EMMA: a free Java code coverage tool
http://emma.sourceforge.net

EMMA is an open-source toolkit for measuring and reporting Java code coverage.


SYSTEMATIC SOFTWARE TESTING


GraphWalker
http://graphwalker.org

GraphWalker is an implementation of Model-based testing built in Java. It allows you to generate test sequences from a finite-state machine (graph). The test sequences can be created statically, or run dynamically.


Java PathFinder
http://javapathfinder.sourceforge.net/

Java PathFinder is a system to verify executable Java bytecode programs. In its basic form, it is a Java Virtual Machine (JVM) that is used as an explicit state software model checker.


Spec Explorer
https://www.microsoft.com/en-us/research/project/model-based-testing-with-specexplorer/

Spec Explorer is a software-development tool for advanced model-based specification and conformance testing. Spec Explorer can help software-development teams detect errors in the design, the specification, and the implementation of their systems. The tool is intended to be used by software testers, designers, and implementers.


REGRESSION TEST


Expect / Dejagnu
http://expect.sourceforge.net/

programmed dialogue with interactive programs


http://www.gnu.org/software/dejagnu/

DejaGnu is a framework for testing other programs. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. A Test Harness is the testing infrastructure that is created to support a specific program or tool.


Junit
http://www.junit.org

JUnit is a regression testing framework written by Erich Gamma and Kent Beck. It is used by the developer who implements unit tests in Java.


CppUnit
http://sourceforge.net/projects/cppunit

CppUnit is the C++ port of the famous JUnit framework for unit testing. Test output is in XML or text format for automatic testing and GUI based for supervised tests.


Jameleon
http://jameleon.sourceforge.net/

Jameleon is a data-driven automated testing tool that is easily extensible via plug-ins. Features of applications are automated in Java and tied together independently in XML, creating self-documenting automated test cases.


csUnit
http://www.csunit.org/

csUnit is a unit testing tool for the Microsoft .NET Framework. You can use it with all .NET languages including C#, Visual Basic .NET, J#, and managed C++.


MbUnit
https://code.google.com/p/mb-unit/ (outdated) :

MbUnit is an extensible .Net test framework. As in NUnit, tests are created at runtime using Reflection and custom attributes. MbUnit differentiates itself from NUnit in it's extensibility model. It contains a number of tests that go beyond the simple unit testing, such as combinatorial testing, data oriented testing, etc...


Zanebug
http://zanebug.soft112.com

Zanebug was developed to provide a unified platform for unit and integration testing. The current release addresses component-level unit testing, with a fair amount of development time focused on performance statistics and a flexible architecture. The long-term view for Zanebug is a tool that provides integrated component, form and web-based testing both locally and remotely with all the necessary performance information necessary for enterprise development.


MUTATION TEST


Jester
http://jester.sourceforge.net/

Jester - the JUnit test tester. Jester finds code that is not covered by tests. Jester makes some change to your code, runs your tests, and if the tests pass Jester displays a message saying what it changed. Jester includes a script for generating web pages that show the changes made that did not cause the tests to fail.


DETERMINING SECURITY DEFECTS


Material

Paper Linux journal http://www.linuxjournal.com//article.php?sid=5673

Tools
FlawFinder
http://www.dwheeler.com/flawfinder
Rats
http://www.securesw.com/download_rats.htm
PSCAN
http://deployingradius.com/pscan/ (outdated?)
bfbtester
http://bfbtester.sourceforge.net/

MEMORY CHECKS


Memory checker / leaktracer
http://freshmeat.net/projects/leaktracer/

LeakTracer is a small tool for checking a C++ program for memory leaks. To use LeakTracer, run your program using the provided LeakCheck script. It uses the LD_PRELOAD feature to "overlay" some functions on top of your functions (no recompile needed). LeakTracer uses gdb to print out the exact line where the memory was allocated and not freed - this of course means you have to free all dynamically allocated data. LeakTracer also overrides the global operator new and operator delete.


Valgrind
http://valgrind.org

Valgrind is an instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.


FUZZ TESTING


american fuzzy lop (afl)
Article
LWN 2015, Wikipedia, Wikipedia/afl
Tool
http://lcamtuf.coredump.cx/afl/

American fuzzy lop is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. This substantially improves the functional coverage for the fuzzed code. The compact synthesized corpora produced by the tool are also useful for seeding other, more labor- or resource-intensive testing regimes down the road.


GUI TEST


JFCUnit
http://sourceforge.net/projects/jfcunit/
http://jfcunit.sourceforge.net

[-jfcUnit enables developers to write test cases for Java Swing based applications. It provides support for: 1.Obtaining handles on Windows/Dialogs opened by the Java code. 2.Locating components within a component hierarchy that occur within the containers found above. 3.Ra\\

the end

Any comments or questions are welcome. Please direct them to monika [period] heiner [snail] b-tu [period] de Privacy Policy