Logo

teaching

data structures and software dependability

computer science department

brandenburg university of technology cottbus - senftenberg

Testen von Software / Software Testing

Lecture, Prof. Dr.-Ing. M. Heiner, WS 2019/2020
This course will be in English.

Module
11885 Software Testing
12408 Testen von Software

latest update: April 25, 2021, at 06:23 PM

I N D E X

Target group
Summary
Objectives
Preconditions
Organisation
About the exercises
Assessment criteria
References
Table of Contents - progress line
Materials - progress line
Exercises

Target group

  • Students of Computer Science (Informatics) - Bachelor, Master
  • Students of Information and Media Technique (IMT) - Bachelor, Master
  • Students of Cyber Security - Compulsory Elective Module for Computer Science
  • Students of (Applied) Mathematics

Summary

  • Category: Practical Computer Science, 8 Credits
  • We start with getting to know the possibilities and limits of testing, specifically in comparison with (mathematically thorough) verification techniques.
  • Next we acquire an overview of state-of-the art methods for systematic testing of sequential software.
  • Finally, we pay attention to the special problems in systematic testing of object-oriented and concurrent software.

Objectives

  • overview of methods for systematic software testing;
  • assessment from a software engineer's point of view, supporting the selection of useful methods in a given situation;
  • some experience in dealing with (public-domain) test systems;

Preconditions

  • basic knowledge of programming (imperative, object-oriented, concurrent software)
  • basic knowledge in software engineering

Organisation

Remark

According to the General Examination and Study Regulations, students have to register for modules (without a limited number of participants) during the first 3 weeks of the lecture period of each semester. You can withdraw your registration until one week before beginning of the examination period in which the examination is offered for the first time. Both steps are handled via the Online Portal. Please note, if you registered for a module, you are automatically registered for the corresponding exam.

The course consists of two blocks of Lectures and one block Exercise per week.

Lectures

Fridays, 4. and 5. block (13.15 – 16.30), VG1c, R 2.01;
There are no lectures on Friday, November, 1st.

Exercises

Wednesdays, 4. block (13.45–15.15), VG 1c, R 2.01; begin: 2nd teaching week
see exercises’ webpage for details

responsible assistant: Jacek [period] Chodak [snail] b-tu [period] de

The exercises will be managed using Moodle; please register for the course: 11885 Software Testing | WiSe 19/20

Exam

Written, without any auxiliary means; you will need paper and pencil, nothing else.
when: the Monday in the second week of the exam period February 2020 (might be subject to change)

Assessment criteria

The assessment of this course (to obtain the credits) comprises two parts:
(1) Participation in the exercises accompanying the course AND
(2) Examination at the end of the course.
Both criteria have to be passed.

(1) Participation in the exercises

The exercises will be in seminar style, where students will take turns in presenting software test tools.

A successful participation in the exercises requires a critical assessment, presentation and short documentation (data sheet) of a software test tool, as well as active participation in all exercises. Allocation of the software test tool to be presented is done in the first exercise (2nd teaching week).

(2) Examination

A written exam is marked as "passed", if at least 50 % of the maximal amount of points have been obtained. The grading follows the usual grading scheme. No auxiliary means are allowed during a written exam; this also excludes the use of mobiles. Dictionaries are provided on request. Duration of the written exam: 2h.

An oral exam comprises two parts: 1h preparation time + 1h oral exam. No auxiliary means are allowed during the whole time; this also excludes the use of mobiles. Dictionaries are provided on request. An oral exam is evaluated as "passed", if at least 50 % of the expected solutions are found. The grading follows the usual grading scheme.

The decision among written or oral examination will be taken at the begin of the semester (depending on the amount of participants).

Note

A credit corresponds to 30 working hours (see, e.g., Bachelor study regulation, §7), yielding the following calculation:

A course with 8 credits = 240 working hours; assuming 16 weeks (15 weeks lecturing + 1 week exam period) makes 15 working hours per week! Of those 15 hours, we have 4.5 of so-called contact hours, which leaves 10.5 hours per week to deepen the understanding of the lectures and preparing the solutions for the homework.

This calculation of expected work load has been taken into consideration when compiling the amount of material covered in the course and the design of the homework assignments.

References

General interest

  • Balzert, Helmut:
    Lehrbuch der Software-Technik: Band 1, Software-Entwicklung;
    Spektrum Akademischer Verlag, 2001.
  • Balzert, Helmut:
    Lehrbuch der Software-Technik: Band 2, Software-Management,
    Software-Qualitätssicherung, Unternehmensmodellierung;
    Spektrum Akademischer Verlag, 1998.
  • Beizer, Boris:
    Software Testing Techniques;
    International Thomson Computer Press, 1990.
  • Beizer, Boris:
    Black-Box Testing: Techniques for Functional Testing of Software and Systems;
    Wiley & Sons, 1995.
    (and many more of the same autor)
  • Beydeda, Sami:
    The Self-Testing COTS Components Method;
    Diss. Uni Leipzig, 12/2003.
  • Binder, Robert V.:
    Testing Object-oriented Systems; Models, Patterns, and Tools;
    Addison-Wesley, 4th edition, 2003.
  • Dijkstra, E. W.:
    Notes on Structured Programming;
    Academic Press, London, 1972, pp. 1-82.
  • Ehrenberger, W.:
    Softwareverifikation, Verfahren für den Zuverlässigkeitsnachweis von Software;
    Hanser 2002.
  • Grams, T.:
    Denkfallen und Programmierfehler;
    Springer 1990.
  • Grams, T.:
    Klüger irren – Denkfallen vermeiden mit System;
    Springer 2016.
  • Liggesmeyer, Peter:
    Software-Qualität; Testen, Analysieren und Verifizieren von Software;
    Spektrum 2002.
  • Myers, Glenford J.:
    The Art of Software Testing;
    Wiley & Sons, 1979.

    → deutsche Fassung:
    Methodisches Testen von Programmen;
    R. Oldenbourg Verlag, 1991.
  • Riedemann,E. H.; Schippers, H.:
    Testmethoden für sequentielle und nebenläufige Software-Systeme;
    Teubner 1998.
  • Sneed, Harry M.; Winter, Mario:
    Testen objektorientierter Software;
    Das Praxishandbuch für den Test objektorientierter Client/Server-Systeme;
    Hanser 2002.
  • Sommerville, Ian:
    Software Engineering;
    Addison-Wesley, fifth Edition, 1995.
    (Hint: The latest (10th) edition was published in April 2015.)
  • Spillner, Andreas; Linz, Tilo:
    Basiswissen Softwaretest
    dpunkt.verlag GmbH, 2007.
    (Hinweis: korrespondiert zum Lehrplan "Certified Tester Foundation Level")
  • Vigenschow, Uwe:
    Objektorientiertes Testen und Testautomatisierung in der Praxis;
    dpunkt.verlag 2005

Testing of concurrent systems

  • Baumgarten, Bernd; Giessler, Alfred:
    OSI-Testmethodik und TTCN;
    GMD-Bericht Nr. 202, R. Oldenbourg Verlag, 1992.
  • Baur, Marcel Christian:
    Instrumenting Java Bytecode to replay Execution Traces of Multithreaded Programs;
    Diplomarbeit ETH Zürich, April 2003.
  • Eugster, Pascal:
    Java Virtual Machine with Rollback Procedure allowing Systematic and Exhaustive Testing og Multi-threaded Java Programs;
    Diplomarbeit ETH Zürich, April 2003.
  • Henniger, Olaf:
    Testgenerierung aus Spezifikationen in Estelle und SDL;
    Diss. BTU Cottbus 06/2002, Shaker-Verlag Aachen 2003.
  • Ulrich, Andreas W.:
    Testfallableitung und Testrealisierung in verteilten Systemen;
    Diss. BTU Cottbus, 09/1998, Shaker-Verlag, Aachen 1998.

Links of interest

Table of Contents

Remark: all keywords given below represent exam-related aspects.

1. Introduction

sw quality management, basic terminology; validation vs testing - an overview;

2. Program verification

basic ideas, pros/cons:
program proofs: logical reasoning over assertions (pre/post conditions), Hoare formulas;
model checking, temporale logics;

3. Static Testing (Human Testing)

basic ideas, pros/cons;
variants: Fagan inspection, walkthrough, desktop testing, peer-rating, reviews;
typical examples of test criteria (checklists);
demarcation from static analysis;

4. White Box Testing (implementation-oriented testing, developer testing)

control structure based testing: control structure graphs for sequential programs, Petri nets for concurrent programs;
coverage measures: node/branch/path coverage;
several condition coverages: simple/multiple/minimal multiple condition coverage, MC/DC;
code instrumentation for coverage measures;
data flow based testing: defs/uses techniques;

5. Black Box Testing (specification-oriented testing, user testing)

basic techniques:
equivalence classes, boundary values, error guessing;
model-based techniques:
cause/effect graphs and their evaluation by help of Petri net T-invariants;
automata, syntax diagrams and their coverage with Euler paths;

6. Testing of object-oriented software

the new challenges: (data encapsulation), inheritance, polymorphy and dynamic binding;
class testing approaches: class testing driver, build-in testing, assertion testing, state testing; class modality and its influence on the testing approach;
testing of inheritance hierarchies, design/testing by contract, testing of polymorph operations;

7. Testing of concurrent software

three problems: complexity of the state space, probe effect, non-determinism;
dedicated approaches/tools: (1) Instant Replay, (2) Verifsoft, (3) Java Replay, (4) Java Milestone/Rollback;
comparison with static analysis techniques;

--------------------- progress line --------------------- progress line ---------------------

8. Summary

four simple criteria when to stop testing: mutation testing, fault injection method, testing by two independent teams, Jelinsky-Moranda method;

9. Testen im Großen

Integrationsteststrategien: Bing-Bang, Backbone, Top-down, Bottom-up, Jo-Jo, Sandwich;
Systemtest, Ziele:: Funktion, Leistung, Benutzbarkeit, Sicherheit, Interoperabilität, Installation, Wiederinbetriebnahme; Abnahmetest;

10. Management des Testens

DIN EN ISO 9000, Normen zum Qualitätsmanagement;
DIN EN ISO 9001, Qualitätsmanagementsysteme, Anforderungen

Materials used in the lectures

The green line indicates the current progress of the lectures. Materials above the green line are released, materials below the green line are preliminary information possibly subject to changes.

1. Introduction

2. Program verification

Hoare logic [wikipedia.org]

Model checking

self-assessment
supplementary material

for more Petri net examples see

related tools

  • Snoopy – Petri net editor and animator
  • Charlie – includes (traditional) explicit CTL model checker
  • Marcie – includes symbolic CTL model checker

3. Static Testing (Human Testing)

checklists, some examples

supplementary material
  • T Grams (2014) Das System der Denkfallen (in German)
  • Keynote Fagan (2001) slides,
    in M Broy, E Denert (Eds): Pioneers and Their Contributions to Software Engineering, sd&m Conference on Software Pioneers, Bonn, June 28/29, 2001
  • Riedemann (2002) Kap. 12, speziell 12.1 Informelle Analyse (in German)

4. White Box Testing

running example

supplementary material

5. Black Box Testing

cause effect graphing using Petri nets techniques slides

Petri nets

FSM-based testing

supplementary material

6. Testing of object-oriented software

demo oo programm (slides, Rauh) (in German)
challenges of oo testing (slides) class contracts (slides, Santen) (in German)
(non/uni/quasi) modal classes (slides, Santen) (in German)
testing of polymorph operations (slides, Santen) (in German)

supplementary material

7. Testing of concurrent software

countCharSystem

Java examples

on the influence of time on system behaviour

VeriSoft - the approach, download
lazy state space exploration - stubborn sets

Java Pathfinder - example for race condition

supplementary material

8. Exam prep's

self-control questions.pdf

--------------------- progress line --------------------- progress line ---------------------

… the end …

Any comments or questions are welcome. Please direct them to monika [period] heiner [snail] b-tu [period] de Privacy Policy