hminhit/awesome-software-quality

List of free software testing and verification resources

List of Free Software Testing Resources

Telegram: t.me/sqaunderhood

Introduction

This page collects resources for anyone considering the use of software testing and formal methods.

There are many axes along which one can organize such a list, such as the level of expertise of the intended audience (from experts to the public at large) or disciplinary orientation (computer science, mathematics, mathematical logic, etc.). Here I have chosen to classify the material by type of subject matter.

If you would like add something, just send the text as you would like it to appear to [email protected], preferrably already formatted in markdown.

Subscribe to receive notificatons.

Index

• Books
• Courses
• Podcasts
• Conferences
• Workshops and interactive learning tools
• Learned lessons
• Awesome lists

Books

In English

Software Testing Theory

• Perspectives on Agile Software Testing
• A Practical Guide to Testing in DevOps
• Programs and Proofs. Mechanizing Mathematics with Dependent Types. - Ilya Sergey @ilyasergey
• Measure Software Quality using Application Security (WIP) - Dinis Cruz
• The "A" Word - Alan Page
• Professional Software Development (Chapter: "5. Software Quality") - Mike G. Miller
• Acceptance Test Engineering Guide, Vol. I - RC1
• A Software Testing Primer (PDF)
• Classic Testing Mistakes (PDF) - Brian Marick
• Code Coverage Analysis - Steve Cornett
• Practical Software Testing - STH in association with Chindam Damodar
• A Tutorial in Exploratory Testing (PDF)
• Essential Acceptance Testing
• Embedded Software Testing Methods Juho Lepistö
• Foundations of Software Testing: Fundamental Algorithms and Techniques Aditya P. Mathur
• Introduction to software testing (PDF)
• Manual Testing Help
• Mobile Testing: Ready Reckoner (PDF) - Ajay Balamurugadas, Sundaresan Krishnaswami
• Performance Testing Guidance
• Random testing book DRAFT
• Software Testing: A Comprehensive Approach - Bill Laboon
• Software Testing Tutorial
• Software Testing Dictionary
• The Way of Testivus
• The Essential Guide to Mobile App Testing (PDF)
• UI and UX Testing: Ready Reckoner (PDF) - Ajay Balamurugadas
• What If: A collection of tips from a software tester (PDF) - Ajay Balamurugadas
• What If: 50+ tips to win testing contests (PDF) - Ajay Balamurugadas
• What If: 50+ tips to boost your productivity (PDF) - Ajay Balamurugadas
• What If: 50+ tips to improve tester-programmer relationship (PDF) - Ajay Balamurugadas
• Handbook of Software Reliability Engineering
• Mutation Testing: Better Code by Making Bugs - Filip van Laenen
• The Tao of Testing. A Field Manual for Software Engineers. - Jason Polites
• Observing and Reasoning About Errors
• The Little Black Book On Test Design -- Rikard Endgren
• Generating Software Tests. Breaking Software for Fun and Profit. -- by Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler
• Software Testing and Analysis: Process, Principles, and Techniques -- Mauro Pezze, Michal Young (slides)
• Software Testing Glossary

Standards In software Testing

• Testing Standards Working Party
• The International Software Testing Standard
• Standard for Software Component Testing (PDF)
• Everything You Need to Know About Software Test Formats (in Russian)
• Software Testing Standards in NASA
• ECSS-Q-ST-80C Rev.1 – Software product assurance
• СИСТЕМНАЯ И ПРОГРАММНАЯ ИНЖЕНЕРИЯ: ГОСТ Р 56922—2016/ISO/IEC/IEEE 29119- 3:2013
• DO-278
• DO-178B

Testing of Free and OpenSource Software

• Quality Improvement in Volunteer Free and Open Source Software Project (PDF)

Testing by different programming languages

• PHPUnit PHP Test-Driven Development - Automated Tools to Improve Your PHP Code Quality
• Practical PHP testing
• Testing Erlang
• Testing in Scala
• Test-Driven Development - Extensive Tutorial - Grzegorz Gałęzowski
• Testing and Debugging JavaScript
• Web Application Testing in Ruby Željko Filipin

Testing Tools

• The Evolving Art of Fuzzing (PDF) - Jared DeMott
• AccelTest (PDF)
• A Step-by-Step Guide to Functional Testing with TestComplete
• HP Quality Center Tutorial
• HP QuickTest Professional Tutorial
• .NET Performance Testing and Optimization - The Complete Guide (zip) - RedGate, By Paul Glavich and Chris Farrell
• Record-Playback Test Automation: Sahi & Selenium IDE
• Selenium 2.0 Using the WebDriver API to Create Robust User Acceptance Tests
• SoapUI 101: Beginner's Guide to Functional Testing
• TestNG Tutorial
• JUnit Tutorial
• PropEr Testing - Fred Hebert
• Better Unit Testing with Microsoft Fakes
• Testing for Continuous Delivery with Visual Studio 2012

Formal methods

• Learn TLA+ - Hillel Wayne @hwayne
• Specifying Systems - Leslie Lamport
• The TLA+ Hyperbook - Leslie Lamport
• Books and papers about TLA+ and formal verification by Leslie Lamport
• Alloy Documentation - Daniel Jackson
• Introduction to Spin - Multiple Authors
• Software Foundations: 4.0, current, draft , epub - Benjamin C. Pierce & Co
• Verified Functional Algorithms - Andrew W. Appel
• Foundations of Computer Science - Dr Larry Paulson
• Principles of Model Checking - Christel Baier, Joost-Pieter Katoen
• Certified Programming with Dependent Types - Adam Chlipala @achlipala
• Formal Reasoning About Programs - Adam Chlipala @achlipala
• Concrete Semantics - Tobias Nipkow and Gerwin Klein @lsf37
• ML for the Working Programmer, 2nd Edition - Lawrence C. Paulson
• Proofs and Types - Jean-Yves Girard, Yves Lafont and Paul Taylor
• Proof assistants: History, ideas and future - H. Geuvers
• Introduction to Logic - Michael Genesereth, Eric Kao (Stanford University)
• An Introduction to Formal Logic - P.D. Magnus (University at Albany)
• A Problem Course in Mathematical Logic - Stefan Bilaniuk (Trent University)
• Language, Proof, and Logic - Jon Barwise, John Etchemendy
• Mathematical Logic - Helmut Schwichtenberg
• Mathematical Logic - Stephen G. Simpson (Pennsylvania State University)
• Formal Logic - Miguel Palomino
• Is Parallel Programming Hard, And, If So, What Can You Do About It? (chapters "Validation" and "Formal verification") - Paul E. McKenney @paulmckrcu
• More books...
• Formal Methods of Software Design

In Russian

Управление проектами

• Черная книга менеджера (pdf, epub, mobi) Слава Панкратов
• Белая книжная полка менеджера (epub, mobi, zip)
• Как стать менеджером (pdf, epub, mobi) - Слава Панкратов, Александр Орлов
• It starts with a great product

Тестирование ПО

• Test-Driven Development in Practice - Сергей Борисов
• Тестирование программного обеспечения. Базовый курс. - Святослав Куликов
• Всё, что вам нужно знать о форматах отчётов в тестировании ПО - Сергей Бронников
• Руководство по тестированию пользовательского опыта для сайтов, мобильных приложений и прототипов
• Автоматизация тестирования от «А» до «Ы» - Gennadiy Alpaev
• Учебник по SilkTest - Gennadiy Alpaev
• Учебник по TestComplete - Gennadiy Alpaev

Формальные методы

• Верификация программ методом Model Checking - А.М.Миронов
• http://is.ifmo.ru/verification/
• Практикум по математической логике. Coq.

Courses

In English

• Software Engineering: Introduction
• Software Debugging
• Software Analysis & Testing
• Fundamentals of TDD
• Critical Reasoning for Beginners
• Reasoning Across the Disciplines
• Discrete Mathematics and Probability Theory
• Mathematical Logic
• Engineering Ethics
• Engineering Ethics
• Software Development Process: Part 1, Part 2, Part 3

Software Testing

• Black Box Software Testing course - Cem Kaner, J.D. Ph.D.
• Software Testing
• Embedded Software Testing
• Software Testing Fundamentals
• Software Testing Methods
• QA Academy
• QA and Automation course
• Software Testing and Verification
• Introduction to Software Testing
• Software Testing - Black-box Strategies and White-box Testing
• Diploma in Software Testing
• Software Testing - Testing Levels and Object-Oriented Program Testing
• Software Testing - Condition Coverage and Mutation Testing Strategies
• Software Testing Training
• Quality Center Training
• QTP Training
• Selenium Tutorials
• Selenium WebDriver Course
• Software Testing - Mauro Pezz

Fuzz testing

• Intro: Unknown vulnerability management and discovery using fuzzing
• Part 1: What are unknown vulnerabilities and why should I care
• Part 2: What is fuzz testing, and where does it fit in the world of software?
• Part 3: How and why fuzz testing, and managing your unknown vulnerabilities saves money
• Part 4: Fuzz testing techniques: unfuzzing the fuzzing

Software Testing Management

• Software Testing Management

User Testing

• Introduction to User Testing

Security Testing

• Software security

Performance Testing

• Performance Engineering of Software Systems
• LoadRunner Tutorials
• Computer Systems Analysis

Formal Software Verification

• Formal Specification
• Formal Software Verification and Formal Software Verification (previous page of the course)
• Logic, Languages, Compilation, and Verification (Youtube)
• A survey of automated theorem proving
• Tutorials and advanced lectures
• Types Project
• Introduction to Formal Systems and Computation
• CSE 814 Formal Methods in Software Engineering Slides
• A survey of automated theorem proving - John Harrison

Coq

• Coq in a Hurry
• Programs and Proofs in the Coq Proof Assistant
• An Introduction to the Coq Proof Assistant
• Introduction to the Coq Proof Assistant
• Coq Intensive - DeepSpec Summer School
• Interactive Computer Theorem Proving
• Coq tutorials
• Type Theory and Coq
• Logical Verification
• Modelling and verifying algorithms in Coq: an introduction
• Formal Reasoning About Programs
• Software foundations in Coq 0.1 - Benjamin Pierce
• Compiler Certification — Xavier Leroy

Isabelle/HOL

• Advanced Topics in Software Verification
• Theorem Proving - Principles, Techniques, Applications

Agda

• Programming Language Foundations in Agda Book

Model Checking

• SPIN/Promela Model Checking Concurrent Programs
• Software Verification - Natasha Sharygina
• Alloy Courses
• The TLA+ Video Course
• Dr. TLA+ Series
• Theorem Proving and Model Checking in PVS - Edmund M. Clarke
• mCRL2 System Validation: Automata and behavioural equivalences
• mCRL2 System Validation (2): Model process behaviour
• mCRL2 System Validation (3): Requirements by modal formulas
• mCRL2 System Validation (4): Modelling Software, Protocols, and other behaviour

In Russian

• Основы программной инженерии, Youtube - Владимир Ицыксон

• Основы программной инженерии (SWEBOK 2004 на русском) в EPUB и HTML

• Введение в программную инженерию
• Введение в технологию программирования
• Юнит-тестирование и метрики покрытия кода тестами
• Методы анализа и обеспечения качества ПО

Тестирование ПО

• Основы тестирования программного обеспечения
• Основы тестирования ПО
• Основы тестирования программного обеспечения
• Школа автоматизации процессов разработки
• Школа тестирования баннерной системы
• Software Testing
• Software Testing 101 (на английском, на русском)
• Software Testing 102
• Software Testing 103
• Тестирование ПО
• Школа тестирования: Ручное тестирование и Автоматизация тестирования (Youtube)
• Тестирование на основе моделей (слайды), Youtube - Кулямин Виктор
• Серия семинаров по тестированию (автоматизация) - Сергей Андреев
• Тестирование ПО: Базовый уровень - курс основан на официальной программе обучения "Сертифицированный тестировщик ПО Базового уровня" ISTQB и предназначен для подготовки к сдаче сертификационных экзаменов

Формальная верификация ПО

• Верификация программ на моделях
• Верификация параллельных и распределенных программных систем (PDF), Youtube - Юрий Карпов
• Формальные методы обеспечения качества ПО
• Верификация программного обеспечения
• Software Verification
• Введение в формальные методы верификации программ -- А.С. Камкин
• Методы верификации программного обеспечения -- В.В. Кулямин
• Формальная спецификация и верификация программ (практикум)

Workshops and interactive learning tools

• Anki:
  • Software Testing
  • Foundations of Software Testing - Glossary
  • ISTQB Glossary of Terms in Software testing
  • Coding and Design
  • Design Patterns
  • Object Oriented Design Patterns
  • Regular Expressions
  • 106 Linux Commands
  • Bash Command Line For Linux
  • LPIC 101 and 102
  • SQL basics
  • Distributed Computing
  • Learn how to design large-scale systems
• Testing Mnemonics Game
• The Deadlock Empire: Slay dragons, learn concurrency!
• Learn TLA
• tla_workshop
• test-anything
• https://math-comp.github.io/mcb/
• LibFuzzer Wokshop
• LibFuzzer Tutorial
• Interactive Z3 Guide
• Code-defenders game (mutation testing)
• RoboBUG: A Debugging Game
• Robot ON! A Program Understanding Game
• FireFinder
• Threaded Paws: A Concurrency Game
• Verigames
• Edukera

Conferences

• Software Testing Conferences

Podcasts

see podcasts

Bugs and Learned Lessons

• NASA Public Lessons Learned System
• Flight Software Engineering Lessons
• Forum on Risks to the Public in Computers and Related Systems
• CPU Bugs
• Illustrative Risks to the Public in the Use of Computer Systems and Related Technology
• Wikipedia: List of software bugs
• Postmortems

Awesome Lists

• Resources for Teaching with Formal Methods
• Fuzzing testing
• Mutation testing
• Chaos Engineering
• Static analysis
• TAP (Test Anything Protocol)
• Awesome Testing
• Awesome Selenium
• Symbolic execution
• History of symbolic execution
• Coq Proof Assistant
• awesome-provable
• Visual regression testing
• List of verification and synthesis tools
• Testing of distributed systems
• A curated list of useful resources for AutoIt
• SecLists - collection of multiple types of lists used during security assessments, collected in one place.
• Awesome Falsehood

License

To the extent possible under law, Sergey Bronnikov has waived all copyright and related or neighboring rights to this work.