Software Testing with Generative AI

Speed up your testing and deliver exceptional product quality with the power of AI tools.

The more you test, the more you learn about your software. Software Testing with Generative AI shows you how you can expand, automate, and enhance your testing with Large Language Model (LLM)-based AI. Your team will soon be delivering higher quality tests, all in less time.

In Software Testing with Generative AI you’ll learn how to:

• Spot opportunities to improve test quality with AI
• Construct test automation with the support of AI tools
• Formulate new ideas during exploratory testing using AI tools
• Use AI tools to aid the design process of new features
• Improve the testability of a context with the help of AI tools
• Maximize your output with prompt engineering
• Create custom LLMs for your business’s specific needs

Software Testing with Generative AI is full of hype-free advice for supporting your software testing with AI. In it, you’ll find strategies from bestselling author Mark Winteringham to generate synthetic testing data, implement automation, and even augment and improve your test design with AI.

Foreword by Nicola Martin.

Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications.

About the technology

There’s a simple rule in software testing: the more you test, the more you learn. And as any testing pro will tell you, good testing takes time. By integrating large language models (LLMs) and generative AI into your process, you can dramatically automate and enhance testing, improve quality and coverage, and deliver more meaningful results.

About the book

Software Testing with Generative AI shows you how AI can elevate every aspect of testing—automation, test data management, test scripting, exploratory testing, and more! Learn how to use AI coding tools like Copilot to guide test-driven development, get relevant feedback about your applications from ChatGPT, and use the OpenAI API to integrate AI into your data generation. You’ll soon have higher-quality testing that takes up less of your time.

What's inside

• Improve test quality and coverage
• AI-powered test automation
• Build agents that act as testing assistants

About the author

Mark Winteringham is an experienced software tester who teaches many aspects of software testing. He is the author of Testing Web APIs.

The technical editor on this book was Robert Walsh.

Table of Contents

Part 1
1 Enhancing testing with large language models
2 Large language models and prompt engineering
3 Artificial intelligence, automation, and testing
Part 2
4 AI-assisted testing for developers
5 Test planning with AI support
6 Rapid data creation using AI
7 Accelerating and improving UI automation using AI
8 Assisting exploratory testing with artificial intelligence
9 AI agents as testing assistants
Part 3
10 Introducing customized LLMs
11 Contextualizing prompts with retrieval-augmented generation
12 Fine-tuning LLMs with business domain knowledge
Appendix A Setting up and using ChatGPT
Appendix B Setting up and using GitHub Copilot
Appendix C Exploratory testing notes

About the Author

Mark Winteringham is the DojoBoss at Ministry of Testing. He teaches many aspects of software testing, and he supports and curates testing training from others in the testing community. He is the co-founder of Automation in Testing and Ministry of Testing Essentials training pathways.

Ebook License

End-User Warranty And License Agreement

1. Grant Of License

Manning Has Authorized The Download By You Of An Unrestricted Number Of Copies Of The Electronic Book (Ebook) In Any Of The Available Formats. Manning Grants You A Nonexclusive, Nontransferable License To Use The Ebook According To The Terms And Conditions Herein. This License Agreement Permits You To Install The Ebook On Any And All Your Devices For Your Personal Use Only.

2. Restrictions

You Shall Not: (1) Share, Resell, Rent, Assign, Timeshare, Distribute, Or Transfer All Or Part Of The Ebook Or Any Rights Granted Hereunder To Any Other Person; (2) Duplicate The Ebook, Except For A Single Backup Or Archival Copy; (3) Remove Any Proprietary Notices, Labels, Or Marks From The Ebook; (4) Transfer Or Sublicense Title To The Ebook To Any Other Party.

3. Intellectual Property Protection

The Ebook Is Owned By Manning And Is Protected By United States And International Copyright And Other Intellectual Property Laws. Manning Reserves All Rights In The Ebook Not Expressly Granted Herein. This License And Your Right To Use The Ebook Terminate Automatically If You Violate Any Part Of This Agreement. In The Event Of Termination, You Must Remove The Original And Any Copies Of The Ebook From All Your Devices.

4. Source Code Supplementary Material

Any Source Code Files Provided As A Supplement To The Book Are Freely Available To The Public For Download. Reuse Of The Code Is Permitted, In Whole Or In Part, Including The Creation Of Derivative Works, Provided That You Acknowledge That You Are Using It And Identify The Source: Title, Publisher And Year.

5. Limited Warranty

Manning Warrants That The Ebook Files, A Copy Of Which You Are Authorized To Download, Are Free From Defects In The Operational Sense That They Can Be Read By A Pdf Reader Or Epub Reader, Or Other. Except For This Express Limited Warranty, Manning Makes And You Receive No Warranties, Express, Implied, Statutory Or In Any Communication With You, And Manning Specifically Disclaims Any Other Warranty Including The Implied Warranty Of Merchantability Or Fitness Or A Particular Purpose. Manning Does Not Warrant That The Operation Of The Ebook Will Be Uninterrupted Or Error Free. If The Ebook Was Purchased In The United States, The Above Exclusions May Not Apply To You As Some States Do Not Allow The Exclusion Of Implied Warranties. In Addition To The Above Warranty Rights, You May Also Have Other Rights That Vary From State To State.

6. Limitation Of Liability

In No Event Will Manning Be Liable For Any Damages, Whether Arising For Tort Or Contract, Including Loss Of Data, Lost Profits, Or Other Special, Incidental, Consequential, Or Indirect Damages Arising Out Of The Use Or Inability To Use The Ebook.

7. General

This Agreement Constitutes The Entire Agreement Between You And Manning And Supersedes Any Prior Agreement Concerning The Ebook. This Agreement Is Governed By The Laws Of The State Of New York