Transformers in Action

Understand the architecture that underpins today’s most powerful AI models.

Transformers are the superpower behind large language models (LLMs) like ChatGPT, Gemini, and Claude. Transformers in Action gives you the insights, practical techniques, and extensive code samples you need to adapt pretrained transformer models to new and exciting tasks.

Inside Transformers in Action you’ll learn:

• How transformers and LLMs work
• Modeling families and architecture variants
• Efficient and specialized large language models
• Adapt HuggingFace models to new tasks
• Automate hyperparameter search with Ray Tune and Optuna
• Optimize LLM model performance
• Advanced prompting and zero/few-shot learning
• Text generation with reinforcement learning
• Responsible LLMs

Transformers in Action takes you from the origins of transformers all the way to fine-tuning an LLM for your own projects. Author Nicole Koenigstein demonstrates the vital mathematical and theoretical background of the transformer architecture practically through executable Jupyter notebooks. You’ll discover advice on prompt engineering, as well as proven-and-tested methods for optimizing and tuning large language models. Plus, you’ll find unique coverage of AI ethics, specialized smaller models, and the decoder encoder architecture.

About the technology

Transformers are the beating heart of large language models (LLMs) and other generative AI tools. These powerful neural networks use a mechanism called self-attention, which enables them to dynamically evaluate the relevance of each input element in context. Transformer-based models can understand and generate natural language, translate between languages, summarize text, and even write code—all with impressive fluency and coherence.

About the book

Transformers in Action introduces you to transformers and large language models with careful attention to their design and mathematical underpinnings. You’ll learn why architecture matters for speed, scale, and retrieval as you explore applications including RAG and multi-modal models. Along the way, you’ll discover how to optimize training and performance using advanced sampling and decoding techniques, use reinforcement learning to align models with human preferences, and more. The hands-on Jupyter notebooks and real-world examples ensure you’ll see transformers in action as you go.

what's inside

• Optimizing LLM model performance
• Adapting HuggingFace models to new tasks
• How transformers and LLMs work under the hood
• Mitigating bias and responsible ethics in LLMs

Ebook License

End-User Warranty And License Agreement

1. Grant Of License

Manning Has Authorized The Download By You Of An Unrestricted Number Of Copies Of The Electronic Book (Ebook) In Any Of The Available Formats. Manning Grants You A Nonexclusive, Nontransferable License To Use The Ebook According To The Terms And Conditions Herein. This License Agreement Permits You To Install The Ebook On Any And All Your Devices For Your Personal Use Only.

2. Restrictions

You Shall Not: (1) Share, Resell, Rent, Assign, Timeshare, Distribute, Or Transfer All Or Part Of The Ebook Or Any Rights Granted Hereunder To Any Other Person; (2) Duplicate The Ebook, Except For A Single Backup Or Archival Copy; (3) Remove Any Proprietary Notices, Labels, Or Marks From The Ebook; (4) Transfer Or Sublicense Title To The Ebook To Any Other Party.

3. Intellectual Property Protection

The Ebook Is Owned By Manning And Is Protected By United States And International Copyright And Other Intellectual Property Laws. Manning Reserves All Rights In The Ebook Not Expressly Granted Herein. This License And Your Right To Use The Ebook Terminate Automatically If You Violate Any Part Of This Agreement. In The Event Of Termination, You Must Remove The Original And Any Copies Of The Ebook From All Your Devices.

4. Source Code Supplementary Material

Any Source Code Files Provided As A Supplement To The Book Are Freely Available To The Public For Download. Reuse Of The Code Is Permitted, In Whole Or In Part, Including The Creation Of Derivative Works, Provided That You Acknowledge That You Are Using It And Identify The Source: Title, Publisher And Year.

5. Limited Warranty

Manning Warrants That The Ebook Files, A Copy Of Which You Are Authorized To Download, Are Free From Defects In The Operational Sense That They Can Be Read By A Pdf Reader Or Epub Reader, Or Other. Except For This Express Limited Warranty, Manning Makes And You Receive No Warranties, Express, Implied, Statutory Or In Any Communication With You, And Manning Specifically Disclaims Any Other Warranty Including The Implied Warranty Of Merchantability Or Fitness Or A Particular Purpose. Manning Does Not Warrant That The Operation Of The Ebook Will Be Uninterrupted Or Error Free. If The Ebook Was Purchased In The United States, The Above Exclusions May Not Apply To You As Some States Do Not Allow The Exclusion Of Implied Warranties. In Addition To The Above Warranty Rights, You May Also Have Other Rights That Vary From State To State.

6. Limitation Of Liability

In No Event Will Manning Be Liable For Any Damages, Whether Arising For Tort Or Contract, Including Loss Of Data, Lost Profits, Or Other Special, Incidental, Consequential, Or Indirect Damages Arising Out Of The Use Or Inability To Use The Ebook.

7. General

This Agreement Constitutes The Entire Agreement Between You And Manning And Supersedes Any Prior Agreement Concerning The Ebook. This Agreement Is Governed By The Laws Of The State Of New York.

About the Author

Nicole Koenigstein is a distinguished Data Scientist and Quantitative Researcher. She is presently the Chief Data Scientist and Head of AI & Quantitative Research at Wyden Capital.