Spark in Action, 2/e

Summary

The Spark distributed data processing platform provides an easy-to-implement tool for ingesting, streaming, and processing data from any source. In Spark in Action, Second Edition, you’ll learn to take advantage of Spark’s core features and incredible processing speed, with applications including real-time computation, delayed evaluation, and machine learning. Spark skills are a hot commodity in enterprises worldwide, and with Spark’s powerful and flexible Java APIs, you can reap all the benefits without first learning Scala or Hadoop.

Foreword by Rob Thomas.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology
Analyzing enterprise data starts by reading, filtering, and merging files and streams from many sources. The Spark data processing engine handles this varied volume like a champ, delivering speeds 100 times faster than Hadoop systems. Thanks to SQL support, an intuitive interface, and a straightforward multilanguage API, you can use Spark without learning a complex new ecosystem.

About the book
Spark in Action, Second Edition, teaches you to create end-to-end analytics applications. In this entirely new book, you’ll learn from interesting Java-based examples, including a complete data pipeline for processing NASA satellite data. And you’ll discover Java, Python, and Scala code samples hosted on GitHub that you can explore and adapt, plus appendixes that give you a cheat sheet for installing tools and understanding Spark-specific terms.

What's inside

Writing Spark applications in Java
Spark application architecture
Ingestion through files, databases, streaming, and Elasticsearch
Querying distributed datasets with Spark SQL

About the reader
This book does not assume previous experience with Spark, Scala, or Hadoop.

Table of Contents

PART 1 - THE THEORY CRIPPLED BY AWESOME EXAMPLES

1 So, what is Spark, anyway?

2 Architecture and flow

3 The majestic role of the dataframe

4 Fundamentally lazy

5 Building a simple app for deployment

6 Deploying your simple app

PART 2 - INGESTION

7 Ingestion from files

8 Ingestion from databases

9 Advanced ingestion: finding data sources and building

your own

10 Ingestion through structured streaming

PART 3 - TRANSFORMING YOUR DATA

11 Working with SQL

12 Transforming your data

13 Transforming entire documents

14 Extending transformations with user-defined functions

15 Aggregating your data

PART 4 - GOING FURTHER

16 Cache and checkpoint: Enhancing Spark’s performances

17 Exporting data and building full data pipelines

18 Exploring deployment

About the Author

An experienced consultant and entrepreneur passionate about all things data, Jean-Georges Perrin was the first IBM Champion in France, an honor he's now held for ten consecutive years. Jean-Georges has managed many teams of software and data engineers.

eBook License

End-User Warranty and License Agreement

1. Grant of License

Manning has authorized the download by you of an unrestricted number of copies of the electronic book (ebook) in any of the available formats. Manning grants you a nonexclusive, nontransferable license to use the ebook according to the terms and conditions herein. This License Agreement permits you to install the ebook on any and all your devices for your personal use only.

2. Restrictions

You shall not: (1) share, resell, rent, assign, timeshare, distribute, or transfer all or part of the ebook or any rights granted hereunder to any other person; (2) duplicate the ebook, except for a single backup or archival copy; (3) remove any proprietary notices, labels, or marks from the ebook; (4) transfer or sublicense title to the ebook to any other party.

3. Intellectual Property Protection

The ebook is owned by Manning and is protected by United States and international copyright and other intellectual property laws. Manning reserves all rights in the ebook not expressly granted herein. This license and your right to use the ebook terminate automatically if you violate any part of this Agreement. In the event of termination, you must remove the original and any copies of the ebook from all your devices.

4. Source Code Supplementary Material

Any source code files provided as a supplement to the book are freely available to the public for download. Reuse of the code is permitted, in whole or in part, including the creation of derivative works, provided that you acknowledge that you are using it and identify the source: title, publisher and year.

5. Limited Warranty

Manning warrants that the ebook files, a copy of which you are authorized to download, are free from defects in the operational sense that they can be read by a PDF Reader or ePub reader, or other. EXCEPT FOR THIS EXPRESS LIMITED WARRANTY, MANNING MAKES AND YOU RECEIVE NO WARRANTIES, EXPRESS, IMPLIED, STATUTORY OR IN ANY COMMUNICATION WITH YOU, AND MANNING SPECIFICALLY DISCLAIMS ANY OTHER WARRANTY INCLUDING THE IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS OR A PARTICULAR PURPOSE. MANNING DOES NOT WARRANT THAT THE OPERATION OF THE EBOOK WILL BE UNINTERRUPTED OR ERROR FREE. If the ebook was purchased in the United States, the above exclusions may not apply to you as some states do not allow the exclusion of implied warranties. In addition to the above warranty rights, you may also have other rights that vary from state to state.

6. Limitation of Liability

IN NO EVENT WILL MANNING BE LIABLE FOR ANY DAMAGES, WHETHER ARISING FOR TORT OR CONTRACT, INCLUDING LOSS OF DATA, LOST PROFITS, OR OTHER SPECIAL, INCIDENTAL, CONSEQUENTIAL, OR INDIRECT DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE EBOOK.

7. General

This Agreement constitutes the entire agreement between you and Manning and supersedes any prior agreement concerning the ebook. This Agreement is governed by the laws of the State of New York.