Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

Preferences
Contact Us
Feedback
AWS Documentation
Create an AWS Account
Using the Spark structured streaming Amazon Kinesis Data Streams connector - Amazon EMR

Using the Spark structured streaming Amazon Kinesis Data Streams connector

PDFRSS

Amazon EMR releases 7.1.0 and higher include a spark structured streaming Amazon Kinesis Data Streams connector in the release image. With this connector, you can use Spark on Amazon EMR to process data that's stored in Amazon Kinesis Data Streams. The connector supports both consumer types of GetRecords (shared throughput) and SubscribeToShard (enhanced fan-out). This integration is based on the spark-sql-kinesis-connector. For details about how to get started using the connector, see the README.

The following example demonstrates how to use the connector to launch a Spark application with Amazon EMR

spark-submit my_kinesis_streaming_script.py

Need help?

PrivacySite termsCookie preferences
© 2025, Amazon Web Services, Inc. or its affiliates. All rights reserved.