AWS Manged Streaming For Apache Kafka - 30

Yeswanth Chauhan
AWS managed streaming for Apache kafka -
EFFICIENTLY SCALE YOUR DATA STREAMING

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that
makes it easy for you to build and run applications that use Apache Kafka to process streaming
data. Apache Kafka is an open-source platform for building real-time streaming data pipelines
and applications.
With Amazon MSK, you can use Apache Kafka APIs to populate data lakes, stream changes to
and from databases, and power machine learning and analytics applications. Apache Kafka
clusters are challenging to set up, scale, and manage in production. When you run Apache
Kafka on your own, you need to provision servers, configure Apache Kafka manually, replace
servers when they fail, orchestrate server patches and upgrades, architect the cluster for high
availability, ensure data is durably stored and secured, setup monitoring and alarms, and
carefully plan to scale events to support load changes. Amazon MSK makes it easy for you to
build and run production applications on Apache Kafka without needing Apache Kafka
infrastructure management expertise. That means you spend less time managing infrastructure
and more time building applications.
With a few clicks in the Amazon MSK console you can create highly available Apache Kafka
clusters with settings and configurations based on Apache Kafka’s deployment best practices.
Amazon MSK automatically provisions and runs your Apache Kafka clusters. Amazon MSK
continuously monitors cluster health and automatically replaces unhealthy nodes with no
downtime to your application. In addition, Amazon MSK secures your Apache Kafka cluster by
encrypting data at rest.
FULLY MANAGED
Provisioning, configuration, and maintenance of Apache Kafka clusters and Apache ZooKeeper
nodes are managed by Amazon MSK. Also, key Apache Kafka performance metrics could be
shown in the dedicated AWS web console. No need to worry about the operational overhead of
managing your Apache Kafka environment because Amazon MSK allows users to focus on the
creation of your streaming applications.
HIGHLY AVAILABLE
Automatic recovery is provided by monitoring the health of the clusters and replacing unhealthy
brokers without downtime for the applications. Amazon MSK manages the availability of Apache
ZooKeeper nodes so you will not need to start, stop, or directly access the nodes yourself.
Amazon MSK also deploys software patches as needed to keep the cluster up to date-and
running smoothly. Amazon MSK uses also multi-Availability Zone replication for high availability.
strip-1
strip-2
HIGHLY SECURE
Yeswanth Chauhan
Amazon MSK provides multiple levels of security for your Apache Kafka clusters including VPC
network isolation, AWS IAM for control-plane API authorization, encryption at rest, TLS
encryption in-transit.
FULLY COMPATIBLE
Amazon MSK runs and manages Apache Kafka for you. This makes it easy for you to migrate
and run your existing Apache Kafka applications on AWS without changes to the application
code. By using Amazon MSK, you maintain open source compatibility and can continue to use
familiar custom and community-built tools such as MirrorMaker.
strip-3
strip-4
HIGHLY SCALABLE
There are two kind of scaling: broker scaling, it is possible to increase or decrease the number
of the brokers of the cluster, and storage scaling, by modifying the amount of storage
provisioned per broker to match changes in storage requirements.
With Amazon MSK, you can use Apache Kafka APIs to populate data lakes, stream changes to
and from databases, and power machine learning and analytics applications. Amazon Managed
Streaming for Kafka makes it easy for you to build and run production applications on Apache
Kafka without needing Apache Kafka infrastructure management expertise.
Amazon Managed Streaming for Kafka is a tool in the Kafka as a Service category of a tech
stack.
Assuming you are referring to AWS Managed Kafka, here are some pros and cons:
Pros:
Easy to use: AWS Managed Kafka simplifies the process of deploying and managing Apache
Kafka clusters, as it takes care of the infrastructure and software management for you.
Highly scalable: AWS Managed Kafka can easily handle large amounts of data and supports
high throughput and low latency requirements.
High availability: AWS Managed Kafka offers a highly available and fault-tolerant platform that
can continue to operate even if one or more nodes fail.
Integrated with AWS services: AWS Managed Kafka is integrated with many AWS services such
as Amazon S3, Amazon EMR, and Amazon Redshift, making it easy to integrate with other
AWS services.
Security: AWS Managed Kafka provides a secure environment with encryption in transit and at
rest, identity and access management, and network isolation.
Cons:
Yeswanth Chauhan
Cost: AWS Managed Kafka can be expensive, especially for small workloads or infrequent use
cases.
Limited flexibility: As AWS Managed Kafka is a managed service, there may be some limitations
in terms of customization and configuration options.
Learning curve: Although AWS Managed Kafka simplifies the process of deploying and
managing Kafka clusters, it still requires some understanding of the underlying infrastructure
and software.
Vendor lock-in: As with any managed service, there is a risk of vendor lock-in, which may limit
your ability to switch to another provider or technology stack in the future.
Reliance on AWS: AWS Managed Kafka relies on the stability and reliability of the AWS
infrastructure, which may be a concern for some users.
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan
Yeswanth Chauhan

AWS Manged Streaming For Apache Kafka - 30

Uploaded by

AWS Manged Streaming For Apache Kafka - 30

Uploaded by

Yeswanth Chauhan

AWS managed streaming for Apache kafka -

EFFICIENTLY SCALE YOUR DATA STREAMING

You might also like