Seven NoSQL Databases in a Week: Get up and running with the fundamentals and functionalities of seven of the most popular NoSQL databases

Seven NoSQL Databases in a Week

Get up and running with the fundamentals and functionalities of seven of the most popular NoSQL databases

Aaron Ploetz

Devram Kandhare

Sudarshan Kadambi

Xun (Brian) Wu

BIRMINGHAM - MUMBAI

Seven NoSQL Databases in a Week

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Amey Varangaonkar

Acquisition Editor: Prachi Bisht

Content Development Editor: Eisha Dsouza

Technical Editor: Nirbhaya Shaji

Copy Editors: Laxmi Subramanian and Safis Editing

Project Coordinator: Kinjal Bari

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Graphics: Jisha Chirayil

Production Coordinator: Aparna Bhagat

First published: March 2018

Production reference: 1270318

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-78728-886-7

www.packtpub.com

To my wife, Coriene, for all of her constant love and support. And to my parents, Brian and Mary Jo Ploetz and Christine and Rick Franda, for all the sacrifices they made to ensure that I always had access to a computer while growing up.

– Aaron Ploetz

To Goraksha Bharam for his sacrifices and support in my life. To my teachers, who introduced me to this beautiful world of knowledge. And to my grandmother.

– Devram Kandhare

I would like to thank my parents for providing me the foundation that make work like this possible. I would like to thank my sister, wife, and son for their support and encouragement and for the joy and happiness they bring to my life.

– Sudarshan Kadambi

I would like to thank my parents who always give me the inspiration, drive, and encouragement. I would also like to thank my wife and kids, Bridget and Charlotte, for their patience and support throughout this endeavor.

– Xun (Brian) Wu

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

PacktPub.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the authors

Aaron Ploetz is the NoSQL Engineering Lead for Target, where his DevOps team supports Cassandra, MongoDB, Redis, and Neo4j. He has been named a DataStax MVP for Apache Cassandra three times, and has presented at multiple events, including the DataStax Summit and Data Day Texas. He earned a BS in Management/Computer Systems from the University of Wisconsin-Whitewater, and a MS in Software Engineering from Regis University. He and his wife, Coriene, live with their three children in the Twin Cities area.

I'd like to thank Dr. David Munro, who inspired me, Ron Bieber, who believed in me and introduced me to the world of distributed databases, and my lovely wife, Coriene, who provided constant support and encouragement during this endeavor. Seriously, without Coriene none of this would have been possible.

Devram Kandhare has 4 years of experience of working with the SQL database—MySql and NoSql databases—MongoDB and DynamoDB. He has worked as database designer and developer. He has developed various projects using the Agile development model. He is experienced in building web-based applications and REST API.

I'd like to thank my grandparents, Mamaji, Shriniwas Gadre, and Mudit Tyagi, for their love and guidance. Most importantly I want to thank my parents, my sister, Amruta, my loving and supportive wife, Ashvini, and my friend Prashant; without them none of this would have been possible.

Sudarshan Kadambi has a background in distributed systems and database design. He has been a user and contributor to various NoSQL databases and is passionate about solving large-scale data management challenges.

I would like to thank my parents for providing me the foundations that make work like this possible. I would like to thank my sister, wife, and son for their support and encouragement and for the joy and happiness they bring to my life.

Xun (Brian) Wu has more than 15 years of experience in web/mobile development, big data analytics, cloud computing, blockchain, and IT architecture. He holds a master's degree in computer science from NJIT. He is always enthusiastic about exploring new ideas, technologies, and opportunities that arise. He has previously reviewed more than 40 books from Packt Publishing. 

I would like to thank my parents who always give me inspiration, drive and encouragement. I would also like to thank my wife and kids, Bridget and Charlotte, for their patience and support throughout this endeavor.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Title Page

Copyright and Credits

Seven NoSQL Databases in a Week

Dedication

Packt Upsell

Why subscribe?

PacktPub.com

Contributors

About the authors

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Introduction to NoSQL Databases

Consistency versus availability

ACID guarantees

Hash versus range partition

In-place updates versus appends

Row versus column versus column-family storage models

Strongly versus loosely enforced schemas

Summary

MongoDB

Installing of MongoDB

MongoDB data types

The MongoDB database

MongoDB collections

MongoDB documents

The create operation

The read operation

Applying filters on fields

Applying conditional and logical operators on the filter parameter

The update operation

The delete operation

Data models in MongoDB

The references document data model

The embedded data model

Introduction to MongoDB indexing

The default _id index

Replication

Replication in MongoDB

Automatic failover in replication

Read operations

Sharding

Sharded clusters

Advantages of sharding

Storing large data in MongoDB

Summary

Neo4j

What is Neo4j?

How does Neo4j work?

Features of Neo4j

Clustering

Neo4j Browser

Cache sharding

Help for beginners

Evaluating your use case

Social networks

Matchmaking

Network management

Analytics

Recommendation engines

Neo4j anti-patterns

Applying relational modeling techniques in Neo4j

Using Neo4j for the first time on something mission-critical

Storing entities and relationships within entities

Improper use of relationship types

Storing binary large object data

Indexing everything

Neo4j hardware selection, installation, and configuration

Random access memory

CPU

Disk

Operating system

Network/firewall

Installation

Installing JVM

Configuration

High-availability clustering

Causal clustering

Using Neo4j

Neo4j Browser

Cypher

Python

Java

Taking a backup with Neo4j

Backup/restore with Neo4j Enterprise

Backup/restore with Neo4j Community

Differences between the Neo4j Community and Enterprise Editions

Tips for success

Summary

References 

Redis

Introduction to Redis

What are the key features of Redis?

Performance

Tunable data durability

Publish/Subscribe

Useful data types

Expiring data over time

Counters

Server-side Lua scripting

Appropriate use cases for Redis

Data fits into RAM

Data durability is not a concern

Data at scale

Simple data model

Features of Redis matching part of your use case

Data modeling and application design with Redis

Taking advantage of Redis' data structures

Queues

Sets

Notifications

Counters

Caching

Redis anti-patterns

Dataset cannot fit into RAM

Modeling relational data

Improper connection management

Security

Using the KEYS command

Unnecessary trips over the network

Not disabling THP

Redis setup, installation, and configuration

Virtualization versus on-the-metal

RAM

CPU

Disk

Operating system

Network/firewall

Installation

Configuration files

Using Redis

redis-cli

Lua

Python

Java

Taking a backup with Redis

Restoring from a backup

Tips for success

Summary

References

Cassandra

Introduction to Cassandra

What problems does Cassandra solve?

What are the key features of Cassandra?

No single point of failure

Tunable consistency

Data center awareness

Linear scalability

Built on the JVM

Appropriate use cases for Cassandra

Overview of the internals

Data modeling in Cassandra

Partition keys

Clustering keys

Putting it all together

Optimal use cases

Cassandra anti-patterns

Frequently updated data

Frequently deleted data

Queues or queue-like data

Solutions requiring query flexibility

Solutions requiring full table scans

Incorrect use of BATCH statements

Using Byte Ordered Partitioner

Using a load balancer in front of Cassandra nodes

Using a framework driver

Cassandra hardware selection, installation, and configuration

RAM

CPU

Disk

Operating system

Network/firewall

Installation using apt-get

Tarball installation

JVM installation

Node configuration

Running Cassandra

Adding a new node to the cluster

Using Cassandra

Nodetool

CQLSH

Python

Java

Taking a backup with Cassandra

Restoring from a snapshot

Tips for success

Run Cassandra on Linux

Open ports 7199, 7000, 7001, and 9042

Enable security

Use solid state drives (SSDs) if possible

Configure only one or two seed nodes per data center

Schedule weekly repairs

Do not force a major compaction

Remember that every mutation is a write

The data model is key

Consider a support contract

Cassandra is not a general purpose database

Summary

References

HBase

Architecture

Components in the HBase stack

Zookeeper

HDFS

HBase master

HBase RegionServers

Reads and writes

The HBase write path

HBase writes – design motivation

The HBase read path

HBase compactions

System trade-offs

Logical and physical data models

Interacting with HBase – the HBase shell

Interacting with HBase – the HBase Client API

Interacting with secure HBase clusters

Advanced topics

HBase high availability

Replicated reads

HBase in multiple regions

HBase coprocessors

SQL over HBase

Summary

DynamoDB

The difference between SQL and DynamoDB

Setting up DynamoDB

Setting up locally

Setting up using AWS

The difference between downloadable DynamoDB and DynamoDB web services

DynamoDB data types and terminology

Tables, items, and attributes

Primary key

Secondary indexes

Streams

Queries

Scan

Data types

Data models and CRUD operations in DynamoDB

Limitations of DynamoDB

Best practices

Summary

InfluxDB

Introduction to InfluxDB

Key concepts and terms of InfluxDB

Data model and storage engine

Storage engine

Installation and configuration

Installing InfluxDB

Configuring InfluxDB

Production deployment considerations

Query language and API

Query language

Query pagination

Query performance optimizations

Interaction via Rest API

InfluxDB API client

InfluxDB with Java client

InfluxDB with a Python client

InfluxDB with Go client

InfluxDB ecosystem

Telegraf

Telegraf data management

Kapacitor

InfluxDB operations

Backup and restore

Backups

Restore

Clustering and HA

Retention policy

Monitoring

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

The book will help you understand the fundamentals of each database, and understand how their functionalities differ, while still giving you a common result – a database solution with speed, high performance, and accuracy.

Who this book is for

If you are a budding DBA or a developer who wants to get started with the fundamentals of NoSQL databases, this book is for you. Relational DBAs who want to get insights into the various offerings of the popular NoSQL databases will also find this book to be very useful.

What this book covers

Chapter 1, Introduction to NoSQL Databases, introduces the topic of NoSQL and distributed  databases. The design principles and trade-offs involved in NoSQL database design are described. These design principles provide context around why individual databases covered in the following chapters are designed in a particular way and what constraints they are trying to optimize for.

Chapter 2, MongoDB, covers installation and basic CRUD operations. High-level concepts such as indexing allow you to speed up database operations, sharding, and replication. Also, it covers data models, which help us with application database design.

Chapter 3, Neo4j, introduces the Neo4j graph database. It discusses Neo4j's architecture, use cases, administration, and application development.

Chapter 4, Redis, discusses the Redis data store. Redis’ unique architecture and behavior will be discussed, as well as installation, application development, and server-side scripting with Lua.

Chapter 5, Cassandra, introduces the Cassandra database. Cassandra’s highly-available, eventually consistent design will be discussed along with the appropriate use cases. Known anti-patterns will also be presented, as well as production-level configuration, administration, and application development.

Chapter 6, HBase, introduces HBase, that is, the Hadoop Database. Inspired by Google's Bigtable, HBase is a widely deployed key-value store today. This chapter covers HBase's architectural internals, data model, and API.

Chapter 7, DynamoDB, covers how to set up a local and AWS DynamoDB service and perform CRUD operations. It also covers how to deal with partition keys, sort keys, and secondary indexes. It covers various advantages and disadvantages of DynamoDB over other databases, which makes it easy for developers to choose a database for an application.

Chapter 8, InfluxDB, describes InfluxDB and its key concepts and terms. It also covers InfluxDB installation and configuration. It explores the query language and APIs. It helps you set up Telegraf and Kapacitor as an InfluxDB ecosystem's key components to collect and process data. At the end of the chapter, you will also find information about InfluxDB operations.

To get the most out of this book

This book assumes that you have access to hardware on which you can install, configure, and code against a database instance. Having elevated admin or sudo privileges on the aforementioned machine will be essential to carrying out some of the tasks described.

Some of the NoSQL databases discussed will only run on a Linux-based operating system. Therefore, prior familiarity with Linux is recommended. As OS-specific system administration is not within the scope of this book, readers who are new to Linux may find value in seeking out a separate tutorial prior to attempting some of the examples.

The Java Virtual Machine (JVM)-based NoSQL databases will require a Java Runtime Environment (JRE) to be installed. Do note that some of them may require a specific version of the JRE to function properly. This will necessitate updating or installing a new JRE, depending on the database.

The Java coding examples will be easier to do from within an Integrated Developer Envorinment (IDE), with Maven installed for dependency management.  You may need to look up additional resources to ensure that these components are configured properly.

In Chapter 6, HBase, you can install the Hortonworks sandbox to get a small HBase cluster set up on your laptop.  The sandbox can be installed for free from https://hortonworks.com/products/sandbox/.

In Chapter 8, InfluxDB, to run the examples you will need to install InfluxDB in a UNIX or Linux environment. In order to run different InfluxDB API client examples, you also need to install a programming language environment and related InfluxDB client packages:

Run the InfluxDB Java client: Install JDK and an editor (Eclipse or IntelliJ).

Run the InfluxDB Python client: Install Python.

Run the InfluxDB Go client: Install Go and the InfluxDB Go client; you can use JetBrains Goland to run the Go code.

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

Log in or register at www.packtpub.com.

Select the SUPPORT tab.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR/7-Zip for Windows

Zipeg/iZip/UnRarX for Mac

7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Seven-NoSQL-Databases-in-a-Week. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/SevenNoSQLDatabasesinaWeek_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Now is also a good time to change the initial password. Neo4j installs with a single default admin username and password of neo4j/neo4j.

A block of code is set as follows:

# Paths of directories in the installation.

#dbms.directories.data=data

#dbms.directories.plugins=plugins

#dbms.directories.certificates=certificates

#dbms.directories.logs=logs

#dbms.directories.lib=lib

#dbms.directories.run=run

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

# Paths of directories in the installation.

#dbms.directories.data=data

#dbms.directories.plugins=plugins

#dbms.directories.certificates=certificates

#dbms.directories.logs=logs

#dbms.directories.lib=lib

#dbms.directories.run=run

Any command-line input or output is written as follows:

sudo mkdir /local

sudo chown $USER:$USER /local

cd /local

mv ~/Downloads/neo4j-community-3.3.3-unix.tar.gz .

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: To create a table, click on the Create table button. This will take you to the Create table screen.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.

Introduction to NoSQL Databases

Over the last decade, the volume and velocity with which data is generated within organizations has grown exponentially. Consequently, there has been an explosion of database technologies that have been developed to address these growing data needs. These databases have typically had distributed implementations, since the volume of data being managed far exceeds the storage capacity of a single node. In order to support the massive scale of data,