Practical Convolutional Neural Networks: Implement advanced deep learning models using Python
()
About this ebook
Convolutional Neural Network (CNN) is revolutionizing several application domains such as visual recognition systems, self-driving cars, medical discoveries, innovative eCommerce and more.You will learn to create innovative solutions around image and video analytics to solve complex machine learning and computer vision related problems and implement real-life CNN models.
This book starts with an overview of deep neural networkswith the example of image classification and walks you through building your first CNN for human face detector. We will learn to use concepts like transfer learning with CNN, and Auto-Encoders to build very powerful models, even when not much of supervised training data of labeled images is available.
Later we build upon the learning achieved to build advanced vision related algorithms for object detection, instance segmentation, generative adversarial networks, image captioning, attention mechanisms for vision, and recurrent models for vision.
By the end of this book, you should be ready to implement advanced, effective and efficient CNN models at your professional project or personal initiatives by working on complex image and video datasets.
Related to Practical Convolutional Neural Networks
Related ebooks
Deep Learning with PyTorch: A practical approach to building neural network models using PyTorch Rating: 0 out of 5 stars0 ratingsHands-On Deep Learning for Images with TensorFlow: Build intelligent computer vision applications using TensorFlow and Keras Rating: 0 out of 5 stars0 ratingsMastering PyTorch: Build powerful neural network architectures using advanced PyTorch 1.x features Rating: 0 out of 5 stars0 ratingsDeep Learning By Example: A hands-on guide to implementing advanced machine learning algorithms and neural networks Rating: 0 out of 5 stars0 ratingsAdvanced Deep Learning with Python: Design and implement advanced next-generation AI solutions using TensorFlow and PyTorch Rating: 0 out of 5 stars0 ratingsHands-On Neural Network Programming with C#: Add powerful neural network capabilities to your C# enterprise applications Rating: 0 out of 5 stars0 ratingsArtificial Intelligence By Example: Develop machine intelligence from scratch using real artificial intelligence use cases Rating: 0 out of 5 stars0 ratingsHands-On Convolutional Neural Networks with TensorFlow: Solve computer vision problems with modeling in TensorFlow and Python Rating: 0 out of 5 stars0 ratingsR Deep Learning Essentials.: A step-by-step guide to building deep learning models using TensorFlow, Keras, and MXNet Rating: 0 out of 5 stars0 ratingsMastering Machine Learning for Penetration Testing: Develop an extensive skill set to break self-learning systems using Python Rating: 0 out of 5 stars0 ratingsKeras 2.x Projects: 9 projects demonstrating faster experimentation of neural network and deep learning applications using Keras Rating: 0 out of 5 stars0 ratingsKeras Deep Learning Cookbook: Over 30 recipes for implementing deep neural networks in Python Rating: 0 out of 5 stars0 ratingsScala Machine Learning Projects: Build real-world machine learning and deep learning projects with Scala Rating: 0 out of 5 stars0 ratingsReinforcement Learning Algorithms with Python: Learn, understand, and develop smart algorithms for addressing AI challenges Rating: 0 out of 5 stars0 ratingsPython Data Analysis: Perform data collection, data processing, wrangling, visualization, and model building using Python Rating: 0 out of 5 stars0 ratingsDeep Learning for Computer Vision: Expert techniques to train advanced neural networks using TensorFlow and Keras Rating: 0 out of 5 stars0 ratingsArchitecting Data-Intensive Applications: Develop scalable, data-intensive, and robust applications the smart way Rating: 0 out of 5 stars0 ratingsR Machine Learning Projects: Implement supervised, unsupervised, and reinforcement learning techniques using R 3.5 Rating: 0 out of 5 stars0 ratingsHands-On Machine Learning with C#: Build smart, speedy, and reliable data-intensive applications using machine learning Rating: 0 out of 5 stars0 ratingsDeep Learning Essentials: Your hands-on guide to the fundamentals of deep learning and neural network modeling Rating: 0 out of 5 stars0 ratings
Computers For You
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL Rating: 4 out of 5 stars4/5The Invisible Rainbow: A History of Electricity and Life Rating: 5 out of 5 stars5/5The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution Rating: 4 out of 5 stars4/5Slenderman: Online Obsession, Mental Illness, and the Violent Crime of Two Midwestern Girls Rating: 4 out of 5 stars4/5101 Awesome Builds: Minecraft® Secrets from the World's Greatest Crafters Rating: 4 out of 5 stars4/5The ChatGPT Millionaire Handbook: Make Money Online With the Power of AI Technology Rating: 4 out of 5 stars4/5Procreate for Beginners: Introduction to Procreate for Drawing and Illustrating on the iPad Rating: 0 out of 5 stars0 ratingsTor and the Dark Art of Anonymity Rating: 5 out of 5 stars5/5Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics Rating: 4 out of 5 stars4/5Elon Musk Rating: 4 out of 5 stars4/5Mastering ChatGPT: 21 Prompts Templates for Effortless Writing Rating: 5 out of 5 stars5/5Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are Rating: 4 out of 5 stars4/5Alan Turing: The Enigma: The Book That Inspired the Film The Imitation Game - Updated Edition Rating: 4 out of 5 stars4/5The Hacker Crackdown: Law and Disorder on the Electronic Frontier Rating: 4 out of 5 stars4/5How to Create Cpn Numbers the Right way: A Step by Step Guide to Creating cpn Numbers Legally Rating: 4 out of 5 stars4/5CompTIA Security+ Get Certified Get Ahead: SY0-701 Study Guide Rating: 5 out of 5 stars5/5CompTIA IT Fundamentals (ITF+) Study Guide: Exam FC0-U61 Rating: 0 out of 5 stars0 ratingsUncanny Valley: A Memoir Rating: 4 out of 5 stars4/5Deep Search: How to Explore the Internet More Effectively Rating: 5 out of 5 stars5/5Grokking Algorithms: An illustrated guide for programmers and other curious people Rating: 4 out of 5 stars4/5Master Builder Roblox: The Essential Guide Rating: 4 out of 5 stars4/5Creating Online Courses with ChatGPT | A Step-by-Step Guide with Prompt Templates Rating: 4 out of 5 stars4/5The Best Hacking Tricks for Beginners Rating: 4 out of 5 stars4/5
Reviews for Practical Convolutional Neural Networks
0 ratings0 reviews
Book preview
Practical Convolutional Neural Networks - Mohit Sewak
Practical Convolutional Neural Networks
Implement advanced deep learning models using Python
Mohit Sewak
Md. Rezaul Karim
Pradeep Pujari
BIRMINGHAM - MUMBAI
Practical Convolutional Neural Networks
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Sunith Shetty
Acquisition Editor: Vinay Argekar
Content Development Editor: Cheryl Dsa
Technical Editor: Sagar Sawant
Copy Editor: Vikrant Phadke, Safis Editing
Project Coordinator: Nidhi Joshi
Proofreader: Safis Editing
Indexer: Tejal Daruwale Soni
Graphics: Tania Dutta
Production Coordinator: Arvindkumar Gupta
First published: February 2018
Production reference: 1230218
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78839-230-3
www.packtpub.com
mapt.io
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Mohit Sewak is a senior cognitive data scientist with IBM, and a PhD scholar in AI and CS at BITS Pilani. He holds several patents and publications in AI, deep learning, and machine learning. He has been the lead data scientist for some very successful global AI/ ML software and industry solutions and was earlier engaged in solutioning and research for the Watson Cognitive Commerce product line. He has 14 years of rich experience in architecting and solutioning with TensorFlow, Torch, Caffe, Theano, Keras, Watson, and more.
Md. Rezaul Karim is a research scientist at Fraunhofer FIT, Germany. He is also a PhD candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a researcher at the Insight Center for Data Analytics, Ireland. He was a lead engineer at Samsung Electronics, Korea.
He has 9 years of R&D experience with C++, Java, R, Scala, and Python. He has published research papers on bioinformatics, big data, and deep learning. He has practical working experience with Spark, Zeppelin, Hadoop, Keras, Scikit-Learn, TensorFlow, Deeplearning4j, MXNet, and H2O.
Pradeep Pujari is machine learning engineer at Walmart Labs and a distinguished member of ACM. His core domain expertise is in information retrieval, machine learning, and natural language processing. In his free time, he loves exploring AI technologies, reading, and mentoring.
About the reviewer
Sumit Pal is a published author with Apress. He has more than 22 years of experience in software, from start-ups to enterprises, and is an independent consultant working with big data, data visualization, and data science. He builds end-to-end data-driven analytic systems.
He has worked for Microsoft (SQLServer), Oracle (OLAP Kernel), and Verizon. He advises clients on their data architectures and build solutions in Spark and Scala. He has spoken at many conferences in North America and Europe and has developed a big data analyst training for Experfy. He has an MS and BS in computer science.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Practical Convolutional Neural Networks
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the authors
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Deep Neural Networks – Overview
Building blocks of a neural network
Introduction to TensorFlow
Installing TensorFlow
For macOS X/Linux variants
TensorFlow basics
Basic math with TensorFlow
Softmax in TensorFlow
Introduction to the MNIST dataset
The simplest artificial neural network
Building a single-layer neural network with TensorFlow
Keras deep learning library overview
Layers in the Keras model
Handwritten number recognition with Keras and MNIST
Retrieving training and test data
Flattened data
Visualizing the training data
Building the network
Training the network
Testing
Understanding backpropagation
Summary
Introduction to Convolutional Neural Networks
History of CNNs
Convolutional neural networks
How do computers interpret images?
Code for visualizing an image
Dropout
Input layer
Convolutional layer
Convolutional layers in Keras
Pooling layer
Practical example – image classification
Image augmentation
Summary
Build Your First CNN and Performance Optimization
CNN architectures and drawbacks of DNNs
Convolutional operations
Pooling, stride, and padding operations
Fully connected layer
Convolution and pooling operations in TensorFlow
Applying pooling operations in TensorFlow
Convolution operations in TensorFlow
Training a CNN
Weight and bias initialization
Regularization
Activation functions
Using sigmoid
Using tanh
Using ReLU
Building, training, and evaluating our first CNN
Dataset description
Step 1 – Loading the required packages
Step 2 – Loading the training/test images to generate train/test set
Step 3- Defining CNN hyperparameters
Step 4 – Constructing the CNN layers
Step 5 – Preparing the TensorFlow graph
Step 6 – Creating a CNN model
Step 7 – Running the TensorFlow graph to train the CNN model
Step 8 – Model evaluation
Model performance optimization
Number of hidden layers
Number of neurons per hidden layer
Batch normalization
Advanced regularization and avoiding overfitting
Applying dropout operations with TensorFlow
Which optimizer to use?
Memory tuning
Appropriate layer placement
Building the second CNN by putting everything together
Dataset description and preprocessing
Creating the CNN model
Training and evaluating the network
Summary
Popular CNN Model Architectures
Introduction to ImageNet
LeNet
AlexNet architecture
Traffic sign classifiers using AlexNet
VGGNet architecture
VGG16 image classification code example
GoogLeNet architecture
Architecture insights
Inception module
ResNet architecture
Summary
Transfer Learning
Feature extraction approach
Target dataset is small and is similar to the original training dataset
Target dataset is small but different from the original training dataset
Target dataset is large and similar to the original training dataset
Target dataset is large and different from the original training dataset
Transfer learning example
Multi-task learning
Summary
Autoencoders for CNN
Introducing to autoencoders
Convolutional autoencoder
Applications
An example of compression
Summary
Object Detection and Instance Segmentation with CNN
The differences between object detection and image classification
Why is object detection much more challenging than image classification?
Traditional, nonCNN approaches to object detection
Haar features, cascading classifiers, and the Viola-Jones algorithm
Haar Features
Cascading classifiers
The Viola-Jones algorithm
R-CNN – Regions with CNN features
Fast R-CNN – fast region-based CNN
Faster R-CNN – faster region proposal network-based CNN
Mask R-CNN – Instance segmentation with CNN
Instance segmentation in code
Creating the environment
Installing Python dependencies (Python2 environment)
Downloading and installing the COCO API and detectron library (OS shell commands)
Preparing the COCO dataset folder structure
Running the pre-trained model on the COCO dataset
References
Summary
GAN: Generating New Images with CNN
Pix2pix - Image-to-Image translation GAN
CycleGAN
Training a GAN model
GAN – code example
Calculating loss
Adding the optimizer
Semi-supervised learning and GAN
Feature matching
Semi-supervised classification using a GAN example
Deep convolutional GAN
Batch normalization
Summary
Attention Mechanism for CNN and Visual Models
Attention mechanism for image captioning
Types of Attention
Hard Attention
Soft Attention
Using attention to improve visual models
Reasons for sub-optimal performance of visual CNN models
Recurrent models of visual attention
Applying the RAM on a noisy MNIST sample
Glimpse Sensor in code
References
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
CNNs are revolutionizing several application domains, such as visual recognition systems, self-driving cars, medical discoveries, innovative e-commerce, and many more. This book gets you started with the building blocks of CNNs, while also guiding you through the best practices for implementing real-life CNN models and solutions. You will learn to create innovative solutions for image and video analytics to solve complex machine learning and computer vision problems.
This book starts with an overview of deep neural networks, with an example of image classification, and walks you through building your first CNN model. You will learn concepts such as transfer learning and autoencoders with CNN that will enable you to build very powerful models, even with limited supervised (labeled image) training data.
Later we build upon these learnings to achieve advanced vision-related algorithms and solutions for object detection, instance segmentation, generative (adversarial) networks, image captioning, attention mechanisms, and recurrent attention models for vision.
Besides giving you hands-on experience with the most intriguing vision models and architectures, this book explores cutting-edge and very recent researches in the areas of CNN and computer vision. This enable the user to foresee the future in this field and quick-start their innovation journey using advanced CNN solutions.
By the end of this book, you should be ready to implement advanced, effective, and efficient CNN models in your professional projects or personal initiatives while working on complex images and video datasets.
Who this book is for
This book is for data scientists, machine learning, and deep learning practitioners, and cognitive and artificial intelligence enthusiasts who want to move one step further in building CNNs. Get hands-on experience with extreme datasets and different CNN architectures to build efficient and smart ConvNet models. Basic knowledge of deep learning concepts and Python programming language is expected.
What this book covers
Chapter 1, Deep Neural Networks - Overview, it gives a quick refresher of the science of deep neural networks and different frameworks that can be used to implement such networks, with the mathematics behind them.
Chapter 2, Introduction to Convolutional Neural Networks, it introduces the readers to convolutional neural networks and shows how deep learning can be used to extract insights from images.
Chapter 3, Build Your First CNN and Performance Optimization, constructs a simple CNN model for image classification from scratch, and explains how to tune hyperparameters and optimize training time and performance of CNNs for improved efficiency and accuracy respectively.
Chapter 4, Popular CNN Model Architectures, shows the advantages and working of different popular (and award winning) CNN architectures, how they differ from each other, and how to use them.
Chapter 5, Transfer Learning, teaches you to take an existing pretrained network and adapt it to a new and different dataset. There is also a custom classification problem for a real-life application using a technique called transfer learning.
Chapter 6, Autoencoders for CNN, introduces an unsupervised learning technique called autoencoders. We walk through different applications of autoencoders for CNN, such as image compression.
Chapter 7, Object Detection and Instance Segmentation with CNN, teaches the difference between object detection, instance segmentation, and image classification. We then learn multiple techniques for object detection and instance segmentation with CNNs.
Chapter 8, GAN—Generating New Images with CNN, explores generative CNN Networks, and then we combine them with our learned discriminative CNN networks to create new images with CNN/GAN.
Chapter 9, Attention Mechanism for CNN and Visual Models, teaches the intuition behind attention in deep learning and learn how attention-based models are used to implement some advanced solutions (image captioning and RAM). We also understand the different types of attention and the role of reinforcement learning with respect to the hard attention mechanism.
To get the most out of this book
This book is focused on building CNNs with Python programming language. We have used Python version 2.7 (2x) to build various applications and the open source and enterprise-ready professional software using Python, Spyder, Anaconda, and PyCharm. Many of the examples are also compatible with Python 3x. As a good practice, we encourage users to use Python virtual environments for implementing these codes.
We focus on how to utilize various Python and deep learning libraries (Keras, TensorFlow, and Caffe) in the best possible way to build real-world applications. In that spirit, we have tried to keep all of the code as friendly and readable as possible. We feel that this will enable our readers to easily understand the code and readily use it in different scenarios.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at www.packtpub.com.
Select the SUPPORT tab.
Click on Code Downloads & Errata.
Enter the name of the book in the Search box and follow the onscreen instructions.
Once