Architecture-Aware Optimization Strategies in Real-time Image Processing
By Chao Li, Souleymane Balla-Arabe and Fan Yang
()
About this ebook
In the field of image processing, many applications require real-time execution, particularly those in the domains of medicine, robotics and transmission, to name but a few. Recent technological developments have allowed for the integration of more complex algorithms with large data volume into embedded systems, in turn producing a series of new sophisticated electronic architectures at affordable prices. This book performs an in-depth survey on this topic. It is primarily written for those who are familiar with the basics of image processing and want to implement the target processing design using different electronic platforms for computing acceleration. The authors present techniques and approaches, step by step, through illustrative examples. This book is also suitable for electronics/embedded systems engineers who want to consider image processing applications as sufficient imaging algorithm details are given to facilitate their understanding.
Related to Architecture-Aware Optimization Strategies in Real-time Image Processing
Related ebooks
Embedded Software Design and Programming of Multiprocessor System-on-Chip: Simulink and System C Case Studies Rating: 0 out of 5 stars0 ratingsElectronic Structure Calculations on Graphics Processing Units: From Quantum Chemistry to Condensed Matter Physics Rating: 0 out of 5 stars0 ratingsInternet of Things: Architectures, Protocols and Standards Rating: 0 out of 5 stars0 ratingsResponsible Data Science Rating: 0 out of 5 stars0 ratingsSiP System-in-Package Design and Simulation: Mentor EE Flow Advanced Design Guide Rating: 0 out of 5 stars0 ratingsMultidisciplinary Design Optimization Supported by Knowledge Based Engineering Rating: 0 out of 5 stars0 ratingsEvolutionary Algorithms for Mobile Ad Hoc Networks Rating: 0 out of 5 stars0 ratingsBig Data Analytics for Large-Scale Multimedia Search Rating: 0 out of 5 stars0 ratingsMulticore DSP: From Algorithms to Real-time Implementation on the TMS320C66x SoC Rating: 0 out of 5 stars0 ratingsColor Profile: Exploring Visual Perception and Analysis in Computer Vision Rating: 0 out of 5 stars0 ratingsRaster Graphics Editor: Transforming Visual Realities: Mastering Raster Graphics Editors in Computer Vision Rating: 0 out of 5 stars0 ratingsDigital Raster Graphic: Unveiling the Power of Digital Raster Graphics in Computer Vision Rating: 0 out of 5 stars0 ratingsMachine Learning in the AWS Cloud: Add Intelligence to Applications with Amazon SageMaker and Amazon Rekognition Rating: 0 out of 5 stars0 ratingsRaster Graphics: Understanding the Foundations of Raster Graphics in Computer Vision Rating: 0 out of 5 stars0 ratingsStatistical Data Cleaning with Applications in R Rating: 0 out of 5 stars0 ratingsVolume Rendering: Exploring Visual Realism in Computer Vision Rating: 0 out of 5 stars0 ratingsData-Variant Kernel Analysis Rating: 0 out of 5 stars0 ratingsBeginning Software Engineering Rating: 5 out of 5 stars5/5Real-Time Embedded Systems Rating: 0 out of 5 stars0 ratingsDaily Knowledge Valuation in Organizations: Traceability and Capitalization Rating: 0 out of 5 stars0 ratingsText Mining in Practice with R Rating: 0 out of 5 stars0 ratingsStack Computers: The New Wave Rating: 0 out of 5 stars0 ratingsMultimedia Networks: Protocols, Design and Applications Rating: 0 out of 5 stars0 ratingsRobot Learning by Visual Observation Rating: 0 out of 5 stars0 ratingsAdvanced Backend Code Optimization Rating: 0 out of 5 stars0 ratingsData Mining and Machine Learning in Building Energy Analysis Rating: 0 out of 5 stars0 ratingsKeras to Kubernetes: The Journey of a Machine Learning Model to Production Rating: 0 out of 5 stars0 ratingsMetaheuristics for String Problems in Bio-informatics Rating: 0 out of 5 stars0 ratingsDreamcast Architecture: Architecture of Consoles: A Practical Analysis, #9 Rating: 0 out of 5 stars0 ratingsMachine Learning for iOS Developers Rating: 0 out of 5 stars0 ratings
Technology & Engineering For You
A Night to Remember: The Sinking of the Titanic Rating: 4 out of 5 stars4/5The Art of War Rating: 4 out of 5 stars4/5Mother of God: An Extraordinary Journey into the Uncharted Tributaries of the Western Amazon Rating: 4 out of 5 stars4/5The 48 Laws of Power in Practice: The 3 Most Powerful Laws & The 4 Indispensable Power Principles Rating: 5 out of 5 stars5/5The Art of War Rating: 4 out of 5 stars4/5The Invisible Rainbow: A History of Electricity and Life Rating: 5 out of 5 stars5/5Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time Rating: 4 out of 5 stars4/5The Big Book of Hacks: 264 Amazing DIY Tech Projects Rating: 4 out of 5 stars4/5Vanderbilt: The Rise and Fall of an American Dynasty Rating: 4 out of 5 stars4/580/20 Principle: The Secret to Working Less and Making More Rating: 5 out of 5 stars5/5The Wuhan Cover-Up: And the Terrifying Bioweapons Arms Race Rating: 4 out of 5 stars4/5The Big Book of Maker Skills: Tools & Techniques for Building Great Tech Projects Rating: 4 out of 5 stars4/5Ultralearning: Master Hard Skills, Outsmart the Competition, and Accelerate Your Career Rating: 4 out of 5 stars4/5Death in Mud Lick: A Coal Country Fight against the Drug Companies That Delivered the Opioid Epidemic Rating: 4 out of 5 stars4/5Summary of Nicolas Cole's The Art and Business of Online Writing Rating: 4 out of 5 stars4/5The Systems Thinker: Essential Thinking Skills For Solving Problems, Managing Chaos, Rating: 4 out of 5 stars4/5Selfie: How We Became So Self-Obsessed and What It's Doing to Us Rating: 4 out of 5 stars4/5The Fast Track to Your Technician Class Ham Radio License: For Exams July 1, 2022 - June 30, 2026 Rating: 5 out of 5 stars5/5Understanding Media: The Extensions of Man Rating: 4 out of 5 stars4/5Broken Money: Why Our Financial System is Failing Us and How We Can Make it Better Rating: 5 out of 5 stars5/5Pilot's Handbook of Aeronautical Knowledge (Federal Aviation Administration) Rating: 4 out of 5 stars4/5Artificial Intelligence Revolution: How AI Will Change our Society, Economy, and Culture Rating: 5 out of 5 stars5/5A History of the American People Rating: 4 out of 5 stars4/5How to Disappear and Live Off the Grid: A CIA Insider's Guide Rating: 0 out of 5 stars0 ratingsLogic Pro X For Dummies Rating: 0 out of 5 stars0 ratingsSeeing Further: The Story of Science and the Royal Society Rating: 4 out of 5 stars4/5Ghost Rider: Travels on the Healing Road Rating: 4 out of 5 stars4/5The CIA Lockpicking Manual Rating: 5 out of 5 stars5/5
Related categories
Reviews for Architecture-Aware Optimization Strategies in Real-time Image Processing
0 ratings0 reviews
Book preview
Architecture-Aware Optimization Strategies in Real-time Image Processing - Chao Li
Preface
In the image processing field, a lot of applications require real-time execution in the domains of medical technology, robotics and transmission, etc. Recently, real-time image processing fields have made a lot of progress. Technological developments allow engineers to integrate more complex algorithms with large data volumes onto embedded systems, and produce series of new sophisticated electronic architectures at an affordable price. At the same time, industrial and academic researchers have proposed new methods and provided new tools in order to facilitate real-time image processing realization at different levels. It is necessary to perform a deepened educational and synthetic survey on this topic. We will present electronic platforms, methods, and strategies to reach this objective.
This book consists of seven chapters ranging from the fundamental conceptual introduction of real-time image processing to future perspectives in this area. We describe hardware architectures and different optimization strategies for real-time purposes. The latter consists of a survey of software and hardware co-design tools at different levels. Two real-time applications will be presented in detail in order to illustrate the proposed approaches.
The major originalities of this book include (1) algorithm architecture mapping: we select methods and tools that treat simultaneously the application and its electronic platform in order to perform fast and optimal design space exploration (DSE), (2) each approach will be illustrated by concrete examples and (3) two of the chosen algorithms have been only recently advanced in their domain.
This book is written primarily for those who are familiar with the basics of image processing and want to implement the target image processing design using different electronic platforms for computing acceleration. It accomplishes this by presenting the techniques and approaches step by step, the algorithm and architecture conjointly, and by notions of description and example illustration. This concerns both the software engineer and the hardware engineer.
This book will also be adequate for those who are familiar with programming and applying embedded systems to other problems and are considering image processing applications. Much of the focus and many of the examples are taken from image processing applications. Sufficient detail is given to make algorithms and their implementation clear.
Chao LI
Souleymane BALLA-ARABE
Fan YANG
August 2017
1
Introduction of Real-time Image Processing
1.1. General image processing presentation
The traditional view of an image derives heavily from experience in photography, television and the like. This means that an image is a two-dimensional (2D) structure, a representation and also a structure with meaning to a visual response system. This view of an image only accepts spatial variation (a static image). In parallel, a dynamic image has spatial and temporal variation. In most contexts, this is usually referred to as video. This more complex structure needs to be viewed as a sequence of images each representing a particular instance in time. On the other hand, an image can be formed by taking a sampling plane through that volume and so the variation in three dimensions is observed. This may be referred to as a volume image. An image linked to a volume that changes with time is a further possibility. This has particular significance in medical imaging applications [MYE 09].
Image processing is a method to convert an image into digital form and perform some operations on it in order to get an improved image or to extract some useful information from it. A digital image described in a 2D space is usually considered as 2D signals while applying already set signal methods to it. Image processing forms a core research area within engineering and computer science too.
Image processing is an enabling technology for a wide range of applications including remote sensing, security, image data sets, digital television, robotics and medical imaging, etc. The goal of this technology can be divided into three levels: low, medium and high. Low-level image processing techniques are mathematical or logical operators that perform simple processing tasks. This processing
level possesses both image_in and image_out. Medium-level image processing combines the simple low-level operators to perform feature extraction and pattern recognition functions. Using an image_in, this analysis
level produces measurements_out (parameters). High-level image processing uses combinations of medium-level functions to perform interpretation. This understanding
level treats measurements_in for high-level description_out. Usually, apart from these three levels, it is also necessary to apply preprocessing techniques that are designed to remove distortions introduced by sensors.
Figure 1.1. Overview of the typical image acquisition process (see [MOE 12])
Before any image processing can commence, an image must be captured by a camera and converted into a manageable entity. This is the image acquisition process (see Figure 1.1), which consists of three steps; energy reflected from the object of interest, an optical system that focuses on the energy and finally a sensor that measures the amount of energy. In order to capture an image, a camera requires some sort of measurable energy. The energy of interest in this context is light or electromagnetic waves. Each wave can have different wavelengths (or different energy levels or different frequencies). The human visual spectrum is in the range of approximately 400–700 nm.
After having illuminated the object of interest, the light reflected from the object now has to be captured by the camera composed of an optical system and an image sensor. The role of the first is to focus the light reflected from the object onto the second (a material sensitive to the reflected light). An image sensor consists of a 2D array of cells. Each of these cells is denoted a pixel and is capable of measuring the amount of incident light and convert that into a voltage, which in turn is converted into a digital number. The more incident light, the higher the voltage and the higher the digital number.
In order to transform the information from the sensor into an image, each cell content is now converted into a pixel value in the range: (0, 255). Such a value is interpreted as the amount of light hitting a cell. This is denoted the intensity of a pixel. It is visualized as a shade of gray denoted gray-level value ranging from black (0) to white (255). Standardly, a monochromatic, static image corresponds to a matrix of m rows and n columns. Therefore, the camera records m × n pixels of 8 bit values.
In order to capture a color image, the color camera must record three matrices of three primary colors red, green and blue. Recently, a lot of applications are realized using multispectral image processing, since the multispectral cameras are more available with reasonable prices. According to application needs, multispectral images are captured using different wavelengths (bands). They can be considered as a cube formed by 2D gray-level images. Figure 1.2 displays two typical test images in the area of image processing research. Figure 1.3 gives two multispectral cube examples; the right image is captured for a skin lesion assessment application.
Figure 1.2. Lena (gray-level image) and landscape (color image). For a color version of the figure, see www.iste.co.uk/li/image.zip
Figure 1.3. Two multispectral image cubes. For a color version of the figure, see www.iste.co.uk/li/image.zip
Certain tools are central to the processing of digital images. These include mathematical tools, statistical descriptions and manipulative tools. We can cite some more used such as convolution, filtering in spatial and frequency domains, morphological operations and image transforms, etc. The types of basic operations that can be applied to digital images to transform an input image A(m, n) into an output image B(m, n) (or another representation) can be classified into three categories:
– Point: the output value at a specific matrix coordinate is dependent only on the input value at that same coordinate. In this case, generic operation complexity per pixel is constant.
– Local: the output value at a specific coordinate is dependent on the input values in the neighborhood of that same coordinate. In most cases, the type of neighborhood is rectangular with 8-connected (3 × 3 mask) or 24-connected (5 × 5 mask pixels). The complexity per pixel is proportional to the square of neighborhood size.
– Global: the output value at a specific coordinate is dependent on all the values in the input image. The complexity per pixel is equal to N × N (image size = N).
The complexity per pixel of each operation type participates in the total complexity of an image processing chain for a target application. This total complexity per image decides the processing speed (time performance).
1.2. Real-time image processing
Real-time image processing is the subfield of image processing focused on producing and analyzing images in real time. Generally, a real-time system has the following three interpretations within different senses [KEH 06]:
– real time in the perceptual sense, which is used to describe the interaction between a human and a computer device for a near instantaneous response of the device to an input by a human user;
– real time in the software engineering sense, which is used to describe a concept of bounded response time in the computer device. With this constraint, the device must satisfy both the correctness of the outputs and their timeliness;
– real time in the signal processing sense, which is used to describe the constraint within which the computer device has to complete processing in the time available between successive input samples.
Since image processing is a subfield of signal processing, in this book we base the interpretation of real time
on the signal processing sense.
In order to satisfy the constraint of real time
, Kehtarnavaz [KEH 11] points out that the total instruction count of a real-time algorithm must be less than the number of instructions that can be executed between two consecutive samples
, and Ackenhusen et al. [ACK 99] describe the real-time processing
as a computation of a certain number of operations upon a required amount of input data within a specified interval of time
as well. Therefore, the key technique of real-time image processing is to ensure that the amount of time for completing all the requisite transferring and processing of image data is less than the allotted time for processing. For example, if the algorithm is aimed at an entire frame (a static image) and the frame frequency of the system is 30 frames per second (fps), the processing of a single frame should be finished during 33