Design Industrial Robot Applications From Perception To Motion Haui
Design Industrial Robot Applications From Perception To Motion Haui
Design Industrial Robot Applications From Perception To Motion Haui
Perception to Motion
Authorized
As the Sole Reseller in Southeast Asia for MathWorks Inc,
Distr ibuto
developer of the MATLAB® and Simulink® family of products since
1996, we provide organizations and businesses with a wide spectrum
of the best tools, products and services to facilitate innovations.
2
DATA QUADRANT AWARDS 2022
Machine Learning
Image
Artificial Wireless Computational Processing & Predictive Machine
Control Systems
Intelligence Systems Biology Computer Vision Maintenance Learning
Robotics
Robotics Power Electronics Internet of Power System Signal
Data Science Mechatronics
Control Design Things (IoT) Analysis & Design Processing
Automated Test & Embedded FPGA, ASIC & SoC Mixed Signal Enterprise &
Driving Systems Deep Learning Measurement Systems Development System IT Systems
5
Agenda
▪ Introduction
– Pick-and-Place Manipulation Application
▪ Trends in Industrial Robotics
– Landscape of AI adoption in Robotics
– Why Use AI for Robotics?
▪ Autonomous Robotics System Development
– How to get data for training & labeling
– AI modeling
– AI within entire systems
– Deployment
▪ Robotics and Controls Made Easier
▪ Resources for further learning
7
Autonomous Robot Development with MATLAB & Simulink
8
Trends in Industrial Robotics
Factories are:
• Incorporating more advanced systems
• Taking advantage of new technology
• Increasing connectivity between components
• Enhancing infrastructure to reach Industrial
IoT
Collaborative Robots
AI-based Robots
9
What we will discuss today
01
Advanced Robots in
Smart Factory Autonomous Robotics
02
System Development
• Platform Design & Simulation
Use case: Pick-and- • Autonomous Application
03 • Hardware connection & deployment
Place Manipulation
Application using
MATLAB and Simulink 04
Concluding Remarks
10
Smart Factory Concept: Asset, Operation, & Workforce Optimization
IoT
Factory Cloud + AI
Local 5G Operation
Big Data
Process Management・Quality Increase Management
Robot Optimization
✔?
✘?
OT
Worker AMR
Optimization
Autonomy Monitor
Update
11
Smart Factory Concept: Asset, Operation, & Workforce Optimization
What we are talking about today
IoT
Factory Cloud + AI
Local 5G Operation
Big Data
Process Management・Quality Increase Management
Robot Optimization
✔?
✘?
OT
Worker AMR
Optimization
Autonomy Monitor
Update
12
Core of autonomous system design
13 13
Core of autonomous system design
Computer vision /
Mission planning Motion control
Image processing
14
But there are challenges…..
15
Enables
Robotics AI
Full Autonomy
• AI in robots increase the capabilities
High Autonomy AI-powered robots of robots in performing complex
tasks
Conditional
Autonomy
• Cobots combine the individual strengths
Cobots of humans and robots for flexible and
Partial Autonomy
agile manufacturing
16
There are two ways to get a robot to do what you want
Traditional Programming
Data
Output
Program
17
There are two ways to get a robot to do what you want
Data
Program
Output
18
There are two ways to get a robot to do what you want
Data
Model
Output
19
There are two ways to get a robot to do what you want
Machine Learning
Artificial Intelligence
Model
Reinforcement
Learning Output
20
Autonomous Robot Development with MATLAB & Simulink
Autonomous
Platform C/C++ Code
Applications
Perception Environment
ROS Node
GPU Code
Planning
Robot Model
Control
Development Environment
Design Simulate Analyze Implement Test
21
Autonomous Robot Development with MATLAB & Simulink
Autonomous
Platform C/C++ Code
Applications
Perception Environment
ROS Node
GPU Code
Planning
Robot Model
Control
Development Environment
Design Simulate Analyze Implement Test
22
Physical & Kinematic Modeling
Creating a physical model Automatic import from CAD Tools URDF / Robot Library
%% Import robot from URDF
smimport('GEN3_FOR_URDF_ARM_V11_MOD.urdf');
CAD Model
Multibody
Model
%% Use robot library
Robot = loadrobot('abbYuMi’);
Show(robot);
24
Landscape of AI adoption in Robotics
❑Computer Vision
Image Industrial Inspection
Pixels Deep Learning
Abnormal with Robots / UAVs
25
Landscape of AI adoption in Robotics
❑Point Cloud Processing
Point Pose Estimation
Deep Learning Structure Localization
Clouds for Robots
26
Landscape of AI adoption in Robotics
Sense & Plan &
Control
Perceive Decide
Voice Command to
Robots
Industrial Inspection
with Robots / UAVs
Pose Estimation
Localization
for Robots
27
Landscape of AI adoption in Robotics
Sense & Plan &
Control
Perceive Decide
Voice Command to
Robots
Pose Estimation
Localization
for Robots
28
Landscape of AI adoption in Robotics
Reinforcement Learning
Look-up table
Control
29
Landscape of AI adoption in Robotics
Exploring
Make Decision
Reinforcement
Learning for
Making Decision &
Control
31
Case study: developing pick & place robot arm
32
AI-driven robotics system
design workflow
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
AI-driven robotics system
design workflow
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
How to get data for training?
Data
1
Preparation • •
AI
• •
2 • •
Modeling
Simulation
3
and Test
4 Deployment
Data
1
Preparation Semantic Segmentation from
Unreal for UAV
AI UAV Toolbox
2
Modeling Simulink Lidar Sensor Model: Simulate
lidar sensor and generate
Simulation
3 point cloud data
and Test
Lidar Toolbox Gazebo Co-simulation with a
Pretrained Deep Learning
4 Deployment Model to Detect Recyclable
Parts
Robotics System Toolbox
ROS Toolbox
36
Automated labeling apps save you weeks to months
AI
2
Modeling
Simulation
3
and Test
4 Deployment
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
Start with a complete set of algorithms and pre-built models
Object Detection
with YOLOv4
Instance Segmentation
with Mask R-CNN
Data
1
Preparation
Semantic Segmentation
2
AI with U-Net
Modeling
Simulation
3
and Test
4 Deployment
39
AI modeling Apps automate training, tuning, visualization...
Data
1
Preparation
Deep Network Designer app to
AI build, visualize, and edit deep Classification Learner app to
2
Modeling learning networks. try different classifiers and find Experiment Manager app to
the best fit for data sets. run deep learning experiments
Simulation to train networks and compare
3
and Test results.
4 Deployment
40
Access AI models from the broader AI community
TensorFlow Importer
Data
1
Preparation Caffe Importer
AI
2
Modeling
Simulation
3
and Test
4 Deployment Other
Frameworks
41
AI-driven robotics system
design workflow
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
Use AI within entire systems Robot Platform
(Simulation or
Real HW)
Sensing
Data AI model
1
Preparation
Perception
AI
2
Modeling
Simulation
3
and Test
Control
4 Deployment
Planning
43
Use AI within entire systems
Data AI model
1
Preparation
AI
2
Modeling
Simulation
3
and Test
4 Deployment
44
Use AI within entire systems
Data AI model
1
Preparation
AI
2
Modeling
Simulation
3
and Test
4 Deployment
45
AI-driven robotics system
design workflow
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
AI models are useful everywhere
Data
1
Preparation
AI
2
Modeling
Simulation
3
and Test
4 Deployment
47
Deploy to any processor with zero coding errors
CPU
Data
1
Preparation Code
Generation GPU
AI
2
Modeling
Simulation
3
and Test
4 Deployment FPGA
48
Deploy to Jetson device as CUDA ROS node
Other ROS
nodes
AI Generation
2
Modeling
Simulation
3
and Test
4 Deployment
NVIDIA ® Jetson™
49
Key Takeaways
50
Why Use AI for Robotics?
51
Why MATLAB and Simulink for AI in Robotics?
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
▪ Create more and ▪ Use a compete set of ▪ Integrate AI into ▪ Make deployment
better data sets AI algorithms and pre- system-wide context, of AI models easy
▪ Speed up data built models simulate, and verify to accelerate AI
labeling effectiveness system
52
Get Started with AI in MATLAB
No download, no installation, just your browser and you…
53
Industry application examples – User Stories / Presentation
Robotic Arm for Removing Verification of Picking System for Autonomous Humanoid Robot with
Nuclear Fuel Debris Industrial Robot Using ROS and Model-Based Design
Mitsubishi Heavy Industries MATLAB German Aerospace Center (DLR)
Yaskawa Electric Corp. Robotics and Mechatronics Center
54
54
Resources
Examples Videos Webinars
Reinforcement Learning
Tech Talk Deploy Deep Neural Networks
55
Line official to get a trial license...