Technology Challenges to IC

Industry for Next Decade

聯發科技
陸國宏博士
Dr. K. Lawrence Loh
Corporate Senior VP
MediaTek Inc.
 Chips Everywhere to Enable ICT Industry
Metaverse Mobile Comm Satellite Comm Environment

Smart Home Automotive Medical Education

Security Industrial Transport Finance

2022 VLSI-TSA and VLSI-DAT 2

Leading SoC Technologies : Enhance and Enrich Everyone’s Life

AI

Compute

Modem Multimedia

RF &
Connectivity Analog
Software
& System

2022 VLSI-TSA and VLSI-DAT 3

Semiconductor Revenue – To Reach $1T in Next Decade
Global Semiconductor Revenue 1,000
$Bn 1,000
Revenue ($Bn)
800

583
600
466

400
298
204
200
51 Projection

0
1990

1995

2000

2005

2010

2015

2020
2021

2030
2022 VLSI-TSA and VLSI-DAT Source: Gartner, WSTS 4
 Growth Opportunities Everywhere from Cloud to Edge

10010101010001010101010100001010010101000
Computing 10100101010101000010111010101011101010101 CLOUD
10110010101110101010100101010001011101011
CPU APU 01010010100101111000110001001010100010100
10010101110101010010101000101011101011010
AI Communication
10010101010001001010100010101010100001010
AI
Camera Audio10100101010111010101010000101011101010101
10010101000110010101110101010101011101011 Computing &
01010010100101111000110001010101010000100
AR/VR/XR 10010101110010101000101010101011101011010 Communication
5G/B5G, WIFI 6/7, xDSL, PON,
Ethernet, etc. Networking/Storage

Edge & End Devices Connectivity Data Centers

2022 VLSI-TSA and VLSI-DAT 5

 Challenges ahead : Data Deluge
Volume of data created and replicated worldwide
Worldwide data created is
200
181
180 overwhelming – Demanding not only
Data Volume in Zettabytes

160

140
147
for more storage, but also for:
120
120 - More Computing performance to
97
100

80
79 process data
64.2
60
41
- More Communications bandwidth
40 33
26
20 12.5 15.5
18 to move data
6.5 9
2 5
0
2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

2025

2022 VLSI-TSA and VLSI-DAT Source: IDC 6

 Challenges ahead : Demands in Communication Bandwidth
Evolution of Communication Standards
108 Ethernet
10TbE, CPO, Coherent Optics
107
1.6TbE WiFi
106 400GbE Converging to cellular? High QAM
Data Rate (Mbps)

100GbE High BW, mMIMO

105
104
10GbE 11ay WiFi7 Cellular
11ad
1GbE WiFi6 B5G&6G: ~100Gbps, mMIMO
103 5G
100MbE WiFi5 (mmWave) >7GHz, Sub-THz, SIC, NTN
102 5G Sensing, AI-Assisted, Open RAN
10Mbps 11n (Sub-6GHz)
11a/g
101
802.11 11b 4G
100
10-1
3G
1G 2G
10-2
1980 1990 2000 2010 2020 2030

2022 VLSI-TSA and VLSI-DAT 7

 Challenges ahead : Demands for High Efficiency
Coverage NTN Non-Terrestrial Network
Space
Coverage - Covering Broader
Sea Sky Space
- Accessing Faster
- Operating Efficiently
Extreme
Coverage

B5G & 6G
High
Quality ~100Gbps

5G Extreme
Sensing Data Rate

Power 3G 4G High Data Rate / Capacity

Efficiency
3GHz 10GHz 30GHz 100GHz 300GHz 1THz
Power Efficiency Spectrum Efficiency Frequency
2022 VLSI-TSA and VLSI-DAT Ref: NTT Docomo, 2020 Symposia on VLSI Technology and Circuits; 6G Flagship, Univ. Oulu 8
 Challenges ahead : Design Complexity (Cellular Modem)
# of Pages of RF Specifications Modem Gate Count & Area Cost
14000 10000
1G
240x Gate Count 6G: 2000x ?
12000 13000 Pages
2G from 2G to 5G
→ 1.3m Thick 1000
10000 WCDMA 5G
(mmWave)
TDSCMA Gate
8000 5G Count
C2K (Sub-6GHz)
100
6000 LTE/LTE-A/LTE-Pro
4G
NR
4000 Area
10
3G
2000 5nm
2G 16/12nm
0 1 28nm
2005 2010 2015 2020 2025 2030
2022 VLSI-TSA and VLSI-DAT 9
Source: www.3gpp.org/specifications
Challenges ahead : Demands in Computing
Computing Capacity used in Training AI Systems 3.4-month doubling
100
10
1

AI Training 0.1
in Modern Era 0.01
0.001
0.0001
0.00001
0.000001
0.0000001
1960 70 80 90 2000 10 20 Days spent calculating at
one petaflop/s computer,
log scale

2022 VLSI-TSA and VLSI-DAT Source: OpenAI, The Economist 10

Moore’s Law – Push Hands of Modern Technologies
a Transistors (K)
- Moore’s Law still (sort of)
continues as of today, but
- Single Thread performance
Single Thread Performance
improves slowly
(SpecINT x 103) - Frequency and Power walls
Frequency (MHz) ended Denard’s Scaling
- Number of logical cores:
Typical Power (Watt)
hitting the wall due to
Number of thermal, packaging &
Logical Cores
Interconnect constraints

2022 VLSI-TSA and VLSI-DAT Source: : K. Rupp, 2021 ; Original data up to 2010 collected by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, and C. Batten; 11
New data collected for 2010-2021 by K. Rupp
Advanced Technology - For Lower Power
Industry’s Expectation
from Node to Node 100%

Power 80%

-30%
60%
Performance
+20% 40%

Area 20%

-50%
0%
Cost 16/14nm 10nm 7nm 5nm 3nm 2nm 1.5nm 1nm
Expectation Practical Target
-30%
2022 VLSI-TSA and VLSI-DAT * Power reduction at same Speed 12
Advanced Technology - For Higher Performance
Industry’s Expectation
from Node to Node 400%

Power
-30% 300%

Performance
+20%
200%

Area
-50%
100%
Cost 16/14nm 10nm 7nm 5nm 3nm 2nm 1.5nm 1nm

-30% Expectation Practical Target

2022 VLSI-TSA and VLSI-DAT * Performance improvement at same power 13

Advanced Technology - For Smaller Die Size
Industry’s Expectation
from Node to Node 100%

Power 80%
-30%
60%
Performance
+20% 40%

Area 20%
-50%
0%
Cost 16/14nm 10nm 7nm 5nm 3nm 2nm 1.5nm 1nm

-30% Expectation Practical Target

2022 VLSI-TSA and VLSI-DAT * Area ratios refer to the first node 14
Advanced Technology - For Lower Cost
Industry’s Expectation
from Node to Node 100%

Power 80%
-30%
60%
Performance
+20% 40%

Area 20%
-50%
0%
Cost 16/14nm 10nm 7nm 5nm 3nm 2nm 1.5nm 1nm

-30% Expectation Practical Target

2022 VLSI-TSA and VLSI-DAT * Transistor cost Area ratios refer to the first node 15
Computing and Communication Energy Consumptions
New Trajectories Required to Sustain
1.E+22
Computation trajectory while
improving the device energy
World's energy production performance.
Energy in J/year

1.E+20
Communication trajectory
1 ZIPS while improving the device
energy performance.
1aJ/bit
0.01 ZIPS
1.E+18
10aJ/bit Market-dynamics limited
scenario –
100aJ/bit
Stopping further increase in
the world’s computing and
communication capacity
1.E+16 Resulting in flattening curve
2010 2020 2030 2040
2022 VLSI-TSA and VLSI-DAT *Decadal Plan model (validated by independent estimates) Source: SRC, Decadal Plan for Semiconductors, Full Report 16
**Decadal Plan model (validation on-going)
 A Small Break
Challenges of Technology Development – for IC industry
• Energy efficiency should be always considered/designed on top priorities,
followed by performance (speed/capacity) and costs (i.e. PPC)
• Moore’s Laws used to bring all PPC benefits - need extra efforts to
maintain the trends
Examples – Foundation/Critical Technologies for Computing and
Communications
• Computing Architectures – Near Memory and In-Memory Computing
• Digital (wired) Interconnects – to connect Digital-Digital worlds
• Data Converters (A/Ds and D/As) – to connect Analog-Digital worlds

2022 VLSI-TSA and VLSI-DAT 17

Near-Memory/In-Memory Compute for Machine Learning
Processor Memory
Moving Data is more expensive than Von Neumann
Computing Data ALU Compute
Bottleneck Traditional
Ctrl “Von Neumann Architecture”

ALU

Compute
Computing Near-Memory
Ctrl

C C C C
ALU
C C C C
C C C C
Ctrl C C C C

Computing in-Memory
ALU C C C C
C C C C
C C C C
Ctrl C C C C

2022 VLSI-TSA and VLSI-DAT Ref: : Y.-H. Chen, et al. “Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks,” ISCA, 2016 18
A. Sebastian, “Memory devices and applications for in-memory computing”, Nature nanotechnology, July. 2020.
 Performance vs. power of AI accelerators
107 AI chips with Computing-in-Memory
Peak Performance (GOPs/sec)

architecture have potential

to achieve better power efficiency.
106 (10s~100s TeraOPs/W)

Still need to handle technical

105 issues like:
• Area overhead
• NN model migration
104 • Analog CIM
• Process/Device variations
• ADC requirement
103
Peak performance vs. power scatter plot
of publicly announced AI accelerators and
102 processors.
10-2 10-1 100 101 102 103 104 = AI Chips with Computing-in-Memory
Peak Power (W) = AI Chips by other architecture

2022 VLSI-TSA and VLSI-DAT REF: A. Reuther, et al, "Survey of Machine Learning Accelerators," 2021 IEEE HPEC, 2021 19
 Connecting Digital-Digital and Digital-Analog Worlds
Wireless
Camera Mic A/D
Network Computing Storage
D/A A/D
1001010100011001

Sensor 1001010111001010100010101 10010101000110010101110101010101011101011

M-LINK, UCI-e, HBM, etc. Chiplet

1001010111001010100010101 10010101000110010101110101010101011101011 Die-to-Die
SerDes links – Ethernet, PCI-E On Board
1001010100011001 D/A Wired A/D
Display Fiber and Copper Off Board
D/A

Actuator Speaker - Digital (wired) Interconnect (DDR, SerDes) - Between digital worlds
- Data Converters (A/D&D/A)
- Between analog (physical) and digital worlds
- Digital Communications (wired and wireless)
Edge / Devices Data Communication Data Centers / Infrastructures

2022 VLSI-TSA and VLSI-DAT 20

 Interconnect Technologies to Connect Digital Worlds
FoM = BW Density  Energy Efficiency (Gbps/mm)  1/(pJ/bit)
106 In package On board Off board
105
M-Link 3.0
104 M-Link 2.0
112G XSR
FoM

103 HBM
224G LR
(Target)
102 SiPh and CPO (Target)
112G LR
101
Optics on
LVDS PCIe Board
100 Optical 100GbE SR4
Backplane 100GbE CPRI QSFP28
100 GBASE SR4
10-1
10-4 10-3 10-2 10-1 100 101 102
Source: DARPA, Photonics in the
2022 VLSI-TSA and VLSI-DAT Maximum Interconnect Distance (m) Package for Extreme Scalability (PIPES) 21
MediaTek
Co-Packaged Optics to Reach High Bandwidth Efficiencies

Off-board Modules
DSP on-chip ➢ Long-range (LR) backplanes up to 1m
to boost ➢ Retimer maybe required
performance ➢ Limited speed
Challenges ➢ Poor power efficiency
– Thermo
– Optics expertise
– Mechanical
alignments
Direct Optical Drive
➢ Short Distance (~mm)
➢ Higher Speed
➢ Better Power Efficiency
2022 VLSI-TSA and VLSI-DAT 22
Digital Interconnects for Chiplets with Advanced Packaging

Homogeneous
Integration

Heterogeneous
Integration

On-Board Interconnect

In-Package Interconnect

2022 VLSI-TSA and VLSI-DAT 23

 2D/2.5D/3D Integrations - For Power and Performance
Architecture Chip Stacking / Advanced Packaging

Traditional
Long Distance

Onchip
Memory
3D
(SOC)
Issues:
Signal Integrity , Power Delivery,
2D Whole system design, Simulation,
2.5D Heterogeneous material interface
Thermal/Cooling, Physical effect
Embedded Bridge Manufactory variations, Logistics, etc.

2022 VLSI-TSA and VLSI-DAT REF: Douglas Yu, TSMC Packaging Technologies for Chiplets and 3D, HotChips, 2021 24
 Connecting Digital-Digital and Digital-Analog Worlds
Wireless
Camera Mic A/D
Network Computing Storage
D/A A/D
1001010100011001

Sensor 1001010111001010100010101 10010101000110010101110101010101011101011

M-LINK, UCI-e, HBM, etc. Chiplet

1001010111001010100010101 10010101000110010101110101010101011101011 Die-to-Die
SerDes links – Ethernet, PCI-E On Board
1001010100011001 D/A Wired A/D
Display Fiber and Copper Off Board
D/A
- Digital (wired) Interconnect (DDR, SerDes) - Between digital worlds
Actuator Speaker
- Data Converters (A/D&D/A)
- Between analog (physical) and digital worlds
- Digital Communications (wired and wireless)
Edge / Devices Data Communication Data Centers / Infrastructures

2022 VLSI-TSA and VLSI-DAT 25

 Data Converters
Data Converters (A/D
(A/D &
& D/A)
D/A)
Between Analog
Between (Physical)
Analog and
(Physical) and Digital
Digital WorldsWorlds

22
Sensor Interface ADC/DACs
20
(audio, biomedical)
18 Basic Comm.
Resolution (Bits)

16 (xDSL, 3G)
Wireless Comm.
14 (LTE/5G, WiFi, mmWave)

12 Infrastructure
(OpenRAN)
10 OthersOthers
Target – Highspeed
High Performance
8 Others Energy-efficient
Others
6 Multimedia SerDes
(video) (112G)
4
[B. Murmann, ADC Performance Survey 1997-2021]
2
1kHz 10k 100k 1M 10M 100M 1G 10G 100G

2022 VLSI-TSA and VLSI-DAT Sampling Rate (Hz) 26

 Data Converters (A/D & D/A)
Between Analog (Physical) and Digital Worlds
Schreier FoM = SNDR + 10 log10(Bandwidth / Power)
190
ADCs
180
Figure of Merit (dB)

SAR
(Biomedical,
170 Time-Interleaving
WiFi, Video)
(WiFi6/7, 5G, SerDes,
Infrastructure/OpenRAN)
160 Target - Highspeed
CT-DSM High performance
Energy-efficient
(Audio, xDSL,3G,LTE)
150 Pipeline
(10GBASE-T)

140 Flash
(mmWave)
[B. Murmann, ADC Performance Survey 1997-2021]
130
1kHz 10k 100k 1M 10M 100M 1G 10G 100G

2022 VLSI-TSA and VLSI-DAT Sampling Rate (Hz) 27

 Digital Massive MIMO – Enabling Future Wireless
OpenRAN (B5G/6G)
Digital Massive MIMO

CU DU RU 32T32R

16T16R

4T4R

2022 VLSI-TSA and VLSI-DAT 28

All-Digital RF Transceivers – Enabled by Data Converters

• Sub-7GHz Direct RF Sampling - All-

digital 4T/4R/1FB RF Transceivers
• Key Technology to enable Full-DIGITAL
SerDes massive MIMO systems (for B5G and
Interface 6G) : Multiple Beam Forming/Beam
Steering
• Four Rx 12b/16GSPS A/Ds
• Four Tx 14b/16GSPS D/As
• One ORx/Feedback 12b/16GSPS A/D
• Multiple-Band DDC/DUC

2022 VLSI-TSA and VLSI-DAT 29

Process Consideration for A/D and D/A Design
SNDR=36 to 47 dB 6~8 bits

SNDR=47.1 to 59 dB 8~10 bits

SNDR=59.1 to 72 dB 10~12 bits

Despite transistors not

65n 32n 28n getting much faster, A/Ds
40/45n 16n
(and D/As) have been
7n
breaking speed limits!

2022 VLSI-TSA and VLSI-DAT Source: Gabriele Manganaro, MediaTek, “Ultra-High-Data-Rate ADCs and DACs: Architectures and Implementations”, ISSCC 2022 30
 Summary
• Overwhelming data generations result in high demands of computing &
communication capacity
• Moore’s Law may be continuing, BUT
• Current trends of CMOS scaling for power and performance cannot sustain future energy
consumption trends
• Cost trends for new technologies may become prohibitedly high - causing inflations?
• Chiplets and advanced packaging (2.5D/3D) – essential to keep technology going
• Capacity of computing and communication would be limited by the world’s
energy production – human technology could be stalled – if no new (green)
energy can be found
• Key technologies introduced – for energy efficiencies
• Computing Architectures
• Digital (wired) Connectivity – Chiplets, advanced packages and CPO
• Data Converters – Enabling Future Wireless
2022 VLSI-TSA and VLSI-DAT 31
Acknowledgement
Special Thanks to Dr. Bor-Sung Liang
& Dr. Zheng Zeng, Yunshiang Shu, Gabriele Manganaro, Tamer Ali, CM Hung and Jon Strange

2022 VLSI-TSA and VLSI-DAT 32