Automatic mixed precision and distributed data-parallel training for satellite image classification using CNN

Yohanes Nuwara; Wei K. Wong; Filbert Hilman Juwono; Gregory Ollivierre

doi:10.1117/12.2679828

7 June 2023 Automatic mixed precision and distributed data-parallel training for satellite image classification using CNN

Yohanes Nuwara, Wei K. Wong, Filbert Hilman Juwono, Gregory Ollivierre

Author Affiliations +

Proceedings Volume 12701, Fifteenth International Conference on Machine Vision (ICMV 2022); 127011O (2023) https://doi.org/10.1117/12.2679828
Event: Fifteenth International Conference on Machine Vision (ICMV 2022), 2022, Rome, Italy

Abstract

Deep learning models for computer vision in remote sensing such as Convolutional Neural Network (CNN) has benefited acceleration from the usage of multiple CPUs and GPUs. There are several ways to make the training stage more effective in terms of utilizing multiple cores at the same time by processing different image mini-batches with a duplicated model called Distributed Data Parallelization (DDP) and computing the parameters in a lower precision floating-point number called Automatic Mixed Precision (AMP). We would like to investigate the impact of DDP and AMP training modes on the overall utilization and memory consumption of CPU and GPU, as well as the accuracy of a CNN model. The study is performed on the EuroSAT dataset, a Sentinel-2-based benchmark satellite image dataset for image classification of land covers. We compare training using 1 CPU, using DDP, and using both DDP and AMP over 100 epochs using ResNet-18 architecture. The hardware that we used are Intel Xeon Silver 4116 CPU with 24 cores and an NVIDIA v100 GPU. We find that although parallelization of CPUs or DDP takes less time to train on the images, it can take 50 MB more memory than using only a single CPU. The combination of DDP and AMP can release memory up to 160 MB and reduce computation time by 20 seconds. The test accuracy is slightly higher for both DDP and DDP-AMP at 90.61% and 90.77% respectively than without DDP and AMP at 89.84%. Hence, training using Distributed Data Parallelization (DDP) and Automatic Mixed Precision (AMP) has more benefits in terms of lower GPU memory consumption, faster training execution time, faster convergence towards solutions, and finally, higher accuracy.

Citation Download Citation

Yohanes Nuwara, Wei K. Wong, Filbert Hilman Juwono, and Gregory Ollivierre "Automatic mixed precision and distributed data-parallel training for satellite image classification using CNN", Proc. SPIE 12701, Fifteenth International Conference on Machine Vision (ICMV 2022), 127011O (7 June 2023); https://doi.org/10.1117/12.2679828

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Education and training

Amplifiers

Deep learning

Satellites

Earth observing sensors

Satellite imaging

Data modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years