PERFORMANCE ENHANCEMENT OF CUDA APPLICATIONS BY OVERLAPPING DATA TRANSFER AND KERNEL EXECUTION

K.  Raju; Niranjan  N Chiplunkar

doi:10.23743/acs-2021-17

Open full text

pdf

Published: Sep 30, 2021

DOI: https://doi.org/10.23743/acs-2021-17

Issue Vol. 17 No. 3 (2021)

Articles

PERFORMANCE ENHANCEMENT OF CUDA APPLICATIONS BY OVERLAPPING DATA TRANSFER AND KERNEL EXECUTION
K. Raju, Niranjan N Chiplunkar

5-18
BREAST CANCER DIAGNOSIS USING WRAPPER-BASED FEATURE SELECTION AND ARTIFICIAL NEURAL NETWORK
Nawazish NAVEED, Hayan T. MADHLOOM, Mohd Shahid HUSAIN

19-30
APPLICATION OF A FUZZY CONTROLLER IN THE PROCESS OF AUTOMATED POLYETHYLENE FILM THICKNESS CONTROL
Andrzej Jardzioch, Wioletta Marczak

31-46
OPTIMAL SLIDING MODE CONTROLLER DESIGN BASED ON WHALE OPTIMIZATION ALGORITHM FOR LOWER LIMB REHABILITATION ROBOT
Noor SABAH, Ekhlas HAMEED, Muayed S AL-HUSEINY

47-59
BACKWARD MOTION PLANNING AND CONTROL OF MULTIPLE MOBILE ROBOTS MOVING IN TIGHTLY COUPLED FORMATIONS
Kuppan Chetty RAMANATHAN, Manju MOHAN, Joshuva AROCKIA DHANRAJ

60-72
RGB-D FACE RECOGNITION USING LBP-DCT ALGORITHM
Sunil Kumar B L, Sharmila Kumari M

73-81
IMPLEMENTATION OF DYNAMIC AND FAST MINING ALGORITHMS ON INCREMENTAL DATASETS TO DISCOVER QUALITATIVE RULES
Pannangi Naresh, R. Suguna

82-91
INNOVATIVE DEVICE FOR TENSILE STRENGTH TESTING OF WELDED JOINTS: 3D MODELLING, FEM SIMULATION AND EXPERIMENTAL VALIDATION OF TEST RIG – A CASE STUDY
Mateusz Sawa, Mirosław Szala, Weronika Henzler

92-105

DOI

https://doi.org/10.23743/acs-2021-17

Authors

K. Raju

rajuk@nitte.edu.in

Department of CSE, NMAM Institute of Technology, Nitte, India

Niranjan N Chiplunkar

nchiplunkar@nitte.edu.in

Department of CSE, NMAM Institute of Technology, Nitte, India

Abstract

The CPU-GPU combination is a widely used heterogeneous computing system in which the CPU and GPU have different address spaces. Since the GPU cannot directly access the CPU memory, prior to invoking the GPU function the input data must be available on the GPU memory. On completion of GPU function, the results of computation are transferred to CPU memory. The CPU-GPU data transfer happens through PCI-Express bus. The PCI-E bandwidth is much lesser than that of GPU memory. The speed at which the data is transferred is limited by the PCI-E bandwidth. Hence, the PCI-E acts as a performance bottleneck. In this paper two approaches are discussed to minimize the overhead of data transfer, namely, performing the data transfer while the GPU function is being executed and reducing the amount of data to be transferred to GPU. The effectiveness of these approaches on the execution time of a set of CUDA applications is realized using CUDA streams. The results of our experiments show that the execution time of applications can be minimized with the proposed approaches.

Keywords:

CPU-GPU, High-performance computing, Kernel, Data transfer, CUDA streams

References

Antoniadis, N., & Sifaleras, A. (2017). A hybrid CPU-GPU parallelization scheme of variable neighborhood search for inventory optimization problems. Electronic Notes in Discrete Mathematics, 58, 47–54. https://doi.org/10.1016/j.endm.2017.03.007 DOI: https://doi.org/10.1016/j.endm.2017.03.007

Dhake, A.A., & Walunj, S.M. (2019). Transfer Time Optimization Between CPU and GPU for Virus Signature Scanning. In A. Luhach, D. Jat, K. Hawari, X.Z. Gao & P. Lingras (Eds.), Advanced Informatics for Computing Research. ICAICR 2019. Communications in Computer and Information Science (vol. 1076, pp. 70–78). Springer Singapore. https://doi.org/https://doi.org/10.1007/978-981-15-0111-1_6 DOI: https://doi.org/10.1007/978-981-15-0111-1_6

Fang, J., Chen, H., & Mao, J. (2018). Understanding data partition for applications on CPU-GPU integrated processors. In Communications in Computer and Information Science (vol. 747). Springer Singapore. https://doi.org/10.1007/978-981-10-8890-2_32 DOI: https://doi.org/10.1007/978-981-10-8890-2_32

Fu, C., Wang, Z., & Zhai, Y. (2017). A CPU-GPU Data Transfer Optimization Approach Based on Code Migration and Merging. Proceedings - 2017 16th International Symposium on Distributed Computing and Applications to Business, Engineering and Science, DCABES 2017, 2018-Septe (pp. 23–26). IEEE. https://doi.org/10.1109/DCABES.2017.13 DOI: https://doi.org/10.1109/DCABES.2017.13

Gowanlock, M., & Karsin, B. (2019). A hybrid CPU/GPU approach for optimizing sorting throughput. Parallel Computing, 85, 45–55. https://doi.org/10.1016/j.parco.2019.01.004 DOI: https://doi.org/10.1016/j.parco.2019.01.004

Gregg, C., & Hazelwood, K. (2011). Where is the Data ? Why You Cannot Debate CPU vs. GPU Performance Without the Answer. IEEE International Symposium on Performance Analysis of Systems and Software. (pp. 134–144). IEEE. https://doi.org/10.1109/ISPASS.2011.5762730 DOI: https://doi.org/10.1109/ISPASS.2011.5762730

Hascoet, T., Zhuang, W., Febvre, Q., Ariki, Y., & Takiguchi, T. (2019). Reducing the Memory Cost of Training Convolutional Neural Networks by CPU Offloading. Journal of Software Engineering and Applications, 12(08), 307–320. https://doi.org/10.4236/jsea.2019.128019 DOI: https://doi.org/10.4236/jsea.2019.128019

Huang, W., Yu, L., Ye, M., Chen, T., & Hu, T. (2012). A CPU-GPGPU scheduler based on data transmission bandwidth of workload. Parallel and Distributed Computing, Applications and Technologies, PDCAT Proceedings (pp. 610–613). IEEE. https://doi.org/10.1109/PDCAT.2012.15 DOI: https://doi.org/10.1109/PDCAT.2012.15

Lázaro-Muñoz, A.J., González-Linares, J.M., Gómez-Luna, J., & Guil, N. (2017). A tasks reordering model to reduce transfers overhead on GPUs. Journal of Parallel and Distributed Computing, 109, 258–271. https://doi.org/10.1016/j.jpdc.2017.06.015 DOI: https://doi.org/10.1016/j.jpdc.2017.06.015

Lee, C., Woo, W.R., & Gaudiot, J. (2014). Boosting CUDA Applications with CPU – GPU Hybrid Computing. International Journal of Parallel Programming, 42, 384–404. https://doi.org/10.1007/s10766-013-0252-y DOI: https://doi.org/10.1007/s10766-013-0252-y

Lee, J., Samadi, M., Park, Y., & Mahlke, S. (2015). SKMD: Single kernel on multiple devices for transparent CPU-GPU collaboration. ACM Transactions on Computer Systems, 33(3). https://doi.org/10.1145/2798725 DOI: https://doi.org/10.1145/2798725

Li, T., Dong, Q., Wang, Y., Gong, X., & Yang, Y. (2017). Dual buffer rotation four-stage pipeline for CPU – GPU cooperative computing. Soft Computing, 23, 859–869. https://doi.org/10.1007/s00500-017-2795-0 DOI: https://doi.org/10.1007/s00500-017-2795-0

Luley, R.S., & Qiu, Q. (2016). Effective utilization of CUDA hyper-Q for improved power and performance efficiency. Proceedings – 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016 (pp. 1160–1169). IEEE. https://doi.org/10.1109/IPDPSW.2016.154 DOI: https://doi.org/10.1109/IPDPSW.2016.154

Lutz, C., Breß, S., Zeuch, S., Rabl, T., & Markl, V. (2020). Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects. Proceedings of the ACM SIGMOD International Conference on Management of Data (pp. 1633–1649). ACM Digital Library. https://doi.org/10.1145/3318464.3389705 DOI: https://doi.org/10.1145/3318464.3389705

NVIDIA TITAN V. (n.d.). NVIDIA Corporation. Retrieved May 8, 2021 from https://www.nvidia.com NVIDIA. (2015). CUDA C Programming Guide v 9.1. NVIDIA.

Pandit, P., & Govindarajan, R. (2014). Fluidic kernels: Cooperative execution of openCL programs on multiple heterogeneous devices. Proceedings of the 12th ACM/IEEE International Symposium on Code Generation and Optimization, CGO 2014 (pp. 273–283). ACM Digital Library. https://doi.org/10.1145/2544137.2544163 DOI: https://doi.org/10.1145/2544137.2544163

Patil, S.V., & Kulkarni, D.B. (2021). Data transfer optimization in CPU/GPGPU Communication. Turkish Journal of Computer and Mathematics Education, 12(13), 1920–1923.

Piao, X., Kim, C., Oh, Y., Li, H., Kim, J., Kim, H., & Lee, J.W. (2015). JAWS: A JavaScript framework for adaptive CPU-GPU work sharing. Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP, 2015-Janua (pp. 251–252). ACM Digital Library. https://doi.org/10.1145/2688500.2688525 DOI: https://doi.org/10.1145/2858788.2688525

Raju, K., & Chiplunkar, N.N. (2018). A survey on techniques for cooperative CPU-GPU computing. Sustainable Computing: Informatics and Systems, 19, 72–85. https://doi.org/10.1016/j.suscom.2018.07.010 DOI: https://doi.org/10.1016/j.suscom.2018.07.010

Sabet, A.H.N., Zhao, Z., & Gupta, R. (2020). Subway: Minimizing data transfer during out-of-GPU-memory graph processing. Proceedings of the 15th European Conference on Computer Systems, EuroSys 2020 (pp. 1–16). ACM Digital Library. https://doi.org/10.1145/3342195.3387537 DOI: https://doi.org/10.1145/3342195.3387537

Siklosi, B., Reguly, I.Z., & Mudalige, G.R. (2019). Heterogeneous CPU-GPU execution of stencil applications. Proceedings of P3HPC 2018: International Workshop on Performance, Portability and Productivity in HPC, Held in Conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 71–80). IEEE. https://doi.org/10.1109/P3HPC.2018.00010 DOI: https://doi.org/10.1109/P3HPC.2018.00010

Werkhoven, B. Van, Maassen, J., Seinstra, F.J., & Bal, H.E. (2014). Performance models for CPU-GPU data transfers. Proceedings – 14th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2014 (pp. 11–20). IEEE. https://doi.org/10.1109/CCGrid.2014.16 DOI: https://doi.org/10.1109/CCGrid.2014.16

Yang, W., Li, K., & Li, K. (2017). A hybrid computing method of SpMV on CPU–GPU heterogeneous computing systems. Journal of Parallel and Distributed Computing, 104, 49–60. https://doi.org/10.1016/j.jpdc.2016.12.023 DOI: https://doi.org/10.1016/j.jpdc.2016.12.023

Raju, K. ., & N Chiplunkar, N. . (2021). PERFORMANCE ENHANCEMENT OF CUDA APPLICATIONS BY OVERLAPPING DATA TRANSFER AND KERNEL EXECUTION. Applied Computer Science, 17(3), 5–18. https://doi.org/10.23743/acs-2021-17

PERFORMANCE ENHANCEMENT OF CUDA APPLICATIONS BY OVERLAPPING DATA TRANSFER AND KERNEL EXECUTION

Issue Vol. 17 No. 3 (2021)

Archives

DOI

Authors

Abstract

Keywords:

References

License

Article Sidebar

Issue Vol. 17 No. 3 (2021)

Archives

Main Article Content

DOI

Authors

Abstract

Keywords:

References

Article Details

License