Efficient communication in heterogeneous architectures

Abstract

Advanced CMOS manufacturing technologies and limitations of power scaling have ushered processing chip architectures into the era of multi-core and many core systems. Simultaneously, there has been an emergence of compute and memory intensive applications like machine learning, cloud computing, artificial intelligence, etc. along with scientific applications, that require ever increasing performance demands. With diminishing returns from complex superscalar architectures, heterogeneous many core architectures have gained significant interest to meet the performance demands of these wide range of applications. They integrate processing cores of different architectures like x86, Field Programmable Gate Arrays (FPGAs), Graphic Processing Units (GPUs), application specific architectures, etc. in a single system. They have several heterogeneous cores working together in parallel on different application kernels towards finding good balance in meeting each kernel’s respective processing requirements and achieving high performance. Especially, single chip and multichip Heterogeneous System Architectures (HSA), that combine GPUs with x86 processors have become wide spread for meeting performance demands of highly parallel general purpose applications. Integration of heterogeneous architectures in a single system, while efficient and becoming wide spread, still poses several challenges for cohesive design and ubiquitous deployment. These challenges range from (i) circuit level for low power integration to (ii) system architectural challenges like shared resources, resource allocation per heterogeneous architecture, etc. for power and performance efficiency to (iii) software and application level challenges for seamless use of heterogeneous resources for achieving high performance. One such major challenge for heterogeneous architectures and the focus of this thesis is the communication across heterogeneous components and the shared system resources of on-chip interconnection network and memory/cache hierarchy. In the context of this thesis, communication in heterogeneous architectures is considered to be broadly comprised of data and coherence control communication. Communication and interconnection topologies are one of the major performance and power bottlenecks v for architectures that integrate many cores, whether homogeneous or heterogeneous. As core counts increase, processing architectures shift from computation to communication centric systems and data and coherence communication become the significant part of chip performance and power. Heterogeneous architectures pose additional communication challenges due to the differences in computing modalities of each architecture. The major of these being the highly diverse ways in which each of these architectures request for data and communicate with other cores/memory. These different and potentially conflicting traffics on the shared interconnection backbone lead to interference and suboptimal performance among the integrated architectures. Other challenges arise from presence of shared data between different computing architectures (coherence), operational heterogeneity, inefficient memory sharing, application kernel partitioning and communication policies, etc. To overcome the major challenge of communication diversity in heterogeneous architectures, we propose HyWin interconnection, built around customized networks and wireless links. HyWin decouples different architectures from one another to minimize interference among them and improve performance across all architectures. Besides improving communication performance, HyWin reduces overheads in presence of operational heterogeneity and reduces design complexity when extending to different heterogeneous architectures. It is robust and easily scales from heterogeneous single chip processors to multi-die and multi-chip systems, irrespective of the integrated architectures. Utilizing the HyWin architecture, we provide scalable and efficient cache coherence through SpyDir and WiSH protocols. SpyDir is a hierarchical hybrid coherence, that combines local directory with global snoopy protocol to minimize coherence storage and energy overheads, while providing necessary performance at any system size. WiSH further reduces coherence overheads by incorporating application sharing characteristics through logical share aware segmentation of caches. On-chip wireless links form the backbone for both HyWin interconnection and SpyDir coherence and are pivotal for realizing high performance and energy efficiency. Towards this end, we perform comprehensive analysis of on-chip wireless channel and design robust and high bandwidth wireless links for use in proposed approaches. HyWin and WiSH addresses the major challenges of communication diversity on shared network and coherence communication in heterogeneous architectures and provides a heterogeneity agnostic interconnection design and coherence protocol that leverages it to provide efficient communication in heterogeneous systems.