NCCL閫氳繃鎻愪緵涓€绯诲垪楂樺害浼樺寲鐨勯泦浣撻€氫俊鍘熻锛堝AllReduce銆丅roadcast銆丷educe銆丄llGather銆丷educeScatter绛夛級锛屼互鍙婄偣瀵圭偣閫氫俊鍔熻兘锛屼娇寰楀紑鍙戣€呰兘澶熷湪澶氫釜GPU涔嬮棿瀹炵幇楂樻晥鐨勬暟鎹紶杈撳拰鍚屾銆?

澶фā鍨嬭缁冮噷缁忓父閲囩敤鐨勪富娴佺殑娣卞害瀛︿範妗嗘灦(渚嬪 Cafe2銆丆hainer銆丮xNet銆丳yTorch鍜?TensorFlow)宸查泦鎴?NCCL锛屼究鏄负浜嗗湪澶?GPU澶氳妭鐐圭殑绯荤粺涓婂姞蹇繁搴﹀涔犺缁冮€熷害銆侼CCL鏃㈠彲浣滀负 NVIDIA HPC SDK鐨勪竴閮ㄥ垎涓嬭浇锛屼篃鍙綔涓洪€傜敤浜?Ubuntu 鍜?Red Hat鐨勫崟鐙蒋浠跺寘涓嬭浇銆?

NCCL涓昏鍋氬嚑浠朵簨锛氭帰娴嬭绠楄妭鐐圭殑缃戠粶璁惧鍜屾嫇鎵戠粨鏋勶紝浣跨敤绠楁硶鑷姩璋冧紭閫夋嫨涓€涓渶浼樼殑閫氫俊鏂瑰紡銆?/strong>

涓轰粈涔堣鍋氭嫇鎵戞帰娴嬶紵鍥犱负姣忎釜璁＄畻鑺傜偣鐨勮澶囨儏鍐靛樊寮傛瘮杈冨ぇ锛屾瘡涓绠楄妭鐐瑰彲鑳芥湁鑷繁鐗瑰畾鐨勭綉鍗IC锛屽彲鑳芥槸InfiniBand涔熷彲鑳芥槸RoCE锛屾瘡涓绠楄妭鐐逛笂鐨凣PU鍙兘鏄疦VLink锛屼篃鍙兘鏄疨CIe銆備负浜嗚揪鍒版渶浼樼殑浼犺緭鏁堢巼锛孨CCL鍏堣鎽告竻褰撳墠璁＄畻鑺傜偣鐨勭綉缁溿€丆PU鍜孏PU鎯呭喌銆備箣鍚庝娇鐢ㄨ皟浼樺伐鍏凤紝杩涜璋冧紭锛屼粠浼楀閫氫俊鏂瑰紡涓€夋嫨涓€涓渶浼樻柟寮忋€?/span>

鍩轰簬姝わ紝鎴戜滑鍙互寰楀嚭NCCL閫氳繃浼樺寲鐨勯€氫俊璺緞鍜屾暟鎹紶杈撶瓥鐣ュ疄鐜颁簡闈炲父浣庣殑寤惰繜鍜岄珮甯﹀锛岃繖瀵逛簬闇€瑕侀绻佷氦鎹㈡暟鎹殑娣卞害瀛︿範妯″瀷璁粌鑷冲叧閲嶈銆傝€孏ooxi鏈€鏂版帹鍑虹殑AMD Milan 鍙岃矾4U8鍗I鏈嶅姟鍣紝鏀寔2棰楅珮鎬ц兘AMD EPYC 7003/7002绯诲垪澶勭悊鍣紝閲囩敤CPU-GPU鐩磋繛閫氫俊锛屾槸鍏煎叿鎬ц兘涓庢垚鏈殑AI鏈嶅姟鍣ㄦ柟妗堛€傚畠鏀寔13涓狿ICe鎵╁睍妲戒綅锛屽彲閰嶇疆8寮犲弻瀹藉叏楂樺叏闀縂PU锛?strong style="-webkit-tap-highlight-color:transparent;margin:0px;padding:0px;outline:0px;max-width:100%;box-sizing:border-box !important;overflow-wrap:break-word !important;">鏁存満NCCL甯﹀鏈€楂樺彲杈?7.22GB/S锛?/span>鎻愪緵澶氬崱鏋佽嚧绠楀姏锛屽彲涓哄ぇ瑙勬ā鏁版嵁闆嗕笂璁粌澶嶆潅鐨?AI 妯″瀷鎻愪緵寮哄姴鐨勭畻鍔涙敮鎸併€傚畠鑳藉蹇€熸墽琛屽ぇ瑙勬ā骞惰璁＄畻锛屾樉钁楃缉鐭ā鍨嬭缁冩椂闂达紝澶у箙鎻愬崌澶фā鍨嬭缁冪殑鎬ц兘鍜屾晥鐜囥€?/span>