From d0b8f62643e50bb6bc5ab7ea7652c21384f22548 Mon Sep 17 00:00:00 2001 From: admin Date: Fri, 28 Aug 2026 07:11:07 +0000 Subject: [PATCH] =?UTF-8?q?v1.0=20=E5=90=8C=E6=AD=A5HTML?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...祦姘存満鍒禵UnitFlag涓嶥oubleBuffer瀵规瘮鍒嗘瀽_v1.0.html | 242 ++++++++++++++++++ 1 file changed, 242 insertions(+) create mode 100644 BMM/BMM绠楀瓙浼樺寲鍒嗘瀽_Release/L0C娴佹按鏈哄埗_UnitFlag涓嶥oubleBuffer瀵规瘮鍒嗘瀽_v1.0.html diff --git a/BMM/BMM绠楀瓙浼樺寲鍒嗘瀽_Release/L0C娴佹按鏈哄埗_UnitFlag涓嶥oubleBuffer瀵规瘮鍒嗘瀽_v1.0.html b/BMM/BMM绠楀瓙浼樺寲鍒嗘瀽_Release/L0C娴佹按鏈哄埗_UnitFlag涓嶥oubleBuffer瀵规瘮鍒嗘瀽_v1.0.html new file mode 100644 index 0000000..a17837a --- /dev/null +++ b/BMM/BMM绠楀瓙浼樺寲鍒嗘瀽_Release/L0C娴佹按鏈哄埗_UnitFlag涓嶥oubleBuffer瀵规瘮鍒嗘瀽_v1.0.html @@ -0,0 +1,242 @@ + + + + + +L0C 娴佹按鏈哄埗锛歎nitFlag 涓 Double Buffer 瀵规瘮鍒嗘瀽 v1.0 + + + + + +
+

L0C 娴佹按鏈哄埗瀵规瘮鍒嗘瀽锛歎nitFlag 涓 Double Buffer

+

鐗堟湰锛歷1.0

+

鏃ユ湡锛2026-08-28

+

鑼冨洿锛氭槆鑵 950PR Cube 鏍革紝BatchMatMulV3 绠楀瓙鏁版嵁娴佸満鏅

+

瀹氫綅锛氬洖绛"UnitFlag 杩欎箞濂斤紝涓轰粈涔堣繕闇瑕 L0C 鍙岀紦鍐诧紙dbL0C=2锛"鈥斺斾粠鏈哄埗銆佹椂寤舵ā鍨嬨佹簮鐮佷簨瀹炰笁涓眰闈㈢粰鍑轰袱鑰呯殑閫傜敤杈圭晫銆

+
+

鎽樿

+

UnitFlag 涓 L0C 鍙岀紦鍐茶В鍐崇殑鏄悓涓涓棶棰橈細MMAD锛圕ube 璁＄畻鍐 L0C锛変笌 Fixpipe锛圠0C 璇诲嚭鍙戦 GM锛変箣闂寸殑娴佹按鎺╃洊锛屼絾鏈哄埗瀹屽叏涓嶅悓鈥斺斿弻缂撳啿鐢"涓ゅ潡 buffer 浜ゆ浛"瀹炵幇 tile 闂寸矖绮掑害娴佹按锛孶nitFlag 鐢"512B 鍧楃骇鏍囧織浣"瀹炵幇 tile 鍐呯粏绮掑害娴佹按銆傛湰鏂囧厛鍒嗗埆瑙ｆ瀽涓ょ鏈哄埗锛屽啀寤虹珛涓夋柟妗堬紙鍗曠紦鍐蹭覆琛 / 鍙岀紦鍐 / UnitFlag锛夋椂寤舵ā鍨嬪姣旓紝鏈鍚庣粨鍚 batch_mat_mul_v3 婧愮爜浜嬪疄缁欏嚭閫傜敤杈圭晫鍒ゅ畾閾俱

+

鏍稿績缁撹锛

+
    +
  1. 绋虫佹椂寤锛氬弻缂撳啿涓 UnitFlag 閮借揪鍒 $\max(T_{MMAD}, T_{FIX})$锛屼袱鑰呮墦骞筹紱
    2. +
  2. UnitFlag 鐨勭湡瀹炰紭鍔锛氱渷鎺夌浜屼唤 buffer 鈫 tile 涓婇檺浠 $L0C/2$ 缈诲嶅埌 $L0C$ 鈫 L2 閲嶅璇诲噺灏戙佸垏鍒 padding 鍑忓皯銆佸熬宸翠粠"1 涓 tile"缂╁埌"1 涓 512B 鍧"锛
    3. +
  3. dbL0C=2 浠嶅瓨鍦ㄧ殑鍘熷洜锛氭簮鐮佹湭鍚敤 UnitFlag锛堣瘯楠岀壒鎬э級锛涗笖 UnitFlag 鏈夌‖绾︽潫鈥斺擬mad/Fixpipe 鍧楀垝鍒嗛』瀵归綈銆丗ixpipe 甯 NZ2ND 绛夊彉鎹㈡椂瑕佹眰 Mmad N 鍚戜紭鍏堛佸 batch 杈撳嚭鍒嗗尯锛圢BatchOut锛夊満鏅鐞嗗鏉傦紱MTE2 Bound 鍦烘櫙涓よ呴兘涓嶉渶瑕侊紱
    4. +
  4. 杈圭晫鍒ゅ畾锛歁TE2 Bound 鈫 鍗曠紦鍐叉棤 UnitFlag锛涢渶鎺╃洊涓 tile 宸茶秴 $L0C/2$ 鈫 UnitFlag 鏄敮涓鎵嬫锛涢渶鎺╃洊涓斿甫鍙樻崲/澶 batch 鍒嗗尯 鈫 鍙岀紦鍐诧紱鍏朵綑閲嶅彔鍖 鈫 UnitFlag 鍗曠紦鍐插ぇ tile 鏇翠紭銆
    5. +
+
+

涓銆侀棶棰樿儗鏅細涓や釜鏈哄埗瑙ｅ喅鍚屼竴闂

+

Cube 鏍告墽琛 matmul 鐨勬暟鎹祦锛堝崟 batch銆佸崟鏍歌瑙掞級锛

+
GM --MTE2--> L1(A,B) --MTE1--> L0A/L0B --Cube(MMAD)--> L0C --Fixpipe--> GM
+

鍏朵腑 L0C 鏄 Cube 鐨勭疮鍔犲櫒锛歁MAD 娌 K 杩唬绱姞鍐 L0C锛孠 杩唬鍏ㄩ儴瀹屾垚鍚 L0C 涓墠鏈夊畬鏁寸粨鏋滐紝闅忓悗 Fixpipe 鎶 L0C 璇诲嚭鍙戦 GM銆

+

璁惧崟 tile 鐨勮绠楁椂寤 $T_{MMAD}$锛堝叏閮 K 杩唬鐨 MMAD 鏃堕棿锛変笌鍐欏嚭鏃跺欢 $T_{FIX}$锛團ixpipe 鎼嚭鏁翠釜 tile 鐨勬椂闂达級銆傝嫢涓よ涓茶锛屽崟 tile 鏃跺欢涓 $T_{MMAD} + T_{FIX}$銆傞棶棰樼殑鏈川鏄細L0C 鍙湁涓浠 buffer 鏃讹紝Fixpipe 璇 tile N 涓 MMAD 鍐 tile N+1 浜夌敤鍚屼竴鍧楀湴鍧绌洪棿锛屾棤娉曞苟琛岋紱瑕佽涓よ呭苟琛岋紝瑕佷箞缁欎袱浠界┖闂达紙鍙岀紦鍐诧級锛岃涔堣璇诲啓鍚屾绮掑害瓒冲缁嗐佸厑璁稿湪鍚屼竴鍧楃┖闂翠笂"杩藉熬"锛圲nitFlag锛夈

+

鐢辨绔嬪嵆鍙锛鍙岀紦鍐蹭笌 UnitFlag 鏄悓涓闂鐨勪袱涓В锛屽湪"鑳藉惁鎺╃洊"涓婄瓑浠凤紝宸紓鍦"鐢ㄤ粈涔堜唬浠锋帺鐩"浠ュ強鍚勮嚜鐨勭害鏉熸潯浠銆

+
+

浜屻乁nitFlag 鏈哄埗璇﹁В锛堝畼鏂硅涔夛級

+

渚濇嵁瀹樻柟鏂囨。 [UnitFlag-Mmad璁＄畻鍏抽敭鐗规ц鏄嶿(https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/latest/API/ascendcopapi/docs/zh/api/SIMD-API/%E5%9F%BA%E7%A1%80API/cube_compute_TensorAPI/Mmad%E8%AE%A1%E7%AE%97%E5%85%B3%E9%94%AE%E7%89%B9%E6%80%A7%E8%AF%B4%E6%98%8E/UnitFlag.md)锛圕ANN 9.2.0-beta.1锛岃瘯楠岀壒鎬э級銆

+

2.1 鏈哄埗锛512B 鍧楃骇鏍囧織浣

+

寮鍚 UnitFlag 鍚庯紝L0C Buffer 涓瘡涓 512B 鍐呭瓨鍧鏈変竴涓崟鍏冩爣蹇椾綅锛屾寚绀鸿鍧楀綋鍓"鍙"杩樻槸"鍙啓"锛

+ + + + +
鎿嶄綔鏍囧織浣嶈涔鎵ц瀹屾垚鍚庣殑鏍囧織浣
鍐欐搷浣滐紙Mmad锛夛紝unitFlag=2flag=0 鐩存帴鍐欙紱flag=1 绛夊緟鑷 0淇濇寔 0锛堝潡浠嶈鍗犵敤锛
鍐欐搷浣滐紙Mmad锛夛紝unitFlag=3flag=0 鐩存帴鍐欙紱flag=1 绛夊緟鑷 0缃 1锛堝啓瀹岋紝鍙锛
璇绘搷浣滐紙Copy/Fixpipe锛夛紝unitFlag=2flag=1 鐩存帴璇伙紱flag=0 绛夊緟鑷 1淇濇寔 1锛堝潡浠嶈鍗犵敤锛
璇绘搷浣滐紙Copy/Fixpipe锛夛紝unitFlag=3flag=1 鐩存帴璇伙紱flag=0 绛夊緟鑷 1缃 0锛堣瀹岋紝鍙啓锛
+

璇箟褰掔撼锛unitFlag=2 鏄"淇濇寔鍗犵敤"锛寀nitFlag=3 鏄"鍐欏畬/璇诲畬閲婃斁"銆傚悓姝ョ矑搴︿粠"鏁翠釜 L0C tile"缁嗗寲涓"512B 鍧"銆

+

2.2 涓ょ鏍囧噯浣跨敤妯″紡

+

瀹樻柟鏂囨。缁欏嚭涓や釜鍏稿瀷妯″紡锛

+

妯″紡涓锛欿 杩唬绱Н鍚庝竴娆℃惉鍑锛堢ず渚 A 128脳1024銆丅 1024脳128锛宐aseK=128 杩唬 8 娆★紝8 娆 MMAD 瀵瑰簲 1 娆 Fixpipe锛夛細

+ +

妯″紡浜岋細鍗曟 MMAD 鍒嗗娆℃惉鍑锛圡MAD 缁撴灉 128脳256锛屾部 N 鍒嗕袱娆 Fixpipe 鎼嚭锛夛細

+ +

2.3 娴佹按濡備綍褰㈡垚锛氳法 tile 鐨 512B 杩藉熬

+

缁撳悎 搂涓 鐨勬暟鎹祦锛孶nitFlag 鐨勬祦姘村彂鐢熷湪 K 杩唬瀹屾垚鍚庣殑杈撳嚭鍧楁祦涓婏細

+
    +
  1. tile N 鐨勬渶鍚庝竴鏉 MMAD 娌 K 杩唬閫 512B 鍧楀畬鎴 鈫 姣忓潡 flag 缃 1锛
    2. +
  2. Fixpipe 閫愬潡璇诲嚭鍙戦 GM 鈫 姣忓潡璇诲畬 flag 缃 0锛
    3. +
  3. tile N+1 鐨 MMAD 閫愬潡杩界潃鍐欙紙璇ュ潡 flag 宸茬疆 0 鎵嶅彲鍐欙級鈫 鍙堥愬潡缃 1 鈫 寰幆銆
    4. +
+

浜庢槸 Fixpipe 涓 MMAD 鍦鍚屼竴浠 L0C 涓婁互 512B 鍧椾负绮掑害杩藉熬骞惰锛岀瓑鏁堜簬 512B 绮掑害鐨勪箳涔撱傝繖鍥炵瓟浜"涓轰粈涔堝崟缂撳啿涔熻兘娴佹按"锛氫簰鏂ョ殑鍗曚綅浠 tile 缂╁皬鍒 512B 鍧楋紝璇诲啓鍙渶閿欏紑涓涓潡鍗冲彲骞惰銆

+

2.4 瀹樻柟鏂囨。鏄庣‘鍒楀嚭鐨勭害鏉

+
    +
  1. Mmad 涓 Copy 鎺ュ彛椤诲悓姝ュ紑鍚 unitFlag锛屾墠鑳芥甯哥敓鏁堬紱
    2. +
  2. 寤鸿 Mmad 璁＄畻鏁版嵁閲忎笌 Fixpipe 鎼嚭鏁版嵁閲忎繚鎸佷竴鑷鈥斺旇嫢 MMAD 绠 128脳128 鑰 Fixpipe 鍙惉 64脳64锛屽彲鑳藉鑷存墽琛屽紓甯革紝闇閫氳繃 ResetL0CState 鎺ュ彛閲嶇疆 L0C 鐘舵併傚嵆涓よ呯殑鍧楀垝鍒嗗繀椤诲榻锛
    3. +
  3. Fixpipe 浣胯兘 NZ2ND 鎴 ChannelMerge 绛 layout 鍙樻崲鏃讹紝MMAD 璁＄畻鏂瑰悜椤昏涓 N 鏂瑰悜浼樺厛锛涙棤鍙樻崲鏃 M 鏂瑰悜浼樺厛銆傚嵆甯﹀彉鎹㈢殑 Fixpipe 鏁版嵁閫氳矾瀵 MMAD 鏈夐澶栨柟鍚戠害鏉锛
    4. +
  4. 杩炵画澶氭潯鎸囦护鎿嶄綔鍚屼竴鍧楃┖闂存椂锛屽墠 n-1 鏉 unitFlag=2銆佹渶鍚庝竴鏉 unitFlag=3 鐨"鍗犵敤閾"蹇呴』瀹屾暣锛
    5. +
  5. 璇ョ壒鎬ф爣娉ㄤ负璇曢獙鐗规銆
    6. +
+
+

涓夈丩0C 鍙岀紦鍐诧紙dbL0C锛夋満鍒朵笌 BMM 婧愮爜浜嬪疄

+

3.1 鏈哄埗锛歵ile 闂寸矖绮掑害涔掍箵

+

dbL0C=2 鏃 L0C 鍒嗕负涓や唤 buffer锛歁MAD 鍦 buffer 0 绠 tile N 鐨勫悓鏃讹紝Fixpipe 浠 buffer 1 鎼嚭 tile N-1銆傚悓姝ュ崟浣嶆槸鏁翠釜 tile锛坆aseM脳baseN 绾э級锛屼簰鏂ュ彧鍦 tile 浜ゆ浛澶勫彂鐢燂紝tile 鍐呴儴涓嶉渶瑕佷换浣曞潡绾ф爣蹇椼備唬浠锋槸 L0C 鏈夋晥瀹归噺鍑忓崐锛

+
$$S_{tile} = \text{baseM} \times \text{baseN} \times 4\text{B} \le L0C/2$$
+

3.2 婧愮爜浜嬪疄锛歞bL0C 鍐崇瓥鏄函瀹归噺鍒ゆ柇

+

[batch_mat_mul_v3 婧愮爜](https://gitcode.com/cann/ops-nn/tree/master/matmul/batch_mat_mul_v3)锛坥p_host/op_tiling锛変腑 dbL0C 鐨勮祴鍊煎湪 AL1FullLoadTiling / BL1FullLoadTiling / arch35 ASW tiling 涓夊锛屽叕寮忓畬鍏ㄤ竴鑷达細

+
tilingData.dbL0C = (baseM * baseN * 4 * NUM_TWO > l0CSize) ? 1 : NUM_TWO; // 4 is data size within l0c
+

鍗筹細tile 鐨 2 鍊嶆斁寰楄繘 L0C 灏卞紑鍙岀紦鍐诧紝鏀句笉杩涘氨鍗曠紦鍐鈥斺斿彧鐪嬪閲忥紝涓嶇湅 bound 绫诲瀷銆

+

閰嶅悎鍙︿竴澶勬簮鐮佷簨瀹炩斺旀櫘閫 BMM 璺緞锛圖oCommonTiling 鈫 TuneBaseMKN 鈫 CalcBaseMN锛夊 tile 鐨勭害鏉燂細

+
baseN = std::min(LastPower2(32768UL / baseM), bestBaseN); // L0C澶у皬闄愬埗baseM * baseN <= 32768
+

$32768 \text{ 鍏冪礌} \times 4\text{B} = 128\text{KB} = L0C/2$銆鏅 BMM 璺緞浠庢牴涓婂氨鎶 tile 闄愬埗鍦 L0C 涓鍗婁互鍐咃紝鍗抽粯璁ゆ寜鍙岀紦鍐茶鍒鈥斺旇繖瑙ｉ噴浜嗕负浠涔 batch_mat_mul_v3 涓ぇ閲忓満鏅 dbL0C=2锛氫笉鏄煇绉 bound 鍒嗘瀽鐨勭粨鏋滐紝鑰屾槸 CalcBaseMN 鐨 tile 涓婇檺鏈韩灏辨槸鍙岀紦鍐插閲忥紝瀹归噺鍏紡鑷劧鍥 2銆

+

鑰 ASW 璺緞锛坅rch35/batch_matmul_v3_asw_al1_full_load_basic_tiling.cpp 绛夛級缁 ResetBaseDav3510 缃 baseM=baseN=256 鍚庯紝$256 \times 256 \times 4\text{B} \times 2 = 512\text{KB} > 256\text{KB} = L0C$锛屽閲忓叕寮忚嚜鍔ㄩ鍖 dbL0C=1銆

+

3.3 澶 batch 杈撳嚭锛圢BatchOut锛夊 dbL0C 鐨勪緷璧

+

DoMultiBatchOutTiling 涓細

+
uint64_t batchOutCnt = compileInfo_.l0CSize / (baseN * baseM * dbL0C * 4);
+bool isNBatchOut = batchOutCnt > 1UL && batchInfo_.batchC > 1000UL;
+

澶 batch 杈撳嚭鍦烘櫙锛坆atchC > 1000 鐨勫皬鐭╅樀 BMM锛夋妸 L0C 鍒掑垎涓 batchOutCnt 涓垎鍖猴紝姣忓垎鍖烘斁涓涓 batch 鐨勮緭鍑 tile锛dbL0C 鐩存帴鍙備笌鍒嗗尯鏁拌绠銆傛鏃 L0C 鐨勮涔変粠"鍚屼竴 tile 鐨勫弻缂撳啿"鍙樹负"澶 batch 杈撳嚭鐨勫鍒嗗尯"锛宼ile 闂存祦姘撮潬 batch 闂翠氦閿欏疄鐜帮紝UnitFlag 鐨勫潡绾ф爣蹇楀湪澶氬垎鍖鸿竟鐣屼笂浼氬紩鍏ヨ法 batch 鐨勭鐞嗗鏉傚害銆

+

3.4 婧愮爜鏈惎鐢 UnitFlag

+

鏈伐浣滃尯淇濆瓨鐨 batch_mat_mul_v3 婧愮爜鍓湰涓紝鎵鏈 MMAD/Fixpipe 璋冪敤鍧囨棤 unitFlag 鍙傛暟鈥斺旇璇曢獙鐗规у湪 BMM 绠楀瓙涓湭鍚敤銆傚嵆婧愮爜鍦 dbL0C=1 鐨勫満鏅紙ASW 澶 tile锛変笅锛孧MAD 涓 Fixpipe 鏄 tile 绾т覆琛岀殑锛堥潬 MTE2 Bound 鎺╃洊锛岃 搂5锛夈

+
+

鍥涖佷笁鏂规鏃跺欢寤烘ā瀵规瘮

+

4.1 绗﹀彿瀹氫箟

+

璁惧崟鏍稿鐞嗕竴涓緭鍑 tile锛坆aseM脳baseN锛孎P32 杈撳嚭 4B锛孠 缁村叏閲忚凯浠ｏ級锛

+ + + + + + +
绗﹀彿瀹氫箟
$T_{MMAD}$鍗 tile 鍏ㄩ儴 K 杩唬鐨 Cube 璁＄畻鏃跺欢
$T_{FIX}$鍗 tile 鐨 Fixpipe 鍐欏嚭鏃跺欢
$T_{MTE2}$鍗 tile 鐨 GM鈫扡1 鎼叆鏃跺欢锛堜笌 L0C 鏂规鏃犲叧锛
$n_{tile}$鍗曟牳澶勭悊鐨 tile 鎬绘暟
$Q_{core}$鍗曟牳 Cube 璁＄畻閫熺巼锛團LOP/s锛
$W_{pc}$鍗曟牳 Fixpipe 鍐 GM 甯﹀锛圔/s锛
+
$$T_{MMAD} = \frac{2 \cdot \text{baseM} \cdot \text{baseN} \cdot K}{Q_{core}},\qquad T_{FIX} = \frac{\text{baseM} \cdot \text{baseN} \cdot 4\text{B}}{W_{pc}}$$
+

4.2 鏂规涓锛氬崟缂撳啿涓茶锛坉bL0C=1锛屾棤 UnitFlag锛

+

L0C 鍙湁涓浠 buffer 涓旀棤鍧楃骇鏍囧織锛欶ixpipe 蹇呴』绛夋暣涓 tile 绠楀畬锛屼笖 Fixpipe 璇 tile N 鏈熼棿 MMAD 涓嶈兘鍐 tile N+1锛堝悓涓鍦板潃锛夈傚畬鍏ㄤ覆琛岋細

+
$$T_1 = n_{tile} \cdot (T_{MMAD} + T_{FIX})$$
+

4.3 鏂规浜岋細鍙岀紦鍐诧紙dbL0C=2锛

+

tile N 璁＄畻涓 tile N-1 鍐欏嚭骞惰锛岀ǔ鎬佸崟 tile 鏃跺欢 $\max(T_{MMAD}, T_{FIX})$锛屽彟鏈夐 tile 璁＄畻鍚姩涓庡熬 tile 鍐欏嚭鎺掔┖鍚勪竴娆★細

+
$$T_2 = n_{tile}' \cdot \max(T_{MMAD}', T_{FIX}') + T_{MMAD}' + T_{FIX}'$$
+

鍏朵腑甯︽拠閲忎负鍙岀紦鍐蹭笅鐨勫弬鏁帮細tile 涓婇檺鍑忓崐锛$\text{baseM}' \times \text{baseN}' \le 32768$锛夛紝$n_{tile}'$ 鐩稿簲澧炲ぇ銆

+

4.4 鏂规涓夛細UnitFlag 鍗曠紦鍐

+

Fixpipe 浠 512B 鍧楃矑搴﹁拷灏 MMAD锛岀ǔ鎬佸悓鏍 $\max(T_{MMAD}, T_{FIX})$锛屾帓绌哄熬宸翠粎涓鏈鍚 1 涓 512B 鍧鐨 Fixpipe 鏃堕棿 $\tau_{blk}$锛

+
$$T_3 = n_{tile} \cdot \max(T_{MMAD}, T_{FIX}) + \tau_{blk},\qquad \tau_{blk} = \frac{512\text{B}}{W_{pc}}$$
+

4.5 绋虫佺瓑浠锋ц璇

+

鍙岀紦鍐 tile 鍑忓崐鍚 $T_{MMAD}', T_{FIX}'$ 鍚勫噺鍗婏紙涓よ呴兘涓 baseM脳baseN 鎴愭姣旓級锛$n_{tile}'$ 缈诲嶏紙蹇界暐瀵归綈 padding 鏃讹級锛屾晠绋虫侀」锛

+
$$n_{tile}' \cdot \max(T_{MMAD}', T_{FIX}') = 2 n_{tile} \cdot \frac{1}{2}\max(T_{MMAD}, T_{FIX}) = n_{tile} \cdot \max(T_{MMAD}, T_{FIX})$$
+

绋虫佹椂寤朵袱鑰呬弗鏍肩浉鍚銆傚樊寮備粎鍦細鈶犻灏惧熬宸达紙鍙岀紦鍐 $T_{MMAD}' + T_{FIX}'$ 涓涓 tile 閲忕骇 vs UnitFlag $\tau_{blk}$ 涓涓 512B 鍧楅噺绾э級锛涒憽tile 涓婇檺瀵艰嚧鐨勫垏鍒 padding 涓 L2 閲嶅璇伙紙搂4.6锛夈

+

4.6 鏁板煎疄渚嬶紙闂 鈫 鎬濊矾 鈫 鍏紡 鈫 瀹炰緥锛

+

鍦烘櫙锛歁=N=2048銆並=512銆佸崟 batch銆丅F16 杈撳叆銆丗P32 杈撳嚭鐩村啓 GM銆傝姱鐗囧弬鏁帮細$Q_{core} = 486\text{TFLOPS}/32 = 15.2\text{TFLOPS}$锛$W_{pc} = 1.6\text{TB/s}/32 = 50\text{GB/s}$锛堝亣璁惧潎鎽婏紝鏁伴噺绾т及璁★級锛孡0C=256KB銆

+

tile 閫夋嫨锛

+ +

鍗 tile 鏃跺欢锛

+
$$ +T_{MMAD} = \frac{2 \times 256 \times 256 \times 512}{15.2 \times 10^{12}} = 4.41\,\mu s,\qquad T_{FIX} = \frac{256 \times 256 \times 4}{50 \times 10^9} = 5.24\,\mu s +$$
+
$$ +T_{MMAD}' = \frac{2 \times 181 \times 181 \times 512}{15.2 \times 10^{12}} = 2.21\,\mu s,\qquad T_{FIX}' = \frac{181 \times 181 \times 4}{50 \times 10^9} = 2.62\,\mu s +$$
+

tile 鏁颁笌 padding锛

+ +

涓夋柟妗堟绘椂寤锛

+
$$ +T_1 = 64 \times (4.41 + 5.24) = 618\,\mu s +$$
+
$$ +T_2 = 144 \times \max(2.21, 2.62) + 2.21 + 2.62 = 382\,\mu s +$$
+
$$ +T_3 = 64 \times \max(4.41, 5.24) + \frac{512}{50 \times 10^9} = 335.4\,\mu s + 10\,ns +$$
+

瑙ｈ锛

+
    +
  1. 涓茶 鈫 鍙岀紦鍐/UnitFlag锛618 鈫 382/335 渭s锛屾帺鐩栨敹鐩婂法澶э紙璇ュ満鏅 $T_{FIX} > T_{MMAD}$ 鍐欏嚭 Bound锛夛紱
    2. +
  2. 鍙岀紦鍐 vs UnitFlag锛氱ǔ鎬佺浉鍚岋紝浣 UnitFlag 鍥 tile 缈诲嶈幏寰闆 padding锛堢渷 42渭s锛12%锛夊拰鍙拷鐣ョ殑灏惧反锛10ns vs 鍙岀紦鍐查灏 4.8渭s锛夛紱
    3. +
  3. 鑻ュ満鏅敼涓鸿绠 Bound锛圞=2048銆丮=N=512锛夛紝鍙畻寰 $T_1 = 92\mu s$銆$T_2 = 79.6\mu s$銆$T_3 = 70.7\mu s$鈥斺斿樊寮傚悓鏍锋潵鑷弻缂撳啿鐨 padding 铏氬銆
    4. +
+

tile 澶у皬鐨勮繛甯︽晥搴旓紙鍐冲畾鎬у樊寮傦級锛歎nitFlag 鐨 tile 涓婇檺鏄弻缂撳啿鐨 2 鍊嶃傛寜鏈灏戝垏鍒嗗師鍒欙紝tile 瓒婂ぇ 鈫 mCnt脳nCnt 瓒婂皬 鈫 姣忚 A/B 鐨 L2 閲嶅璇昏秺灏 鈫 $T_{MTE2}$ 瓒婂皬銆傝繖姝ｆ槸 UnitFlag 鐩稿鍙岀紦鍐茬殑缁撴瀯鎬т紭鍔锛岃屼笉鏄ǔ鎬佹祦姘翠笂鐨勪紭鍔裤

+
+

浜斻佷负浠涔 batch_mat_mul_v3 涓繕鏈 dbL0C=2 鍦烘櫙

+

缁煎悎 搂浜寏搂鍥涳紝閫愭潯鍥炵瓟锛

+
    +
  1. 婧愮爜鍘嬫牴娌″惎鐢 UnitFlag锛埪3.4锛夈傝瘯楠岀壒鎬ф湭琚敓浜т唬鐮侀噰鐢紝dbL0C 鏄簮鐮佷腑鍞竴鐨勬祦姘存墜娈碉紱
    2. +
  2. dbL0C 鍐崇瓥涓嶅垎鏋 bound锛氬閲忓叕寮忓洖 2 鐨勫敮涓鏉′欢鏄 tile 鈮 L0C/2銆傛櫘閫 BMM 璺緞 CalcBaseMN 寮哄埗 tile 鈮 32768 鍏冪礌锛埪3.2锛夛紝鎵浠ヨ繖浜涘満鏅蹇呯劧 dbL0C=2鈥斺旇繖鏄"鎸夊弻缂撳啿瑙勫垝 tile"鐨勮璁℃儻鎬э紝鑰岄潪"璇ュ満鏅弻缂撳啿鏇翠紭"鐨勭粨璁猴紱
    3. +
  3. 澶 batch 杈撳嚭鍒嗗尯锛圢BatchOut锛锛歜atchC>1000 鐨勫皬鐭╅樀鍦烘櫙 L0C 琚垝鍒嗕负澶 batch 杈撳嚭鍖猴紙搂3.3锛夛紝鍒嗗尯杈圭晫涓 UnitFlag 鍧楃骇鏍囧織鐨勭鐞嗕細浜掔浉骞叉壈锛宐atch 闂翠氦閿欏凡鎻愪緵绮楃矑搴︽祦姘达紝UnitFlag 鏀剁泭杈归檯灏忥紱
    4. +
  4. Fixpipe 甯﹀竷灞鍙樻崲锛歂Z 杈撳嚭 / ND2NZ on-the-fly 绛夊彉鎹㈣姹 MMAD N 鏂瑰悜浼樺厛锛埪2.4 绾︽潫 3锛夛紝涓庨粯璁 M 鏂瑰悜浼樺厛鐨 BMM 鏁版嵁娴佸啿绐侊紝鍙岀紦鍐叉棤姝ょ害鏉燂紱
    5. +
  5. Mmad/Fixpipe 鍧楀垝鍒嗗榻愮害鏉锛埪2.4 绾︽潫 2锛夛細BMM 鐨 Fixpipe 鎼嚭绮掑害闅 epilogue銆乥ias銆乥atch stride 鍙樺寲锛屽己鍒朵笌 MMAD 鍧楀榻愪細闄愬埗 tiling 鐏垫椿鎬э紝鍙岀紦鍐蹭笅 Fixpipe 鍙惉浠绘剰瀛愬尯鍩燂紱
    6. +
  6. MTE2 Bound 鍦烘櫙涓よ呴兘涓嶉渶瑕锛$T_{MTE2} > T_{MMAD}, T_{FIX}$ 鏃剁摱棰堝湪 GM鈫扡1 鎼叆锛孧MAD/Fixpipe 涓茶涔熻鎼Щ鎺╃洊锛屽崟缂撳啿 dbL0C=1 鍗冲彲鈥斺旇繖姝ｆ槸 ASW 澶 tile 璺緞鐨勭幇鐘躲
    7. +
+
+

鍏侀傜敤杈圭晫鍒ゅ畾閾

+

璁 $\beta = \max(T_{MMAD}, T_{FIX})$ 涓洪渶鎺╃洊鐨勭摱棰堥」锛$S_{tile} = \text{baseM} \times \text{baseN} \times 4\text{B}$ 涓 tile 瀛楄妭鏁般傚垽瀹氫粠 bound 绫诲瀷寮濮嬶細

+

鍒ゅ畾 1锛氭槸鍚 MTE2 Bound

+
$$T_{MTE2} \ge \beta \;\Longrightarrow\; \text{鍗曠紦鍐 dbL0C=1锛屼笉寮 UnitFlag}$$
+

鐡堕鍦ㄦ惉鍏ワ紝MMAD/Fixpipe 涓茶琚帺鐩栵紝鍙岀紦鍐茬櫧鐧芥氮璐逛竴鍗 L0C 瀹归噺锛孶nitFlag 鏃犻澶栨敹鐩娿娉ㄦ剰锛氭簮鐮 dbL0C 瀹归噺鍏紡鍦ㄦ鍦烘櫙鎭板ソ姝ｇ‘锛坱ile 澶 鈫 瀹归噺鏀句笉涓嬩袱浠 鈫 鑷姩 1锛夛紝鏄"瀹归噺鍒ゆ柇涓 bound 鍒ゆ柇娈婇斿悓褰"鐨勫阀鍚堬紝涓嶈兘鍊掓灉涓哄洜璁や负婧愮爜鍋氫簡 bound 鍒嗘瀽銆

+

**鍒ゅ畾 2锛氶渶瑕佹帺鐩栵紙$T_{MTE2} < \beta$锛夋椂锛屽弻缂撳啿鏄惁鍙**

+
$$2 \cdot S_{tile} \le L0C \;\Longleftrightarrow\; \text{鍙岀紦鍐插彲琛寎$$
+

鍒ゅ畾 3锛歎nitFlag 鏄惁鍙锛埪2.4 绾︽潫鏄犲皠鍒 BMM 鍦烘櫙锛

+ + + + +
绾︽潫涓嶆弧瓒崇殑 BMM 鍦烘櫙
MMAD 涓 Fixpipe 鍧楀垝鍒嗗榻Fixpipe 甯 epilogue 閾 / bias / batch stride 瀵艰嚧鎼嚭绮掑害涓 512B 鍧楅敊浣
甯 NZ2ND/ChannelMerge 鏃 MMAD 椤 N 浼樺厛NZ 杈撳嚭銆丯D2NZ on-the-fly 鍙樻崲鍦烘櫙
鍗 tile 鍒嗗尯璇箟绠鍗澶 batch 杈撳嚭 NBatchOut 鍒嗗尯鍦烘櫙
璇曢獙鐗规у彲鎺ュ彈鐢熶骇浠ｇ爜淇濆畧绛栫暐
+

鍒ゅ畾 4锛氬垎鏀夋嫨

+ + + + +
鏉′欢缁撹
$T_{MTE2} \ge \beta$dbL0C=1锛屾棤 UnitFlag锛堟簮鐮 ASW 鐜扮姸锛
$T_{MTE2} < \beta$ 涓 $2 S_{tile} > L0C$鍙岀紦鍐蹭笉鍙 鈫 UnitFlag 鏄敮涓娴佹按鎵嬫锛涗笉鐢ㄥ垯鎺ュ彈 $T_{MMAD}+T_{FIX}$ 涓茶
$T_{MTE2} < \beta$ 涓 $2 S_{tile} \le L0C$ 涓 UnitFlag 涓嶅彲琛鍙岀紦鍐诧紙婧愮爜鏅 BMM 璺緞鐜扮姸锛
$T_{MTE2} < \beta$ 涓 $2 S_{tile} \le L0C$ 涓 UnitFlag 鍙涓よ呴兘鍙紝UnitFlag 鍗曠紦鍐插ぇ tile 鏇翠紭锛坱ile 缈诲 鈫 L2 閲嶅璇烩啌銆乸adding鈫撱佸熬宸粹啌锛
+

閲嶅彔鍖猴紙鏈锛夋槸鍞竴"鍙互閫"鐨勫尯鍩燂紝鍏朵綑鍖哄煙鐢辩害鏉熷敮涓鍐冲畾銆傞噸鍙犲尯涓 UnitFlag 鐩稿鍙岀紦鍐茬殑浼樺娍鍏ㄩ儴鏉ヨ嚜 tile 涓婇檺缈诲嶏紙搂4.6锛夛紝鑻 tiling 鏈韩鏃 padding 涓 L2 閲嶅璇讳笉鏁忔劅锛屼袱鑰呭樊寮傜缉鑷冲熬宸撮噺绾с

+
+

涓冦佺粨璁轰笌瀵 BMM 浼樺寲鐨勫缓璁

+
    +
  1. UnitFlag 涓嶈兘瀹屽叏鍙栦唬鍙岀紦鍐锛歁TE2 Bound 鍦烘櫙涓よ呴兘涓嶉渶瑕侊紱甯 NZ2ND 鍙樻崲銆丯BatchOut 澶氬垎鍖恒佸潡鍒掑垎鏃犳硶瀵归綈鐨勫満鏅 UnitFlag 涓嶅彲琛岋紝鍙岀紦鍐叉槸鍞竴鎵嬫锛
    2. +
  2. 婧愮爜 dbL0C=2 鏄璁℃儻鎬ц岄潪鏈浼橀夋嫨锛欳alcBaseMN 鎶 tile 涓婇檺閿佹鍦 L0C/2锛堝弻缂撳啿瀹归噺锛夛紝瀵艰嚧鏅 BMM 璺緞 tile 鍏ㄩ儴鍋忓皬銆佸垏鍒嗗亸澶氥傝嫢鎸 UnitFlag 鍗曠紦鍐茶鍒掞紙tile 涓婇檺 L0C锛夛紝tile 鍙斁澶 2 鍊嶏紝L2 閲嶅璇讳笌 padding 鍚屾涓嬮檷锛
    3. +
  3. BMM 浼樺寲寤鸿锛堟寜浼樺厛绾э級锛
    4. +
      +
    • 璁＄畻/鍐欏嚭 Bound + tile 宸茶秴 L0C/2 鐨 ASW 鍦烘櫙锛鍚敤 UnitFlag锛屽皢鍗 tile 鏃跺欢浠 $T_{MMAD}+T_{FIX}$ 闄嶄负 $\max(T_{MMAD}, T_{FIX})$鈥斺旇繖鏄綋鍓嶆簮鐮佹渶澶х殑鐩存帴鏀剁泭鐐癸紙姝ゅ墠鍒嗘瀽宸叉寚鍑 ASW kernel 鏈惎鐢 UnitFlag锛夛紱
      • +
    • 璁＄畻/鍐欏嚭 Bound + 灏 tile + 鏃犲彉鎹/鍗 batch锛tile 鏀惧ぇ + UnitFlag锛屾浛浠 dbL0C=2锛
      • +
    • NZ 杈撳嚭 / NBatchOut / 鍧楀垝鍒嗘棤娉曞榻愶細缁存寔鍙岀紦鍐锛
      • +
    • MTE2 Bound锛氱淮鎸佸崟缂撳啿锛屼笉蹇呭紩鍏 UnitFlag 澶嶆潅搴︼紱
      • +
    +
  4. 宸ョ▼钀藉湴椤哄簭锛氬厛鍦ㄦ棤鍙樻崲銆佸崟 batch銆佸潡鍒掑垎瑙勬暣鐨 ASW 璺緞鍚敤锛堢害鏉熸渶灏戙佹敹鐩婃渶澶э級锛岄獙璇佸悗鍐嶅悜甯﹀彉鎹㈠満鏅墿灞曘
    5. +
+
+

鍙傝冩枃鐚

+
    +
  1. [UnitFlag-Mmad璁＄畻鍏抽敭鐗规ц鏄庯紙CANN 9.2.0-beta.1 瀹樻柟鏂囨。锛塢(https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/latest/API/ascendcopapi/docs/zh/api/SIMD-API/%E5%9F%BA%E7%A1%80API/cube_compute_TensorAPI/Mmad%E8%AE%A1%E7%AE%97%E5%85%B3%E9%94%AE%E7%89%B9%E6%80%A7%E8%AF%B4%E6%98%8E/UnitFlag.md)
    2. +
  2. [batch_mat_mul_v3 婧愮爜浠擄紙CANN ops-nn锛塢(https://gitcode.com/cann/ops-nn/tree/master/matmul/batch_mat_mul_v3)锛歰p_host/op_tiling/batch_mat_mul_v3_base_tiling.cpp锛圕alcBaseMN銆丄L1FullLoadTiling/BL1FullLoadTiling 鐨 dbL0C 璧嬪笺丏oMultiBatchOutTiling锛夈乷p_host/op_tiling/arch35/batch_matmul_v3_asw_al1_full_load_basic_tiling.cpp
    3. +
  3. 鏄囪吘 950PR 瑙勬牸锛歀0C 256KB/鏍搞丩1 512KB/鏍搞丆ube 486 TFLOPS@BF16锛32 鏍稿潎鎽 15.2 TFLOPS/鏍革級銆丟M 甯﹀ 1.6TB/s锛32 鏍稿潎鎽 50GB/s锛屼及璁″硷級
    4. +
+
+ + \ No newline at end of file