admin/matmul-analysis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix #36: MergeBatch 合并搬移效率收益建模 (move_eff) + t_cmd_ns 置 0

Browse Source 
用户澄清: MergeBatch vs IterBatch 的本质区别不只是 DMA 命令数 —— 合并 b0 个
batch 的左/右矩阵一起搬入 L1, 使单块 tile = nValue*dValue*dt 放大 b0 倍 (堆叠
方向视转置: A ND 非转置沿 M(nValue), B ND 非转置沿 N(dValue)), 搬移效率更高,
即便 T_cmd=0 也有效益。

- models.move_eff: 单命令搬移效率 eff = min(1, tile/min_TileSize) (16KB 饱和,
  与进入条件4效率下限语义同源); gm_move_time 按 A/B 两侧字节加权
  t = (V_A/eff_A + V_B/eff_B)/BW_gm; 只影响时间列, GM 字节量仍 = V_in
- IterBatch: l1_form 补驻留侧返回; move_tiles 分侧口径 (a/b 双侧整K, c 驻留侧
  整K+对侧k_l1, d 双侧k_l1), evaluate 接入效率加权
- MergeBatch: 合并 tile 放大 b0 倍接入效率加权; beats_iterbatch 净收益 =
  命令节省(cmds差×T_cmd) + 效率节省(t_data差) − drain惩罚, K截断且效率打平且
  T_cmd>0 时严格退化为 v1.1 §4.5 闭式; 退役 T_cmd<=0 策略特判
- router: 退役 "T_cmd<=0 策略优先 MergeBatch" 覆盖, 时延模型统一终审
- hardware: t_cmd_ns 50 -> 0 (未标定按 0; 合并收益不再依赖 T_cmd 估计值)
- 作用域: 仅切B 两分支接入 (逐命令 tile 小、效率差显著); ASW/StreamK 单命令
  tile 通常已饱和, 极端小 tile 走 issue#34 效率降级标注通道
- 用户 case 家族 B=128,M=1~16,N=128,K=512: m=1~8 -> MergeBatch (效率节省
  ~0.61us > drain), m=16 -> IterBatch (iter A tile 恰达 16KB 饱和, 效率打平,
  drain 决定); 分界与时延全家族一致
- demo: merge_demo_k_trunc 形状 (2048,32,32,256)->(2048,16,64,128) (原形状
  两侧 tile 均已 16KB 饱和, t_cmd=0 下无收益转 IterBatch; 新形状 iter A tile
  4KB eff=0.25 vs 合并 16KB eff=1.0, 保持 MergeBatch 胜出演示且仍 K截断)
- 测试: 74/74 (新增 TestIssue36 5 例: 效率曲线/字节不变/效率差胜出/家族;
  TestArbitration/TestZeroCmdHandling 按 t_cmd=0+效率语义重写; TestIssue35
  家族期望更新)
- 文档: 01_MergeBatch §4/§5 效率模型+泛化净收益; 02_IterBatch 口径注;
  00_总纲胜出条件; 01_软件架构 T_cmd 标定说明; 05 时间列效率口径注; README 要点
- 验证: examples 重生成可复现 0 diff; 压力 10000 例 0 崩溃/0 NaN/0 违规/
  0 GM<V_in, 七分支覆盖 (MergeBatch 386 例)
This commit is contained in:
admin
2026-09-07 21:09:49 +08:00
parent fdd3c8883c
commit b0b48b9073
16 changed files with 277 additions and 139 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
BMM/BMM_Theory/bmm_theory/branches/iter_batch.py
View File

@@ -12,7 +12,7 @@ batch 间流水靠 L1 双 buffer / 驻留侧预取 (c 形态半预算) 掩盖.
from __future__ import annotations
from ..hardware import NpuSpec, ASCEND950PR
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down, gm_move_time
from ..timing import assemble_timing, output_to_l2
from .base import Branch, BranchResult, ConditionCheck
@@ -23,11 +23,34 @@ class IterBatchBranch(Branch):
def __init__(self, spec: NpuSpec = ASCEND950PR):
super().__init__(spec)
# ------------------------------------------------------------------
# 单命令搬移 tile (issue#36 效率模型口径, evaluate 与 MergeBatch 仲裁共用)
# ------------------------------------------------------------------
def move_tiles(self, case: BmmCase) -> tuple:
"""返回 (A侧单命令tile字节, B侧单命令tile字节).
a/b 形态: 双侧整 K 一次搬入; c 形态: 驻留侧整 K + 对侧 k_l1 分块;
d 形态: 双侧 k_l1 分块。tile = nValue*dValue*dt (乘积与转置/排布无关)。
"""
m, n, k = case.m, case.n, case.k
dt = case.dtype_in_bytes
form, k_l1, _, resident = self.l1_form(case)
if form in ("a", "b"):
return m * k * dt, k * n * dt
if form == "c":
return (m * k * dt, k_l1 * n * dt) if resident == "A" \
else (m * k_l1 * dt, k * n * dt)
return m * k_l1 * dt, k_l1 * n * dt # d 或 None (兜底)
# ------------------------------------------------------------------
# L1 驻留形态判定 (v0.98 §六 条件 3, 四选一)
# ------------------------------------------------------------------
def l1_form(self, case: BmmCase) -> tuple:
"""返回 (形态 'a'/'b'/'c'/'d'/None, k_l1, 说明)."""
"""返回 (形态 'a'/'b'/'c'/'d'/None, k_l1, 说明, 驻留侧 'A'/'B'/None).
驻留侧仅 c 形态非 None (issue#36: 效率模型需要区分两侧单命令 tile —
c 形态驻留侧整侧一次搬入 (全 K), 对侧按 k_l1 分块).
"""
s = self.spec
m, n, k = case.m, case.n, case.k
dt = case.dtype_in_bytes
@@ -36,11 +59,11 @@ class IterBatchBranch(Branch):
# a) 单 batch 全驻留
if b_core == 1 and single <= s.l1_bytes:
return "a", k, f"单batch全驻留: (MK+KN)*dtype={single/1024:.0f}KB <= L1"
return "a", k, f"单batch全驻留: (MK+KN)*dtype={single/1024:.0f}KB <= L1", None
# b) 双 batch 乒乓
if b_core > 1 and 2 * single <= s.l1_bytes:
return "b", k, f"双batch乒乓: 2*(MK+KN)*dtype={2*single/1024:.0f}KB <= L1"
return "b", k, f"双batch乒乓: 2*(MK+KN)*dtype={2*single/1024:.0f}KB <= L1", None
# c) 一侧驻留 + 对侧切 K, 预算按 b_core 分档
# dValue 守卫与条件 4 / constraints 同源 (issue#19): 转置感知连续维判据
@@ -61,7 +84,7 @@ class IterBatchBranch(Branch):
return "c", k_l1, (
f"一侧驻留({side})+对侧切K: {side}驻留{resident/1024:.0f}KB, "
f"预算L1/{min(b_core,2)}, k_L1={k_l1}, "
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B")
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B"), side
# b_core>=2 时另一半 L1 预取下一 batch 驻留侧, 边界无气泡
# d) 两侧都切 K (兜底)
@@ -69,9 +92,9 @@ class IterBatchBranch(Branch):
dv_a, dv_b = dvalue_contig_dims(case, k_l1)
if (k_l1 >= s.fractal and dv_a >= s.dvalue_min and dv_b >= s.dvalue_min):
return "d", k_l1, (f"两侧都切K: k_L1={k_l1}, K段成对流水, batch边界天然无缝; "
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B")
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B"), None
return None, 0, "L1 四形态均不满足 (M/N 相对 L1 过大)"
return None, 0, "L1 四形态均不满足 (M/N 相对 L1 过大)", None
# ------------------------------------------------------------------
# 进入条件 (v0.98 §六, 四条同时满足)
@@ -95,7 +118,7 @@ class IterBatchBranch(Branch):
"2_负载均衡: B mod C == 0 或 >= minCoreNum",
c2, f"B mod C={rem}, minCoreNum={s.min_core_num}"))
form, k_l1, form_desc = self.l1_form(case)
form, k_l1, form_desc, _resident = self.l1_form(case)
checks.append(ConditionCheck(
"3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读)",
form is not None, form_desc))
@@ -131,7 +154,7 @@ class IterBatchBranch(Branch):
m, n, k = case.m, case.n, case.k
dt = case.dtype_in_bytes
b_core = -(-case.batch_c // s.aic_num)
form, k_l1, form_desc = self.l1_form(case)
form, k_l1, form_desc, _resident = self.l1_form(case)
form = form or "d"
# L0 级 tile (BaseM x BaseN): 核内 L1->L0 的切分, 与核间切分无关.
@@ -223,7 +246,11 @@ class IterBatchBranch(Branch):
# n_K/k_L1 只决定 DMA 命令次数 (T_cmd) 与双缓冲调度, 不放大数据量.
# dma_cmds 为单核命令数 (各核并行, issue#29 口径)
dma_cmds = b_core * n_k
t_mte2_data = case.input_bytes / s.bw_gm
# 搬移效率 (issue#36): 单命令 tile = nValue*dValue*dt 决定有效带宽
# (达 min_TileSize 饱和, 之下线性退化), 分侧口径见 move_tiles
tile_a, tile_b = self.move_tiles(case)
t_mte2_data = gm_move_time(b * m * k * dt, b * k * n * dt,
tile_a, tile_b, s)
t_dma_cmd = dma_cmds * s.t_cmd
t_mte2 = t_mte2_data + t_dma_cmd

85
BMM/BMM_Theory/bmm_theory/branches/merge_batch.py
View File

@@ -4,13 +4,13 @@
- 《BMM算子优化分析 v0.98》§五 (进入条件 + 实现方案 Step1~3)
- 《MergeBatch_vs_IterBatch分析 v1.1》§三/§四 (执行模型 + 分界条件)
核心思想: 合并 b0 个 batch 的 A'[b0*M,K] @ B'[K,b0*N] 为单次 DMA 搬入,
减少 GM->L1 搬移命令数 (省 b0 倍 T_cmd); 交叉项被算出但丢弃 (冗余比例 (b0-1)/b0),
进入条件 5 保证 case 为访存 Bound, 冗余算力被搬移时延掩盖.
T_cmd=0 时 (无命令固定时延/未标定) 命令时延收益不可量化, 但合并仍可减少
搬移命令/主机指令数 (指令发射/调度/同步收益未建模) —— beats_iterbatch 按既定
策略裁决: 合并后实际每核命令数更少即优先 MergeBatch (模型内代价仅 drain 惩罚),
命令数打平恒劣 (issue#35 泛化口径, 覆盖 dValue 512B cap 截断等第三情形).
核心思想: 合并 b0 个 batch 的 A'[b0*M,K] @ B'[K,b0*N] 为单次 DMA 搬入。
合并的核心收益有二 (用户澄清, issue#36): (1) 搬移命令数减少 (省 T_cmd);
(2) **单块 tile = nValue*dValue*dt 放大 b0 倍 -> 搬移效率提升** (堆叠方向视
转置/排布: A ND 非转置沿 M(nValue) 堆叠, B ND 非转置沿 N(dValue) 堆叠) ——
即便 T_cmd=0 (950PR 默认, 未标定按 0) 收益依然成立, 由 move_eff 效率模型刻画。
核心劣势是交叉项冗余算力 ((b0-1)/b0 被算出但丢弃) 与 drain 暴露放大 b0 倍;
进入条件 5 保证 case 为访存 Bound, 冗余算力被搬移时延掩盖。
"""
from __future__ import annotations
@@ -18,7 +18,7 @@ from __future__ import annotations
import math
from ..hardware import NpuSpec, ASCEND950PR
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down, gm_move_time
from ..timing import assemble_timing, output_to_l2
from .base import Branch, BranchResult, ConditionCheck
@@ -212,7 +212,12 @@ class MergeBatchBranch(Branch):
n_k = -(-k // k_l1)
dma_cmds = -(-b_core // b0) * n_k
t_dma_cmd = dma_cmds * s.t_cmd
t_mte2_data = case.input_bytes / s.bw_gm
# 搬移效率 (issue#36, 用户澄清): 合并 b0 个 batch 使单命令 tile 放大 b0 倍
# (A': b0*M x k_l1^m, B': k_l1^m x b0*N; 堆叠方向视转置/排布, 乘积不变),
# 小 M/N case 下相对 IterBatch 逐 batch 搬移的效率收益显著, T_cmd=0 时仍成立
k_eff = min(k_l1, k)
t_mte2_data = gm_move_time(b * m * k * dt, b * k * n * dt,
b0 * m * k_eff * dt, b0 * n * k_eff * dt, s)
t_mte2 = t_mte2_data + t_dma_cmd
# Cube (每核口径): 合并计算含冗余 (b0^2 输出, 有效 b0), 每核 b_core/b0 步,
@@ -244,18 +249,19 @@ class MergeBatchBranch(Branch):
def beats_iterbatch(self, case: BmmCase) -> tuple:
"""返回 (MergeBatch是否更优, 说明).
v1.1 §4.5 统一分界的泛化口径 (issue#35): 直接比较两分支**实际每核 DMA
命令数**与 drain 惩罚 ——
v1.1 §4.5 统一分界的泛化口径 (issue#35 命令数 / issue#36 搬移效率):
cmds_iter = b_core * ⌈K/k_l1_iter⌉ (IterBatch 逐 batch 逐 K 段一条)
cmds_mb = ⌈b_core/b0⌉ * ⌈K/k_l1^m⌉ (合并组数 x 每组 K 段数)
搬移节省 = (cmds_iter - cmds_mb) * T_cmd
命令节省 = (cmds_iter - cmds_mb) * T_cmd
效率节省 = t_data_iter - t_data_mb (合并放大单块 tile b0 倍 ->
搬移效率提升, issue#36 用户澄清: 即便 T_cmd=0 也有收益)
drain 惩罚 = (b0-1) * (T_comp + T_write) (T_comp 按未合并基线分块)
MergeBatch 最优 ⟺ 搬移节省 > drain 惩罚.
MergeBatch 最优 ⟺ 命令节省 + 效率节省 > drain 惩罚.
与 v1.1 §4.5 闭式的等价性:
与 v1.1 §4.5 闭式的等价性 (T_cmd>0 且效率打平时):
- K 截断 (k_l1^m = K): cmds_mb = b_core/b0, 退化为文档闭式
b_core > b0*(T_comp+T_write)/T_cmd;
- L1 绑定理想情形 (k_l1^m = k_l1^iter/b0): 命令数相同, 节省=0 -> 恒劣;
- L1 绑定理想情形 (k_l1^m = k_l1^iter/b0): 命令数相同, 命令节省=0;
- dValue 512B 推荐值截断等第三情形 (文档二分未覆盖): 按实际命令数比较.
截断判定用**合并后** plan.k_l1 >= K (与 make_plan/evaluate 同源,
@@ -263,25 +269,30 @@ class MergeBatchBranch(Branch):
且受 dValue 512B 推荐值截断), v1.1 line 219 的字面定义 (未合并口径) 与
line 161 的合并公式矛盾时以后者为准.
T_cmd<=0 (无命令时延/未标定) 时阈值不可量化: 按既定策略, 合并后每核命令数
更少 (cmds_mb < cmds_iter, 结构性指令/调度收益未建模) 即判 MergeBatch 优先;
命令数打平则恒劣 (合并只放大 drain).
T_cmd=0 (950PR 默认, 未标定): 命令节省项为 0, 由效率节省与 drain 惩罚
决定 —— 合并 tile 放大带来的效率收益被显式建模, 不再需要"T_cmd<=0 时
K截断即优先"的策略覆盖 (issue#36 退役)。
"""
s = self.spec
m, n, k = case.m, case.n, case.k
dt = case.dtype_in_bytes
out_b = case.dtype_out_bytes
b_core = case.batch_c // s.aic_num
# IterBatch 基线的每核命令数 (与 iter_batch.evaluate 同源: 形态 a/b 时
# k_l1=K 一次一条; c/d 形态按 l1_form 判定的 k_l1 分段)
# IterBatch 基线: 每核命令数 + 单命令 tile (与 iter_batch.evaluate 同源)
from .iter_batch import IterBatchBranch
_, k_l1_iter, _ = IterBatchBranch(s).l1_form(case)
ib = IterBatchBranch(s)
_, k_l1_iter, _, _ = ib.l1_form(case)
if not k_l1_iter:
k_l1_iter = k
n_k_iter = -(-k // min(k_l1_iter, k))
cmds_iter = b_core * n_k_iter
tile_a_ib, tile_b_ib = ib.move_tiles(case)
bb = case.batch_c
t_data_ib = gm_move_time(bb * m * k * dt, bb * k * n * dt,
tile_a_ib, tile_b_ib, s) # 整芯片口径 (量纲一致)
# MergeBatch 实际每核命令数 (issue#35: 合并组数 x 每组 K 段数)
# MergeBatch 实际每核命令数 (issue#35) + 合并 tile (issue#36)
plan = self.make_plan(case)
b0 = plan.merge_b0
n_k_mb = -(-k // min(plan.k_l1, k))
@@ -289,6 +300,9 @@ class MergeBatchBranch(Branch):
k_truncated = plan.k_l1 >= k # 合并口径截断判定 (issue#35)
regime = f"K截断(k_l1^m={plan.k_l1}>=K)" if k_truncated else \
f"L1绑定(k_l1^m={plan.k_l1}<K={k})"
k_eff = min(plan.k_l1, k)
t_data_mb = gm_move_time(bb * m * k * dt, bb * k * n * dt,
b0 * m * k_eff * dt, b0 * n * k_eff * dt, s)
# T_comp 按输入 dtype 算力 (issue#28), 未合并基线分块 (v1.1 §4.1 符号);
# T_write 保持 v1.1 直写 GM 语义
@@ -297,28 +311,15 @@ class MergeBatchBranch(Branch):
t_write = m * n * out_b / s.bw_pc
drain_pen = (b0 - 1) * (t_comp + t_write)
save_cmds = cmds_iter - cmds_mb
savings_cmd = save_cmds * s.t_cmd
savings_eff = t_data_ib - t_data_mb
if s.t_cmd <= 0:
# T_cmd=0: 命令时延收益不可量化 -> 按结构性命令数比较的策略裁决
# (合并把搬移/计算命令序列并少, 指令发射/调度/同步收益存在但未量化);
# 命令数打平时合并只放大 drain -> 恒劣.
if cmds_mb < cmds_iter:
detail = (f"{regime}; T_cmd=0: 命令时延收益不可量化(阈值=+inf), "
f"合并后每核命令数 {cmds_mb} < IterBatch {cmds_iter} "
f"(结构性收益, 未量化) -> 策略优先 MergeBatch; "
f"模型内代价 drain 惩罚={drain_pen*1e6:.2f}us")
return True, detail
detail = (f"{regime}; T_cmd=0: 每核命令数 MergeBatch={cmds_mb} 不少于 "
f"IterBatch={cmds_iter}, 合并只放大 drain 惩罚="
f"{drain_pen*1e6:.2f}us -> 恒劣")
return False, detail
savings = save_cmds * s.t_cmd
win = savings > drain_pen
detail = (f"{regime}; 每核命令数 MergeBatch={cmds_mb} vs IterBatch={cmds_iter}, "
f"搬移节省={savings*1e6:.2f}us vs drain惩罚=(b0-1)*(T_comp+T_write)="
win = (savings_cmd + savings_eff) > drain_pen
detail = (f"{regime}; 每核命令数 MergeBatch={cmds_mb} vs IterBatch={cmds_iter} "
f"(命令节省={savings_cmd*1e6:.2f}us) + 搬移效率节省="
f"{savings_eff*1e6:.2f}us vs drain惩罚=(b0-1)*(T_comp+T_write)="
f"{drain_pen*1e6:.2f}us -> {'MergeBatch优' if win else 'IterBatch优'}")
if k_truncated:
if k_truncated and s.t_cmd > 0:
detail += (f" (闭式等价: b_core={b_core} vs 阈值 "
f"b0*(T_comp+T_write)/T_cmd={b0 * (t_comp + t_write) / s.t_cmd:.1f})")
return win, detail