Compare commits
79 Commits
3a6c952301
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 5db73dfa26 | |||
| ec4fee99b8 | |||
| 8e784a748c | |||
| 8d42d958e4 | |||
| f9d5da0c0c | |||
| b0b48b9073 | |||
| fdd3c8883c | |||
| 0cd47f93cb | |||
| 05ca91e291 | |||
| 18f59599e7 | |||
| b9e07edc1d | |||
| 4843053ad3 | |||
| f4b23d9f05 | |||
| 5051e00fa0 | |||
| a071d87831 | |||
| 9afe6eec02 | |||
| f5a0b6fe81 | |||
| 08bf74d8b8 | |||
| 2d9d9f3116 | |||
| c9edf8d6b0 | |||
| e0ee3d1c94 | |||
| 3ff9f7d205 | |||
| 588e8928c0 | |||
| a51571b20d | |||
| 6d5e452fbc | |||
| 5888333298 | |||
| 7ebe391b8f | |||
| 59a7f8df44 | |||
| 9c96fab9f6 | |||
| 1d1a9187c9 | |||
| 759a51e92a | |||
| 1e8f9102d0 | |||
| 6a33c3b1f9 | |||
| 657cc6639f | |||
| c05cbeafad | |||
| 49397260f6 | |||
| 4f90f76db9 | |||
| af0a90bc25 | |||
| 517441de6e | |||
| 27e0fd85ba | |||
| 2881fdca38 | |||
| 9cb1eb91c3 | |||
| c14310d9cb | |||
| d27278bbb6 | |||
| e7db521122 | |||
| a7356fec99 | |||
| 4d5c95c3e2 | |||
| eb16b71387 | |||
| ff0bac9b45 | |||
| db8399b62c | |||
| c2028a41ed | |||
| f3d6bc7003 | |||
| ebf77d9b48 | |||
| 98223e0e12 | |||
| 32febc1f62 | |||
| ed084a4be0 | |||
| 340ae2c0d4 | |||
| b4003d8246 | |||
| 7309215720 | |||
| d8457be420 | |||
| 7f318199ab | |||
| 60d37a83d2 | |||
| d4a249e0f8 | |||
| 9c805659c4 | |||
| 0198f60f4e | |||
| 5fcfb38adf | |||
| 7644ac53f0 | |||
| 0440342dd9 | |||
| 88b1e4946c | |||
| f4e8eff040 | |||
| 3090d76a92 | |||
| b8247af9ee | |||
| 72769f4b90 | |||
| 0f474b3b65 | |||
| 69d09e1f73 | |||
| 23780c5c67 | |||
| ae3ef6f7e4 | |||
| d0a6df0f8e | |||
| 623f20259e |
9
.gitignore
vendored
9
.gitignore
vendored
@@ -12,6 +12,15 @@ venv/
|
|||||||
*~
|
*~
|
||||||
*.swp
|
*.swp
|
||||||
|
|
||||||
|
# BMM_Theory 调试/复现用临时 csv (issue#21, 防再次误提交; examples/ 下正式样例不受影响)
|
||||||
|
/BMM/BMM_Theory/bug*.csv
|
||||||
|
/BMM/BMM_Theory/bug*_out.csv
|
||||||
|
/BMM/BMM_Theory/bug*_p.csv
|
||||||
|
/BMM/BMM_Theory/t.csv
|
||||||
|
/BMM/BMM_Theory/t_*.csv
|
||||||
|
/BMM/BMM_Theory/t*.csv
|
||||||
|
/BMM/BMM_Theory/p2.csv
|
||||||
|
|
||||||
# IDE / OS
|
# IDE / OS
|
||||||
.idea/
|
.idea/
|
||||||
.vscode/
|
.vscode/
|
||||||
|
|||||||
@@ -49,10 +49,15 @@ python -m unittest discover -s tests -v
|
|||||||
|
|
||||||
### 时延模型要点 (读结果前必看)
|
### 时延模型要点 (读结果前必看)
|
||||||
|
|
||||||
- **搬入 MTE2 分两段**: `t_mte2_gm` (GM→L1 直读, 按 GM 带宽 1.6TB/s; 无论是否随路驻留 L2 都不再累加 L2→L1) 与 `t_mte2_l2` (数据驻留 L2 后重复读命中, 按 L2 带宽 5.2TB/s);
|
- **GM 是读写共享总线 (1.6TB/s)**: MTE2 的 GM 读与 Fixpipe 直写 GM 并发时按 **(读+写)/1.6TB/s 累加**计入 MTE2 搬移链 (issue#23);
|
||||||
- **Cube 计算** `t_mmad`: 单核算力 ≈15.2 TFLOPS, MergeBatch 的冗余计算计入;
|
- **搬入 MTE2 分两段、同链相加**: `t_mte2_gm` (首读按 GM 带宽; 随路驻留 L2 不重复累加) + `t_mte2_l2` (共享块驻留 L2 后被其它核重复读, 按 L2 读口 5.2TB/s) + DMA 命令开销 (issue#24; **T_cmd 默认 0**, 未标定按 0 处理, issue#36);
|
||||||
- **Fixpipe 搬出** `t_fixpipe`: 数据量按 **C 矩阵 dtype** 计 (fp16/fp8 时随路转换、写出量减半); StreamK 中间部分和为防精度丢失按 4B (L0C dtype) 计;
|
- **搬移效率模型 (issue#36)**: 切B 两分支 (IterBatch/MergeBatch) 的 GM→L1 数据时延按单命令 tile 效率加权 —— eff = min(1, tile/min_TileSize), tile = nValue×dValue×dtype 达 16KB 饱和; **MergeBatch 合并 b0 个 batch 使单块 tile 放大 b0 倍, 即便 T_cmd=0 也比 IterBatch 逐 batch 搬移效率高** (堆叠方向视转置: A ND 非转置沿 M(nValue), B ND 非转置沿 N(dValue)); 只影响时间列, GM 字节量不变;
|
||||||
- **总时延** `t_total = max(各级) + t_drain` (双缓冲稳态取最大 + 末级排空暴露);
|
- **Cube 计算** `t_mmad`: 芯片算力 = 白皮书 Cube-only 432 TFLOPS (单核 13.5 TFLOPS, issue#40), MergeBatch 冗余计算计入;
|
||||||
|
- **Fixpipe 搬出**: 直写 GM 计入 GM 共享总线; 驻留 L2 走 5.2TB/s 写口 (独立计时); 数据量按 **C 矩阵 dtype** 计 (fp16/fp8 随路转换减半); StreamK 部分和按 4B (L0C dtype);
|
||||||
|
- **总时延** `t_total = max(MTE2搬移链, MMAD, Fixpipe-L2) + t_drain` (稳态取最大 + 末级排空暴露; REDUCE 串行追加);
|
||||||
|
- **GM 读取下限**: 每输入字节至少从 GM 读一次 (R1); L2 只吸收"驻留后的再次读取"; 整 case 输入+输出 ≤ L2 时 GM 恰读一次、输出全驻留 L2 (issue#29 设计文档 docs/05);
|
||||||
|
- **Fixpipe 输出落点**: 整 case (输入+输出+workspace) 可驻留 L2 → 输出写 L2 写口 5.2TB/s、GM 写流量 0; 否则输入优先保 L2, 输出直写 GM 计入共享总线 (issue#30);
|
||||||
|
- **算力按输入 dtype**: Cube BF16/FP16 432TFLOPS 为基准 (Cube-only 口径, 白皮书表3-1 "Cube算力" 单行, issue#40), fp8=2x/fp4=4x (白皮书), fp32=1/2 (假设待标定); AIV 逐元素通量同理 (issue#28);
|
||||||
- **瓶颈交换**: 搬移瓶颈可牺牲算力换搬移效率 (MergeBatch), 计算瓶颈可牺牲搬移换计算效率 (ASW_Basic 切 M/N).
|
- **瓶颈交换**: 搬移瓶颈可牺牲算力换搬移效率 (MergeBatch), 计算瓶颈可牺牲搬移换计算效率 (ASW_Basic 切 M/N).
|
||||||
|
|
||||||
## 目录结构
|
## 目录结构
|
||||||
@@ -63,7 +68,8 @@ BMM_Theory/
|
|||||||
├── bmm_theory/ # 软件包
|
├── bmm_theory/ # 软件包
|
||||||
│ ├── __main__.py # CLI 入口 (recommend / evaluate)
|
│ ├── __main__.py # CLI 入口 (recommend / evaluate)
|
||||||
│ ├── models.py # 数据模型: BmmCase / ImplPlan(标准结构体) / HardwareTiming
|
│ ├── models.py # 数据模型: BmmCase / ImplPlan(标准结构体) / HardwareTiming
|
||||||
│ ├── hardware/ascend950pr.py# 950PR 硬件参数表 (换芯片只换这份)
|
│ ├── hardware/ # 硬件参数表 (NpuSpec; 950PR 32/28核 + 950DT 36/32/28核,
|
||||||
|
│ │ # SPECS/get_spec 索引, 默认 ASCEND950PR; 换芯片只加这份)
|
||||||
│ ├── timing.py # 时延评估引擎 (MTE2/Cube/Fixpipe/Reduce 模型)
|
│ ├── timing.py # 时延评估引擎 (MTE2/Cube/Fixpipe/Reduce 模型)
|
||||||
│ ├── constraints.py # 单一约束源 (生成与校验共用, L0C/L0A/L0B/L1/dValue/核数)
|
│ ├── constraints.py # 单一约束源 (生成与校验共用, L0C/L0A/L0B/L1/dValue/核数)
|
||||||
│ ├── router.py # 分支决策路由 + 重叠区仲裁 + 生成后自检
|
│ ├── router.py # 分支决策路由 + 重叠区仲裁 + 生成后自检
|
||||||
@@ -88,6 +94,9 @@ BMM_Theory/
|
|||||||
│ │ ├── 06_ASW_Basic分支.md # 尾轮策略已内化为其必要环节
|
│ │ ├── 06_ASW_Basic分支.md # 尾轮策略已内化为其必要环节
|
||||||
│ │ └── 07_尾轮处理策略.md # 尾轮完整推导 (参考)
|
│ │ └── 07_尾轮处理策略.md # 尾轮完整推导 (参考)
|
||||||
│ └── 03_测评报告/ # 外部测评报告 (v1.0/v2.0 及后续复评)
|
│ └── 03_测评报告/ # 外部测评报告 (v1.0/v2.0 及后续复评)
|
||||||
|
│ ├── 04_差异对照_bmmv3_vs_BMM_Theory.md # bmmv3 行为预测器 vs 理论推导器对照
|
||||||
|
│ ├── 05_L2驻留GM读写与dtype算力口径_设计分析.md # GM/L2/输出落点/dtype算力统一口径 (issue#27-#30)
|
||||||
|
│ └── 06_落地验证计划_理论方案_vs_bmmv3源码.md # 理论落地 bmmv3 源码的比对/修改/上板验证计划
|
||||||
├── examples/ # 示例输入输出
|
├── examples/ # 示例输入输出
|
||||||
└── tests/ # 单元测试 (固化文档边界 case + issue 回归)
|
└── tests/ # 单元测试 (固化文档边界 case + issue 回归)
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
"""ASW_Basic 分支: 核间切 M/N (或混合切) 的兜底分支, 含尾轮处理.
|
"""ASW_Basic 分支: 核间切 M/N (或混合切) 的兜底分支, 含尾轮处理.
|
||||||
|
|
||||||
理论依据:
|
理论依据:
|
||||||
- 《BMM算子优化分析 v0.98》§八 + docs/02_分支理论/06_ASW_Basic分支.md
|
- 《ASW_Basic分支分析 v1.91》(L0C 单缓冲方形 Base tile §5.1 + SingleCoreM/N 有界
|
||||||
- 《BMM尾轮处理策略对比分析 v1.5》+ docs/02_分支理论/07_尾轮处理策略.md
|
枚举 §5.2) + docs/02_分支理论/06_ASW_Basic分支.md
|
||||||
|
- 《BMM尾轮处理策略对比分析 v1.5》(单块搬入稳态口径 K(sM+sN)·dt/BW, k_L1 约掉)
|
||||||
|
+ docs/02_分支理论/07_尾轮处理策略.md
|
||||||
|
|
||||||
核心: 兜底分支, 核间切 M/N, 重复读交给 L2 + swizzle. 尾轮处理是必要组成环节:
|
核心: 兜底分支, 核间切 M/N, 重复读交给 L2 + swizzle. 尾轮处理是必要组成环节:
|
||||||
默认方案 B (工程简洁, 主流场景与 A1b 严格打平), 周长型且 rho>=rho_dv 时 A1b.
|
默认方案 B (工程简洁, 主流场景与 A1b 严格打平), 周长型且 rho>=rho_dv 时 A1b.
|
||||||
@@ -13,7 +15,7 @@ from __future__ import annotations
|
|||||||
import math
|
import math
|
||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming, ceil_div, align_down
|
from ..models import BmmCase, ImplPlan, HardwareTiming, ceil_div, align_down, align_up
|
||||||
from ..timing import assemble_timing
|
from ..timing import assemble_timing
|
||||||
from .base import Branch, ConditionCheck
|
from .base import Branch, ConditionCheck
|
||||||
|
|
||||||
@@ -60,47 +62,44 @@ class AswBasicBranch(Branch):
|
|||||||
return self._plan_reduced_core(case, p)
|
return self._plan_reduced_core(case, p)
|
||||||
|
|
||||||
# ---- 正常模式: Step 0~6 ----
|
# ---- 正常模式: Step 0~6 ----
|
||||||
# Step 0: BaseM/BaseN 用满 L0C (32768 元素双缓冲)
|
# Step 0: BaseM/BaseN (v1.91 §5.1): UnitFlag 单缓冲 (tile 内 16x16x16 细粒度
|
||||||
base_mn = int(math.sqrt(s.l0c_bytes // 8)) # L0C/(2*4B)
|
# 流水替代 tile 间粗粒度双缓冲), BaseM*BaseN = L0C/4B = 65536 元素, 方形
|
||||||
base_m = align_down(base_mn, s.fractal)
|
# 优先 (256x256, L0A/L0B 同时装满、baseK 加倍); 仅当 M/N < 方形边长时被迫
|
||||||
base_n = align_down(base_mn, s.fractal)
|
# 跟随 M/N (另一侧按 L0C 面积余量放大); BaseK 由 L0A/L0B (双缓冲) 反推.
|
||||||
base_k = align_down(int(min(
|
from ..constraints import clamp_base_k
|
||||||
s.l0a_bytes / (2 * base_m * dt),
|
cap_l0c = s.l0c_bytes // 4 # 65536 元素 (单缓冲)
|
||||||
s.l0b_bytes / (2 * base_n * dt),
|
sq = align_down(int(math.sqrt(cap_l0c)), s.fractal) # 256
|
||||||
)), s.fractal)
|
# L0A/L0B 在 base_k>=16 (fractal) 下的单边上限 (v1.91 §5.7 核内约束;
|
||||||
|
# 防止例外路径 (如 M=16 -> BaseN=4096) 超 L0B)
|
||||||
|
side_cap = s.l0a_bytes // (2 * s.fractal * dt) # L0A=L0B=64KB
|
||||||
|
if m >= sq and n >= sq:
|
||||||
|
base_m = base_n = sq
|
||||||
|
elif m < sq:
|
||||||
|
base_m = max(align_down(m, s.fractal), s.fractal)
|
||||||
|
base_n = max(align_down(min(cap_l0c // base_m, n, side_cap),
|
||||||
|
s.fractal), s.fractal)
|
||||||
|
else: # n < sq
|
||||||
|
base_n = max(align_down(n, s.fractal), s.fractal)
|
||||||
|
base_m = max(align_down(min(cap_l0c // base_n, m, side_cap),
|
||||||
|
s.fractal), s.fractal)
|
||||||
|
base_k = clamp_base_k(base_m, base_n, dt, case.k, s) # v1.91 §5.7
|
||||||
|
|
||||||
# Step 1: SingleCoreM/N 尽量大 (满足并行度下限)
|
# Step 1+2+6: SingleCoreM/N + k_L1 + mCnt/nCnt 有界枚举 (v1.91 §5.2):
|
||||||
min_blocks = ceil_div(s.aic_num, b)
|
(single_m, single_n, k_l1, m_cnt, n_cnt, pick_note) = \
|
||||||
single_m, single_n = self._pick_single_core(m, n, min_blocks, base_m, base_n, dt)
|
self._pick_single_core(case, base_m, base_n)
|
||||||
|
|
||||||
# Step 2: mCnt/nCnt
|
|
||||||
m_cnt = ceil_div(m, single_m)
|
|
||||||
n_cnt = ceil_div(n, single_n)
|
|
||||||
|
|
||||||
# Step 4: swizzle 窗口 W = max{d | d|C, d <= floor(sqrt(C))}
|
# Step 4: swizzle 窗口 W = max{d | d|C, d <= floor(sqrt(C))}
|
||||||
swizzle_w = self._swizzle_w()
|
swizzle_w = self._swizzle_w()
|
||||||
|
|
||||||
# Step 5: L2 分组判断
|
# Step 5: L2 场景判定 (issue#24/#30/#32, 设计文档 docs/05 §4, 芯片 L2 128MB)
|
||||||
s_in = b * (m * k + k * n) * dt
|
# S_A 整case全驻留: V_in+V_out <= L2 -> 输出驻留 L2 (R4, GM 写=0);
|
||||||
s_out = b * m * n * case.dtype_out_bytes
|
# S_B 单batch可驻留 (全驻留或单侧全驻留+对侧滑窗): GM 每字节一次,
|
||||||
if s_in + s_out <= s.l2_bytes:
|
# 共享块重复读全部命中 L2 读口, 输出直写 GM;
|
||||||
l2_scene = "A_全驻留"
|
# S_C 双侧超L2: 最小替换 2D 分组 (组间落空 GM, 窗口 L2).
|
||||||
l2_out = "resident(输出驻留L2异步回写)"
|
scene = self._l2_scene(case, single_m, single_n, m_cnt, n_cnt)
|
||||||
r_in = 1.0
|
l2_out = ("resident(整case全驻留S_A: 输出驻留L2异步回写, GM写=0)"
|
||||||
elif s_in <= s.l2_bytes:
|
if scene["to_l2"] else
|
||||||
l2_scene = "B_输入驻留输出直写GM"
|
"direct_gm(输出直写GM, 输入优先驻留L2)")
|
||||||
l2_out = "direct_gm(输出直写GM不占L2)"
|
|
||||||
r_in = 1.0
|
|
||||||
else:
|
|
||||||
l2_scene = "C_输入超L2分组执行"
|
|
||||||
l2_out = "direct_gm(输出直写GM不占L2)"
|
|
||||||
r_in = self._l2_group_r_in(case, single_m, single_n)
|
|
||||||
|
|
||||||
# Step 6: k_l1 (GM->L1 K 向粒度, 须 >= dValue 下限; K 小于下限时整 K 一次搬入不切)
|
|
||||||
dv_min_elems = max(s.dvalue_hw_min // dt, 1)
|
|
||||||
k_l1 = min(k, s.dvalue_recommend // dt)
|
|
||||||
if k_l1 < dv_min_elems:
|
|
||||||
k_l1 = k # K 本身小于 dValue 下限: 不切 K, 整段搬入 (K 非连续维, dValue 由 M/N 保证)
|
|
||||||
|
|
||||||
# ---- 尾轮决策 (必要组成环节) ----
|
# ---- 尾轮决策 (必要组成环节) ----
|
||||||
n_blk = b * m_cnt * n_cnt
|
n_blk = b * m_cnt * n_cnt
|
||||||
@@ -122,9 +121,15 @@ class AswBasicBranch(Branch):
|
|||||||
tail_m_cnt=tail["tail_m_cnt"], tail_n_cnt=tail["tail_n_cnt"],
|
tail_m_cnt=tail["tail_m_cnt"], tail_n_cnt=tail["tail_n_cnt"],
|
||||||
tail_k_cnt=1, tail_m_main=tail["tail_m_main"], tail_n_main=tail["tail_n_main"],
|
tail_k_cnt=1, tail_m_main=tail["tail_m_main"], tail_n_main=tail["tail_n_main"],
|
||||||
tail_block_cnt=tail["r"], tail_wave_num=tail["n_wave"],
|
tail_block_cnt=tail["r"], tail_wave_num=tail["n_wave"],
|
||||||
fixpipe_unitflag=True,
|
fixpipe_unitflag=True, # UnitFlag 单缓冲: tile 内 16x16x16 细粒度流水
|
||||||
out_dtype_bytes=case.dtype_out_bytes,
|
out_dtype_bytes=case.dtype_out_bytes,
|
||||||
note=f"L2场景{l2_scene}, r_in={r_in:.2f}; 尾轮: {tail['reason']}",
|
note=(f"tile枚举: {pick_note}; Base tile {base_m}x{base_n} "
|
||||||
|
f"(L0C 单缓冲方形用满); "
|
||||||
|
f"L2场景: {scene['label']} (V_in={case.input_bytes/1048576:.1f}MB, "
|
||||||
|
f"V_out={case.output_bytes/1048576:.1f}MB, "
|
||||||
|
f"L2={s.l2_bytes/1048576:.0f}MB); "
|
||||||
|
+ (scene["note_extra"] + "; " if scene["note_extra"] else "")
|
||||||
|
+ f"尾轮: {tail['reason']}"),
|
||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -142,6 +147,7 @@ class AswBasicBranch(Branch):
|
|||||||
single_m, single_n = clamp_base_mn_l0c(single_m, single_n, False, s)
|
single_m, single_n = clamp_base_mn_l0c(single_m, single_n, False, s)
|
||||||
# base_k 由 L0A/L0B 容量按 dtype 反推 (issue#5: 原硬编码 min(K,64) 在 fp32 下溢出)
|
# base_k 由 L0A/L0B 容量按 dtype 反推 (issue#5: 原硬编码 min(K,64) 在 fp32 下溢出)
|
||||||
base_k = clamp_base_k(single_m, single_n, dt, case.k, s)
|
base_k = clamp_base_k(single_m, single_n, dt, case.k, s)
|
||||||
|
out_l2 = case.input_bytes + case.output_bytes <= s.l2_bytes # R4 (issue#30)
|
||||||
return ImplPlan(
|
return ImplPlan(
|
||||||
case_id=case.case_id, branch=self.name + "_降核",
|
case_id=case.case_id, branch=self.name + "_降核",
|
||||||
used_core_num=used,
|
used_core_num=used,
|
||||||
@@ -154,7 +160,9 @@ class AswBasicBranch(Branch):
|
|||||||
l1_form="标准核内流水",
|
l1_form="标准核内流水",
|
||||||
base_m=single_m, base_n=single_n,
|
base_m=single_m, base_n=single_n,
|
||||||
base_k=base_k,
|
base_k=base_k,
|
||||||
l2_policy_in="allocate", l2_policy_out="direct_gm",
|
l2_policy_in="allocate",
|
||||||
|
l2_policy_out=("resident(整case全驻留S_A)"
|
||||||
|
if out_l2 else "direct_gm(输出直写GM)"),
|
||||||
swizzle_w=0, workspace_bytes=0,
|
swizzle_w=0, workspace_bytes=0,
|
||||||
tail_strategy="不涉及(每核一块无尾轮)",
|
tail_strategy="不涉及(每核一块无尾轮)",
|
||||||
fixpipe_unitflag=True, out_dtype_bytes=case.dtype_out_bytes,
|
fixpipe_unitflag=True, out_dtype_bytes=case.dtype_out_bytes,
|
||||||
@@ -162,24 +170,126 @@ class AswBasicBranch(Branch):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def _pick_single_core(self, m, n, min_blocks, base_m, base_n, dt):
|
def _pick_single_core(self, case: BmmCase, base_m: int, base_n: int) -> tuple:
|
||||||
"""Step 1: 满足并行度下限前提下 SingleCoreM/N 尽量大, 长宽比跟随 M/N."""
|
"""SingleCoreM/N + k_L1 + mCnt/nCnt 有界枚举 (v1.91 §5.2 + v1.5 §2.1 修正口径).
|
||||||
|
|
||||||
|
- P = ⌈C/B⌉; P=1 (B>=C): 先试不切分 (mCnt=nCnt=1, tile 跟随 M/N, 情形1),
|
||||||
|
约束不满足则进入枚举强制切分;
|
||||||
|
- 枚举空间: mCnt ∈ [1, ⌈M/BaseM⌉], nCnt ∈ [1, ⌈N/BaseN⌉] (约束4 上界:
|
||||||
|
SingleCore 为 Base 整数倍), 且 B·mCnt·nCnt >= C (约束1 并行度);
|
||||||
|
- 约束 2 (L1 双缓冲): k_L1 = min(K, ⌊L1/(2(sM+sN)·dt)⌋16), 须 ≥ 一个
|
||||||
|
fractal 且满足 dValue 下限;
|
||||||
|
- 约束 3 (搬移效率, 转置感知): A 非转置 dValue = k_L1·dt / A 转置 = sM·dt;
|
||||||
|
B 非转置 dValue = sN·dt / B 转置 = k_L1·dt; 均须 ≥ 256B (dValue 硬件
|
||||||
|
突发下限); 且 sM·k_L1·dt 与 k_L1·sN·dt ≥ min_TileSize (16KB);
|
||||||
|
- 目标 (v1.5 §2.1 修正: 稳态流水下单块搬入 = K·(sM+sN)·dt, 与分次粒度
|
||||||
|
k_L1 无关, k_L1 只进约束): 每 batch 搬入量 K·dt·(nCnt·M + mCnt·N) 最小
|
||||||
|
(共享块重复读最少 = tile 尽可能大/少); 并列取 r = B·mCnt·nCnt mod C 最大
|
||||||
|
(尾轮块越多, 重切收益越大, v1.91 §5.2.d);
|
||||||
|
返回 (single_m, single_n, k_l1, m_cnt, n_cnt, note).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
# 从大到小枚举 (m_cnt*n_cnt >= min_blocks), 取最大 tile
|
b = case.batch_c
|
||||||
best = (base_m, base_n)
|
m, n, k = case.m, case.n, case.k
|
||||||
for m_cnt in range(1, int(math.sqrt(min_blocks)) + 2):
|
dt = case.dtype_in_bytes
|
||||||
n_cnt = ceil_div(min_blocks, m_cnt)
|
c = s.aic_num
|
||||||
if m_cnt * n_cnt < min_blocks:
|
p_min = ceil_div(c, b) # 最少切分块数 (约束1)
|
||||||
continue
|
dv_min = s.dvalue_hw_min # 256B
|
||||||
sm = align_down(ceil_div(m, m_cnt), base_m) or base_m
|
min_tile = s.min_tile_size # 16KB
|
||||||
sn = align_down(ceil_div(n, n_cnt), base_n) or base_n
|
|
||||||
# 约束 2: L1 容量 2(sM+sN)*k_l1*dt <= L1, k_l1 取 256B/dt
|
def eval_tile(sm: int, sn: int):
|
||||||
k_l1_min = s.dvalue_hw_min // dt
|
"""约束 2/3 + 目标值; 返回 (ok, k_l1, traffic_per_batch)."""
|
||||||
if 2 * (sm + sn) * k_l1_min * dt > s.l1_bytes:
|
k_cap = int(s.l1_bytes / (2 * (sm + sn) * dt))
|
||||||
continue
|
k_l1 = align_down(min(k, k_cap), s.fractal)
|
||||||
if sm * sn > best[0] * best[1]:
|
if k_l1 < s.fractal:
|
||||||
best = (sm, sn)
|
return False, 0, None
|
||||||
return best
|
# dValue 下限 (转置感知连续维, 同 issue#19 口径)
|
||||||
|
dv_a = sm * dt if case.trans_a else k_l1 * dt
|
||||||
|
dv_b = k_l1 * dt if case.trans_b else sn * dt
|
||||||
|
if dv_a < dv_min or dv_b < dv_min:
|
||||||
|
return False, 0, None
|
||||||
|
# 单次搬移量下限
|
||||||
|
if sm * k_l1 * dt < min_tile or k_l1 * sn * dt < min_tile:
|
||||||
|
return False, 0, None
|
||||||
|
mc = ceil_div(m, sm)
|
||||||
|
nc = ceil_div(n, sn)
|
||||||
|
traffic = k * dt * (nc * m + mc * n) # 每 batch 总搬入字节
|
||||||
|
return True, k_l1, traffic
|
||||||
|
|
||||||
|
# 情形 1: P=1 (B >= C) 先试不切分, tile 跟随 M/N
|
||||||
|
if p_min == 1:
|
||||||
|
sm = max(align_up(m, s.fractal), s.fractal)
|
||||||
|
sn = max(align_up(n, s.fractal), s.fractal)
|
||||||
|
ok, k_l1, traffic = eval_tile(sm, sn)
|
||||||
|
if ok:
|
||||||
|
return (sm, sn, k_l1, 1, 1,
|
||||||
|
"P=1(B>=C) 不切分, tile 跟随 M/N (v1.91 §5.2 情形1)")
|
||||||
|
|
||||||
|
# 情形 2: 有界枚举
|
||||||
|
best = None
|
||||||
|
m_max = ceil_div(m, base_m)
|
||||||
|
n_max = ceil_div(n, base_n)
|
||||||
|
for mc in range(1, m_max + 1):
|
||||||
|
sm = align_up(ceil_div(m, mc), base_m)
|
||||||
|
for nc in range(1, n_max + 1):
|
||||||
|
sn = align_up(ceil_div(n, nc), base_n)
|
||||||
|
ok, k_l1, traffic = eval_tile(sm, sn)
|
||||||
|
if not ok:
|
||||||
|
continue
|
||||||
|
mc_r = ceil_div(m, sm)
|
||||||
|
nc_r = ceil_div(n, sn)
|
||||||
|
# 约束 1: 并行度 (真实块数, 考虑 align 上取后的收缩)
|
||||||
|
if b * mc_r * nc_r < c:
|
||||||
|
continue
|
||||||
|
r = (b * mc_r * nc_r) % c
|
||||||
|
key = (traffic, -r) # 主键搬入量最小; 并列 r 最大
|
||||||
|
if best is None or key < best[0]:
|
||||||
|
best = (key, sm, sn, k_l1, mc_r, nc_r, traffic, r)
|
||||||
|
if best is not None:
|
||||||
|
_, sm, sn, k_l1, mc_r, nc_r, traffic, r = best
|
||||||
|
return (sm, sn, k_l1, mc_r, nc_r,
|
||||||
|
f"P={p_min}, 有界枚举最优 mCnt={mc_r} x nCnt={nc_r} "
|
||||||
|
f"(tile {sm}x{sn}, 每batch搬入{traffic/1048576:.1f}MB, r={r})")
|
||||||
|
|
||||||
|
# 情形 3: 放开约束 4 (Base 整数倍), 16 对齐网格 + 硬下限 (dValue>=128B) 再搜
|
||||||
|
# —— 兜底分支恒出方案 (issue#34): 256B 偏好无解时按硬下限给最优可行 tile,
|
||||||
|
# 标注效率降级 (搬移效率低于模型假设, 时延可能低估)
|
||||||
|
dv_hard = s.dvalue_min # 128B 硬下限
|
||||||
|
best2 = None
|
||||||
|
for sm in range(s.fractal, align_up(min(m, 1024), s.fractal) + 1, s.fractal):
|
||||||
|
for sn in range(s.fractal, align_up(min(n, 1024), s.fractal) + 1, s.fractal):
|
||||||
|
k_cap = int(s.l1_bytes / (2 * (sm + sn) * dt))
|
||||||
|
k_l1 = align_down(min(k, k_cap), s.fractal)
|
||||||
|
if k_l1 < s.fractal:
|
||||||
|
continue
|
||||||
|
dv_a = sm * dt if case.trans_a else k_l1 * dt
|
||||||
|
dv_b = k_l1 * dt if case.trans_b else sn * dt
|
||||||
|
if dv_a < dv_hard or dv_b < dv_hard:
|
||||||
|
continue
|
||||||
|
mc_r = ceil_div(m, sm)
|
||||||
|
nc_r = ceil_div(n, sn)
|
||||||
|
if b * mc_r * nc_r < c:
|
||||||
|
continue
|
||||||
|
traffic = k * dt * (nc_r * m + mc_r * n)
|
||||||
|
r = (b * mc_r * nc_r) % c
|
||||||
|
key = (traffic, -r)
|
||||||
|
if best2 is None or key < best2[0]:
|
||||||
|
best2 = (key, sm, sn, k_l1, mc_r, nc_r, traffic, r)
|
||||||
|
if best2 is not None:
|
||||||
|
_, sm, sn, k_l1, mc_r, nc_r, traffic, r = best2
|
||||||
|
return (sm, sn, k_l1, mc_r, nc_r,
|
||||||
|
f"效率降级(放开约束4, dValue 按 128B 硬下限, 搬移效率低于模型假设, "
|
||||||
|
f"时延可能低估): P={p_min}, mCnt={mc_r} x nCnt={nc_r} "
|
||||||
|
f"(tile {sm}x{sn}, 每batch搬入{traffic/1048576:.1f}MB, r={r})")
|
||||||
|
|
||||||
|
# 情形 4: 极端兜底 —— 兜底分支恒出方案 (issue#34): 效率下限物理不可满足
|
||||||
|
# 也照常给方案 + 标注效率降级 (搬移效率崩塌), 不判违规/不产 None
|
||||||
|
k_l1 = max(align_down(min(k, int(s.l1_bytes /
|
||||||
|
(2 * (base_m + base_n) * dt))),
|
||||||
|
s.fractal), s.fractal)
|
||||||
|
return (base_m, base_n, k_l1, ceil_div(m, base_m), ceil_div(n, base_n),
|
||||||
|
"效率降级(极端形状: 效率下限物理不可满足, 搬移效率崩塌, 时延可能低估; "
|
||||||
|
"方案供参考, 建议调整 dtype/布局或转置)")
|
||||||
|
|
||||||
def _swizzle_w(self) -> int:
|
def _swizzle_w(self) -> int:
|
||||||
s = self.spec
|
s = self.spec
|
||||||
@@ -189,17 +299,102 @@ class AswBasicBranch(Branch):
|
|||||||
w = d
|
w = d
|
||||||
return w
|
return w
|
||||||
|
|
||||||
def _l2_group_r_in(self, case, sm, sn) -> float:
|
def _l2_scene(self, case: BmmCase, sm: int, sn: int,
|
||||||
"""场景 C: L2 分组的重复读倍率 r_in = (n_grp*M + m_grp*N)/(M+N)."""
|
m_cnt: int | None = None, n_cnt: int | None = None) -> dict:
|
||||||
|
"""L2 场景判定 (make_plan 与 evaluate 共用, 设计文档 docs/05 §4.1, issue#32).
|
||||||
|
|
||||||
|
判定顺序 S_A -> S_B -> S_C (L2 为整芯片 128MB; 返回"每 batch"流量):
|
||||||
|
S_A 整case全驻留: V_in + V_out <= L2 -> 输出驻留 L2 (R4, GM 写=0);
|
||||||
|
S_B 单batch可驻留: 单 batch 输入可全驻留 (a_b+bb_b<=L2) 或**单侧全驻留 +
|
||||||
|
对侧滑窗** (驻留侧 <= L2 且 + 2x对侧单块 <= L2) -> GM 每字节恰读一次
|
||||||
|
(V_in), 共享块重复读全部命中 L2 读口 (n_cnt-1)/(m_cnt-1) 次;
|
||||||
|
S_C 双侧均不可全驻留: 整 L2 容量约束下搜索**最小 GM** 的 2D 分组
|
||||||
|
(力求最小 L2 替换), 组间共享块落空计 GM、组内窗口复用计 L2.
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
m, n, k, b = case.m, case.n, case.k, case.batch_c
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
d = s.l2_bytes / (b * k * dt)
|
a_b = m * k * dt
|
||||||
m_grp = max(1, int(d / (2 * sm)))
|
bb_b = k * n * dt
|
||||||
n_grp = max(1, int(d / (2 * sn)))
|
if m_cnt is None:
|
||||||
m_cnt = ceil_div(m, sm)
|
m_cnt = max(ceil_div(m, max(sm, 1)), 1)
|
||||||
n_cnt = ceil_div(n, sn)
|
if n_cnt is None:
|
||||||
return (ceil_div(n_cnt, n_grp) * m + ceil_div(m_cnt, m_grp) * n) / (m + n)
|
n_cnt = max(ceil_div(n, max(sn, 1)), 1)
|
||||||
|
blk_a = a_b / max(m_cnt, 1) # A 行块字节 (tile 行宽 x K)
|
||||||
|
blk_b = bb_b / max(n_cnt, 1) # B 列块字节 (K x tile 列宽)
|
||||||
|
if case.input_bytes + case.output_bytes <= s.l2_bytes:
|
||||||
|
return {"code": "S_A",
|
||||||
|
"label": "A_整case全驻留(输入+输出<=L2)",
|
||||||
|
"to_l2": True,
|
||||||
|
"gm_batch": a_b + bb_b,
|
||||||
|
"l2_batch": (n_cnt - 1) * a_b + (m_cnt - 1) * bb_b,
|
||||||
|
"note_extra": ""}
|
||||||
|
if (a_b + bb_b <= s.l2_bytes or # 双侧全驻留
|
||||||
|
bb_b + 2 * blk_a <= s.l2_bytes or # B 驻留 + A 行块滑窗
|
||||||
|
a_b + 2 * blk_b <= s.l2_bytes): # A 驻留 + B 列块滑窗
|
||||||
|
return {"code": "S_B",
|
||||||
|
"label": "B_单batch可驻留(全驻留或单侧驻留+对侧滑窗)",
|
||||||
|
"to_l2": False,
|
||||||
|
"gm_batch": a_b + bb_b,
|
||||||
|
"l2_batch": (n_cnt - 1) * a_b + (m_cnt - 1) * bb_b,
|
||||||
|
"note_extra": ""}
|
||||||
|
g = self._l2_group_min_gm(case, m_cnt, n_cnt, blk_a, blk_b)
|
||||||
|
return {"code": "S_C",
|
||||||
|
"label": "C_双侧超L2: 最小替换2D分组(组间落空GM, 窗口L2)",
|
||||||
|
"to_l2": False,
|
||||||
|
"gm_batch": g["gm_batch"],
|
||||||
|
"l2_batch": g["l2_batch"],
|
||||||
|
"note_extra": g["note"]}
|
||||||
|
|
||||||
|
def _l2_group_min_gm(self, case: BmmCase, m_cnt: int, n_cnt: int,
|
||||||
|
blk_a: float, blk_b: float) -> dict:
|
||||||
|
"""场景 C: 整 L2 容量约束下的最小 GM 2D 分组 (issue#32, docs/05 §4.3b).
|
||||||
|
|
||||||
|
容量约束: m_grp*blk_a + n_grp*blk_b <= L2 (组工作集占满整 L2, 不再对半);
|
||||||
|
目标: GM = ceil(n_cnt/n_grp)*a_b + ceil(m_cnt/m_grp)*bb_b 最小
|
||||||
|
(A 行块每列组一次 GM 首读; B 列块每行组一次 GM 首读);
|
||||||
|
窗口 L2: (n_cnt - ceil(n_cnt/n_grp))*a_b + (m_cnt - ceil(m_cnt/m_grp))*bb_b
|
||||||
|
(组内共享块其余次复用走 L2 读口, 与 S_B 口径一致, issue#32 失真b);
|
||||||
|
无可行分组 (单块对都超 L2) 时保守回落: 每共享块独立落 GM, 窗口不计.
|
||||||
|
"""
|
||||||
|
s = self.spec
|
||||||
|
m, n, k = case.m, case.n, case.k
|
||||||
|
dt = case.dtype_in_bytes
|
||||||
|
a_b = m * k * dt
|
||||||
|
bb_b = k * n * dt
|
||||||
|
best = None
|
||||||
|
for mg in range(1, m_cnt + 1):
|
||||||
|
for ng in range(1, n_cnt + 1):
|
||||||
|
if mg * blk_a + ng * blk_b > s.l2_bytes:
|
||||||
|
continue
|
||||||
|
ga = ceil_div(n_cnt, ng)
|
||||||
|
gb = ceil_div(m_cnt, mg)
|
||||||
|
gm = ga * a_b + gb * bb_b
|
||||||
|
if best is None or gm < best[0]:
|
||||||
|
best = (gm, mg, ng, ga, gb)
|
||||||
|
if best is None:
|
||||||
|
return {"gm_batch": n_cnt * a_b + m_cnt * bb_b, "l2_batch": 0.0,
|
||||||
|
"note": "分组无可行解(单块对超L2): 保守每共享块独立落GM"}
|
||||||
|
gm, mg, ng, ga, gb = best
|
||||||
|
l2 = (n_cnt - ga) * a_b + (m_cnt - gb) * bb_b
|
||||||
|
r = gm / (a_b + bb_b)
|
||||||
|
return {"gm_batch": gm, "l2_batch": l2,
|
||||||
|
"note": f"最小替换分组 m_grp={mg}x n_grp={ng} (GM倍率{r:.2f}, "
|
||||||
|
f"窗口L2/batch={l2/1048576:.1f}MB)"}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
def _block_times(self, case: BmmCase, sm: int, sn: int) -> tuple:
|
||||||
|
"""单块三段时延 (尾轮文档 v1.5 §2.1 口径: k_L1 稳态约掉, L2 命中带宽).
|
||||||
|
|
||||||
|
返回 (t_mmad_blk, t_mte2_blk, t_fix_blk); _decide_tail 主导项判定与
|
||||||
|
evaluate 尾轮残余 drain (issue#37) 同源共用.
|
||||||
|
"""
|
||||||
|
s = self.spec
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b) # issue#28
|
||||||
|
t_mmad = 2 * sm * sn * case.k / qc
|
||||||
|
t_mte2 = case.k * (sm + sn) * case.dtype_in_bytes / s.bw_l2_pc
|
||||||
|
t_fix = sm * sn * case.dtype_out_bytes / s.bw_pc
|
||||||
|
return t_mmad, t_mte2, t_fix
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def _decide_tail(self, case, sm, sn, n_blk, k_l1) -> dict:
|
def _decide_tail(self, case, sm, sn, n_blk, k_l1) -> dict:
|
||||||
@@ -210,7 +405,6 @@ class AswBasicBranch(Branch):
|
|||||||
r = n_blk % c
|
r = n_blk % c
|
||||||
rho = r / c
|
rho = r / c
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
|
||||||
|
|
||||||
base = dict(r=r, n_wave=n_wave, tail_m_cnt=1, tail_n_cnt=1,
|
base = dict(r=r, n_wave=n_wave, tail_m_cnt=1, tail_n_cnt=1,
|
||||||
tail_m_main=0, tail_n_main=0)
|
tail_m_main=0, tail_n_main=0)
|
||||||
@@ -218,11 +412,8 @@ class AswBasicBranch(Branch):
|
|||||||
if r == 0:
|
if r == 0:
|
||||||
return {**base, "strategy": "A0", "reason": "r=0 无尾轮"}
|
return {**base, "strategy": "A0", "reason": "r=0 无尾轮"}
|
||||||
|
|
||||||
# 主导项判定
|
# 主导项判定 (块级三段, 与 evaluate drain 同源 _block_times)
|
||||||
bw_eff = s.bw_l2_pc # L2 命中
|
t_mmad, t_mte2, t_fix = self._block_times(case, sm, sn)
|
||||||
t_mmad = 2 * sm * sn * case.k / s.q16
|
|
||||||
t_mte2 = case.k * (sm + sn) * dt / bw_eff
|
|
||||||
t_fix = sm * sn * out_b / s.bw_pc
|
|
||||||
t_block = max(t_mmad, t_mte2, t_fix)
|
t_block = max(t_mmad, t_mte2, t_fix)
|
||||||
area_dominated = t_block != t_mte2 # 面积型 = MMAD 或 FIX 主导
|
area_dominated = t_block != t_mte2 # 面积型 = MMAD 或 FIX 主导
|
||||||
|
|
||||||
@@ -261,37 +452,82 @@ class AswBasicBranch(Branch):
|
|||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
|
"""MTE2 两段块级模型 (issue#24 用户口径 + #28/#29/#30/#32):
|
||||||
|
|
||||||
|
- 首读走 GM (按 GM 带宽, 不叠加 L2); 共享块 (A 行块被 n_cnt 个 tile 读、
|
||||||
|
B 列块被 m_cnt 个 tile 读) 驻留 L2 后其余 (n_cnt-1)/(m_cnt-1) 次读走
|
||||||
|
L2 读口 (5.2TB/s 独享);
|
||||||
|
- 场景 (与 make_plan 同源 _l2_scene, 返回每 batch GM/L2 流量):
|
||||||
|
S_A/S_B: GM = V_in 一次 (S_B 含单侧全驻留+对侧滑窗, issue#32);
|
||||||
|
S_C: 最小替换 2D 分组 (整 L2 容量约束下最小 GM), 组间落空计 GM、
|
||||||
|
组内窗口复用计 L2;
|
||||||
|
- 输出落点 R4 (issue#30): 仅 S_A (整 case 输入+输出 <= L2) 驻留 L2
|
||||||
|
(5.2 写口, GM 写 = 0); S_B/S_C 直写 GM —— GM 读写共享总线累加由
|
||||||
|
assemble 的 MTE2 链处理 (issue#23);
|
||||||
|
- 字节列整芯片口径 (issue#29); Cube 算力按输入 dtype (issue#28);
|
||||||
|
- t_drain = 尾轮残余闭式 (issue#37, v1.5 §3.3/§4.3/§5): A0 (1-ρ)·T_block;
|
||||||
|
周长型 A1b/方案B 残余见下; 面积型与 r=0 恒 0 (与稳态严格相等).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
b = case.batch_c
|
b = case.batch_c
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
|
used = max(plan.used_core_num, 1)
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
|
||||||
flops = 2.0 * b * m * n * k
|
flops = 2.0 * b * m * n * k
|
||||||
t_mmad = flops / (plan.used_core_num * s.q16)
|
t_mmad = flops / (used * qc)
|
||||||
|
|
||||||
in_bytes = b * (m * k + k * n) * dt
|
# ---- 字节量 (整芯片口径) ----
|
||||||
out_bytes = b * m * n * out_b
|
out_all = b * m * n * out_b # 输出总字节
|
||||||
# r_in 从 note 里解析困难, 重新计算
|
m_cnt = max(plan.m_cnt, 1)
|
||||||
s_in = in_bytes
|
n_cnt = max(plan.n_cnt, 1)
|
||||||
s_out = out_bytes
|
|
||||||
if s_in + s_out <= s.l2_bytes or s_in <= s.l2_bytes:
|
|
||||||
r_in = 1.0
|
|
||||||
else:
|
|
||||||
r_in = self._l2_group_r_in(case, plan.single_core_m, plan.single_core_n)
|
|
||||||
|
|
||||||
t_mte2 = r_in * in_bytes / (plan.used_core_num * s.bw_pc)
|
# ---- L2 场景 (S_A/S_B/S_C, 与 make_plan 同源; 每 batch 流量 x B) ----
|
||||||
to_l2 = "resident" in plan.l2_policy_out
|
scene = self._l2_scene(case, plan.single_core_m, plan.single_core_n,
|
||||||
t_fix = out_bytes / (plan.used_core_num * (s.bw_l2_pc if to_l2 else s.bw_pc))
|
m_cnt, n_cnt)
|
||||||
|
to_l2 = scene["to_l2"]
|
||||||
|
gm_read = b * scene["gm_batch"]
|
||||||
|
l2_read = b * scene["l2_batch"]
|
||||||
|
t_gm = gm_read / (used * s.bw_pc)
|
||||||
|
t_l2 = l2_read / (used * s.bw_l2_pc)
|
||||||
|
|
||||||
# drain: 尾轮暴露 (方案 B 已均匀重切, drain 小; A1b 尾轮凑满, drain 小; A0 尾轮 r 核空转)
|
# ---- Fixpipe (R4) ----
|
||||||
|
t_fix = out_all / (used * (s.bw_l2_pc if to_l2 else s.bw_pc))
|
||||||
|
|
||||||
|
# ---- drain: 尾轮残余时延 (issue#37, 尾轮 v1.5 §3.3/§4.3/§5 闭式) ----
|
||||||
|
# 稳态聚合 t_steady ≡ (n_wave-1+rho)·T_block (均匀分块下 N_blk/used 恒等),
|
||||||
|
# drain 只计尾轮结构相对该稳态的残余:
|
||||||
|
# A0 (r>0): (1-rho)·T_block (T_A0 = n_wave·T_block)
|
||||||
|
# 面积型 A1b/方案B: 0 (§4.3 总量守恒, 与稳态严格相等)
|
||||||
|
# 周长型 A1b: (sqrt(rho)-rho)·T_load (§5: (n_wave-1+√ρ)·T_load)
|
||||||
|
# 周长型 方案B: (√(n_wave(n_wave-1+rho))-(n_wave-1+rho))·T_load
|
||||||
|
# 块级时延与 _decide_tail 同源 (_block_times, L2 命中口径); 未建模策略
|
||||||
|
# (如 A1a) 维持原口径 0。首块填充/末块排空 (v1.91 §3.1 O(T_comp+T_write))
|
||||||
|
# 在 UnitFlag 16-granule 细粒度流水下为 granule 级小量, 量级未标定,
|
||||||
|
# 不入模型 (docs/02_分支理论/06 §3 标注)。
|
||||||
t_drain = 0.0
|
t_drain = 0.0
|
||||||
if plan.tail_strategy == "A0" and plan.tail_block_cnt > 0:
|
r = plan.tail_block_cnt
|
||||||
t_drain = max(t_mmad, t_mte2, t_fix) # 尾轮空转一个整块
|
if r > 0:
|
||||||
|
n_wave = plan.tail_wave_num if plan.tail_wave_num > 0 \
|
||||||
|
else ceil_div(b * m_cnt * n_cnt, used)
|
||||||
|
rho = min(r / used, 1.0)
|
||||||
|
t_mm_b, t_mv_b, t_fx_b = self._block_times(
|
||||||
|
case, plan.single_core_m, plan.single_core_n)
|
||||||
|
t_block = max(t_mm_b, t_mv_b, t_fx_b)
|
||||||
|
if plan.tail_strategy == "A0":
|
||||||
|
t_drain = (1.0 - rho) * t_block
|
||||||
|
elif t_block == t_mv_b: # 仅周长型主导有残余 (面积型严格 0)
|
||||||
|
if plan.tail_strategy == "A1b":
|
||||||
|
t_drain = (math.sqrt(rho) - rho) * t_mv_b
|
||||||
|
elif plan.tail_strategy == "方案B":
|
||||||
|
t_drain = (math.sqrt(n_wave * (n_wave - 1 + rho))
|
||||||
|
- (n_wave - 1 + rho)) * t_mv_b
|
||||||
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_mte2, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
t_mte2_gm=t_gm, t_mte2_l2=t_l2, t_dma_cmd=0.0,
|
||||||
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
||||||
gm_read_bytes=r_in * in_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
gm_read_bytes=gm_read, l2_read_bytes=l2_read, dma_cmd_count=0.0,
|
||||||
cube_flops=flops, fixpipe_bytes=out_bytes,
|
cube_flops=flops, fixpipe_bytes=out_all,
|
||||||
|
fixpipe_to_gm=(not to_l2),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ batch 间流水靠 L1 双 buffer / 驻留侧预取 (c 形态半预算) 掩盖.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down
|
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down, gm_move_time
|
||||||
from ..timing import assemble_timing
|
from ..timing import assemble_timing, output_to_l2
|
||||||
from .base import Branch, BranchResult, ConditionCheck
|
from .base import Branch, BranchResult, ConditionCheck
|
||||||
|
|
||||||
|
|
||||||
@@ -23,11 +23,34 @@ class IterBatchBranch(Branch):
|
|||||||
def __init__(self, spec: NpuSpec = ASCEND950PR):
|
def __init__(self, spec: NpuSpec = ASCEND950PR):
|
||||||
super().__init__(spec)
|
super().__init__(spec)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# 单命令搬移 tile (issue#36 效率模型口径, evaluate 与 MergeBatch 仲裁共用)
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
def move_tiles(self, case: BmmCase) -> tuple:
|
||||||
|
"""返回 (A侧单命令tile字节, B侧单命令tile字节).
|
||||||
|
|
||||||
|
a/b 形态: 双侧整 K 一次搬入; c 形态: 驻留侧整 K + 对侧 k_l1 分块;
|
||||||
|
d 形态: 双侧 k_l1 分块。tile = nValue*dValue*dt (乘积与转置/排布无关)。
|
||||||
|
"""
|
||||||
|
m, n, k = case.m, case.n, case.k
|
||||||
|
dt = case.dtype_in_bytes
|
||||||
|
form, k_l1, _, resident = self.l1_form(case)
|
||||||
|
if form in ("a", "b"):
|
||||||
|
return m * k * dt, k * n * dt
|
||||||
|
if form == "c":
|
||||||
|
return (m * k * dt, k_l1 * n * dt) if resident == "A" \
|
||||||
|
else (m * k_l1 * dt, k * n * dt)
|
||||||
|
return m * k_l1 * dt, k_l1 * n * dt # d 或 None (兜底)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# L1 驻留形态判定 (v0.98 §六 条件 3, 四选一)
|
# L1 驻留形态判定 (v0.98 §六 条件 3, 四选一)
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def l1_form(self, case: BmmCase) -> tuple:
|
def l1_form(self, case: BmmCase) -> tuple:
|
||||||
"""返回 (形态 'a'/'b'/'c'/'d'/None, k_l1, 说明)."""
|
"""返回 (形态 'a'/'b'/'c'/'d'/None, k_l1, 说明, 驻留侧 'A'/'B'/None).
|
||||||
|
|
||||||
|
驻留侧仅 c 形态非 None (issue#36: 效率模型需要区分两侧单命令 tile —
|
||||||
|
c 形态驻留侧整侧一次搬入 (全 K), 对侧按 k_l1 分块).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
@@ -36,31 +59,42 @@ class IterBatchBranch(Branch):
|
|||||||
|
|
||||||
# a) 单 batch 全驻留
|
# a) 单 batch 全驻留
|
||||||
if b_core == 1 and single <= s.l1_bytes:
|
if b_core == 1 and single <= s.l1_bytes:
|
||||||
return "a", k, f"单batch全驻留: (MK+KN)*dtype={single/1024:.0f}KB <= L1"
|
return "a", k, f"单batch全驻留: (MK+KN)*dtype={single/1024:.0f}KB <= L1", None
|
||||||
|
|
||||||
# b) 双 batch 乒乓
|
# b) 双 batch 乒乓
|
||||||
if b_core > 1 and 2 * single <= s.l1_bytes:
|
if b_core > 1 and 2 * single <= s.l1_bytes:
|
||||||
return "b", k, f"双batch乒乓: 2*(MK+KN)*dtype={2*single/1024:.0f}KB <= L1"
|
return "b", k, f"双batch乒乓: 2*(MK+KN)*dtype={2*single/1024:.0f}KB <= L1", None
|
||||||
|
|
||||||
# c) 一侧驻留 + 对侧切 K, 预算按 b_core 分档
|
# c) 一侧驻留 + 对侧切 K, 预算按 b_core 分档
|
||||||
|
# dValue 守卫与条件 4 / constraints 同源 (issue#19): 转置感知连续维判据
|
||||||
|
# (dv_a=M*dt 当 A 转置, 否则 k_l1*dt; dv_b=N*dt 当 B 不转置, 否则 k_l1*dt).
|
||||||
|
from ..models import dvalue_contig_dims
|
||||||
l1_budget = s.l1_bytes / min(b_core, 2)
|
l1_budget = s.l1_bytes / min(b_core, 2)
|
||||||
for resident, side in ((m * k * dt, "A"), (k * n * dt, "B")):
|
for resident, side in ((m * k * dt, "A"), (k * n * dt, "B")):
|
||||||
other = n * dt if side == "A" else m * dt
|
other = n * dt if side == "A" else m * dt
|
||||||
if resident <= l1_budget:
|
if resident <= l1_budget:
|
||||||
k_l1 = min(int((l1_budget - resident) / other / 2), k)
|
k_l1 = min(int((l1_budget - resident) / other / 2), k)
|
||||||
k_l1 = align_down(max(k_l1, s.fractal), s.fractal)
|
k_l1 = align_down(max(k_l1, s.fractal), s.fractal)
|
||||||
if k_l1 >= s.fractal and k_l1 * dt >= s.dvalue_min:
|
dv_a, dv_b = dvalue_contig_dims(case, k_l1)
|
||||||
|
# 双缓冲适配: 驻留侧 + 对侧 k_l1 段 x2 必须 <= 预算
|
||||||
|
# (resident 恰占满预算时 k_l1 被抬到 16 会造成超预算的假方案)
|
||||||
|
if (k_l1 >= s.fractal and
|
||||||
|
resident + 2 * k_l1 * other <= l1_budget and
|
||||||
|
dv_a >= s.dvalue_min and dv_b >= s.dvalue_min):
|
||||||
return "c", k_l1, (
|
return "c", k_l1, (
|
||||||
f"一侧驻留({side})+对侧切K: {side}驻留{resident/1024:.0f}KB, "
|
f"一侧驻留({side})+对侧切K: {side}驻留{resident/1024:.0f}KB, "
|
||||||
f"预算L1/{min(b_core,2)}, k_L1={k_l1}")
|
f"预算L1/{min(b_core,2)}, k_L1={k_l1}, "
|
||||||
|
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B"), side
|
||||||
# b_core>=2 时另一半 L1 预取下一 batch 驻留侧, 边界无气泡
|
# b_core>=2 时另一半 L1 预取下一 batch 驻留侧, 边界无气泡
|
||||||
|
|
||||||
# d) 两侧都切 K (兜底)
|
# d) 两侧都切 K (兜底)
|
||||||
k_l1 = align_down(int(s.l1_bytes / (2 * (m + n) * dt)), s.fractal)
|
k_l1 = align_down(int(s.l1_bytes / (2 * (m + n) * dt)), s.fractal)
|
||||||
if k_l1 >= s.fractal and k_l1 * dt >= s.dvalue_min:
|
dv_a, dv_b = dvalue_contig_dims(case, k_l1)
|
||||||
return "d", k_l1, f"两侧都切K: k_L1={k_l1}, K段成对流水, batch边界天然无缝"
|
if (k_l1 >= s.fractal and dv_a >= s.dvalue_min and dv_b >= s.dvalue_min):
|
||||||
|
return "d", k_l1, (f"两侧都切K: k_L1={k_l1}, K段成对流水, batch边界天然无缝; "
|
||||||
|
f"dValueA={dv_a:.0f}B/dValueB={dv_b:.0f}B"), None
|
||||||
|
|
||||||
return None, 0, "L1 四形态均不满足 (M/N 相对 L1 过大)"
|
return None, 0, "L1 四形态均不满足 (M/N 相对 L1 过大)", None
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# 进入条件 (v0.98 §六, 四条同时满足)
|
# 进入条件 (v0.98 §六, 四条同时满足)
|
||||||
@@ -84,21 +118,28 @@ class IterBatchBranch(Branch):
|
|||||||
"2_负载均衡: B mod C == 0 或 >= minCoreNum",
|
"2_负载均衡: B mod C == 0 或 >= minCoreNum",
|
||||||
c2, f"B mod C={rem}, minCoreNum={s.min_core_num}"))
|
c2, f"B mod C={rem}, minCoreNum={s.min_core_num}"))
|
||||||
|
|
||||||
form, k_l1, form_desc = self.l1_form(case)
|
form, k_l1, form_desc, _resident = self.l1_form(case)
|
||||||
checks.append(ConditionCheck(
|
checks.append(ConditionCheck(
|
||||||
"3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读)",
|
"3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读)",
|
||||||
form is not None, form_desc))
|
form is not None, form_desc))
|
||||||
|
|
||||||
# 条件 4: 搬移效率下限 (c/d 切分后)
|
# 条件 4: 搬移效率下限 (c/d 切分后)
|
||||||
|
# dValue 判定按转置调整连续维 (与 l1_form 守卫/constraints 同源, issue#19):
|
||||||
|
# A 不转置: K 向连续 -> k_l1*dt; A 转置: M 向连续 -> M*dt;
|
||||||
|
# B 不转置: N 向连续 -> N*dt; B 转置: K 向连续 -> k_l1*dt.
|
||||||
|
from ..models import dvalue_contig_dims
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
if form in ("c", "d"):
|
if form in ("c", "d"):
|
||||||
tile_ok = (k_l1 * m * dt >= s.min_tile_size) or (k_l1 * n * dt >= s.min_tile_size)
|
tile_ok = (k_l1 * m * dt >= s.min_tile_size) or (k_l1 * n * dt >= s.min_tile_size)
|
||||||
dv_ok = k_l1 * dt >= s.dvalue_min
|
dv_a, dv_b = dvalue_contig_dims(case, k_l1)
|
||||||
|
# 切 K 时 A/B 两侧分段各自搬移, 两侧连续维 dValue 均须 >= 下限 (与生成守卫/约束同源,
|
||||||
|
# issue#19; 非转置时两侧同为 k_l1*dt, 等价于历史口径)
|
||||||
|
dv_ok = dv_a >= s.dvalue_min and dv_b >= s.dvalue_min
|
||||||
c4 = tile_ok and dv_ok
|
c4 = tile_ok and dv_ok
|
||||||
checks.append(ConditionCheck(
|
checks.append(ConditionCheck(
|
||||||
"4_搬移效率: 搬移分块>=min_TileSize 且 dValue>=128B",
|
"4_搬移效率: 搬移分块>=min_TileSize 且 dValue>=128B (转置调整连续维)",
|
||||||
c4, f"tile={max(k_l1*m*dt, k_l1*n*dt)/1024:.1f}KB, dValue={k_l1*dt}B"))
|
c4, f"tile={max(k_l1*m*dt, k_l1*n*dt)/1024:.1f}KB, dValueA={dv_a}B, dValueB={dv_b}B"))
|
||||||
else:
|
else:
|
||||||
checks.append(ConditionCheck(
|
checks.append(ConditionCheck(
|
||||||
"4_搬移效率(a/b形态不切分, 恒满足)", True, ""))
|
"4_搬移效率(a/b形态不切分, 恒满足)", True, ""))
|
||||||
@@ -113,7 +154,7 @@ class IterBatchBranch(Branch):
|
|||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
b_core = -(-case.batch_c // s.aic_num)
|
b_core = -(-case.batch_c // s.aic_num)
|
||||||
form, k_l1, form_desc = self.l1_form(case)
|
form, k_l1, form_desc, _resident = self.l1_form(case)
|
||||||
form = form or "d"
|
form = form or "d"
|
||||||
|
|
||||||
# L0 级 tile (BaseM x BaseN): 核内 L1->L0 的切分, 与核间切分无关.
|
# L0 级 tile (BaseM x BaseN): 核内 L1->L0 的切分, 与核间切分无关.
|
||||||
@@ -153,6 +194,12 @@ class IterBatchBranch(Branch):
|
|||||||
form_name = {"a": "a_单batch全驻留", "b": "b_双batch乒乓",
|
form_name = {"a": "a_单batch全驻留", "b": "b_双batch乒乓",
|
||||||
"c": "c_一侧驻留+对侧切K", "d": "d_两侧都切K"}[form]
|
"c": "c_一侧驻留+对侧切K", "d": "d_两侧都切K"}[form]
|
||||||
|
|
||||||
|
# 输出落点 (issue#30, R4): 整 case 输入+输出 <= L2 才驻留 L2
|
||||||
|
out_l2 = output_to_l2(case, s)
|
||||||
|
l2_out = ("resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)"
|
||||||
|
if out_l2 else
|
||||||
|
"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)")
|
||||||
|
|
||||||
return ImplPlan(
|
return ImplPlan(
|
||||||
case_id=case.case_id, branch=self.name,
|
case_id=case.case_id, branch=self.name,
|
||||||
used_core_num=s.aic_num,
|
used_core_num=s.aic_num,
|
||||||
@@ -163,54 +210,68 @@ class IterBatchBranch(Branch):
|
|||||||
k_l1=k_l1, b_l1=2 if form == "b" else 1, l1_form=form_name,
|
k_l1=k_l1, b_l1=2 if form == "b" else 1, l1_form=form_name,
|
||||||
base_m=base_m, base_n=base_n, base_k=base_k,
|
base_m=base_m, base_n=base_n, base_k=base_k,
|
||||||
l2_policy_in="allocate(GM->L1随路驻留L2)",
|
l2_policy_in="allocate(GM->L1随路驻留L2)",
|
||||||
l2_policy_out="direct_gm(输出仅写一次,直写GM不占L2)",
|
l2_policy_out=l2_out,
|
||||||
swizzle_w=0, workspace_bytes=0,
|
swizzle_w=0, workspace_bytes=0,
|
||||||
tail_strategy="不涉及(核内不切M/N)",
|
tail_strategy="不涉及(核内不切M/N)",
|
||||||
fixpipe_unitflag=True,
|
fixpipe_unitflag=True,
|
||||||
out_dtype_bytes=case.dtype_out_bytes,
|
out_dtype_bytes=case.dtype_out_bytes,
|
||||||
note=form_desc,
|
note=form_desc + f"; 输出落点: {'L2驻留' if out_l2 else '直写GM'}",
|
||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# 时延评估 (v1.1 §4 端到端模型, IterBatch 侧)
|
# 时延评估 (v1.1 §4 端到端模型, IterBatch 侧)
|
||||||
|
# 口径: dtype 感知算力 (issue#28); GM 每字节恰读一次 = V_in (issue#31);
|
||||||
|
# 字节列整芯片 (issue#29); 输出落点 R4 (issue#30).
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
s = self.spec
|
s = self.spec
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
|
b = case.batch_c
|
||||||
b_core, k_l1 = plan.b_core, plan.k_l1
|
b_core, k_l1 = plan.b_core, plan.k_l1
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
out_l2 = output_to_l2(case, s)
|
||||||
|
w_fix = s.bw_l2_pc if out_l2 else s.bw_pc
|
||||||
|
|
||||||
k_truncated = k_l1 >= k
|
k_truncated = k_l1 >= k
|
||||||
n_k = 1 if k_truncated else -(-k // k_l1)
|
n_k = 1 if k_truncated else -(-k // k_l1)
|
||||||
|
|
||||||
t_load = min(k_l1, k) * (m + n) * dt / s.bw_pc
|
t_comp_chunk = 2.0 * m * n * min(k_l1, k) / qc
|
||||||
t_comp_chunk = 2.0 * m * n * min(k_l1, k) / s.q16
|
t_write = m * n * out_b / w_fix
|
||||||
t_write = m * n * out_b / s.bw_pc
|
|
||||||
|
|
||||||
# 搬移: 每 batch n_K 次 GM->L1, 每次含 T_cmd
|
# 搬移 (issue#31): 每 batch 的 K 段/驻留侧数据互不重叠, 每个输入字节恰好从
|
||||||
|
# GM 读一次 (形态 c 驻留侧每 batch 只搬一次; 切 K 末段按实际剩余计) ->
|
||||||
|
# GM 数据量 = V_in, 数据时延 = V_in/芯片带宽 (全核并发);
|
||||||
|
# n_K/k_L1 只决定 DMA 命令次数 (T_cmd) 与双缓冲调度, 不放大数据量.
|
||||||
|
# dma_cmds 为单核命令数 (各核并行, issue#29 口径)
|
||||||
dma_cmds = b_core * n_k
|
dma_cmds = b_core * n_k
|
||||||
t_mte2_data = dma_cmds * t_load
|
# 搬移效率 (issue#36): 单命令 tile = nValue*dValue*dt 决定有效带宽
|
||||||
|
# (达 min_TileSize 饱和, 之下线性退化), 分侧口径见 move_tiles
|
||||||
|
tile_a, tile_b = self.move_tiles(case)
|
||||||
|
t_mte2_data = gm_move_time(b * m * k * dt, b * k * n * dt,
|
||||||
|
tile_a, tile_b, s)
|
||||||
t_dma_cmd = dma_cmds * s.t_cmd
|
t_dma_cmd = dma_cmds * s.t_cmd
|
||||||
t_mte2 = t_mte2_data + t_dma_cmd
|
t_mte2 = t_mte2_data + t_dma_cmd
|
||||||
|
|
||||||
# Cube: 无冗余
|
# Cube: 无冗余; 每核 flops = b_core*2MNK (整芯片列 = b*2MNK)
|
||||||
flops_pc = b_core * 2.0 * m * n * k
|
flops_pc = b_core * 2.0 * m * n * k
|
||||||
t_mmad = flops_pc / s.q16
|
flops_chip = b * 2.0 * m * n * k
|
||||||
|
t_mmad = flops_pc / qc
|
||||||
|
|
||||||
# Fixpipe: b_core 个 batch 输出, 按 C dtype
|
# Fixpipe (R4): b_core 个 batch 输出, 按 C dtype
|
||||||
fix_bytes_pc = b_core * m * n * out_b
|
fix_bytes_pc = b_core * m * n * out_b
|
||||||
t_fix = fix_bytes_pc / s.bw_pc
|
fix_bytes_chip = b * m * n * out_b
|
||||||
|
t_fix = fix_bytes_pc / w_fix
|
||||||
|
|
||||||
# drain: 末 batch 排空 = T_comp + T_write
|
# drain: 末 batch 排空 = T_comp + T_write
|
||||||
t_drain = t_comp_chunk + t_write
|
t_drain = t_comp_chunk + t_write
|
||||||
|
|
||||||
gm_bytes_pc = b_core * (m * k + k * n) * dt
|
|
||||||
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_mte2_data, t_mte2_l2=0.0, t_dma_cmd=t_dma_cmd,
|
t_mte2_gm=t_mte2_data, t_mte2_l2=0.0, t_dma_cmd=t_dma_cmd,
|
||||||
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
||||||
gm_read_bytes=gm_bytes_pc, l2_read_bytes=0.0,
|
gm_read_bytes=case.input_bytes, l2_read_bytes=0.0,
|
||||||
dma_cmd_count=dma_cmds, cube_flops=flops_pc,
|
dma_cmd_count=dma_cmds, cube_flops=flops_chip,
|
||||||
fixpipe_bytes=fix_bytes_pc,
|
fixpipe_bytes=fix_bytes_chip,
|
||||||
|
fixpipe_to_gm=(not out_l2),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -4,9 +4,13 @@
|
|||||||
- 《BMM算子优化分析 v0.98》§五 (进入条件 + 实现方案 Step1~3)
|
- 《BMM算子优化分析 v0.98》§五 (进入条件 + 实现方案 Step1~3)
|
||||||
- 《MergeBatch_vs_IterBatch分析 v1.1》§三/§四 (执行模型 + 分界条件)
|
- 《MergeBatch_vs_IterBatch分析 v1.1》§三/§四 (执行模型 + 分界条件)
|
||||||
|
|
||||||
核心思想: 合并 b0 个 batch 的 A'[b0*M,K] @ B'[K,b0*N] 为单次 DMA 搬入,
|
核心思想: 合并 b0 个 batch 的 A'[b0*M,K] @ B'[K,b0*N] 为单次 DMA 搬入。
|
||||||
减少 GM->L1 搬移命令数 (省 b0 倍 T_cmd); 交叉项被算出但丢弃 (冗余比例 (b0-1)/b0),
|
合并的核心收益有二 (用户澄清, issue#36): (1) 搬移命令数减少 (省 T_cmd);
|
||||||
进入条件 5 保证 case 为访存 Bound, 冗余算力被搬移时延掩盖.
|
(2) **单块 tile = nValue*dValue*dt 放大 b0 倍 -> 搬移效率提升** (堆叠方向视
|
||||||
|
转置/排布: A ND 非转置沿 M(nValue) 堆叠, B ND 非转置沿 N(dValue) 堆叠) ——
|
||||||
|
即便 T_cmd=0 (950PR 默认, 未标定按 0) 收益依然成立, 由 move_eff 效率模型刻画。
|
||||||
|
核心劣势是交叉项冗余算力 ((b0-1)/b0 被算出但丢弃) 与 drain 暴露放大 b0 倍;
|
||||||
|
进入条件 5 保证 case 为访存 Bound, 冗余算力被搬移时延掩盖。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -14,8 +18,8 @@ from __future__ import annotations
|
|||||||
import math
|
import math
|
||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down
|
from ..models import BmmCase, ImplPlan, HardwareTiming, align_down, gm_move_time
|
||||||
from ..timing import MoveInPlan, assemble_timing
|
from ..timing import assemble_timing, output_to_l2
|
||||||
from .base import Branch, BranchResult, ConditionCheck
|
from .base import Branch, BranchResult, ConditionCheck
|
||||||
|
|
||||||
MIN_B0 = 2 # b0: 合并搬移有收益的最小合并数
|
MIN_B0 = 2 # b0: 合并搬移有收益的最小合并数
|
||||||
@@ -82,6 +86,20 @@ class MergeBatchBranch(Branch):
|
|||||||
"5_访存Bound: 2MN/(M+N) < R16/b0",
|
"5_访存Bound: 2MN/(M+N) < R16/b0",
|
||||||
c5, f"AI={ai:.1f} vs R16/b0={s.r16/MIN_B0:.1f}"))
|
c5, f"AI={ai:.1f} vs R16/b0={s.r16/MIN_B0:.1f}"))
|
||||||
|
|
||||||
|
# 条件 6: 转置对齐 (参考 bmmv3 源码条件 17/20)
|
||||||
|
# A 转置且 M>1 时, tempAlignM = b0 * alignM (M 维按 b0 对齐)
|
||||||
|
# A 转置或 B 不转置时, minBaseK 需按 basic_block_size 对齐
|
||||||
|
from ..models import align_up
|
||||||
|
if case.trans_a and m > 1:
|
||||||
|
temp_align_m = MIN_B0 * align_up(m, s.fractal)
|
||||||
|
l0a_need_trans = temp_align_m * s.fractal * dt * 2
|
||||||
|
c6 = l0a_need_trans <= s.l0a_bytes
|
||||||
|
checks.append(ConditionCheck(
|
||||||
|
"6_A转置对齐: A转置时 tempAlignM=b0*alignM 需驻留 L0A",
|
||||||
|
c6, f"tempAlignM={temp_align_m}, L0A需{l0a_need_trans/1024:.0f}KB(≤{s.l0a_bytes/1024:.0f})"))
|
||||||
|
else:
|
||||||
|
checks.append(ConditionCheck("6_A转置对齐", True, "A不转置或M=1, 无额外对齐要求"))
|
||||||
|
|
||||||
return checks
|
return checks
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -125,9 +143,17 @@ class MergeBatchBranch(Branch):
|
|||||||
))
|
))
|
||||||
b_l1 = max(b_l1, b0)
|
b_l1 = max(b_l1, b0)
|
||||||
|
|
||||||
|
# Step 5: 输出落点 (issue#30): 整 case 输入+输出可驻留 L2 才写 L2 (R4)
|
||||||
|
out_l2 = output_to_l2(case, s)
|
||||||
|
l2_out = ("resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)"
|
||||||
|
if out_l2 else
|
||||||
|
"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)")
|
||||||
|
|
||||||
note = (f"b0={b0} (L0C上限{b0_l0c:.1f}/算存比上限{b0_ai:.1f}/b_core={b_core}); "
|
note = (f"b0={b0} (L0C上限{b0_l0c:.1f}/算存比上限{b0_ai:.1f}/b_core={b_core}); "
|
||||||
f"{'K截断' if k_truncated else 'L1绑定'}; "
|
f"{'K截断' if k_truncated else 'L1绑定'}; "
|
||||||
f"合并后单次DMA搬入 A'[{b0*m},{k_l1}]+B'[{k_l1},{b0*n}]")
|
f"合并后单次DMA搬入 A'[{b0*m},{k_l1}]+B'[{k_l1},{b0*n}]; "
|
||||||
|
f"输出落点: {'L2驻留' if out_l2 else '直写GM'} (整case V_in+V_out"
|
||||||
|
f"={case.input_bytes/1048576:.1f}MB vs L2={s.l2_bytes/1048576:.0f}MB)")
|
||||||
|
|
||||||
return ImplPlan(
|
return ImplPlan(
|
||||||
case_id=case.case_id, branch=self.name,
|
case_id=case.case_id, branch=self.name,
|
||||||
@@ -140,7 +166,7 @@ class MergeBatchBranch(Branch):
|
|||||||
# L0C 双缓冲约束已由进入条件 2 保证 (2*(b0*M)*(b0*N)*4B <= L0C)
|
# L0C 双缓冲约束已由进入条件 2 保证 (2*(b0*M)*(b0*N)*4B <= L0C)
|
||||||
base_m=b0 * m, base_n=b0 * n, base_k=max(min(k_l0, k_l1, k), s.fractal),
|
base_m=b0 * m, base_n=b0 * n, base_k=max(min(k_l0, k_l1, k), s.fractal),
|
||||||
l2_policy_in="allocate(GM->L1随路驻留L2)",
|
l2_policy_in="allocate(GM->L1随路驻留L2)",
|
||||||
l2_policy_out="direct_gm(输出仅写一次,直写GM不占L2)" if not case.out_nd else "direct_gm",
|
l2_policy_out=l2_out,
|
||||||
swizzle_w=0, workspace_bytes=0,
|
swizzle_w=0, workspace_bytes=0,
|
||||||
tail_strategy="不涉及(核内不切M/N)",
|
tail_strategy="不涉及(核内不切M/N)",
|
||||||
fixpipe_unitflag=True,
|
fixpipe_unitflag=True,
|
||||||
@@ -149,55 +175,72 @@ class MergeBatchBranch(Branch):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# 时延评估 (v1.1 §4 端到端模型)
|
# 时延评估 (v1.1 §4 端到端模型; issue#27/#28/#29/#30 口径)
|
||||||
|
# - Cube flops (issue#27 复核): 每合并步计算全网格 (b0*M)x(b0*N)xK
|
||||||
|
# 含交叉项冗余, flops_step = 2*(b0M)*(b0N)*K, 每核步数 = b_core/b0
|
||||||
|
# -> 每核 flops = b_core*b0*2MNK (与 CSV 及文档公式一致, 无修改);
|
||||||
|
# - 算力按输入 dtype (issue#28): q_cube(dtype_a, dtype_b);
|
||||||
|
# - 字节列 = 整芯片口径 (issue#29); GM 首读 = V_in 一次 (合并按组搬入
|
||||||
|
# 各 (batch,K段) 数据互不重叠, 无 L2 重复读);
|
||||||
|
# - 输出落点按整 case 驻留判定 (issue#30, R4).
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
s = self.spec
|
s = self.spec
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
|
b = case.batch_c
|
||||||
b_core, b0, k_l1 = plan.b_core, plan.merge_b0, plan.k_l1
|
b_core, b0, k_l1 = plan.b_core, plan.merge_b0, plan.k_l1
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
out_l2 = output_to_l2(case, s, 0.0)
|
||||||
|
w_fix = s.bw_l2_pc if out_l2 else s.bw_pc
|
||||||
|
|
||||||
k_truncated = k_l1 >= k
|
k_truncated = k_l1 >= k # 保留语义标记 (plan.note/调试用)
|
||||||
# 每 K 分块搬移/计算时延 (未合并基准, v1.1 §4.1 符号)
|
# 每 K 分块计算时延 (未合并基准, v1.1 §4.1 符号) / 单 batch 输出写回 (R4)
|
||||||
t_load = k_l1 * (m + n) * dt / s.bw_pc
|
t_comp_chunk = 2.0 * m * n * k_l1 / qc
|
||||||
t_comp_chunk = 2.0 * m * n * k_l1 / s.q16
|
t_write = m * n * out_b / w_fix
|
||||||
t_write = m * n * out_b / s.bw_pc # 单 batch 输出写回 (单核带宽份额)
|
|
||||||
|
|
||||||
if k_truncated:
|
|
||||||
# K 截断: n_K=1, 合并后单次搬移量 b0 倍
|
|
||||||
n_move = b_core / b0
|
|
||||||
t_mte2_data = n_move * b0 * t_load
|
|
||||||
dma_cmds = n_move
|
|
||||||
else:
|
|
||||||
# L1 绑定: k_L1^m = k_L1/b0, n_K^m = b0*n_K, 搬移次数与 IterBatch 相同
|
|
||||||
n_k = -(-k // k_l1)
|
|
||||||
n_move = b_core * n_k
|
|
||||||
t_mte2_data = n_move * t_load
|
|
||||||
dma_cmds = n_move
|
|
||||||
|
|
||||||
|
# 搬移 (issue#31): 合并组/切 K 各 (组, K段) 数据互不重叠, 每个输入字节恰好
|
||||||
|
# 从 GM 读一次 (K截断与 L1 绑定均如此; 切 K 末段按实际剩余计, 不再整段上取)
|
||||||
|
# -> GM 数据量 = V_in, 数据时延 = V_in/芯片带宽 (全核并发);
|
||||||
|
# 搬移命令数只决定 T_cmd (issue#35 修正: 真实命令数 = 合并组数 x 每组 K 段数,
|
||||||
|
# 不得把"命令数与 IterBatch 相同 = b_core*n_K"(v1.1 §4.4, n_K 为未合并粒度)
|
||||||
|
# 误代入合并后段数 —— 那样会多计 b0 倍):
|
||||||
|
# K截断: n_K^m = 1 -> cmds = ceil(b_core/b0) (比 IterBatch 省 b0 倍)
|
||||||
|
# L1绑定: k_L1^m = k_L1/b0 理想情形退化为 b_core*n_K, 与 IterBatch 相同;
|
||||||
|
# dValue 512B cap 截断等情形按实际 ceil(K/k_l1^m) 计
|
||||||
|
n_k = -(-k // k_l1)
|
||||||
|
dma_cmds = -(-b_core // b0) * n_k
|
||||||
t_dma_cmd = dma_cmds * s.t_cmd
|
t_dma_cmd = dma_cmds * s.t_cmd
|
||||||
|
# 搬移效率 (issue#36, 用户澄清): 合并 b0 个 batch 使单命令 tile 放大 b0 倍
|
||||||
|
# (A': b0*M x k_l1^m, B': k_l1^m x b0*N; 堆叠方向视转置/排布, 乘积不变),
|
||||||
|
# 小 M/N case 下相对 IterBatch 逐 batch 搬移的效率收益显著, T_cmd=0 时仍成立
|
||||||
|
k_eff = min(k_l1, k)
|
||||||
|
t_mte2_data = gm_move_time(b * m * k * dt, b * k * n * dt,
|
||||||
|
b0 * m * k_eff * dt, b0 * n * k_eff * dt, s)
|
||||||
t_mte2 = t_mte2_data + t_dma_cmd
|
t_mte2 = t_mte2_data + t_dma_cmd
|
||||||
|
|
||||||
# Cube: 合并计算含冗余 (b0^2 输出, 有效 b0) -> 计算量 = b_core*b0*2MNK
|
# Cube (每核口径): 合并计算含冗余 (b0^2 输出, 有效 b0), 每核 b_core/b0 步,
|
||||||
|
# 每步 flops = 2*(b0*M)*(b0*N)*K -> 每核 = b_core*b0*2MNK (issue#27 复核一致)
|
||||||
flops_pc = b_core * b0 * 2.0 * m * n * k
|
flops_pc = b_core * b0 * 2.0 * m * n * k
|
||||||
t_mmad = flops_pc / s.q16
|
flops_chip = b * b0 * 2.0 * m * n * k # 整芯片口径列
|
||||||
|
t_mmad = flops_pc / qc
|
||||||
|
|
||||||
# Fixpipe: 只写对角块, 写出量 = b_core*MN*outB (C 矩阵 dtype, 随路转换)
|
# Fixpipe (R4): 只写对角块, 写出量 = b_core*MN*outB (C 矩阵 dtype, 随路转换)
|
||||||
fix_bytes_pc = b_core * m * n * out_b
|
fix_bytes_pc = b_core * m * n * out_b
|
||||||
t_fix = fix_bytes_pc / s.bw_pc
|
fix_bytes_chip = b * m * n * out_b
|
||||||
|
t_fix = fix_bytes_pc / w_fix
|
||||||
|
|
||||||
# drain: 末合并 batch 排空 = b0*(T_comp + T_write)
|
# drain: 末合并 batch 排空 = b0*(T_comp + T_write)
|
||||||
t_drain = b0 * (t_comp_chunk + t_write)
|
t_drain = b0 * (t_comp_chunk + t_write)
|
||||||
|
|
||||||
gm_bytes_pc = b_core * (m * k + k * n) * dt
|
|
||||||
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_mte2_data, t_mte2_l2=0.0, t_dma_cmd=t_dma_cmd,
|
t_mte2_gm=t_mte2_data, t_mte2_l2=0.0, t_dma_cmd=t_dma_cmd,
|
||||||
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=0.0, t_drain=t_drain,
|
||||||
gm_read_bytes=gm_bytes_pc, l2_read_bytes=0.0,
|
gm_read_bytes=case.input_bytes, l2_read_bytes=0.0,
|
||||||
dma_cmd_count=dma_cmds, cube_flops=flops_pc,
|
dma_cmd_count=dma_cmds, cube_flops=flops_chip,
|
||||||
fixpipe_bytes=fix_bytes_pc,
|
fixpipe_bytes=fix_bytes_chip,
|
||||||
|
fixpipe_to_gm=(not out_l2),
|
||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
@@ -206,8 +249,29 @@ class MergeBatchBranch(Branch):
|
|||||||
def beats_iterbatch(self, case: BmmCase) -> tuple:
|
def beats_iterbatch(self, case: BmmCase) -> tuple:
|
||||||
"""返回 (MergeBatch是否更优, 说明).
|
"""返回 (MergeBatch是否更优, 说明).
|
||||||
|
|
||||||
MergeBatch 最优 ⟺ K截断 (k_L1=K) 且 b_core > b0*(T_comp+T_write)/T_cmd
|
v1.1 §4.5 统一分界的泛化口径 (issue#35 命令数 / issue#36 搬移效率):
|
||||||
L1 绑定情形 MergeBatch 恒劣于 IterBatch (搬移次数相同, 只放大 drain).
|
cmds_iter = b_core * ⌈K/k_l1_iter⌉ (IterBatch 逐 batch 逐 K 段一条)
|
||||||
|
cmds_mb = ⌈b_core/b0⌉ * ⌈K/k_l1^m⌉ (合并组数 x 每组 K 段数)
|
||||||
|
命令节省 = (cmds_iter - cmds_mb) * T_cmd
|
||||||
|
效率节省 = t_data_iter - t_data_mb (合并放大单块 tile b0 倍 ->
|
||||||
|
搬移效率提升, issue#36 用户澄清: 即便 T_cmd=0 也有收益)
|
||||||
|
drain 惩罚 = (b0-1) * (T_comp + T_write) (T_comp 按未合并基线分块)
|
||||||
|
MergeBatch 最优 ⟺ 命令节省 + 效率节省 > drain 惩罚.
|
||||||
|
|
||||||
|
与 v1.1 §4.5 闭式的等价性 (T_cmd>0 且效率打平时):
|
||||||
|
- K 截断 (k_l1^m = K): cmds_mb = b_core/b0, 退化为文档闭式
|
||||||
|
b_core > b0*(T_comp+T_write)/T_cmd;
|
||||||
|
- L1 绑定理想情形 (k_l1^m = k_l1^iter/b0): 命令数相同, 命令节省=0;
|
||||||
|
- dValue 512B 推荐值截断等第三情形 (文档二分未覆盖): 按实际命令数比较.
|
||||||
|
|
||||||
|
截断判定用**合并后** plan.k_l1 >= K (与 make_plan/evaluate 同源,
|
||||||
|
issue#35): 未合并 k_l1 截断不代表合并后仍截断 (合并使 L1 占用放大 b0 倍,
|
||||||
|
且受 dValue 512B 推荐值截断), v1.1 line 219 的字面定义 (未合并口径) 与
|
||||||
|
line 161 的合并公式矛盾时以后者为准.
|
||||||
|
|
||||||
|
T_cmd=0 (950PR 默认, 未标定): 命令节省项为 0, 由效率节省与 drain 惩罚
|
||||||
|
决定 —— 合并 tile 放大带来的效率收益被显式建模, 不再需要"T_cmd<=0 时
|
||||||
|
K截断即优先"的策略覆盖 (issue#36 退役)。
|
||||||
"""
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
@@ -215,21 +279,49 @@ class MergeBatchBranch(Branch):
|
|||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
b_core = case.batch_c // s.aic_num
|
b_core = case.batch_c // s.aic_num
|
||||||
|
|
||||||
# IterBatch 基准的 k_L1 (未合并): L1 双缓冲单 batch
|
# IterBatch 基线: 每核命令数 + 单命令 tile (与 iter_batch.evaluate 同源)
|
||||||
k_l1_iter = min(k, s.l1_bytes / (2 * (m + n) * dt))
|
from .iter_batch import IterBatchBranch
|
||||||
k_truncated = k_l1_iter >= k
|
ib = IterBatchBranch(s)
|
||||||
|
_, k_l1_iter, _, _ = ib.l1_form(case)
|
||||||
|
if not k_l1_iter:
|
||||||
|
k_l1_iter = k
|
||||||
|
n_k_iter = -(-k // min(k_l1_iter, k))
|
||||||
|
cmds_iter = b_core * n_k_iter
|
||||||
|
tile_a_ib, tile_b_ib = ib.move_tiles(case)
|
||||||
|
bb = case.batch_c
|
||||||
|
t_data_ib = gm_move_time(bb * m * k * dt, bb * k * n * dt,
|
||||||
|
tile_a_ib, tile_b_ib, s) # 整芯片口径 (量纲一致)
|
||||||
|
|
||||||
|
# MergeBatch 实际每核命令数 (issue#35) + 合并 tile (issue#36)
|
||||||
plan = self.make_plan(case)
|
plan = self.make_plan(case)
|
||||||
b0 = plan.merge_b0
|
b0 = plan.merge_b0
|
||||||
t_comp = 2.0 * m * n * min(k_l1_iter, k) / s.q16
|
n_k_mb = -(-k // min(plan.k_l1, k))
|
||||||
t_write = m * n * out_b / s.bw_pc
|
cmds_mb = -(-b_core // b0) * n_k_mb
|
||||||
threshold = b0 * (t_comp + t_write) / s.t_cmd
|
k_truncated = plan.k_l1 >= k # 合并口径截断判定 (issue#35)
|
||||||
|
regime = f"K截断(k_l1^m={plan.k_l1}>=K)" if k_truncated else \
|
||||||
|
f"L1绑定(k_l1^m={plan.k_l1}<K={k})"
|
||||||
|
k_eff = min(plan.k_l1, k)
|
||||||
|
t_data_mb = gm_move_time(bb * m * k * dt, bb * k * n * dt,
|
||||||
|
b0 * m * k_eff * dt, b0 * n * k_eff * dt, s)
|
||||||
|
|
||||||
win = k_truncated and (b_core > threshold)
|
# T_comp 按输入 dtype 算力 (issue#28), 未合并基线分块 (v1.1 §4.1 符号);
|
||||||
detail = (f"k_L1={'K(截断)' if k_truncated else f'{k_l1_iter:.0f}<K(L1绑定)'}; "
|
# T_write 保持 v1.1 直写 GM 语义
|
||||||
f"b_core={b_core} vs 阈值 b0*(T_comp+T_write)/T_cmd={threshold:.1f}; "
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
f"drain惩罚=(b0-1)*(T_comp+T_write)={((b0-1)*(t_comp+t_write))*1e6:.2f}us, "
|
t_comp = 2.0 * m * n * min(k_l1_iter, k) / qc
|
||||||
f"搬移节省=b_core*(1-1/b0)*T_cmd={b_core*(1-1/b0)*s.t_cmd*1e6:.2f}us")
|
t_write = m * n * out_b / s.bw_pc
|
||||||
|
drain_pen = (b0 - 1) * (t_comp + t_write)
|
||||||
|
save_cmds = cmds_iter - cmds_mb
|
||||||
|
savings_cmd = save_cmds * s.t_cmd
|
||||||
|
savings_eff = t_data_ib - t_data_mb
|
||||||
|
|
||||||
|
win = (savings_cmd + savings_eff) > drain_pen
|
||||||
|
detail = (f"{regime}; 每核命令数 MergeBatch={cmds_mb} vs IterBatch={cmds_iter} "
|
||||||
|
f"(命令节省={savings_cmd*1e6:.2f}us) + 搬移效率节省="
|
||||||
|
f"{savings_eff*1e6:.2f}us vs drain惩罚=(b0-1)*(T_comp+T_write)="
|
||||||
|
f"{drain_pen*1e6:.2f}us -> {'MergeBatch优' if win else 'IterBatch优'}")
|
||||||
|
if k_truncated and s.t_cmd > 0:
|
||||||
|
detail += (f" (闭式等价: b_core={b_core} vs 阈值 "
|
||||||
|
f"b0*(T_comp+T_write)/T_cmd={b0 * (t_comp + t_write) / s.t_cmd:.1f})")
|
||||||
return win, detail
|
return win, detail
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming
|
from ..models import BmmCase, ImplPlan, HardwareTiming
|
||||||
from ..timing import assemble_timing
|
from ..timing import assemble_timing, output_to_l2
|
||||||
from .base import Branch, ConditionCheck
|
from .base import Branch, ConditionCheck
|
||||||
|
|
||||||
|
|
||||||
@@ -24,7 +24,7 @@ class SpecialBranch(Branch):
|
|||||||
c1 = case.k <= 1
|
c1 = case.k <= 1
|
||||||
checks = [ConditionCheck("1_K<=1 (Cube 无用)", c1, f"K={case.k}")]
|
checks = [ConditionCheck("1_K<=1 (Cube 无用)", c1, f"K={case.k}")]
|
||||||
if case.k == 1:
|
if case.k == 1:
|
||||||
# K=1 的 AIV 通路恒可用 (issue#12): B>=2*AIV 开 UB 乒乓; B<128 退化为
|
# K=1 的 AIV 通路恒可用 (issue#12/#17): B>=2*AIV 开 UB 乒乓; B<128 退化为
|
||||||
# AIV 单缓冲 (无乒乓, 逐 batch 串行搬入), 不再是无方案空洞.
|
# AIV 单缓冲 (无乒乓, 逐 batch 串行搬入), 不再是无方案空洞.
|
||||||
b = case.batch_c
|
b = case.batch_c
|
||||||
pingpong = b >= 2 * self.spec.aiv_num
|
pingpong = b >= 2 * self.spec.aiv_num
|
||||||
@@ -68,32 +68,42 @@ class SpecialBranch(Branch):
|
|||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
"""AIV 通路时延: 瓶颈在搬移 (AIV 算力远剩)."""
|
"""AIV 通路时延: 瓶颈在搬移 (AIV 算力远剩).
|
||||||
|
|
||||||
|
口径: AIV 逐元素通量按输入 dtype (issue#28); 输出落点 R4 (issue#30):
|
||||||
|
整 case 输入+输出 <= L2 -> 输出写 L2 (异步回写不占算子时延), 否则直写
|
||||||
|
GM 与读共享总线 (assemble 的 MTE2 链累加, issue#23).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
b = case.batch_c
|
b = case.batch_c
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
|
out_l2 = output_to_l2(case, s)
|
||||||
|
w_fix = s.bw_l2 if out_l2 else s.bw_gm
|
||||||
|
cube_flops = 0.0
|
||||||
|
t_compute = 0.0
|
||||||
|
|
||||||
if k == 0:
|
if k == 0:
|
||||||
# 纯写值: 仅写出
|
# 纯写值: 仅写出 (R1 下 GM 读 = 0, 输入本身为空)
|
||||||
t_in, t_compute, t_out = 0.0, 0.0, b * m * n * out_b / s.bw_gm
|
t_in = 0.0
|
||||||
in_bytes, cube_flops = 0.0, 0.0
|
t_out = b * m * n * out_b / w_fix
|
||||||
|
in_bytes = 0.0
|
||||||
else:
|
else:
|
||||||
# 逐元素乘: 搬入 A+B, 搬出 C, AIV 算力远剩
|
# 逐元素乘: 搬入 A+B (GM 每字节一次), 搬出 C
|
||||||
in_bytes = b * (m * k + k * n) * dt
|
in_bytes = b * (m * k + k * n) * dt
|
||||||
out_bytes = b * m * n * out_b
|
out_bytes = b * m * n * out_b
|
||||||
t_in = in_bytes / s.bw_gm
|
t_in = in_bytes / s.bw_gm
|
||||||
t_out = out_bytes / s.bw_gm
|
t_out = out_bytes / w_fix
|
||||||
# AIV 求积吞吐 (近似按 Q_AIV)
|
# AIV 逐元素吞吐按输入 dtype 取通量 (issue#28: bf16/fp16 x2, int8 x4...)
|
||||||
t_compute = b * m * n / s.q_aiv
|
t_compute = b * m * n / s.aiv_elem_rate(case.dtype_in)
|
||||||
cube_flops = float(b * m * n)
|
cube_flops = float(b * m * n)
|
||||||
|
|
||||||
t_total = max(t_in, t_out, t_compute)
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_in, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
t_mte2_gm=t_in, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
||||||
t_mmad=t_compute, t_fixpipe=t_out, t_reduce=0.0, t_drain=0.0,
|
t_mmad=t_compute, t_fixpipe=t_out, t_reduce=0.0, t_drain=0.0,
|
||||||
gm_read_bytes=in_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
gm_read_bytes=in_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
||||||
cube_flops=cube_flops,
|
cube_flops=cube_flops,
|
||||||
fixpipe_bytes=b * m * n * out_b,
|
fixpipe_bytes=b * m * n * out_b,
|
||||||
|
fixpipe_to_gm=(not out_l2),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import math
|
|||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming, ceil_div
|
from ..models import BmmCase, ImplPlan, HardwareTiming, ceil_div
|
||||||
from ..timing import assemble_timing, eval_streamk_reduce
|
from ..timing import assemble_timing, eval_streamk_reduce, output_to_l2
|
||||||
from .base import Branch, ConditionCheck
|
from .base import Branch, ConditionCheck
|
||||||
|
|
||||||
|
|
||||||
@@ -32,10 +32,14 @@ class StreamKBranch(Branch):
|
|||||||
p = self._p_value(case)
|
p = self._p_value(case)
|
||||||
return int(self.spec.aic_num // max(1, math.ceil(p)))
|
return int(self.spec.aic_num // max(1, math.ceil(p)))
|
||||||
|
|
||||||
def _theta_c(self) -> float:
|
def _theta_c(self, case: BmmCase) -> float:
|
||||||
"""归约代价系数 theta_c = Q16/2 * (8B/W_L2 + 1/Q_AIV) ≈ 12."""
|
"""归约代价系数 theta_c = Q_cube(dtype)/2 * (8B/W_L2 + 1/Q_AIV_fp32).
|
||||||
|
|
||||||
|
Q_AIV 用 fp32 档 (部分和为 fp32, issue#28); Cube 侧按输入 dtype.
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
return s.q16 / 2 * (8 / s.bw_l2 + 1 / s.q_aiv)
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
return qc / 2 * (8 / s.bw_l2 + 1 / s.q_aiv)
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def check_conditions(self, case: BmmCase) -> list:
|
def check_conditions(self, case: BmmCase) -> list:
|
||||||
@@ -61,7 +65,7 @@ class StreamKBranch(Branch):
|
|||||||
|
|
||||||
# 条件 3: K > grid_K^2/(grid_K-1) * theta_c (归约代价可接受)
|
# 条件 3: K > grid_K^2/(grid_K-1) * theta_c (归约代价可接受)
|
||||||
if grid_k >= 2:
|
if grid_k >= 2:
|
||||||
theta_c = self._theta_c()
|
theta_c = self._theta_c(case)
|
||||||
k_thresh = grid_k * grid_k / (grid_k - 1) * theta_c
|
k_thresh = grid_k * grid_k / (grid_k - 1) * theta_c
|
||||||
c3 = case.k > k_thresh
|
c3 = case.k > k_thresh
|
||||||
checks.append(ConditionCheck(
|
checks.append(ConditionCheck(
|
||||||
@@ -129,39 +133,64 @@ class StreamKBranch(Branch):
|
|||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
|
"""芯片口径评估, 按 plan 实际 tile 布局 (issue#25).
|
||||||
|
|
||||||
|
切 K 组 (tile = single_m x single_n) 由 grid_k 核协作: 组内 K 段零重复读
|
||||||
|
(GM 一次); 组间共享 (A 行块被 n_cnt 个 tile 用) 在单 batch 工作集可驻留时
|
||||||
|
走 L2 读口 (与 ASW 同规则). 全部 tile-组在 C 核上分波执行:
|
||||||
|
waves = ceil(b*m_cnt*n_cnt*grid_k / C).
|
||||||
|
- GM 读: 首读总量 = b*(m*k + k*n)*dt (组间共享命中 L2 时), 或按 r_gm 放大
|
||||||
|
(工作集超 L2, 保守同 ASW 场景 C 公式);
|
||||||
|
- MMAD: 芯片总量 2*b*m*n*k / (C核) —— 全核忙稳态;
|
||||||
|
- 归约 (部分和 4B 写 L2/AIV 读回求和/最终写回): 每 tile-组一次、组间串行
|
||||||
|
追加为 drain: t_drain = waves * eval_streamk_reduce(tile, grid_k, out);
|
||||||
|
最终输出写回落点按 R4 (issue#30): 整 case 输入+输出+workspace <= L2
|
||||||
|
时写 L2, 否则直写 GM (drain 内按 GM 带宽).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
b = case.batch_c
|
b = case.batch_c
|
||||||
m, n, k = case.m, case.n, case.k
|
m, n, k = case.m, case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
grid_k = plan.grid_k
|
grid_k = max(plan.grid_k, 1)
|
||||||
|
m_cnt = max(plan.m_cnt, 1)
|
||||||
|
n_cnt = max(plan.n_cnt, 1)
|
||||||
|
tile_m = max(plan.single_core_m, 1)
|
||||||
|
tile_n = max(plan.single_core_n, 1)
|
||||||
|
tile = tile_m * tile_n # 每核实际 tile 元素数
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
out_l2 = output_to_l2(case, s, float(plan.workspace_bytes))
|
||||||
|
|
||||||
# 单 tile (L0C 满载基本块)
|
# ---- MTE2: GM 段 + L2 段 (同 ASW 块级规则) ----
|
||||||
tile_elems = s.l0c_elems # 65536
|
a_b = m * k * dt
|
||||||
t_mmad_tile = 2.0 * k * s.l0c_bytes / 4 / s.q16 # 2K/Q16 * L0C/4B
|
bb_b = k * n * dt
|
||||||
t_mte2_tile = k * (2 * math.sqrt(tile_elems)) * dt / s.bw_pc # 近似方形 tile
|
if a_b + bb_b <= s.l2_bytes:
|
||||||
|
gm_read = b * (a_b + bb_b) # 组间共享命中 L2: GM 每字节一次
|
||||||
|
l2_read = b * ((n_cnt - 1) * a_b + (m_cnt - 1) * bb_b)
|
||||||
|
else:
|
||||||
|
gm_read = b * (a_b + bb_b) # 保守: 共享跨组回落 GM 未另计
|
||||||
|
l2_read = 0.0
|
||||||
|
t_gm = gm_read / s.bw_gm # 芯片口径 (全核并发)
|
||||||
|
t_l2 = l2_read / s.bw_l2
|
||||||
|
t_mte2 = t_gm + t_l2
|
||||||
|
|
||||||
# 切 K 后流水时延缩 grid_k 倍
|
# ---- MMAD: 芯片总量, dtype 感知算力 (issue#28) ----
|
||||||
t_mmad = t_mmad_tile / grid_k
|
flops = 2.0 * b * m * n * k
|
||||||
t_mte2 = t_mte2_tile / grid_k
|
t_mmad = flops / (s.aic_num * qc)
|
||||||
|
|
||||||
# 归约: 部分和 4B 驻留 L2, AIV 归约 (含部分和写/读回/求和/最终按 C dtype 写回)
|
# ---- 归约: 每 tile-组一次, 组间分波串行追加 ----
|
||||||
# 口径 (issue#11): 归约整体为串行追加 (t_drain=t_reduce, reduce_serial=True),
|
waves = ceil_div(b * m_cnt * n_cnt * grid_k, s.aic_num)
|
||||||
# 部分和写出已计入 eval_streamk_reduce 的 t_write_partial —— 稳态 Fixpipe 不再
|
t_reduce_group = eval_streamk_reduce(tile, grid_k, out_b, s,
|
||||||
# 重复计账. 此前按 grid_k*tile*4B/单核带宽份额另计一次, 既重复计账又把整组
|
out_to_gm=not out_l2)
|
||||||
# 部分和串行压到单核写口, 高估 grid_k 倍 (streamk_demo 曾虚高到 55us/FIXPIPE).
|
t_reduce = waves * t_reduce_group
|
||||||
t_reduce = eval_streamk_reduce(tile_elems, grid_k, out_b, s)
|
|
||||||
fix_bytes = 0.0
|
fix_bytes = 0.0
|
||||||
t_fix = 0.0
|
t_fix = 0.0
|
||||||
|
|
||||||
flops_pc = 2.0 * tile_elems * k / grid_k
|
|
||||||
gm_bytes = k * (2 * math.sqrt(tile_elems)) * dt / grid_k
|
|
||||||
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_mte2, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
t_mte2_gm=t_gm, t_mte2_l2=t_l2, t_dma_cmd=0.0,
|
||||||
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=t_reduce,
|
t_mmad=t_mmad, t_fixpipe=t_fix, t_reduce=t_reduce,
|
||||||
t_drain=t_reduce, # 归约串行追加 (reduce_serial 默认 True, 不进稳态 max)
|
t_drain=t_reduce, # 归约串行追加 (reduce_serial 默认 True, 不进稳态 max)
|
||||||
gm_read_bytes=gm_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
gm_read_bytes=gm_read, l2_read_bytes=l2_read, dma_cmd_count=0.0,
|
||||||
cube_flops=flops_pc, fixpipe_bytes=fix_bytes,
|
cube_flops=flops, fixpipe_bytes=fix_bytes,
|
||||||
reduce_serial=True,
|
reduce_serial=True, fixpipe_to_gm=False,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from ..hardware import NpuSpec, ASCEND950PR
|
from ..hardware import NpuSpec, ASCEND950PR
|
||||||
from ..models import BmmCase, ImplPlan, HardwareTiming
|
from ..models import BmmCase, ImplPlan, HardwareTiming
|
||||||
from ..timing import assemble_timing
|
from ..timing import assemble_timing, output_to_l2
|
||||||
from .base import Branch, ConditionCheck
|
from .base import Branch, ConditionCheck
|
||||||
|
|
||||||
|
|
||||||
@@ -71,7 +71,12 @@ class ToMatmulBranch(Branch):
|
|||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
def evaluate(self, case: BmmCase, plan: ImplPlan) -> HardwareTiming:
|
||||||
"""折叠后按 Matmul 粗估 (详细切分待 MM 理论体系打通后接入)."""
|
"""折叠后按 Matmul 粗估 (详细切分待 MM 理论体系打通后接入).
|
||||||
|
|
||||||
|
口径: Cube 算力按输入 dtype (issue#28); GM 首读 = V_in 一次 (R1);
|
||||||
|
输出落点 R4 (issue#30): 整 case 输入+输出 <= L2 时写 L2 写口, 否则
|
||||||
|
直写 GM 与读共享总线 (assemble 累加).
|
||||||
|
"""
|
||||||
s = self.spec
|
s = self.spec
|
||||||
if case.batch_b == 1:
|
if case.batch_b == 1:
|
||||||
fold_m, fold_n, kk = case.batch_a * case.m, case.n, case.k
|
fold_m, fold_n, kk = case.batch_a * case.m, case.n, case.k
|
||||||
@@ -79,13 +84,15 @@ class ToMatmulBranch(Branch):
|
|||||||
fold_m, fold_n, kk = case.m, case.batch_b * case.n, case.k
|
fold_m, fold_n, kk = case.m, case.batch_b * case.n, case.k
|
||||||
dt = case.dtype_in_bytes
|
dt = case.dtype_in_bytes
|
||||||
out_b = case.dtype_out_bytes
|
out_b = case.dtype_out_bytes
|
||||||
|
qc = s.q_cube(case.dtype_a, case.dtype_b)
|
||||||
|
out_l2 = output_to_l2(case, s)
|
||||||
|
|
||||||
flops = 2.0 * fold_m * fold_n * kk
|
flops = 2.0 * fold_m * fold_n * kk
|
||||||
t_mmad = flops / (s.aic_num * s.q16)
|
t_mmad = flops / (s.aic_num * qc)
|
||||||
in_bytes = (fold_m * kk + kk * fold_n) * dt
|
in_bytes = (fold_m * kk + kk * fold_n) * dt
|
||||||
out_bytes = fold_m * fold_n * out_b
|
out_bytes = fold_m * fold_n * out_b
|
||||||
t_mte2 = in_bytes / s.bw_gm
|
t_mte2 = in_bytes / s.bw_gm
|
||||||
t_fix = out_bytes / s.bw_gm
|
t_fix = out_bytes / (s.bw_l2 if out_l2 else s.bw_gm)
|
||||||
|
|
||||||
return assemble_timing(
|
return assemble_timing(
|
||||||
t_mte2_gm=t_mte2, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
t_mte2_gm=t_mte2, t_mte2_l2=0.0, t_dma_cmd=0.0,
|
||||||
@@ -93,4 +100,5 @@ class ToMatmulBranch(Branch):
|
|||||||
t_drain=0.0,
|
t_drain=0.0,
|
||||||
gm_read_bytes=in_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
gm_read_bytes=in_bytes, l2_read_bytes=0.0, dma_cmd_count=0.0,
|
||||||
cube_flops=flops, fixpipe_bytes=out_bytes,
|
cube_flops=flops, fixpipe_bytes=out_bytes,
|
||||||
|
fixpipe_to_gm=(not out_l2),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -36,6 +36,13 @@ def check_plan_constraints(case: BmmCase, plan: ImplPlan,
|
|||||||
s = spec
|
s = spec
|
||||||
v = []
|
v = []
|
||||||
|
|
||||||
|
# 占位/无效方案: used_core_num<1 说明没有真实方案 (如 router._no_plan 的占位),
|
||||||
|
# recommend 侧 advice 已标注"[无方案]", evaluate 侧必须判不可行 (issue#18),
|
||||||
|
# 不得当作可行方案给正常时延.
|
||||||
|
if plan.used_core_num < 1:
|
||||||
|
v.append("used_core_num=0: 占位/未生成方案, 不可评估")
|
||||||
|
return v
|
||||||
|
|
||||||
# AIV 通路: 只校验 AIV 核数
|
# AIV 通路: 只校验 AIV 核数
|
||||||
if plan.branch in AIV_BRANCHES:
|
if plan.branch in AIV_BRANCHES:
|
||||||
if plan.used_core_num > s.aiv_num:
|
if plan.used_core_num > s.aiv_num:
|
||||||
@@ -50,8 +57,9 @@ def check_plan_constraints(case: BmmCase, plan: ImplPlan,
|
|||||||
if plan.used_core_num > s.aic_num:
|
if plan.used_core_num > s.aic_num:
|
||||||
v.append(f"used_core_num={plan.used_core_num} 超 AIC 核数 {s.aic_num}")
|
v.append(f"used_core_num={plan.used_core_num} 超 AIC 核数 {s.aic_num}")
|
||||||
|
|
||||||
# --- L0C --- (ASW 降核每核单份, 其他分支双缓冲)
|
# --- L0C --- (ASW 两分支 UnitFlag 单缓冲: tile 内 16x16x16 细粒度流水
|
||||||
l0c_factor = 1 if plan.branch == "ASW_Basic_降核" else 2
|
# (ASW_Basic分支分析 v1.91 §5.1); IterBatch/MergeBatch 双缓冲)
|
||||||
|
l0c_factor = 1 if plan.branch in ("ASW_Basic", "ASW_Basic_降核") else 2
|
||||||
l0c_need = plan.base_m * plan.base_n * 4 * l0c_factor
|
l0c_need = plan.base_m * plan.base_n * 4 * l0c_factor
|
||||||
if l0c_need > s.l0c_bytes:
|
if l0c_need > s.l0c_bytes:
|
||||||
v.append(f"L0C tile 超容量: BaseM*BaseN*4B*{l0c_factor}={l0c_need}B > {s.l0c_bytes}B")
|
v.append(f"L0C tile 超容量: BaseM*BaseN*4B*{l0c_factor}={l0c_need}B > {s.l0c_bytes}B")
|
||||||
@@ -77,9 +85,24 @@ def check_plan_constraints(case: BmmCase, plan: ImplPlan,
|
|||||||
|
|
||||||
# --- dValue --- (issue#6 口径裁定: 只对"以 K 段为连续维"的方案生效)
|
# --- dValue --- (issue#6 口径裁定: 只对"以 K 段为连续维"的方案生效)
|
||||||
if _k_segment_is_contiguous(plan, case) and plan.k_l1 > 0:
|
if _k_segment_is_contiguous(plan, case) and plan.k_l1 > 0:
|
||||||
dv = plan.k_l1 * case.dtype_in_bytes
|
if plan.branch in ("ASW_Basic", "ASW_Basic_降核"):
|
||||||
if dv < s.dvalue_min:
|
# issue#34: ASW 是兜底分支, 恒出方案; dValue 效率下限不满足只算**效率降级**
|
||||||
v.append(f"dValue={dv}B < 下限 {s.dvalue_min}B, K 段连续维搬移效率崩塌")
|
# (plan.note 含"效率降级"标注, DMA 仍能工作只是效率低), 不判违规.
|
||||||
|
# 真正不可行的只有容量/核数硬约束 (L0C/L0A/L0B/L1 超容, 核数超限).
|
||||||
|
pass
|
||||||
|
elif plan.branch == "IterBatch" and plan.l1_form.startswith(("c_", "d_")):
|
||||||
|
# 转置感知判据与生成守卫/条件 4 同源 (issue#19):
|
||||||
|
# dv_a = M*dt (A 转置) 或 k_l1*dt; dv_b = N*dt (B 不转置) 或 k_l1*dt;
|
||||||
|
# 两侧连续维 dValue 均低于下限才算违规.
|
||||||
|
from .models import dvalue_contig_dims
|
||||||
|
dv_a, dv_b = dvalue_contig_dims(case, plan.k_l1)
|
||||||
|
if dv_a < s.dvalue_min and dv_b < s.dvalue_min:
|
||||||
|
v.append(f"dValueA={dv_a:.0f}B 与 dValueB={dv_b:.0f}B 均 < 下限 "
|
||||||
|
f"{s.dvalue_min}B, 搬移连续维效率崩塌")
|
||||||
|
else:
|
||||||
|
dv = plan.k_l1 * case.dtype_in_bytes
|
||||||
|
if dv < s.dvalue_min:
|
||||||
|
v.append(f"dValue={dv}B < 下限 {s.dvalue_min}B, K 段连续维搬移效率崩塌")
|
||||||
|
|
||||||
# --- 写出 dtype --- (StreamK 部分和 4B 是正确行为)
|
# --- 写出 dtype --- (StreamK 部分和 4B 是正确行为)
|
||||||
if plan.branch not in PARTIAL_SUM_4B_BRANCHES:
|
if plan.branch not in PARTIAL_SUM_4B_BRANCHES:
|
||||||
|
|||||||
@@ -63,14 +63,16 @@ class PlanEvaluator:
|
|||||||
def _advice(self, case: BmmCase, plan: ImplPlan, res: EvalResult) -> str:
|
def _advice(self, case: BmmCase, plan: ImplPlan, res: EvalResult) -> str:
|
||||||
t = res.timing
|
t = res.timing
|
||||||
tips = []
|
tips = []
|
||||||
|
if "效率降级" in plan.note:
|
||||||
|
tips.append("效率降级标注 (plan.note): 搬移效率下限不满足 —— 方案照常给出 "
|
||||||
|
"(兜底), 但实际效率低于模型假设, 时延可能低估; 建议调整 dtype/布局")
|
||||||
if not res.feasible:
|
if not res.feasible:
|
||||||
tips.append("方案违反硬件约束, 需先修正: " + res.violations)
|
tips.append("方案违反硬件约束, 需先修正: " + res.violations)
|
||||||
bn = t.bottleneck
|
bn = t.bottleneck
|
||||||
if bn == "MTE2_GM":
|
if bn == "MTE2":
|
||||||
tips.append("瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, "
|
tips.append("瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile "
|
||||||
"或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)")
|
"提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/"
|
||||||
elif bn == "MTE2_L2":
|
"分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争")
|
||||||
tips.append("瓶颈在 L2 重复读: 优化核间分配/swizzle 窗口压低活跃工作集")
|
|
||||||
elif bn == "MMAD":
|
elif bn == "MMAD":
|
||||||
tips.append("瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 "
|
tips.append("瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 "
|
||||||
"(MergeBatch 交叉项) 可消除")
|
"(MergeBatch 交叉项) 可消除")
|
||||||
|
|||||||
@@ -1,3 +1,24 @@
|
|||||||
from .ascend950pr import ASCEND950PR, NpuSpec
|
from .ascend950pr import ASCEND950PR, ASCEND950PR_C28, NpuSpec
|
||||||
|
from .ascend950dt import ASCEND950DT, ASCEND950DT_C32, ASCEND950DT_C28
|
||||||
|
|
||||||
__all__ = ["ASCEND950PR", "NpuSpec"]
|
# 芯片规格注册表 (issue#38): name -> NpuSpec; 默认主 bin = ASCEND950PR
|
||||||
|
SPECS = {s.name: s for s in (
|
||||||
|
ASCEND950PR, ASCEND950PR_C28,
|
||||||
|
ASCEND950DT, ASCEND950DT_C32, ASCEND950DT_C28,
|
||||||
|
)}
|
||||||
|
|
||||||
|
|
||||||
|
def get_spec(name: str = "Ascend950PR") -> NpuSpec:
|
||||||
|
"""按名称取芯片规格 (默认 Ascend950PR 32 核主 bin); 未知名称抛 KeyError."""
|
||||||
|
try:
|
||||||
|
return SPECS[name]
|
||||||
|
except KeyError:
|
||||||
|
raise KeyError(f"未知芯片规格 {name!r}, 可选: {sorted(SPECS)}") from None
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"NpuSpec",
|
||||||
|
"ASCEND950PR", "ASCEND950PR_C28",
|
||||||
|
"ASCEND950DT", "ASCEND950DT_C32", "ASCEND950DT_C28",
|
||||||
|
"SPECS", "get_spec",
|
||||||
|
]
|
||||||
|
|||||||
57
BMM/BMM_Theory/bmm_theory/hardware/ascend950dt.py
Normal file
57
BMM/BMM_Theory/bmm_theory/hardware/ascend950dt.py
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
"""Ascend950DT 硬件规格参数 (issue#38).
|
||||||
|
|
||||||
|
数值来源: 昇腾950 NPU架构白皮书 表3-1 (系列 SKU 规格) / 表4-2 (Memory 层次);
|
||||||
|
产品形态: Atlas 850E 超节点 / Atlas 650E 服务器 / Atlas 950 SuperPoD 超节点.
|
||||||
|
|
||||||
|
与 950PR **共架构** (第三代 DaVinci), 仅以下维度不同:
|
||||||
|
- AI 子系统 36/32/28 个 (每子系统 = 1 Cube Core + 2 Vector Core);
|
||||||
|
- 高速片上内存 4TB/s (950PR 为 1.6/1.4TB/s), 容量 144/96GB;
|
||||||
|
- L2 128MB 全档一致 (950PR 28 核降 112MB).
|
||||||
|
|
||||||
|
共架构的交叉验证 (表3-1/表4-2 反推, 与 950PR 单核速率一致):
|
||||||
|
- Cube: 432T/32核 = 486T/36核 = 13.5T/核 (= 16^3 MAC/拍 @ ~1.65GHz),
|
||||||
|
cube_peak_tflops 取白皮书表3-1 "Cube算力" 单行 (Cube-only 口径,
|
||||||
|
issue#40 用户裁决), 单核 13.5T 全系列精确一致;
|
||||||
|
- Vector fp32: 27T/64核 ≈ 30T/72核 -> 128 lane/拍 @ 1.65GHz (乘加计 2 次),
|
||||||
|
即 aiv_freq_ghz / aiv_fp32_per_cycle 默认值直接适用;
|
||||||
|
- L1 512KB / L0A/L0B 64KB / L0C 256KB / UB 512KB 每核一致 (表4-2);
|
||||||
|
- L2 带宽 7.5TB/s, 读写各自独享 (issue#39 用户澄清; 白皮书未分档,
|
||||||
|
DT 三档统一); 搬移效率经验值 (dValue/minTile/minDatamountPerCore/
|
||||||
|
T_cmd=0) 同架构沿用 950PR 口径.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .ascend950pr import NpuSpec
|
||||||
|
|
||||||
|
# 950DT 36 核主 bin (白皮书表3-1: 36 AIC / 72 AIV, Cube BF16 486 TFLOPS,
|
||||||
|
# HBM 4TB/s 144GB, L2 128MB; L2 带宽 7.5TB/s 读写各自独享, issue#39)
|
||||||
|
ASCEND950DT = NpuSpec(
|
||||||
|
name="Ascend950DT",
|
||||||
|
aic_num=36, aiv_num=72,
|
||||||
|
cube_peak_tflops=486.0,
|
||||||
|
bw_gm=4.0e12,
|
||||||
|
bw_l2=7.5e12,
|
||||||
|
gm_capacity_gb=144.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 950DT 32 核档 (Cube BF16 432 TFLOPS; HBM 4TB/s 不变;
|
||||||
|
# 容量表3-1 仅给 144/96 两档, 32 核档映射待确认)
|
||||||
|
ASCEND950DT_C32 = NpuSpec(
|
||||||
|
name="Ascend950DT_C32",
|
||||||
|
aic_num=32, aiv_num=64,
|
||||||
|
cube_peak_tflops=432.0,
|
||||||
|
bw_gm=4.0e12,
|
||||||
|
bw_l2=7.5e12,
|
||||||
|
gm_capacity_gb=144.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 950DT 28 核档 (Cube BF16 378 TFLOPS; HBM 4TB/s 不变, 容量 96GB)
|
||||||
|
ASCEND950DT_C28 = NpuSpec(
|
||||||
|
name="Ascend950DT_C28",
|
||||||
|
aic_num=28, aiv_num=56,
|
||||||
|
cube_peak_tflops=378.0,
|
||||||
|
bw_gm=4.0e12,
|
||||||
|
bw_l2=7.5e12,
|
||||||
|
gm_capacity_gb=96.0,
|
||||||
|
)
|
||||||
@@ -1,15 +1,47 @@
|
|||||||
"""Ascend950PR (DAV_3510) 硬件规格参数.
|
"""Ascend950PR (DAV_3510) 硬件规格参数.
|
||||||
|
|
||||||
数值来源: 《BMM算子优化分析 v0.98》§二 + 昇腾950 NPU架构白皮书.
|
数值来源: 《BMM算子优化分析 v0.98》§二 + 昇腾950 NPU架构白皮书 (表3-1 SKU 规格 /
|
||||||
换芯片时逻辑结构不变, 只需新增一份同结构参数表.
|
表4-2 Memory 层次). 换芯片时逻辑结构不变, 只需新增一份同结构参数表
|
||||||
|
(系列其他 SKU 见文末 ASCEND950PR_C28 与 hardware/ascend950dt.py, issue#38).
|
||||||
|
|
||||||
单位约定: 算力 FLOP/s, 带宽 Byte/s, 容量 Byte, 时延 秒.
|
单位约定: 算力 FLOP/s, 带宽 Byte/s, 容量 Byte, 时延 秒.
|
||||||
|
|
||||||
|
dtype 感知算力 (issue#28, 设计文档 docs/05 §2):
|
||||||
|
Cube 算力按输入 dtype 分档, 基准 = BF16 (fp16 同速); 白皮书: FP8/MXFP8/HiF8
|
||||||
|
提供 2x FP16 张量 TFLOPS, MXFP4 提供 4x FP16; FP32/TF32 同代比值为假设值
|
||||||
|
(按 DaVinci 惯例 = 1/2, 白皮书未给同代比值), int8 假设同 FP8, 均待实测标定.
|
||||||
|
AIV 逐元素通量按 lane 位宽等比假设 (16bit x2 / 8bit x4 / 4bit x8, 待标定).
|
||||||
|
|
||||||
|
口径 (issue#40 用户裁决): cube_peak_tflops 采用白皮书表3-1 **"Cube算力"单行**
|
||||||
|
(Cube-only, 不含 Vector): 950PR 32核 = 432, 单核 13.5T = 16³ MAC/拍 × 1.65GHz;
|
||||||
|
全系列 SKU 单核精确统一 13.5T (432/32 = 378/28 = 486/36)。2026-09 前曾沿用
|
||||||
|
v0.98 的 "Cube+Vector 总算力" 口径 (486, 单核 15.1875T), 切换后全模型 MMAD
|
||||||
|
时延 ×486/432 (+12.5%), R16 607.5→540, StreamK θ_c 12.24→10.9,
|
||||||
|
ASW 面积/周长分界 93.5→83.1。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
# Cube 精度因子 (相对 BF16/FP16 基准档; A/B 不一致时取较慢一侧 = min 因子)
|
||||||
|
CUBE_DTYPE_FACTOR = {
|
||||||
|
"fp32": 0.5, "f32": 0.5, "tf32": 0.5, # 假设 = 1/2, 待实测标定
|
||||||
|
"fp16": 1.0, "f16": 1.0, "bf16": 1.0,
|
||||||
|
"fp8": 2.0, "fp8_e4m3": 2.0, "fp8_e5m2": 2.0, "int8": 2.0, # 白皮书 FP8=2xFP16; int8 假设同 FP8
|
||||||
|
"fp4": 4.0, "fp4_e2m1": 4.0, # 白皮书 MXFP4=4xFP16; 普通 fp4 假设同 MXFP4
|
||||||
|
}
|
||||||
|
|
||||||
|
# AIV 逐元素通量因子 (相对 fp32 128 lane/拍/核; 位宽等比假设, 待实测标定)
|
||||||
|
AIV_DTYPE_FACTOR = {
|
||||||
|
"fp32": 1.0, "f32": 1.0, "tf32": 1.0,
|
||||||
|
"fp16": 2.0, "f16": 2.0, "bf16": 2.0,
|
||||||
|
"fp8": 4.0, "fp8_e4m3": 4.0, "fp8_e5m2": 4.0, "int8": 4.0,
|
||||||
|
"fp4": 8.0, "fp4_e2m1": 8.0,
|
||||||
|
}
|
||||||
|
|
||||||
|
_RATE_FALLBACK = 1.0 # 未知 dtype 按基准档 (models.dtype_bytes 已先行校验, 正常不会到达)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class NpuSpec:
|
class NpuSpec:
|
||||||
@@ -18,7 +50,8 @@ class NpuSpec:
|
|||||||
# ---- 核数与算力 ----
|
# ---- 核数与算力 ----
|
||||||
aic_num: int = 32 # C: AIC (Cube) 核数
|
aic_num: int = 32 # C: AIC (Cube) 核数
|
||||||
aiv_num: int = 64 # AIV (Vector) 核数
|
aiv_num: int = 64 # AIV (Vector) 核数
|
||||||
cube_peak_tflops: float = 486.0 # 全芯片 Cube BF16 标称算力 (乘加各计一次)
|
cube_peak_tflops: float = 432.0 # 全芯片 Cube BF16 标称算力 (乘加各计一次,
|
||||||
|
# 白皮书表3-1 Cube 单行, issue#40)
|
||||||
aiv_freq_ghz: float = 1.65
|
aiv_freq_ghz: float = 1.65
|
||||||
aiv_fp32_per_cycle: int = 128 # 单 AIV 每拍 fp32 求和吞吐
|
aiv_fp32_per_cycle: int = 128 # 单 AIV 每拍 fp32 求和吞吐
|
||||||
|
|
||||||
@@ -30,8 +63,9 @@ class NpuSpec:
|
|||||||
|
|
||||||
# ---- L2 / GM ----
|
# ---- L2 / GM ----
|
||||||
l2_bytes: int = 128 * 1024 * 1024 # L2 Cache 128MB
|
l2_bytes: int = 128 * 1024 * 1024 # L2 Cache 128MB
|
||||||
bw_l2: float = 5.2e12 # L2 读写带宽 5.2TB/s
|
bw_l2: float = 5.2e12 # L2 读写带宽 5.2TB/s (白皮书未给, 用户澄清口径)
|
||||||
bw_gm: float = 1.6e12 # GM(HBM) 带宽 1.6TB/s (读写共享)
|
bw_gm: float = 1.6e12 # GM(HBM) 带宽 1.6TB/s (读写共享)
|
||||||
|
gm_capacity_gb: float = 128.0 # GM(HBM) 容量 GB (信息项, 模型暂不消费)
|
||||||
|
|
||||||
# ---- 搬移效率经验约束 (950PR 实测总结) ----
|
# ---- 搬移效率经验约束 (950PR 实测总结) ----
|
||||||
dvalue_recommend: int = 512 # dValue 推荐值 (Byte), 更大无额外收益
|
dvalue_recommend: int = 512 # dValue 推荐值 (Byte), 更大无额外收益
|
||||||
@@ -39,10 +73,19 @@ class NpuSpec:
|
|||||||
dvalue_hw_min: int = 256 # DMA 硬件突发下限 (Byte) —— 尾轮文档 §2.3
|
dvalue_hw_min: int = 256 # DMA 硬件突发下限 (Byte) —— 尾轮文档 §2.3
|
||||||
min_tile_size: int = 16 * 1024 # min_TileSize: 单块搬移最小量 16KB
|
min_tile_size: int = 16 * 1024 # min_TileSize: 单块搬移最小量 16KB
|
||||||
min_datamount_per_core: int = 480 * 1024 # min_DatamountPerCore: 单核搬移总量下限 480KB
|
min_datamount_per_core: int = 480 * 1024 # min_DatamountPerCore: 单核搬移总量下限 480KB
|
||||||
min_core_num_ratio: float = 0.8 # minCoreNum ≈ 0.8 * C
|
min_core_num_ratio: float = 0.8 # minCoreNum ≈ 0.8 * C (经验: 约 3/4 核并发才达 90%+ 带宽利用率)
|
||||||
|
|
||||||
# ---- DMA 固定开销 ----
|
# ---- DMA 固定开销 ----
|
||||||
t_cmd_ns: float = 50.0 # T_cmd: 单次 GM->L1 DMA 命令固定开销 (ns, 估计值, 需实测标定)
|
t_cmd_ns: float = 0.0 # T_cmd: 单次 GM->L1 DMA 命令固定开销 (ns)。
|
||||||
|
# 未标定, 按 0 处理 (issue#36 用户澄清):
|
||||||
|
# MergeBatch 的合并收益由搬移效率模型
|
||||||
|
# (move_eff: 单块 tile=nValue*dValue*dt
|
||||||
|
# 放大 b0 倍) 刻画, 不依赖 T_cmd 估计值.
|
||||||
|
|
||||||
|
# ---- 带宽模型假设 (issue#26) ----
|
||||||
|
# GM 1.6TB/s 为读写共享总线 (读+写累加计时); L2 带宽读写各自独享 5.2TB/s;
|
||||||
|
# active_cores < C 时按"每核份额线性配平"为模型假设 (如 1 核也给 50GB/s),
|
||||||
|
# 真实低核数带宽利用率低于线性, 需 msProf 实测曲线标定后替换该假设.
|
||||||
|
|
||||||
# ---- Cube 计算粒度 ----
|
# ---- Cube 计算粒度 ----
|
||||||
fractal: int = 16 # 16x16x16 基本块
|
fractal: int = 16 # 16x16x16 基本块
|
||||||
@@ -50,13 +93,50 @@ class NpuSpec:
|
|||||||
# ---- 派生量 (属性) ----
|
# ---- 派生量 (属性) ----
|
||||||
@property
|
@property
|
||||||
def q16(self) -> float:
|
def q16(self) -> float:
|
||||||
"""单核 Cube BF16 峰值算力 (FLOP/s)."""
|
"""单核 Cube BF16/FP16 峰值算力 (FLOP/s) = 13.5T (issue#40 Cube-only)."""
|
||||||
return self.cube_peak_tflops * 1e12 / self.aic_num
|
return self.cube_peak_tflops * 1e12 / self.aic_num
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _dtype_key(dtype) -> str:
|
||||||
|
return str(dtype).strip().lower()
|
||||||
|
|
||||||
|
def cube_factor(self, dtype_a, dtype_b=None) -> float:
|
||||||
|
"""按输入 dtype 取 Cube 精度因子 (issue#28).
|
||||||
|
|
||||||
|
A/B 不一致时取**较慢一侧** (因子较小者, 等价字节较大者);
|
||||||
|
Cube 乘法两侧的实际吞吐受较慢精度限制.
|
||||||
|
"""
|
||||||
|
keys = [self._dtype_key(dtype_a)]
|
||||||
|
if dtype_b is not None:
|
||||||
|
keys.append(self._dtype_key(dtype_b))
|
||||||
|
factors = [CUBE_DTYPE_FACTOR.get(k, _RATE_FALLBACK) for k in keys]
|
||||||
|
return min(factors)
|
||||||
|
|
||||||
|
def q_cube(self, dtype_a, dtype_b=None) -> float:
|
||||||
|
"""单核 Cube 峰值算力 (FLOP/s), 按输入 dtype 分档 (issue#28)."""
|
||||||
|
return self.q16 * self.cube_factor(dtype_a, dtype_b)
|
||||||
|
|
||||||
|
def aiv_elem_factor(self, dtype) -> float:
|
||||||
|
"""按 dtype 取 AIV 逐元素通量因子 (相对 fp32 基准)."""
|
||||||
|
return AIV_DTYPE_FACTOR.get(self._dtype_key(dtype), _RATE_FALLBACK)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def aiv_elem_rate_fp32(self) -> float:
|
||||||
|
"""AIV fp32 逐元素吞吐 (元素/s), 64 核合计."""
|
||||||
|
return self.aiv_num * self.aiv_fp32_per_cycle * self.aiv_freq_ghz * 1e9
|
||||||
|
|
||||||
|
def aiv_elem_rate(self, dtype) -> float:
|
||||||
|
"""AIV 逐元素吞吐 (元素/s, 64 核合计), 按 dtype 分档 (issue#28)."""
|
||||||
|
return self.aiv_elem_rate_fp32 * self.aiv_elem_factor(dtype)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def q_aiv(self) -> float:
|
def q_aiv(self) -> float:
|
||||||
"""AIV 向量求和吞吐 (元素/s), 64 核合计."""
|
"""AIV fp32 逐元素吞吐 (元素/s, 64 核合计) = aiv_elem_rate_fp32.
|
||||||
return self.aiv_num * self.aiv_fp32_per_cycle * self.aiv_freq_ghz * 1e9
|
|
||||||
|
注意: 该值只适用于 fp32 数据 (如 StreamK 对 fp32 部分和求和);
|
||||||
|
逐元素运算按输入 dtype 用 aiv_elem_rate(dtype) (issue#28).
|
||||||
|
"""
|
||||||
|
return self.aiv_elem_rate_fp32
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def bw_pc(self) -> float:
|
def bw_pc(self) -> float:
|
||||||
@@ -70,7 +150,11 @@ class NpuSpec:
|
|||||||
|
|
||||||
@property
|
@property
|
||||||
def r16(self) -> float:
|
def r16(self) -> float:
|
||||||
"""16bit 位宽平衡点算存比 R_16 = Cube峰值 / (GM带宽/2B) ≈ 607.5 FLOP/元素."""
|
"""16bit 位宽平衡点算存比 R_16 = Cube峰值 / (GM带宽/2B) = 540 FLOP/元素.
|
||||||
|
|
||||||
|
按 issue#28 速率表该比值对全 dtype 不变 (Cube 因子与元素字节数互成反比),
|
||||||
|
故入口条件沿用单一 R_16. (issue#40 Cube-only 口径: 607.5 -> 540)
|
||||||
|
"""
|
||||||
return self.cube_peak_tflops * 1e12 / (self.bw_gm / 2)
|
return self.cube_peak_tflops * 1e12 / (self.bw_gm / 2)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@@ -90,3 +174,14 @@ class NpuSpec:
|
|||||||
|
|
||||||
|
|
||||||
ASCEND950PR = NpuSpec()
|
ASCEND950PR = NpuSpec()
|
||||||
|
|
||||||
|
# 950PR 28 核衍生版 (白皮书表3-1: 28 AIC / 56 AIV, HBM 1.4TB/s 112GB,
|
||||||
|
# L2 112MB; L2 带宽未分档, 沿用 5.2TB/s 待标定)
|
||||||
|
ASCEND950PR_C28 = NpuSpec(
|
||||||
|
name="Ascend950PR_C28",
|
||||||
|
aic_num=28, aiv_num=56,
|
||||||
|
cube_peak_tflops=378.0,
|
||||||
|
l2_bytes=112 * 1024 * 1024,
|
||||||
|
bw_gm=1.4e12,
|
||||||
|
gm_capacity_gb=112.0,
|
||||||
|
)
|
||||||
|
|||||||
@@ -28,13 +28,15 @@ DTYPE_BYTES = {
|
|||||||
"fp8_e4m3": 1,
|
"fp8_e4m3": 1,
|
||||||
"fp8_e5m2": 1,
|
"fp8_e5m2": 1,
|
||||||
"int8": 1,
|
"int8": 1,
|
||||||
|
"fp4": 0.5, # 半字节 (对齐 bmmv3, 2026-09-03)
|
||||||
|
"fp4_e2m1": 0.5,
|
||||||
}
|
}
|
||||||
|
|
||||||
# Cube 累加器 (L0C) 中元素字节数: 16bit 输入 -> fp32 累加; fp8 输入 -> fp32 累加
|
# Cube 累加器 (L0C) 中元素字节数: 16bit 输入 -> fp32 累加; fp8/fp4 输入 -> fp32 累加
|
||||||
L0C_DTYPE_BYTES = 4
|
L0C_DTYPE_BYTES = 4
|
||||||
|
|
||||||
|
|
||||||
def dtype_bytes(dtype: str) -> int:
|
def dtype_bytes(dtype: str) -> float:
|
||||||
key = dtype.strip().lower()
|
key = dtype.strip().lower()
|
||||||
if key not in DTYPE_BYTES:
|
if key not in DTYPE_BYTES:
|
||||||
raise ValueError(f"不支持的 dtype: {dtype!r}, 支持 {sorted(DTYPE_BYTES)}")
|
raise ValueError(f"不支持的 dtype: {dtype!r}, 支持 {sorted(DTYPE_BYTES)}")
|
||||||
@@ -53,6 +55,53 @@ def align_down(x: int, align: int) -> int:
|
|||||||
return (x // align) * align
|
return (x // align) * align
|
||||||
|
|
||||||
|
|
||||||
|
def dvalue_contig_dims(case: "BmmCase", k_l1: float) -> tuple:
|
||||||
|
"""A/B 两侧 GM->L1 搬移的连续维 dValue (Byte) —— 依转置定连续维 (ND 排布).
|
||||||
|
|
||||||
|
A 不转置存储 [M,K]: 行内 K 连续, 切 K 后每行连续段 k_l1 -> k_l1*dt;
|
||||||
|
A 转置存储 [K,M]: M 连续 -> m*dt;
|
||||||
|
B 不转置存储 [K,N]: N 连续 -> n*dt;
|
||||||
|
B 转置存储 [N,K]: 行内 K 连续, 切 K 后连续段 k_l1 -> k_l1*dt.
|
||||||
|
|
||||||
|
供三处共用 (issue#19): l1_form 生成守卫 / IterBatch 条件 4 / constraints 校验,
|
||||||
|
保证"生成说可行、条件说可行、校验说可行"口径一致.
|
||||||
|
"""
|
||||||
|
dt = case.dtype_in_bytes
|
||||||
|
dv_a = case.m * dt if case.trans_a else k_l1 * dt
|
||||||
|
dv_b = k_l1 * dt if case.trans_b else case.n * dt
|
||||||
|
return dv_a, dv_b
|
||||||
|
|
||||||
|
|
||||||
|
def move_eff(tile_bytes: float, min_tile_size: int) -> float:
|
||||||
|
"""GM->L1 单命令搬移效率 (issue#36, 用户澄清口径).
|
||||||
|
|
||||||
|
单命令搬移效率由单块 tile = nValue × dValue × input_dtype 决定: tile 越大,
|
||||||
|
有效带宽越高; 达到 min_TileSize (16KB, 即进入条件 4 的效率下限常数) 饱和,
|
||||||
|
之下线性退化 —— 与"max(MK,KN)·dt >= min_TileSize 才保证搬移效率"的语义同源
|
||||||
|
(该条件只要求一侧达标, 另一侧小 tile 的低效由此函数量化)。
|
||||||
|
|
||||||
|
MergeBatch 合并 b0 个 batch 后单块 tile 放大 b0 倍 (A 侧: b0·M·k_l1·dt,
|
||||||
|
B 侧: b0·N·k_l1·dt; 堆叠方向视转置/排布而定 —— A ND 非转置时沿 M(nValue)
|
||||||
|
堆叠, B ND 非转置时沿 N(dValue) 堆叠, 乘积口径不变), 因此即便 T_cmd=0
|
||||||
|
合并仍有搬移效率收益。
|
||||||
|
"""
|
||||||
|
if tile_bytes <= 0:
|
||||||
|
return 1.0
|
||||||
|
return min(1.0, tile_bytes / min_tile_size)
|
||||||
|
|
||||||
|
|
||||||
|
def gm_move_time(v_a: float, v_b: float, tile_a: float, tile_b: float,
|
||||||
|
spec) -> float:
|
||||||
|
"""效率加权的 GM->L1 数据时延 (秒, 整芯片口径, issue#36).
|
||||||
|
|
||||||
|
t = (V_A/eff_A + V_B/eff_B) / BW_gm; 只影响**时间**列, GM 字节量不变
|
||||||
|
(仍按每输入字节恰读一次 = V_in 计, issue#31)。
|
||||||
|
"""
|
||||||
|
ea = move_eff(tile_a, spec.min_tile_size)
|
||||||
|
eb = move_eff(tile_b, spec.min_tile_size)
|
||||||
|
return (v_a / ea + v_b / eb) / spec.bw_gm
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Case 输入
|
# Case 输入
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -109,12 +158,19 @@ class BmmCase:
|
|||||||
return max(self.batch_a, self.batch_b)
|
return max(self.batch_a, self.batch_b)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def dtype_in_bytes(self) -> int:
|
def dtype_in_bytes(self) -> float:
|
||||||
# A/B 输入元素字节数 (要求 A/B 同 dtype, 不一致时取较大者并在校验中报 warning)
|
# A/B 输入元素字节数 (要求 A/B 同 dtype, 不一致时取较大者并在校验中报 warning)
|
||||||
|
# 返回 float 以兼容 fp4 (0.5B)
|
||||||
return max(dtype_bytes(self.dtype_a), dtype_bytes(self.dtype_b))
|
return max(dtype_bytes(self.dtype_a), dtype_bytes(self.dtype_b))
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def dtype_out_bytes(self) -> int:
|
def dtype_in(self) -> str:
|
||||||
|
"""输入侧"较慢"dtype (元素字节较大者) — Cube/AIV 速率取慢侧 (issue#28)."""
|
||||||
|
return (self.dtype_a if dtype_bytes(self.dtype_a) >= dtype_bytes(self.dtype_b)
|
||||||
|
else self.dtype_b)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def dtype_out_bytes(self) -> float:
|
||||||
return dtype_bytes(self.dtype_c)
|
return dtype_bytes(self.dtype_c)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@@ -241,23 +297,30 @@ class ImplPlan:
|
|||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class HardwareTiming:
|
class HardwareTiming:
|
||||||
"""各硬件流水级时延 (秒) 与数据量明细."""
|
"""各硬件流水级时延 (秒) 与数据量明细.
|
||||||
|
|
||||||
|
数据量列口径 (issue#29, 设计文档 docs/05 §4.4):
|
||||||
|
gm_read_bytes / l2_read_bytes / fixpipe_bytes / cube_flops = **整芯片**总量
|
||||||
|
(跨分支可比, GM 首读下限断言: gm_read_bytes >= case.input_bytes);
|
||||||
|
dma_cmd_count = **单核** GM->L1 DMA 命令数 (各核引擎并行, 墙钟 T_cmd =
|
||||||
|
单核命令数 x T_cmd), 与字节列口径不同属.
|
||||||
|
"""
|
||||||
|
|
||||||
# 搬入 (MTE2)
|
# 搬入 (MTE2)
|
||||||
gm_read_bytes: float = 0.0 # GM->L1 直读数据量 (不驻留/未命中 L2 的部分)
|
gm_read_bytes: float = 0.0 # GM->L1 直读数据量 (整芯片, 首读/落空重读)
|
||||||
l2_read_bytes: float = 0.0 # L2->L1 数据量 (驻留 L2 后重复读命中部分)
|
l2_read_bytes: float = 0.0 # L2->L1 数据量 (整芯片, 驻留 L2 后重复读命中部分)
|
||||||
t_mte2_gm: float = 0.0 # GM->L1 时延 (按 GM 带宽, 不累加 L2->L1)
|
t_mte2_gm: float = 0.0 # GM->L1 时延 (按 GM 带宽, 不累加 L2->L1)
|
||||||
t_mte2_l2: float = 0.0 # L2->L1 时延 (按 L2 带宽)
|
t_mte2_l2: float = 0.0 # L2->L1 时延 (按 L2 带宽)
|
||||||
t_mte2: float = 0.0 # 搬入合计 = t_mte2_gm + t_mte2_l2 (两者发生在不同数据上)
|
t_mte2: float = 0.0 # 搬入合计 = t_mte2_gm + t_mte2_l2 (两者发生在不同数据上)
|
||||||
dma_cmd_count: float = 0.0 # GM->L1 DMA 命令次数 (T_cmd 分析用)
|
dma_cmd_count: float = 0.0 # 单核 GM->L1 DMA 命令次数 (T_cmd 分析用)
|
||||||
t_dma_cmd: float = 0.0 # DMA 命令固定开销合计
|
t_dma_cmd: float = 0.0 # DMA 命令固定开销合计 (墙钟)
|
||||||
|
|
||||||
# 计算 (Cube MMAD)
|
# 计算 (Cube MMAD)
|
||||||
cube_flops: float = 0.0 # Cube 实际计算量 (MergeBatch 含冗余)
|
cube_flops: float = 0.0 # Cube 实际计算量 (整芯片, MergeBatch 含冗余)
|
||||||
t_mmad: float = 0.0
|
t_mmad: float = 0.0
|
||||||
|
|
||||||
# 搬出 (Fixpipe)
|
# 搬出 (Fixpipe)
|
||||||
fixpipe_bytes: float = 0.0 # 写出数据量 (按 C 矩阵 dtype / StreamK 临时矩阵按 4B)
|
fixpipe_bytes: float = 0.0 # 写出数据量 (整芯片, 按 C 矩阵 dtype / StreamK 临时矩阵按 4B)
|
||||||
t_fixpipe: float = 0.0
|
t_fixpipe: float = 0.0
|
||||||
|
|
||||||
# 归约 (StreamK 专用)
|
# 归约 (StreamK 专用)
|
||||||
|
|||||||
@@ -4,10 +4,12 @@
|
|||||||
1. 前置归约: BatchA=1 或 BatchB=1 -> 转Matmul
|
1. 前置归约: BatchA=1 或 BatchB=1 -> 转Matmul
|
||||||
K=0 / K=1 -> 特殊分支 (AIV 向量通路)
|
K=0 / K=1 -> 特殊分支 (AIV 向量通路)
|
||||||
2. B >= C 且 BatchA==BatchB: 切B -> IterBatch 与 MergeBatch 仲裁
|
2. B >= C 且 BatchA==BatchB: 切B -> IterBatch 与 MergeBatch 仲裁
|
||||||
仲裁规则 (v1.1 §4.5 统一分界):
|
仲裁规则 (v1.1 §4.5 统一分界 + issue#35/#36 泛化):
|
||||||
MergeBatch 最优 <=> K截断(k_L1=K) 且 b_core > b0*(T_comp+T_write)/T_cmd
|
净收益 = 命令节省(cmds 差 × T_cmd) + 搬移效率节省(合并 tile 放大 b0 倍,
|
||||||
L1 绑定时 MergeBatch 恒劣于 IterBatch;
|
move_eff 模型) − drain 惩罚; K截断且效率打平时退化为文档闭式
|
||||||
两分支同时合法时用端到端时延模型 T_total 仲裁.
|
b_core > b0*(T_comp+T_write)/T_cmd;
|
||||||
|
两分支同时合法时用端到端时延模型 T_total 终审 (T_cmd 默认 0, 未标定;
|
||||||
|
合并效率收益已建模, 不再需要策略覆盖).
|
||||||
3. StreamK 检查: P <= C/2 且满足切K条件 -> StreamK (B/M/N 买不满时买 K)
|
3. StreamK 检查: P <= C/2 且满足切K条件 -> StreamK (B/M/N 买不满时买 K)
|
||||||
4. 兜底: ASW_Basic 切 M/N (含降核模式)
|
4. 兜底: ASW_Basic 切 M/N (含降核模式)
|
||||||
"""
|
"""
|
||||||
@@ -84,7 +86,9 @@ class BranchRouter:
|
|||||||
|
|
||||||
arbitration = ""
|
arbitration = ""
|
||||||
if mb.capable and ib.capable:
|
if mb.capable and ib.capable:
|
||||||
# 统一分界条件 + 端到端时延仲裁双保险
|
# 统一分界条件 + 端到端时延仲裁双保险; 时延模型为最终裁决.
|
||||||
|
# (issue#36: 合并的搬移效率收益已由 move_eff 模型量化进时延模型,
|
||||||
|
# T_cmd=0 默认下不再需要"T_cmd<=0 策略优先 MergeBatch"的覆盖逻辑)
|
||||||
mb_win, detail = self.merge_batch.beats_iterbatch(case)
|
mb_win, detail = self.merge_batch.beats_iterbatch(case)
|
||||||
t_mb = mb.timing.t_total
|
t_mb = mb.timing.t_total
|
||||||
t_ib = ib.timing.t_total
|
t_ib = ib.timing.t_total
|
||||||
@@ -94,10 +98,10 @@ class BranchRouter:
|
|||||||
f"两分支均合法, 仲裁: "
|
f"两分支均合法, 仲裁: "
|
||||||
f"[分界条件] MergeBatch最优={mb_win} ({detail}); "
|
f"[分界条件] MergeBatch最优={mb_win} ({detail}); "
|
||||||
f"[时延模型] T_MergeBatch={t_mb*1e6:.2f}us vs T_IterBatch={t_ib*1e6:.2f}us -> {lat_win}更优; "
|
f"[时延模型] T_MergeBatch={t_mb*1e6:.2f}us vs T_IterBatch={t_ib*1e6:.2f}us -> {lat_win}更优; "
|
||||||
f"[裁决] {win}" + ("" if win == lat_win else f" (分界条件与时延模型不一致, 以时延模型为准: {lat_win})")
|
f"[裁决] {lat_win}" + ("" if win == lat_win else
|
||||||
|
f" (分界条件判{win}, 与时延模型不一致, 以时延模型为准)")
|
||||||
)
|
)
|
||||||
if win != lat_win:
|
win = lat_win # 时延模型为最终裁决
|
||||||
win = lat_win # 时延模型为最终裁决
|
|
||||||
elif any(capable.values()):
|
elif any(capable.values()):
|
||||||
win = next(n for n, v in capable.items() if v)
|
win = next(n for n, v in capable.items() if v)
|
||||||
arbitration = f"仅 {win} 条件满足"
|
arbitration = f"仅 {win} 条件满足"
|
||||||
@@ -158,6 +162,9 @@ class BranchRouter:
|
|||||||
if violations:
|
if violations:
|
||||||
note = (note + " [自检违规: " + "; ".join(violations) +
|
note = (note + " [自检违规: " + "; ".join(violations) +
|
||||||
"] —— 方案生成存在缺陷, 需人工复核")
|
"] —— 方案生成存在缺陷, 需人工复核")
|
||||||
|
# issue#34: 兜底分支 (ASW) 效率下限不满足时降级标注 (warning), 不判违规
|
||||||
|
if result.plan is not None and "效率降级" in result.plan.note:
|
||||||
|
note += " [效率降级标注: 搬移效率低于模型假设, 时延可能低估, 见 plan.note]"
|
||||||
return {
|
return {
|
||||||
"branch": result.plan.branch if result.plan else "未知",
|
"branch": result.plan.branch if result.plan else "未知",
|
||||||
"plan": result.plan,
|
"plan": result.plan,
|
||||||
|
|||||||
@@ -24,7 +24,19 @@ from __future__ import annotations
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
|
||||||
from .hardware import NpuSpec, ASCEND950PR
|
from .hardware import NpuSpec, ASCEND950PR
|
||||||
from .models import HardwareTiming
|
from .models import BmmCase, HardwareTiming
|
||||||
|
|
||||||
|
|
||||||
|
def output_to_l2(case: BmmCase, spec: NpuSpec = ASCEND950PR,
|
||||||
|
workspace_bytes: float = 0.0) -> bool:
|
||||||
|
"""Fixpipe 输出落点决策 (issue#30, 设计文档 docs/05 §4.2 R4).
|
||||||
|
|
||||||
|
to_l2 (输出写 L2 写口 5.2TB/s, GM 写流量 = 0, 异步回写不占算子时延)
|
||||||
|
⟺ 整 case 输入 V_in + 输出 V_out [+ StreamK workspace] ≤ L2
|
||||||
|
否则输出**直写 GM**: 输入优先驻留 L2 (输入存在重复读), 输出计入 GM
|
||||||
|
读写共享总线 (与读累加, issue#23).
|
||||||
|
"""
|
||||||
|
return (case.input_bytes + case.output_bytes + workspace_bytes) <= spec.l2_bytes
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -56,9 +68,11 @@ def eval_mte2(move: MoveInPlan, spec: NpuSpec = ASCEND950PR,
|
|||||||
return t_gm, t_l2, t_gm + t_l2 + t_cmd, t_cmd
|
return t_gm, t_l2, t_gm + t_l2 + t_cmd, t_cmd
|
||||||
|
|
||||||
|
|
||||||
def eval_mmad(flops_per_core: float, spec: NpuSpec = ASCEND950PR) -> float:
|
def eval_mmad(flops_per_core: float, spec: NpuSpec = ASCEND950PR,
|
||||||
"""Cube 计算时延: 单核计算量 / 单核算力."""
|
dtype_a=None, dtype_b=None) -> float:
|
||||||
return flops_per_core / spec.q16 if flops_per_core > 0 else 0.0
|
"""Cube 计算时延: 单核计算量 / 单核 dtype 感知算力 (issue#28)."""
|
||||||
|
rate = spec.q_cube(dtype_a, dtype_b)
|
||||||
|
return flops_per_core / rate if flops_per_core > 0 else 0.0
|
||||||
|
|
||||||
|
|
||||||
def eval_fixpipe(bytes_per_core: float, to_l2: bool,
|
def eval_fixpipe(bytes_per_core: float, to_l2: bool,
|
||||||
@@ -75,21 +89,24 @@ def eval_fixpipe(bytes_per_core: float, to_l2: bool,
|
|||||||
|
|
||||||
|
|
||||||
def eval_streamk_reduce(tile_elems: float, grid_k: int, out_dtype_bytes: int,
|
def eval_streamk_reduce(tile_elems: float, grid_k: int, out_dtype_bytes: int,
|
||||||
spec: NpuSpec = ASCEND950PR) -> float:
|
spec: NpuSpec = ASCEND950PR, out_to_gm: bool = False) -> float:
|
||||||
"""StreamK 单 tile 归约时延 (v0.98 §七).
|
"""StreamK 单 tile 归约时延 (v0.98 §七).
|
||||||
|
|
||||||
部分和 dtype = L0C dtype (4B, 防精度丢失), 驻留 L2, AIV 归约:
|
部分和 dtype = L0C dtype (4B, 防精度丢失), 驻留 L2, AIV 归约 (fp32 求和,
|
||||||
|
AIV 按 fp32 通量, 不随输入 dtype 变 — issue#28):
|
||||||
AIC 写部分和 grid_k x tile x 4B / W_L2
|
AIC 写部分和 grid_k x tile x 4B / W_L2
|
||||||
AIV 读回 grid_k x tile x 4B / W_L2
|
AIV 读回 grid_k x tile x 4B / W_L2
|
||||||
AIV 求和 grid_k x tile / Q_AIV
|
AIV 求和 grid_k x tile / Q_AIV(fp32)
|
||||||
写回 tile x outB / W_L2
|
写回 tile x outB / W (最终输出落点, issue#30):
|
||||||
|
整 case 可驻留 (S_A) 时 W = W_L2; 否则直写 GM (W = W_GM)
|
||||||
"""
|
"""
|
||||||
b4 = 4
|
b4 = 4
|
||||||
w_l2 = spec.bw_l2
|
w_l2 = spec.bw_l2
|
||||||
|
w_out = spec.bw_gm if out_to_gm else w_l2
|
||||||
t_write_partial = grid_k * tile_elems * b4 / w_l2
|
t_write_partial = grid_k * tile_elems * b4 / w_l2
|
||||||
t_read_back = grid_k * tile_elems * b4 / w_l2
|
t_read_back = grid_k * tile_elems * b4 / w_l2
|
||||||
t_sum = grid_k * tile_elems / spec.q_aiv
|
t_sum = grid_k * tile_elems / spec.q_aiv
|
||||||
t_write_out = tile_elems * out_dtype_bytes / w_l2
|
t_write_out = tile_elems * out_dtype_bytes / w_out
|
||||||
return t_write_partial + t_read_back + t_sum + t_write_out
|
return t_write_partial + t_read_back + t_sum + t_write_out
|
||||||
|
|
||||||
|
|
||||||
@@ -99,19 +116,31 @@ def assemble_timing(t_mte2_gm: float, t_mte2_l2: float, t_dma_cmd: float,
|
|||||||
gm_read_bytes: float, l2_read_bytes: float,
|
gm_read_bytes: float, l2_read_bytes: float,
|
||||||
dma_cmd_count: float, cube_flops: float,
|
dma_cmd_count: float, cube_flops: float,
|
||||||
fixpipe_bytes: float,
|
fixpipe_bytes: float,
|
||||||
reduce_serial: bool = True) -> HardwareTiming:
|
reduce_serial: bool = True,
|
||||||
|
fixpipe_to_gm: bool = True) -> HardwareTiming:
|
||||||
"""汇总各级时延, 判定瓶颈.
|
"""汇总各级时延, 判定瓶颈.
|
||||||
|
|
||||||
|
带宽端口口径 (issue#23, 用户澄清 + KB):
|
||||||
|
- GM 1.6TB/s 为**读写共享总线**: MTE2 的 GM 读与 Fixpipe 直写 GM 并发时无法
|
||||||
|
拆分读写占用, 时延累加 (读+写)/1.6TB/s, 并入 MTE2 搬移链;
|
||||||
|
- L2 带宽读写各自独享 5.2TB/s: L2 重复读段(读口) 与 Fixpipe→L2 写(写口)
|
||||||
|
互不竞争; MTE2 引擎顺序服务 GM/L2 装载, 两段相加 (官方 T≈HBM/1.6+L2/5.2);
|
||||||
|
- Fixpipe→L2 (resident) 写出独立为 FIXPIPE 级, 不进 GM 总线.
|
||||||
|
|
||||||
归约计账约定 (issue#9): REDUCE 默认**串行追加** (reduce_serial=True, 归约不可
|
归约计账约定 (issue#9): REDUCE 默认**串行追加** (reduce_serial=True, 归约不可
|
||||||
掩盖, 体现在 t_drain 中), 不进稳态 max(); 仅当调用方显式声明归约可流水掩盖
|
掩盖, 体现在 t_drain 中), 不进稳态 max(); 仅当调用方显式声明归约可流水掩盖
|
||||||
(reduce_serial=False) 时才进稳态 max(). 避免"既取最大又串行追加"的双倍计账.
|
(reduce_serial=False) 时才进稳态 max(). 避免"既取最大又串行追加"的双倍计账.
|
||||||
"""
|
"""
|
||||||
t_mte2 = t_mte2_gm + t_mte2_l2 + t_dma_cmd
|
t_mte2 = t_mte2_gm + t_mte2_l2 + t_dma_cmd
|
||||||
|
fix_gm = t_fixpipe if fixpipe_to_gm else 0.0 # Fixpipe 直写 GM 的时延
|
||||||
|
fix_l2 = 0.0 if fixpipe_to_gm else t_fixpipe # Fixpipe→L2 (5.2TB/s 写口)
|
||||||
|
# MTE2 搬移链 = GM 总线(读写共享, 读+直写累加) + L2 重复读段 + DMA 命令开销
|
||||||
|
# (同一 MTE2 引擎顺序服务 GM/L2 两类装载; GM 写由 Fixpipe 并发发起, 共享 GM 总线)
|
||||||
|
mte2_chain = t_mte2_gm + t_dma_cmd + fix_gm + t_mte2_l2
|
||||||
stages = {
|
stages = {
|
||||||
"MTE2_GM": t_mte2_gm + t_dma_cmd,
|
"MTE2": mte2_chain,
|
||||||
"MTE2_L2": t_mte2_l2,
|
|
||||||
"MMAD": t_mmad,
|
"MMAD": t_mmad,
|
||||||
"FIXPIPE": t_fixpipe,
|
"FIXPIPE": fix_l2, # 仅 Fixpipe→L2 (5.2 写口, 与 L2 读口互不竞争)
|
||||||
}
|
}
|
||||||
if not reduce_serial:
|
if not reduce_serial:
|
||||||
stages["REDUCE"] = t_reduce
|
stages["REDUCE"] = t_reduce
|
||||||
@@ -135,8 +164,7 @@ def assemble_timing(t_mte2_gm: float, t_mte2_l2: float, t_dma_cmd: float,
|
|||||||
def bound_type_of(bottleneck: str) -> str:
|
def bound_type_of(bottleneck: str) -> str:
|
||||||
return {
|
return {
|
||||||
"MMAD": "计算Bound",
|
"MMAD": "计算Bound",
|
||||||
"MTE2_GM": "访存Bound(GM)",
|
"MTE2": "访存Bound(GM读写共享+L2重复读)",
|
||||||
"MTE2_L2": "访存Bound(L2)",
|
"FIXPIPE": "写出Bound(L2写口)",
|
||||||
"FIXPIPE": "写出Bound",
|
|
||||||
"REDUCE": "归约Bound",
|
"REDUCE": "归约Bound",
|
||||||
}.get(bottleneck, "")
|
}.get(bottleneck, "")
|
||||||
|
|||||||
@@ -25,7 +25,9 @@
|
|||||||
│ └─ constraints.py 单一约束源 (生成与校验共用) │
|
│ └─ constraints.py 单一约束源 (生成与校验共用) │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ 硬件层 (hardware/) │
|
│ 硬件层 (hardware/) │
|
||||||
│ └─ ascend950pr.py NpuSpec 参数表 (换芯片只换这份) │
|
│ ├─ ascend950pr.py NpuSpec + 950PR 32/28核 SKU │
|
||||||
|
│ ├─ ascend950dt.py 950DT 36/32/28核 SKU (issue#38) │
|
||||||
|
│ └─ __init__.py SPECS 注册表 + get_spec(name) │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ IO 层 (io_csv.py) case/plan/result 的 csv 读写 │
|
│ IO 层 (io_csv.py) case/plan/result 的 csv 读写 │
|
||||||
└─────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────┘
|
||||||
@@ -86,7 +88,7 @@ class Branch:
|
|||||||
|
|
||||||
### 3.3 硬件参数与逻辑分离
|
### 3.3 硬件参数与逻辑分离
|
||||||
|
|
||||||
`hardware/ascend950pr.py` 的 `NpuSpec` 集中所有芯片常数(核数/算力/各级容量/带宽/搬移效率经验值/T_cmd)。所有分支通过 `self.spec` 取参数——换芯片时新增一份参数表即可,分支逻辑零改动。
|
`hardware/` 的 `NpuSpec` 集中所有芯片常数(核数/算力/各级容量/带宽/搬移效率经验值/T_cmd)。所有分支通过 `self.spec` 取参数——换芯片时新增一份参数表即可,分支逻辑零改动。已注册 SKU(issue#38,昇腾950 白皮书表3-1/表4-2):`Ascend950PR`(32核主bin)/`Ascend950PR_C28` 与 `Ascend950DT`(36核主bin)/`Ascend950DT_C32`/`Ascend950DT_C28`,经 `hardware.SPECS` / `get_spec(name)` 索引,默认仍为 `ASCEND950PR`。
|
||||||
|
|
||||||
### 3.4 时延引擎统一在 timing.py
|
### 3.4 时延引擎统一在 timing.py
|
||||||
|
|
||||||
@@ -110,6 +112,6 @@ class Branch:
|
|||||||
当前六分支已全部实现(转Matmul / 特殊 / MergeBatch / IterBatch / StreamK / ASW_Basic)。后续方向:
|
当前六分支已全部实现(转Matmul / 特殊 / MergeBatch / IterBatch / StreamK / ASW_Basic)。后续方向:
|
||||||
|
|
||||||
1. **转Matmul 精切**: 当前折叠后只粗估(`to_matmul.py::evaluate` 按 Matmul 总量),接入 MM 理论体系做精确切分。
|
1. **转Matmul 精切**: 当前折叠后只粗估(`to_matmul.py::evaluate` 按 Matmul 总量),接入 MM 理论体系做精确切分。
|
||||||
2. **换芯片**: 复制 `hardware/ascend950pr.py` 改常数,`NpuSpec` 接口不变。
|
2. **换芯片**: `NpuSpec(name=..., ...)` 派生实例并注册进 `hardware.SPECS`(参考 `ascend950dt.py`),接口不变;默认规格恒为 `ASCEND950PR`。
|
||||||
3. **标定 T_cmd**: 当前取 50ns 估计值,实测后改 `t_cmd_ns` 一处即可。
|
3. **标定 T_cmd**: 当前按 0 处理 (未标定, issue#36; MergeBatch 合并收益已由 move_eff 搬移效率模型刻画, 不依赖 T_cmd 估计值), 实测后改 `t_cmd_ns` 一处即可。
|
||||||
4. **新分支**: 在 `branches/` 下加一个文件实现三接口 + 在 `router.py` 注册 + 在 `evaluator._BRANCH_EVAL` 注册;约束一律走 `constraints.py`,不在分支内另造规则。
|
4. **新分支**: 在 `branches/` 下加一个文件实现三接口 + 在 `router.py` 注册 + 在 `evaluator._BRANCH_EVAL` 注册;约束一律走 `constraints.py`,不在分支内另造规则。
|
||||||
|
|||||||
@@ -18,7 +18,9 @@ $$T_{total} = \max\big(T_{MMAD},\; T_{MTE2},\; T_{MTE1},\; T_{Fixpipe}\;[,\;T_{R
|
|||||||
|
|
||||||
case 固有算存比与 16bit 位宽平衡点:
|
case 固有算存比与 16bit 位宽平衡点:
|
||||||
|
|
||||||
$$AI = \frac{2MN}{M+N},\qquad R_{16} = \frac{486\ \text{TFLOPS}}{1.6\ \text{TB/s}\,/\,2\ \text{B}} \approx 607.5$$
|
$$AI = \frac{2MN}{M+N},\qquad R_{16} = \frac{432\ \text{TFLOPS}}{1.6\ \text{TB/s}\,/2\ \text{B}} = 540$$
|
||||||
|
|
||||||
|
(Cube 算力为白皮书 Cube-only 口径 432 TFLOPS,issue#40;此前 v0.98 沿用总算力 486 → R₁₆=607.5。)
|
||||||
|
|
||||||
AI < R₁₆ → 访存 Bound;反之计算 Bound。
|
AI < R₁₆ → 访存 Bound;反之计算 Bound。
|
||||||
|
|
||||||
@@ -50,7 +52,9 @@ case (B, M, N, K, dtype)
|
|||||||
│
|
│
|
||||||
├─ 单 batch M×N 太小, 合并搬移 ──▶ MergeBatch (冗余算力换搬移效率)
|
├─ 单 batch M×N 太小, 合并搬移 ──▶ MergeBatch (冗余算力换搬移效率)
|
||||||
│ 进入: 5 条件 (L0C/搬移量/tile/算存比/b_core≥2b0)
|
│ 进入: 5 条件 (L0C/搬移量/tile/算存比/b_core≥2b0)
|
||||||
│ 胜出: K 截断 (k_L1=K) 且 b_core > b0(T_comp+T_write)/T_cmd
|
│ 胜出: 命令节省(cmds差×T_cmd) + 搬移效率节省(合并 tile 放大 b0 倍,
|
||||||
|
│ issue#36) > drain 惩罚; K截断且效率打平时退化为
|
||||||
|
│ b_core > b0(T_comp+T_write)/T_cmd (v1.1 §4.5)
|
||||||
│
|
│
|
||||||
└─ 逐 batch 计算 ──────────────▶ IterBatch (无浪费, 最朴素)
|
└─ 逐 batch 计算 ──────────────▶ IterBatch (无浪费, 最朴素)
|
||||||
进入: b_core≥1 + 负载均衡 + L1 四形态之一 + 搬移效率
|
进入: b_core≥1 + 负载均衡 + L1 四形态之一 + 搬移效率
|
||||||
@@ -64,7 +68,7 @@ case (B, M, N, K, dtype)
|
|||||||
│
|
│
|
||||||
▼ P ≤ C/2 (B/M/N 都买不满)
|
▼ P ≤ C/2 (B/M/N 都买不满)
|
||||||
StreamK: 核间切 K + 归约
|
StreamK: 核间切 K + 归约
|
||||||
进入: P ≤ C/2, K/grid_K ≥ 256B/dtype, K > grid_K²/(grid_K-1)·θ_c (θ_c≈12)
|
进入: P ≤ C/2, K/grid_K ≥ 256B/dtype, K > grid_K²/(grid_K-1)·θ_c (θ_c≈10.9, issue#40 Cube-only 口径)
|
||||||
```
|
```
|
||||||
|
|
||||||
## 4. 重叠区仲裁
|
## 4. 重叠区仲裁
|
||||||
|
|||||||
@@ -43,24 +43,51 @@ k_L1 被 512B 截断省出的 L1 空间容纳更多 batch,提升 batch 间流
|
|||||||
|
|
||||||
## 4. 时延模型(v1.1 §4)
|
## 4. 时延模型(v1.1 §4)
|
||||||
|
|
||||||
符号:T_load = k_L1(M+N)·dt/BW_pc(每 K 分块搬移)、T_comp = 2MN·k_L1/Q₁₆(每 K 分块计算)、T_write = MN·outB/W_GM(单 batch 写回)、T_cmd(单次 GM→L1 DMA 固定开销,~50ns)。
|
符号:T_load = k_L1(M+N)·dt/(BW_pc·eff)(每 K 分块搬移,eff=move_eff(单命令tile),见下)、T_comp = 2MN·k_L1/Q₁₆(每 K 分块计算)、T_write = MN·outB/W_GM(单 batch 写回)、T_cmd(单次 GM→L1 DMA 固定开销,**默认按 0**——未标定;issue#36 起合并收益由搬移效率模型刻画,不依赖 T_cmd 估计值)。
|
||||||
|
|
||||||
|
> **搬移效率模型 (issue#36, 用户澄清)**: 单命令搬移效率由单块 tile = nValue×dValue×dt
|
||||||
|
> 决定 —— eff(tile) = min(1, tile/min_TileSize),达 16KB 饱和、之下线性退化 (与进入
|
||||||
|
> 条件 4 的效率下限语义同源)。**合并 b0 个 batch 使单块 tile 放大 b0 倍** (A 侧
|
||||||
|
> b0·M·k_L1^m·dt, B 侧 b0·N·k_L1^m·dt; 堆叠方向视转置/排布: A ND 非转置沿 M(nValue)
|
||||||
|
> 堆叠, B ND 非转置沿 N(dValue) 堆叠, 乘积不变), 相对 IterBatch 逐 batch 搬移
|
||||||
|
> (A 侧 tile = M·k_L1^iter·dt) 效率更高 —— **即便 T_cmd=0 也有收益**。
|
||||||
|
> GM→L1 数据时延按两侧字节加权: t_data = (V_A/eff_A + V_B/eff_B)/W_GM;
|
||||||
|
> GM 字节量不变 (仍 = V_in)。效率模型只接入切B 两分支 (逐命令 tile 小、效率差显著);
|
||||||
|
> ASW/StreamK 单命令 tile 通常 ≥ 数百 KB 已饱和 (极端小 tile 形状有"效率降级"标注通道,
|
||||||
|
> issue#34)。
|
||||||
|
|
||||||
$$T_{mb} = \underbrace{\frac{b_{core}}{b_0}\cdot n_K^m\cdot(T_{load}^m + T_{cmd})}_{\text{搬移(合并)}} + \underbrace{b_0(T_{comp}+T_{write})}_{\text{末合并 batch drain}}$$
|
$$T_{mb} = \underbrace{\frac{b_{core}}{b_0}\cdot n_K^m\cdot(T_{load}^m + T_{cmd})}_{\text{搬移(合并)}} + \underbrace{b_0(T_{comp}+T_{write})}_{\text{末合并 batch drain}}$$
|
||||||
|
|
||||||
两种情形:
|
两种经典情形 (v1.1 §4.3/§4.4) 与第三情形 (issue#35):
|
||||||
|
|
||||||
| 情形 | k_L1^m | n_K^m | 搬移命令数 | 结论 |
|
| 情形 | k_L1^m | n_K^m | 每核搬移命令数 (⌈b_core/b₀⌉·n_K^m) | 结论 |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| **K 截断** (k_L1=K) | K 不减半 | 1 | b_core/b₀(少 b₀ 倍) | MergeBatch 可胜 |
|
| **K 截断** (k_L1^m=K) | K | 1 | ⌈b_core/b₀⌉(比 IterBatch 少 b₀ 倍) | MergeBatch 可胜 |
|
||||||
| **L1 绑定** (k_L1<K) | k_L1/b₀ | b₀·n_K | b_core·n_K(与 IterBatch 相同) | **MergeBatch 恒劣** |
|
| **L1 绑定** (k_L1^m=k_L1^iter/b₀) | k_L1^iter/b₀ | b₀·n_K | b_core·n_K(与 IterBatch 相同) | **MergeBatch 恒劣** |
|
||||||
|
| **dValue cap 截断** (k_L1^m=512B/dt, 文档二分未覆盖) | 512B/dt | ⌈K/k_L1^m⌉ | 按实际比较 (可与 IterBatch 打平或少) | 按泛化分界判定 |
|
||||||
|
|
||||||
L1 绑定情形搬移次数、单次搬移量都与 IterBatch 相同,只放大 drain——证明见 v1.1 §4.4。
|
L1 绑定理想情形搬移次数、单次搬移量都与 IterBatch 相同,只放大 drain——证明见 v1.1 §4.4。
|
||||||
|
第三情形例: IterBatch 走 a/b 形态 (k_L1=K) 而合并侧被 512B 推荐值截断时, n_K^m=n_K,
|
||||||
|
命令数打平 (既不省 b₀ 倍也不放大), 合并仅剩 drain 惩罚 -> 恒劣; 反之若 IterBatch 走
|
||||||
|
c/d 形态切 K 而合并侧每核命令数更少, 则按实际节省判定。
|
||||||
|
|
||||||
|
> **每核命令数口径 (issue#35)**: 真实命令数 = **合并组数 × 每组 K 段数** =
|
||||||
|
> ⌈b_core/b₀⌉·⌈K/k_L1^m⌉。注意 v1.1 §4.4 "命令数与 IterBatch 相同 = b_core·n_K" 中的
|
||||||
|
> n_K 是**未合并**粒度 (⌈K/k_L1^iter⌉); 误代入合并后段数 ⌈K/k_L1^m⌉ 会多计 b₀ 倍
|
||||||
|
> (修复前 evaluate 即此错, L1 绑定情形命令时延虚高 b₀ 倍, 可把仲裁方向翻错)。
|
||||||
|
|
||||||
|
> **GM 数据量口径 (issue#31)**: 各 (合并组, K段) 的数据互不重叠, 每个输入字节恰好从 GM
|
||||||
|
> 读一次 —— K 截断与 L1 绑定两种情形的**芯片 GM 读取量都 = V_in** (末段按实际剩余计,
|
||||||
|
> 无 padding 上取)。上式的 T_load 级联只用于刻画命令/双缓冲调度结构: 数据时延按
|
||||||
|
> V_in/W_GM 计, n_K 只放大 DMA 命令项 ⌈b_core/b₀⌉·n_K^m × T_cmd (issue#35 口径)。
|
||||||
|
|
||||||
## 5. MergeBatch vs IterBatch 净收益
|
## 5. MergeBatch vs IterBatch 净收益
|
||||||
|
|
||||||
$$\text{净收益} = \underbrace{b_{core}\Big(1-\frac{1}{b_0}\Big)T_{cmd}}_{\text{搬移命令节省}} - \underbrace{(b_0-1)(T_{comp}+T_{write})}_{\text{drain 惩罚}}$$
|
泛化分界 (issue#35 命令数 / issue#36 搬移效率, 覆盖三种情形): 直接比较两分支**实际每核 DMA 命令数**与**效率加权搬移时延** ——
|
||||||
|
|
||||||
大 B(b_core 大)且小 MN(T_comp 小)时 MergeBatch 最优。T_cmd 的物理成因:Nd2Nz 描述符配置(7 字段写 DMA 寄存器)+ 地址生成 + 突发启动 + L1 同步握手。
|
$$\text{净收益} = \underbrace{(cmds_{iter}-cmds_{mb})\,T_{cmd}}_{\text{命令节省}} + \underbrace{(t_{data}^{iter}-t_{data}^{mb})}_{\text{效率节省}} - \underbrace{(b_0-1)(T_{comp}+T_{write})}_{\text{drain 惩罚}},\qquad \begin{array}{l}cmds_{iter}=b_{core}\lceil K/k_{L1}^{iter}\rceil\\ cmds_{mb}=\lceil b_{core}/b_0\rceil\lceil K/k_{L1}^m\rceil\end{array}$$
|
||||||
|
|
||||||
|
其中效率节省来自合并 tile 放大 (t_data 按 eff(tile) 加权, 见 §4)。**T_cmd=0 时命令节省为 0, 效率节省依然在项** —— MergeBatch 凭搬移效率胜出 (用户澄清口径)。K 截断且效率打平 (两侧 tile 均 ≥16KB 饱和) 且 T_cmd>0 时严格退化为 v1.1 §4.5 闭式 b_core > b₀(T_comp+T_write)/T_cmd;L1 绑定理想情形 (k_L1^m=k_L1^iter/b₀ 且 tile 均饱和) 命令数与效率均打平、净收益恒负。大 B(b_core 大)且小 MN(T_comp 小、IterBatch 单 batch tile 小未饱和)时 MergeBatch 最优。T_cmd 的物理成因:Nd2Nz 描述符配置(7 字段写 DMA 寄存器)+ 地址生成 + 突发启动 + L1 同步握手 (待 msProf 标定后恢复非零取值)。
|
||||||
|
|
||||||
## 6. 与源码的差异(v1.1 §5.1)
|
## 6. 与源码的差异(v1.1 §5.1)
|
||||||
|
|
||||||
|
|||||||
@@ -40,9 +40,17 @@ $$T_{iter} = \underbrace{b_{core}\cdot n_K\cdot(T_{load} + T_{cmd})}_{\text{搬
|
|||||||
|
|
||||||
每 batch 的 A[M,K]+B[K,N] 作为独立 DMA 操作搬入 L1(源码 `ndNum = curIterBatchL1`,多块独立寻址)。
|
每 batch 的 A[M,K]+B[K,N] 作为独立 DMA 操作搬入 L1(源码 `ndNum = curIterBatchL1`,多块独立寻址)。
|
||||||
|
|
||||||
|
> **GM 数据量口径 (issue#31)**: 形态 a/b/c/d 的 K 段/驻留侧数据互不重叠、每个输入字节
|
||||||
|
> 恰好从 GM 读一次 (形态 c 的驻留侧每 batch 只搬一次; 切 K 末段按实际剩余计, 无 padding
|
||||||
|
> 上取) —— **芯片 GM 读取量 = V_in**, 与 L2 容量无关 (IterBatch 无 L2 级重复读)。
|
||||||
|
> n_K 只决定 DMA 命令数 (T_cmd 项, 950PR 默认 t_cmd=0) 与双缓冲调度, 不放大数据量;
|
||||||
|
> 数据时延按**效率加权** (V_A/eff_A + V_B/eff_B)/W_GM 计 (issue#36: 单命令 tile =
|
||||||
|
> nValue×dValue×dt 越小有效带宽越低, 达 min_TileSize=16KB 饱和; 分侧口径 —— a/b 形态
|
||||||
|
> 双侧整 K、c 形态驻留侧整 K + 对侧 k_L1 分块、d 形态双侧 k_L1 分块)。
|
||||||
|
|
||||||
## 6. 与 MergeBatch 的分界
|
## 6. 与 MergeBatch 的分界
|
||||||
|
|
||||||
见 [01_MergeBatch分支.md](01_MergeBatch分支.md) §4~5。IterBatch 在 **L1 绑定情形恒优**;MergeBatch 仅在 K 截断且 b_core 足够大时胜。
|
见 [01_MergeBatch分支.md](01_MergeBatch分支.md) §4~5。IterBatch 在 **L1 绑定理想情形恒优**;MergeBatch 凭 命令节省 (T_cmd>0 时) 与 **搬移效率节省** (合并 tile 放大 b0 倍, T_cmd=0 时仍成立, issue#36) 胜出 —— 净收益 > 0 即 MergeBatch 优。
|
||||||
|
|
||||||
## 7. 与源码的差异(v1.1 §5.2)
|
## 7. 与源码的差异(v1.1 §5.2)
|
||||||
|
|
||||||
|
|||||||
@@ -21,13 +21,13 @@ K 维度是 Cube(MMAD)存在的意义——`C = Σ_k A[..,k]·B[k,..]` 的
|
|||||||
- 数据流:GM→UB(读 A、B)→ Mul → GM(写 C),全程 AIV;
|
- 数据流:GM→UB(读 A、B)→ Mul → GM(写 C),全程 AIV;
|
||||||
- 时延:`T = max(搬入, 搬出)`,AIV 算力远剩,瓶颈在搬移:`T ≈ B·(MK + KN + MN)·dt / W_GM`;
|
- 时延:`T = max(搬入, 搬出)`,AIV 算力远剩,瓶颈在搬移:`T ≈ B·(MK + KN + MN)·dt / W_GM`;
|
||||||
- **触发条件**:`B ≥ 2×AIV核数 = 128`(开 UB 乒乓需要每核至少 2 个 batch 块)且单 batch 输入输出能驻留 UB;
|
- **触发条件**:`B ≥ 2×AIV核数 = 128`(开 UB 乒乓需要每核至少 2 个 batch 块)且单 batch 输入输出能驻留 UB;
|
||||||
- **B < 128 时并不无解**(issue#12):退化为 **AIV 单缓冲**——无乒乓、逐 batch 串行搬入计算,仍远优于 Cube 通路(K=1 时 Cube 16×16×16 浪费 15/16),只是流水掩盖能力下降。软件 `special.py` 按 `B ≥ 128` 自动选择"UB乒乓 / AIV单缓冲"模式。
|
- **B < 128 时并不无解**(issue#12/#17):退化为 **AIV 单缓冲**——无乒乓、逐 batch 串行搬入计算,仍远优于 Cube 通路(K=1 时 Cube 16×16×16 浪费 15/16),只是流水掩盖能力下降。软件 `special.py` 按 `B ≥ 128` 自动选择"UB乒乓 / AIV单缓冲"模式。
|
||||||
|
|
||||||
## 4. 软件处理
|
## 4. 软件处理
|
||||||
|
|
||||||
`router.py` 前置归约中,`k ≤ 1` 直接路由到特殊分支:
|
`router.py` 前置归约中,`k ≤ 1` 直接路由到特殊分支:
|
||||||
|
|
||||||
- `K=0` → 标注"纯 AIV 写值",评估时延 = 写出时延;
|
- `K=0` → 标注"纯 AIV 写值",评估时延 = 写出时延;
|
||||||
- `K=1` → 标注"AIV 逐元素乘",评估时延 = 搬入/搬出较大者。
|
- `K=1` → 标注"AIV 逐元素乘",评估时延 = 搬入/搬出较大者;plan 按 `B ≥ 128` 自动选择"UB乒乓 / AIV单缓冲"模式(`l1_form` 与 `note` 中可见)。
|
||||||
|
|
||||||
不进入 Cube 切分体系,无 ImplPlan 的 tile 字段(除 used_core_num = AIV 核数外均不适用)。
|
不进入 Cube 切分体系,无 ImplPlan 的 tile 字段(除 used_core_num = AIV 核数外均不适用)。
|
||||||
|
|||||||
@@ -16,7 +16,7 @@
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 1 | `P ≤ C/2` | **并行缺口**:不切 K 时至多一半核有事做,K 是唯一剩余的并行维度。阈值取 C/2 而非 C:StreamK 定义即 grid_K≥2(至少 2 路切 K),grid_K=2 时每块需 2 核,总核数需求 = ⌈P⌉×2 ≤ C |
|
| 1 | `P ≤ C/2` | **并行缺口**:不切 K 时至多一半核有事做,K 是唯一剩余的并行维度。阈值取 C/2 而非 C:StreamK 定义即 grid_K≥2(至少 2 路切 K),grid_K=2 时每块需 2 核,总核数需求 = ⌈P⌉×2 ≤ C |
|
||||||
| 2 | `K/grid_K ≥ 256B/dtype` | **单核 K 段下限**:每核 K 段内轴连续长度不小于 dValue 推荐值,保证段内搬移效率不崩。grid_K = ⌊C/⌈P⌉⌋ |
|
| 2 | `K/grid_K ≥ 256B/dtype` | **单核 K 段下限**:每核 K 段内轴连续长度不小于 dValue 推荐值,保证段内搬移效率不崩。grid_K = ⌊C/⌈P⌉⌋ |
|
||||||
| 3 | `K > grid_K²/(grid_K-1)·θ_c`,θ_c≈12 | **归约代价可接受**:切 K 后总时延须小于不切 K(降核 ASW),推导见 §3 |
|
| 3 | `K > grid_K²/(grid_K-1)·θ_c`,θ_c≈10.9 | **归约代价可接受**:切 K 后总时延须小于不切 K(降核 ASW),推导见 §3 |
|
||||||
| 4 | 确定性等级 ≤ 1 且 ND 格式 | 归约顺序不定引入浮点非确定性,确定性等级 2/3 的业务禁用 |
|
| 4 | 确定性等级 ≤ 1 且 ND 格式 | 归约顺序不定引入浮点非确定性,确定性等级 2/3 的业务禁用 |
|
||||||
|
|
||||||
## 3. 归约代价与 θ_c 推导(条件 3 的来源)
|
## 3. 归约代价与 θ_c 推导(条件 3 的来源)
|
||||||
@@ -34,13 +34,13 @@ $$T_{Reduce}^t = \underbrace{\frac{grid_K\cdot M^t N^t\cdot 4B}{W_{L2}}}_{\text{
|
|||||||
|
|
||||||
**计算 Bound 情形**(T_pipe = T_MMAD^t)代入判据,两边除以 L0C/4B 后 **tile 尺寸消去**——K 阈值不依赖 M/N 具体值:
|
**计算 Bound 情形**(T_pipe = T_MMAD^t)代入判据,两边除以 L0C/4B 后 **tile 尺寸消去**——K 阈值不依赖 M/N 具体值:
|
||||||
|
|
||||||
$$K > \frac{grid_K^2}{grid_K-1}\cdot\theta_c,\qquad \theta_c = \frac{Q_{16}}{2}\Big(\frac{8B}{W_{L2}}+\frac{1}{Q_{AIV}}\Big) \approx 12$$
|
$$K > \frac{grid_K^2}{grid_K-1}\cdot\theta_c,\qquad \theta_c = \frac{Q_{16}}{2}\Big(\frac{8B}{W_{L2}}+\frac{1}{Q_{AIV}}\Big) \approx 10.9$$
|
||||||
|
|
||||||
代入数值:grid_K=2 → K>49;grid_K=4 → K>66;grid_K=8 → K>112。L2 读写(1.54 ps/元素)是主导项,AIV 求和仅占 5%。
|
(Q16 为 Cube-only 口径 13.5T,issue#40;旧总算力口径 15.1875T 时 θ_c≈12。)代入数值:grid_K=2 → K>44;grid_K=4 → K>58;grid_K=8 → K>100。L2 读写(1.54 ps/元素)是主导项,AIV 求和仅占 5%。
|
||||||
|
|
||||||
访存 Bound 情形阈值恒低于计算 Bound(θ_m·[M^tN^t/(M^t+N^t)]/θ_c ≤ 128/304 ≈ 0.42 < 1),故**汇总条件取计算 Bound 阈值**(保守,覆盖两种情形)。
|
访存 Bound 情形阈值恒低于计算 Bound(θ_m·[M^tN^t/(M^t+N^t)]/θ_c ≤ 128/270 ≈ 0.47 < 1),故**汇总条件取计算 Bound 阈值**(保守,覆盖两种情形)。
|
||||||
|
|
||||||
> **注意**:θ_c 对 workspace 落点敏感——部分和落 GM 时读写带宽从 5.2TB/s 降到 ~0.64TB/s,θ_c 升至约 97。**设计时应优先保证 workspace 驻留 L2**。
|
> **注意**:θ_c 对 workspace 落点敏感——部分和落 GM 时读写带宽从 5.2TB/s 降到 ~0.64TB/s,θ_c 升至约 86。**设计时应优先保证 workspace 驻留 L2**。
|
||||||
|
|
||||||
## 4. 实现方案
|
## 4. 实现方案
|
||||||
|
|
||||||
@@ -70,4 +70,4 @@ $$T_{SK} = \max\Big(\frac{T_{MMAD}}{grid_K},\; \frac{T_{MTE2}}{grid_K}\Big) + T_
|
|||||||
- `8192` = C×512B = dValue 推荐值在最大 grid_K=C 下的保障(同为条件 2,取推荐值);
|
- `8192` = C×512B = dValue 推荐值在最大 grid_K=C 下的保障(同为条件 2,取推荐值);
|
||||||
- max 取更严的 8192。
|
- max 取更严的 8192。
|
||||||
|
|
||||||
修正后的归约阈值(θ_c≈12,grid_K=32 时 K>396)远低于 8192,说明 **8192 的绑定约束是 dValue(条件 2),不是归约代价(条件 3)**。源码不动态算 grid_K,用固定阈值保守合并两条条件。本软件按理论动态计算 grid_K 与 θ_c,进入条件更精细。
|
修正后的归约阈值(θ_c≈10.9,grid_K=32 时 K>360)远低于 8192,说明 **8192 的绑定约束是 dValue(条件 2),不是归约代价(条件 3)**。源码不动态算 grid_K,用固定阈值保守合并两条条件。本软件按理论动态计算 grid_K 与 θ_c,进入条件更精细。
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
# ASW_Basic 分支理论(含尾轮处理)
|
# ASW_Basic 分支理论(含尾轮处理)
|
||||||
|
|
||||||
> 整理自《BMM算子优化分析 v0.98》§八 + 《BMM尾轮处理策略对比分析 v1.5》.
|
> 整理自《BMM算子优化分析 v0.98》§八 + 《ASW_Basic分支分析 v1.91》 + 《BMM尾轮处理策略对比分析 v1.5》.
|
||||||
> 尾轮策略的完整推导见 [07_尾轮处理策略.md](07_尾轮处理策略.md), 本文将其结论**内化为 ASW_Basic 最优实现的必要环节**.
|
> 尾轮策略的完整推导见 [07_尾轮处理策略.md](07_尾轮处理策略.md), 本文将其结论**内化为 ASW_Basic 最优实现的必要环节**.
|
||||||
> 对应软件实现: `bmm_theory/branches/asw_basic.py`.
|
> 对应软件实现: `bmm_theory/branches/asw_basic.py`.
|
||||||
|
> 2026-09 更新: Step0/Step1 按 v1.91 §5.1/§5.2 重写为"UnitFlag 单缓冲方形 256 + 有界枚举最小搬入" (issue#33)。
|
||||||
|
|
||||||
## 1. 定位:兜底分支,实践中最常命中
|
## 1. 定位:兜底分支,实践中最常命中
|
||||||
|
|
||||||
@@ -20,15 +21,50 @@ ASW_Basic 是 BMM 的**兜底分支**——核间切 M/N(或混合切),不
|
|||||||
|
|
||||||
## 3. 时延建模
|
## 3. 时延建模
|
||||||
|
|
||||||
核间切 M/N(不切 K),每核处理若干 [singleCoreM, singleCoreN] 输出块:
|
核间切 M/N(不切 K),每核处理若干 [singleCoreM, singleCoreN] 输出块;单 batch 内
|
||||||
|
A 行块被 n_cnt 个列 tile 共享、B 列块被 m_cnt 个行 tile 共享(issue#24 用户口径,
|
||||||
|
最新场景口径见 issue#32 与 docs/05 §4):
|
||||||
|
|
||||||
$$T = \max(T_{MTE2},\; T_{MMAD},\; T_{FIX}) + T_{drain}$$
|
$$T = \max(T_{MTE2},\; T_{MMAD},\; T_{FIX}) + T_{drain}$$
|
||||||
|
|
||||||
其中 `T_MMAD = 2BMNK/(C·Q16)`、`T_FIX = B·MN·outB/(C·W_pc)` 与分配策略无关——**分配策略只影响 T_MTE2**(通过 L2 命中率影响有效 GM 带宽):
|
- `T_MMAD = 2BMNK / (used·Q16(dtype))` —— 与分配策略无关;Q16 按输入 dtype 分档 (issue#28);
|
||||||
|
- **MTE2 两段搬移链**(issue#23/#31/#32,GM 为读写共享总线):
|
||||||
|
- `t_gm = S_in^GM / W_GM`:GM 直读量(首读 + 落空重读;与输出直写 GM 并发时
|
||||||
|
读写累加,输出落点见 docs/05 §4.2 R4);
|
||||||
|
- `t_l2 = S_in^L2 / W_L2`:L2 重复读量(共享块驻留后其余次读取,5.2TB/s 读口独享);
|
||||||
|
- MTE2 链 = t_gm + t_l2 + DMA 命令开销。
|
||||||
|
- `T_FIX`:R4 输出落点——整 case 输入+输出 ≤ L2 时写 L2 写口(异步回写不占算子时延),
|
||||||
|
否则直写 GM 计入共享总线(issue#30)。
|
||||||
|
|
||||||
$$T_{MTE2} = \frac{S_{in}^{GM}}{C\cdot BW_{pc}},\qquad S_{in}^{GM} = r_{in}\cdot S_{in}$$
|
**GM/L2 流量归属 —— L2 场景判定**(与代码 `_l2_scene` 同源,docs/05 §4.1/§4.3):
|
||||||
|
S_A 整case全驻留 / S_B 单batch可驻留(全驻留**或单侧全驻留 + 对侧滑窗**)→
|
||||||
|
GM = V_in 一次(r_in=1),共享块重复读全部命中 L2;S_C 双侧均超 L2 → 最小替换 2D 分组
|
||||||
|
(整 L2 容量约束下最小 GM),组间共享块落空回 GM、组内窗口复用按 L2 计账。
|
||||||
|
|
||||||
`r_in` = 重复读倍率(GM 输入流量/输入总量),r_in≥1,下界 1 表示每字节只从 GM 读一次(后续复用全命中 L2)。**分配策略的全部目标就是让 r_in 尽量接近 1**。
|
`r_in` = GM 输入流量/输入总量,r_in≥1,下界 1 表示每字节只从 GM 读一次(后续复用
|
||||||
|
全命中 L2)。**分配策略的全部目标就是让 r_in 尽量接近 1**:S_B 已识别"单侧可全驻留"
|
||||||
|
调度(不再把可驻留 case 错误地按分组放大 GM);只有双侧都放不下 L2 时(S_C)r_in 才 > 1。
|
||||||
|
|
||||||
|
**尾轮残余 T_drain(闭式, issue#37)**:均匀分块下稳态聚合 ≡ `(n_wave−1+ρ)·T_block`
|
||||||
|
(`N_blk/used = n_wave−1+ρ` 恒等),`T_drain` 只计尾轮结构相对稳态的残余;块级三段
|
||||||
|
时延与 `_decide_tail` 主导项判定同源(L2 命中口径):
|
||||||
|
|
||||||
|
| 策略 / 主导项 | T_drain | 出处 |
|
||||||
|
|---|---|---|
|
||||||
|
| r = 0 | 0 | — |
|
||||||
|
| A0(r>0,尾轮 r 核各 1 整块、C−r 核空转) | `(1−ρ)·T_block` | v1.5 §3.3 `T_A0 = n_wave·T_block` |
|
||||||
|
| 面积型 A1b / 方案B | 0 | v1.5 §4.3 总量守恒严格相等 |
|
||||||
|
| 周长型 A1b | `(√ρ−ρ)·T_load` | v1.5 §5 `T = (n_wave−1+√ρ)·T_load` |
|
||||||
|
| 周长型 方案B | `(√(n_wave(n_wave−1+ρ)) − (n_wave−1+ρ))·T_load` | v1.5 §5 `T = √(n_wave(n_wave−1+ρ))·T_load` |
|
||||||
|
|
||||||
|
边界说明(**不计入模型**的两项, issue#37 决议仅标注):
|
||||||
|
|
||||||
|
- **首块填充/末块排空**:v1.91 §3.1 的 `T_drain = O(T_comp+T_write)` 在 UnitFlag
|
||||||
|
16×16×16 细粒度流水下真实暴露为 granule 级小量(首 k 段搬入 + 末 granule 排空),
|
||||||
|
量级未标定,不入模型——大 n_wave 时相对误差 O(1/n_wave);
|
||||||
|
- **尾轮重切的流量放大**:方案B(全局 tile 缩 1/√g)与 A1b(尾轮区缩 √ρ)会真实
|
||||||
|
增加搬移总量(周长和 ×√g / ×1/√ρ),字节列仍按主 tile 几何计账;仅影响周长型
|
||||||
|
主导角区的精度(面积型 MTE2 非瓶颈,无影响)。
|
||||||
|
|
||||||
## 4. 核间分配策略:B→M→N 线性映射
|
## 4. 核间分配策略:B→M→N 线性映射
|
||||||
|
|
||||||
@@ -42,22 +78,50 @@ $$T_{MTE2} = \frac{S_{in}^{GM}}{C\cdot BW_{pc}},\qquad S_{in}^{GM} = r_{in}\cdot
|
|||||||
|
|
||||||
## 5. 实现方案
|
## 5. 实现方案
|
||||||
|
|
||||||
**Step 0: BaseM/BaseN**(L0 级 tile,先把 L0C 用满)
|
**Step 0: BaseM/BaseN**(L0 级 tile, 先按 UnitFlag 单缓冲把 L0C 用满——v1.91 §5.1)
|
||||||
|
|
||||||
$$\text{BaseM}\times\text{BaseN} = \frac{L0C}{2\times 4B} = 32768\ \text{元素}$$
|
Fixpipe 的 UnitFlag 提供 **tile 内部 16×16×16 细粒度流水**, 取代 tile 间粗粒度双缓冲 →
|
||||||
|
L0C 只需一份 buffer:
|
||||||
|
|
||||||
双缓冲两份、FP32 4B/元素。长宽比跟随 SingleCoreM/N,对齐 16。baseK = min(L0A/(2·BaseM·dt), L0B/(2·BaseN·dt)) 向下 16 对齐(L1→L0 无 dValue 要求)。
|
$$\text{BaseM}\times\text{BaseN} = \frac{L0C}{4\text{B}} = 65536\ \text{元素(UnitFlag 单缓冲)}$$
|
||||||
|
|
||||||
**Step 1: SingleCoreM/N**(每核输出 tile,≥ BaseM/N)
|
长宽比**方形优先**(而非跟随 M/N——跟随不会改善 GM→L1 搬移, 只会使 L0A/L0B 容量利用率
|
||||||
|
失衡、baseK 减半): 默认 BaseM = BaseN = 256(L0A/L0B 同时装满); **例外**: M 或 N 小于
|
||||||
|
方形边长时被迫跟随: BaseM = min(256, ⌊M⌋₁₆), 另一维 = min(65536/BaseM, N) 向下 16 对齐
|
||||||
|
(例:M=128, N=4096 → BaseM=128, BaseN=512)。
|
||||||
|
|
||||||
不受 L0 容量直接约束(内部由若干 BaseM×BaseN L0 tile 组成)。核心影响 **GM→L1 搬移效率和 L2 重复读率**:
|
$$\text{baseK} = \min\Big(\frac{L0A}{2 \cdot \text{BaseM} \cdot dtype},\; \frac{L0B}{2 \cdot \text{BaseN} \cdot dtype}\Big) \text{ 向下 16 对齐}$$
|
||||||
|
|
||||||
- 约束 1 并行度:`mCnt×nCnt ≥ ⌈C/B⌉`;
|
(L0A/L0B 仍开双缓冲; 16 对齐是 Cube K 向粒度要求, L1→L0 搬移无 dValue 要求。)
|
||||||
- 约束 2 L1 容量:`2(sM+sN)·k_L1·dt ≤ L1`,`k_L1·dt ≥ 256B`;
|
|
||||||
- 约束 3 搬移效率:`sM·k_L1·dt ≥ 16KB`,`k_L1·sN·dt ≥ 16KB`;
|
|
||||||
- 约束 4:SingleCoreM/N 是 BaseM/N 的整数倍。
|
|
||||||
|
|
||||||
选取策略:满足约束 1 前提下 SingleCoreM/N 尽量大,长宽比跟随 M/N。
|
**Step 1: SingleCoreM/N + k_L1 + mCnt/nCnt**(有界枚举——v1.91 §5.2, 与尾轮 v1.5 §2.1
|
||||||
|
修正口径一致)
|
||||||
|
|
||||||
|
SingleCoreM/N **不受 L0 容量直接约束**(内部由若干 BaseM×BaseN L0 tile 组成), 核心影响
|
||||||
|
GM→L1 搬移效率与 L2 重复读率:
|
||||||
|
|
||||||
|
- **P = ⌈C/B⌉**(最少切分块数);
|
||||||
|
- **P=1 (B ≥ C)**: 先试不切分 (mCnt=nCnt=1, tile 跟随 M/N); 约束不满足则强制切分, 进入枚举;
|
||||||
|
- **枚举空间**(有界, host 端遍历): mCnt ∈ [1, ⌈M/BaseM⌉], nCnt ∈ [1, ⌈N/BaseN⌉],
|
||||||
|
且 B·mCnt·nCnt ≥ C(约束 1 并行度);
|
||||||
|
- 每候选: sM = align_up(⌈M/mCnt⌉, BaseM)(约束 4: BaseM 整数倍), sN 同理;
|
||||||
|
- **约束 2**(L1 双缓冲): k_L1 = min(K, ⌊L1/(2(sM+sN)·dtype)⌋₁₆)(k_L1 与形状耦合);
|
||||||
|
- **约束 3**(搬移效率, 转置感知): dValue ≥ 256B——A 非转置 k_L1·dt / A 转置 sM·dt;
|
||||||
|
B 非转置 sN·dt / B 转置 k_L1·dt; 且单次搬移量 sM·k_L1·dt 与 k_L1·sN·dt ≥ 16KB;
|
||||||
|
- **目标**: 每 batch 总搬入 `K·dt·(nCnt·M + mCnt·N)` 最小(v1.5 §2.1 修正: 稳态流水下
|
||||||
|
k_L1 约掉——单块搬入 = K(sM+sN)·dt 与分次粒度无关, k_L1 只进约束);
|
||||||
|
**并列时取 r = B·mCnt·nCnt mod C 最大**(尾轮块越多, 重切越省)。
|
||||||
|
|
||||||
|
mCnt = ⌈M/singleCoreM⌉, nCnt = ⌈N/singleCoreN⌉(按实际 tile 反推)。
|
||||||
|
|
||||||
|
> **兜底分支恒出方案 (issue#34)**:枚举无可行候选时,不再产出"违规/不可行"方案——
|
||||||
|
> 先放开约束 4(Base 整数倍)按 16 对齐网格 + dValue 128B 硬下限再搜;仍无解
|
||||||
|
> (极端形状,如 N=8 int8 大 K 时 B 侧 dValue = sN·dt 物理上不可能 ≥ 下限)时退回
|
||||||
|
> Base tile 并标注**效率降级**(warning,搬移效率低于模型假设、时延可能低估,
|
||||||
|
> 建议调 dtype/布局),**不是违规/不可行**:dValue/minTile 是效率下限(DMA 仍能工作),
|
||||||
|
> 真正不可行的只有容量/核数硬约束(L0C/L0A/L0B/L1 超容、used_core_num 超核数)。
|
||||||
|
> 对比:IterBatch/MergeBatch/StreamK 等**有替代分支**的分支仍按违规处理
|
||||||
|
> (不满足条件不该进,由路由另择或落 ASW)。
|
||||||
|
|
||||||
**Step 2: mCnt/nCnt 与核间分配**:`mCnt=⌈M/sM⌉`,`nCnt=⌈N/sN⌉`,总块数 B·mCnt·nCnt,B→M→N 线性映射。
|
**Step 2: mCnt/nCnt 与核间分配**:`mCnt=⌈M/sM⌉`,`nCnt=⌈N/sN⌉`,总块数 B·mCnt·nCnt,B→M→N 线性映射。
|
||||||
|
|
||||||
@@ -67,19 +131,21 @@ $$\text{BaseM}\times\text{BaseN} = \frac{L0C}{2\times 4B} = 32768\ \text{元素}
|
|||||||
|
|
||||||
同一波 C 个核所需的 A 行块 + B 列块集合是"活跃工作集",超 L2 就回 GM 读。swizzle 编排输出块执行顺序,把每一波的活跃工作集压到最小。窗口宽度 `W = max{d | d|C, d ≤ ⌊√C⌋}`(C=32 时 W=4);窗口内先扫 M、扫满 W 行进下一列 N,奇数窗口行 N 向反向(蛇形)。
|
同一波 C 个核所需的 A 行块 + B 列块集合是"活跃工作集",超 L2 就回 GM 读。swizzle 编排输出块执行顺序,把每一波的活跃工作集压到最小。窗口宽度 `W = max{d | d|C, d ≤ ⌊√C⌋}`(C=32 时 W=4);窗口内先扫 M、扫满 W 行进下一列 N,奇数窗口行 N 向反向(蛇形)。
|
||||||
|
|
||||||
**Step 5: L2 分组(工作集超 L2 时)**
|
**Step 5: L2 场景判定与分组(make_plan/evaluate 共用 `_l2_scene`,issue#24/#32)**
|
||||||
|
|
||||||
记 S_in = B(MK+KN)·dt,S_out = B·MN·outB。两个不变量:GM 流量下界 = S_in + S_out;L2 读入可用空间 L2_read = L2 − S_out^resident。
|
记单 batch 输入 a_b = MK·dt、b_b = KN·dt,整 case V_in = S_in、V_out = S_out;
|
||||||
|
A 行块 block_a = a_b/m_cnt、B 列块 block_b = b_b/n_cnt。
|
||||||
|
|
||||||
| 场景 | 条件 | 策略 |
|
| 场景 | 条件 | GM / L2 计账(每 batch) |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| A 全驻留 | S_in + S_out ≤ L2 | 输入读一遍(r_in=1),输出驻留 L2 异步回写,无需切分 |
|
| S_A 整case全驻留 | V_in + V_out ≤ L2 | GM = a_b+b_b;重复读 (n_cnt−1)a_b+(m_cnt−1)b_b 走 L2;输出驻留 L2(GM 写=0) |
|
||||||
| B 输入能驻留,加输出超了 | S_in ≤ L2 < S_in+S_out | **输入驻留、输出直写 GM**,保住 r_in=1;校验总线 (S_in+S_out)/T_MMAD ≤ W_GM |
|
| S_B 单batch可驻留 | a_b+b_b ≤ L2,**或**单侧全驻留+对侧滑窗:b_b+2·block_a ≤ L2 或 a_b+2·block_b ≤ L2 | GM = a_b+b_b(r_in=1);重复读 (n_cnt−1)a_b+(m_cnt−1)b_b 全部命中 L2;输出直写 GM |
|
||||||
| C 输入本身超 | S_in > L2 | 分组执行,每组输入工作集 ≤ L2,输出直写 GM |
|
| S_C 双侧超 L2 | 以上均不满足 | 容量 m_grp·block_a+n_grp·block_b ≤ L2 下最小化 GM = ⌈n_cnt/n_grp⌉·a_b + ⌈m_cnt/m_grp⌉·b_b;组间落空计 GM,窗口 (n_cnt−⌈n_cnt/n_grp⌉)a_b+(m_cnt−⌈m_cnt/m_grp⌉)b_b 计 L2 |
|
||||||
|
|
||||||
场景 C 分组:每组覆盖 M 向 m_grp、N 向 n_grp 个基本块,`m_grp = ⌊D/(2·sM)⌋`,`n_grp = ⌊D/(2·sN)⌋`,D = L2/(B·K·dt);重复读倍率 `r_in = (n_grp·M + m_grp·N)/(M+N)`。块内错位分核(对角线分配)避免同地址并发读串行化。
|
> 注:早期口径(预算 D = L2/(B·K·dt) 且对半切、组内窗口流量零计)已废弃
|
||||||
|
> (issue#24 除总 B 的矛盾、issue#32 对半预算与窗口零计失真);分组不再对半、窗口复用如实计 L2。
|
||||||
|
|
||||||
**Step 6: 核内 tiling**:BaseM×BaseN×4B×DB ≤ L0C;BaseM×k_L0×dt×2 ≤ L0A;k_L0×BaseN×dt×2 ≤ L0B;内轴按 dValue 256B/512B 对齐。
|
**Step 6: 核内 tiling**:BaseM×BaseN×4B ≤ L0C(UnitFlag 单缓冲, 见 Step 0);BaseM×k_L0×dt×2 ≤ L0A;k_L0×BaseN×dt×2 ≤ L0B;内轴按 dValue 256B/512B 对齐。
|
||||||
|
|
||||||
**Step 7: 内部特化**:单边无 batch 且该侧矩阵小时小侧整个常驻 L1 只搬一次。
|
**Step 7: 内部特化**:单边无 batch 且该侧矩阵小时小侧整个常驻 L1 只搬一次。
|
||||||
|
|
||||||
@@ -106,7 +172,7 @@ $$\text{BaseM}\times\text{BaseN} = \frac{L0C}{2\times 4B} = 32768\ \text{元素}
|
|||||||
|
|
||||||
- A1b 与方案 B **理论时延严格相等**(总量守恒 `T = c·B·M·N/C`);
|
- A1b 与方案 B **理论时延严格相等**(总量守恒 `T = c·B·M·N/C`);
|
||||||
- **默认选方案 B(工程简洁:一套 tile、无尾轮分支)**;追求搬移下限选 A1b(周长和恒 ≤ 方案 B,均值不等式);
|
- **默认选方案 B(工程简洁:一套 tile、无尾轮分支)**;追求搬移下限选 A1b(周长和恒 ≤ 方案 B,均值不等式);
|
||||||
- r 小(ρ < (187/s)²)时 A1b 尾轮翻出为周长型,**方案 B 微优 ~6%**;
|
- r 小(ρ < (166/s)²,Cube-only 口径下由 187 修正,issue#40)时 A1b 尾轮翻出为周长型,**方案 B 微优 ~6%**;
|
||||||
- A1a 是 A1b 真子集,仅在 r | C 且 s*=⌊C/r⌋ 完美时打平,工程上可不单列。
|
- A1a 是 A1b 真子集,仅在 r | C 且 s*=⌊C/r⌋ 完美时打平,工程上可不单列。
|
||||||
|
|
||||||
**边角场景(周长型主导,小 tile 或工作集超 L2 的 GM 直读)**:
|
**边角场景(周长型主导,小 tile 或工作集超 L2 的 GM 直读)**:
|
||||||
|
|||||||
@@ -17,12 +17,14 @@ $$T_{MMAD} = \frac{2\cdot sM\cdot sN\cdot K}{Q_{16}},\qquad T_{MTE2} = \frac{K(s
|
|||||||
| 对比 | 判据 | 决定因素 |
|
| 对比 | 判据 | 决定因素 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| MMAD vs MTE2 | `sM·sN/(sM+sN)` ⋛ `dt·Q16/(2·BW_eff)` | **tile 尺寸**(K 约掉) |
|
| MMAD vs MTE2 | `sM·sN/(sM+sN)` ⋛ `dt·Q16/(2·BW_eff)` | **tile 尺寸**(K 约掉) |
|
||||||
| MMAD vs FIX | `K` ⋛ `outB·Q16/(2·BW_pc)` ≈ 304 | **K** |
|
| MMAD vs FIX | `K` ⋛ `outB·Q16/(2·BW_pc)` ≈ 270 | **K** |
|
||||||
|
|
||||||
|
(Q16 为 Cube-only 口径 13.5T(issue#40):MMAD/MTE2 分界 93.5→**83.1**,MMAD/FIX 分界 304→**270**。)
|
||||||
|
|
||||||
**缩放类型二分**:MMAD 与 FIX 都 ∝ 面积 sM·sN,MTE2 ∝ 周长 (sM+sN)。
|
**缩放类型二分**:MMAD 与 FIX 都 ∝ 面积 sM·sN,MTE2 ∝ 周长 (sM+sN)。
|
||||||
|
|
||||||
- **面积型主导**(MMAD 或 FIX 最大):tile 大(sM·sN/(sM+sN)≥93.5,L2 命中)且(K≥304 时 MMAD、K<304 时 FIX)——**主流 prefill/decode case 均属此类**;
|
- **面积型主导**(MMAD 或 FIX 最大):tile 大(sM·sN/(sM+sN)≥83.1,L2 命中)且(K≥270 时 MMAD、K<270 时 FIX)——**主流 prefill/decode case 均属此类**;
|
||||||
- **周长型主导**(MTE2 最大):tile 小(<93.5,L2 命中)或工作集超 L2 的 GM 直读(<304)——边角 case。
|
- **周长型主导**(MTE2 最大):tile 小(<83.1,L2 命中)或工作集超 L2 的 GM 直读(<270)——边角 case。
|
||||||
|
|
||||||
## 2. 四种策略定义
|
## 2. 四种策略定义
|
||||||
|
|
||||||
@@ -54,7 +56,7 @@ $$\big(n_{wave}-1+\sqrt{\rho}\big)^2 \le n_{wave}(n_{wave}-1+\rho) \iff (\sqrt{\
|
|||||||
|
|
||||||
**A1b 周长和恒 ≤ 方案 B**(搬入总量少、L2 重复读少、掩盖余量大)。离散 16 对齐后时延互有胜负(<3%,数值依赖、无系统性方向)。
|
**A1b 周长和恒 ≤ 方案 B**(搬入总量少、L2 重复读少、掩盖余量大)。离散 16 对齐后时延互有胜负(<3%,数值依赖、无系统性方向)。
|
||||||
|
|
||||||
**尾轮翻出修正**(§7.2.4):面积型主导但 r 小(ρ < (187/s)²)时 A1b 尾轮 tile 缩得太小,周长/面积比上升使主导项翻转为周长型(搬入翻出),方案 B 的全局 tile 缩得温和(1/√g > √ρ 恒成立)不翻出——**r 小的面积型 case 方案 B 可微优 ~6%**。
|
**尾轮翻出修正**(§7.2.4):面积型主导但 r 小(ρ < (166/s)²,Cube-only 口径下 187→166,issue#40)时 A1b 尾轮 tile 缩得太小,周长/面积比上升使主导项翻转为周长型(搬入翻出),方案 B 的全局 tile 缩得温和(1/√g > √ρ 恒成立)不翻出——**r 小的面积型 case 方案 B 可微优 ~6%**。
|
||||||
|
|
||||||
## 4. 周长型主导(边角场景)
|
## 4. 周长型主导(边角场景)
|
||||||
|
|
||||||
@@ -79,7 +81,7 @@ $$\big(n_{wave}-1+\sqrt{\rho}\big)^2 \le n_{wave}(n_{wave}-1+\rho) \iff (\sqrt{\
|
|||||||
| 1 | r = 0 | A0 | n_wave·T_block |
|
| 1 | r = 0 | A0 | n_wave·T_block |
|
||||||
| 2 | 面积型,0<r≤C/2,无翻出 | A1a,s*=⌊C/r⌋ | T_block(n_wave−1+1/s*) |
|
| 2 | 面积型,0<r≤C/2,无翻出 | A1a,s*=⌊C/r⌋ | T_block(n_wave−1+1/s*) |
|
||||||
| 3 | 面积型,r>C/2,无翻出 | A1b 或方案 B(严格打平) | T_block(n_wave−1+ρ) |
|
| 3 | 面积型,r>C/2,无翻出 | A1b 或方案 B(严格打平) | T_block(n_wave−1+ρ) |
|
||||||
| 4 | 面积型,ρ<(187/s)²(翻出) | 方案 B | T_block·n_wave/g |
|
| 4 | 面积型,ρ<(166/s)²(翻出,issue#40 口径) | 方案 B | T_block·n_wave/g |
|
||||||
| 5 | 周长型,ρ≥ρ_dv,B 可行 | A1b | T_load(n_wave−1+√ρ) |
|
| 5 | 周长型,ρ≥ρ_dv,B 可行 | A1b | T_load(n_wave−1+√ρ) |
|
||||||
| 6 | 周长型,ρ<ρ_dv 且分界成立且 B 可行 | 方案 B | T_load·n_wave/√g |
|
| 6 | 周长型,ρ<ρ_dv 且分界成立且 B 可行 | 方案 B | T_load·n_wave/√g |
|
||||||
| 7 | 周长型,广义 B 损失>0 | A1b(恒不劣) | 广义枚举 min w×T_block |
|
| 7 | 周长型,广义 B 损失>0 | A1b(恒不劣) | 广义枚举 min w×T_block |
|
||||||
|
|||||||
213
BMM/BMM_Theory/docs/03_测评报告/BMM_Theory软件测评报告_v2.0.md
Normal file
213
BMM/BMM_Theory/docs/03_测评报告/BMM_Theory软件测评报告_v2.0.md
Normal file
@@ -0,0 +1,213 @@
|
|||||||
|
# BMM/BMM_Theory 理论最优实现分析软件 测评报告(第二轮 · 整改后复评)
|
||||||
|
|
||||||
|
- 测评对象:`git.magicnetworld.com/admin/matmul-analysis` 仓库 `BMM/BMM_Theory`(软件包 `bmm_theory`)
|
||||||
|
- 本轮基准:HEAD `99a25a6b42efa63689f21ae53a7cc467c2fca79f`(main,2026-09-03),相对上轮基准 `d36e224` 共 **33 个整改提交**
|
||||||
|
- 测评方式:整改 diff 逐文件审查 + issue 逐条复测 + 全量运行/单测 + 同分布压力回归(与上轮同种子同分布对比)+ 口径深查
|
||||||
|
- 说明:上轮报告为 `docs/03_测评报告/BMM_Theory软件测评报告_v1.0.md`(2026-09-03 入库,commit d36e224);本轮为整改后的复评。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 结论摘要(TL;DR)
|
||||||
|
|
||||||
|
**整改诚意与质量都很好:上轮提出的 7 个 issue(#4–#10)中 6 个实质闭环,1 个(#9 StreamK 计账口径)修复方向正确但引入新疑点,需二次复核。** 软件从"可跑但自相矛盾"升级为"可自检、自洽率大幅提升"的可信工具雏形。
|
||||||
|
|
||||||
|
关键数字对比:
|
||||||
|
|
||||||
|
| 指标 | v1(上轮) | v2(本轮) |
|
||||||
|
|---|---|---|
|
||||||
|
| 单元测试 | 17/17 | **23/23**(新增 6 条 issue 回归) |
|
||||||
|
| P0 崩溃(K=1 且 B<128) | 必崩 AttributeError | **不崩溃**,占位标注"[无方案]" |
|
||||||
|
| 典型 LLM 形状自检不可行率(6000 例) | **18.0%**(1079 例) | **0.0%** |
|
||||||
|
| 宽范围随机自检不可行率(8000 例) | **6.0%**(480 例) | **0.2%**(16 例,全部为极端瘦长 MergeBatch 且自检已显式标注) |
|
||||||
|
| 生成结果可复现性(vs 仓库固化示例) | 0 差异 | 0 差异 |
|
||||||
|
| 推荐→评估回灌 feasible | 94% | **99.8%**(16/8000 标注违规但仍推荐) |
|
||||||
|
| NaN/负时延/崩溃 | 无(P0 区除外) | 无 |
|
||||||
|
|
||||||
|
**遗留问题(详见 §5)**:
|
||||||
|
1. 【P2,建议跟进】issue#9 的修复引入新口径问题:StreamK 部分和写出被**双重计账且按单核串行假设高估 32 倍**——demo case 总时延从 9.96us 变为 55.03us、瓶颈变成 FIXPIPE(51.6us),而同一批字节在 `t_reduce` 内仅计 1.61us。该数字已被作者固化进 examples,建议复核。
|
||||||
|
2. 【P3】K=1 且 B<128 区域仍**无任何理论方案**(占 K=1 域的约一半),现以"无方案占位"标注兜底(崩溃已解决、可接受),建议后续补 AIV 单缓冲/Cube 兜底方案。
|
||||||
|
3. 【P3】MergeBatch 极端瘦长 case(M 或 N 很小 × 大 B)仍有 0.2% 生成不可行方案——`b0` 选择未把 L0A/L0B 容量纳入上限;现已被生成自检显式标注(不再静默矛盾)。
|
||||||
|
4. 【P3】负/零矩阵维度无输入校验,静默产出无意义方案;FIXPIPE 建议文案对 StreamK(部分和 4B)给出 fp16/fp8 减半的误导提示。
|
||||||
|
5. 【P3】过程噪音:整改过程中 11 个 commit 误提交临时文件(bug1.csv/t.csv 等)后删除,建议加 .gitignore。
|
||||||
|
|
||||||
|
**总体评价:6.2 → 约 7.6/10。** 架构与理论映射维持高水准;内部一致性与鲁棒性是本轮最大提升(单一约束源 + 生成自检是正确架构决策);剩余扣分集中在 StreamK 时延口径复核、输入校验与若干边界覆盖。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 本轮整改概览(33 commits)
|
||||||
|
|
||||||
|
| 主题 | 主要提交 | 对应 issue |
|
||||||
|
|---|---|---|
|
||||||
|
| 新增 `bmm_theory/constraints.py`(170 行,单一约束源 + 生成侧收敛辅助) | 16a84ac | #5 #6 共性根因 |
|
||||||
|
| `router.py`:生成后自检 `_wrap_checked` + 无方案兜底 `_no_plan` | dad4585 | #4 #5 |
|
||||||
|
| `evaluator.py`:约束校验委托单一约束源 | 0c19d99 | #5 |
|
||||||
|
| `__main__.py`:plan=None / timing=None / 自检违规打印防护 | b57ba57 | #4 |
|
||||||
|
| `io_csv.py`:out_nd 解析 + A/B dtype 不一致告警 + 建模边界说明 | 9ef240f | #7 #8 |
|
||||||
|
| `timing.py`:REDUCE 移出稳态 max()(`reduce_serial` 约定) | 933b428 | #9 |
|
||||||
|
| `branches/stream_k.py`:fixpipe 改按部分和 4B、显式 reduce_serial | f756729 | #9 |
|
||||||
|
| `branches/asw_basic.py`:降核 base_k 按 dtype 容量反推(clamp 同源);k_l1 小 K 不切 | e6aa792 | #5 |
|
||||||
|
| `branches/iter_batch.py`:L0 tile 长宽比跟随 + L0C/L0A/L0B 同源收敛 | c0b358f | #5 |
|
||||||
|
| `branches/merge_batch.py`:新增条件 2b(合并后最小 K 粒度下 L0A/L0B 可驻留) | 1385d01 | #5 |
|
||||||
|
| README / docs/01_软件架构.md 同步 6 分支现状 + 约束模块/归约口径文档 | 7bcacef / 593ffcf | #10 |
|
||||||
|
| examples 两个结果 csv 按新模型重生成 | bbef360 / cff5665 | — |
|
||||||
|
| tests 新增 `TestIssueRegression` 6 条 | 87b7be5 | #4–#9 |
|
||||||
|
| 误提交临时 csv 后清理 | 0e644ac…99a25a6(11 commits) | — |
|
||||||
|
|
||||||
|
净改动:15 个文件,+408/−92 行。无第三方依赖新增。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. issue 逐条闭环验证
|
||||||
|
|
||||||
|
### #4 【P0】K=1 且 batch<128 崩溃 → ✅ 已闭环(标注式兜底)
|
||||||
|
|
||||||
|
复测:`B=64 M=8192 N=32 K=1 int8` 走 `recommend`(-v / 不带 -v / 输出 plans.csv 三条路径)均**不再崩溃**:
|
||||||
|
|
||||||
|
```
|
||||||
|
=== bug_k1_smallB: ... K=1 int8 -> [特殊分支]
|
||||||
|
仲裁: [无方案] K=1逐元素乘...; 但进入条件不满足 (2_K=1的AIV触发: B >= 2*AIV核数), 该区域暂无理论方案...
|
||||||
|
方案: 核数=0 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
实现:`router.py:52-58` capable=False 或 plan=None 时走 `_no_plan`(占位 ImplPlan,used_core_num=0,timing=None);`__main__.py:85-90` 增加空值防护。
|
||||||
|
|
||||||
|
**残留(P3)**:该区域仍未给出任何实现方案。扫描 B=1..255 × K=1 × 若干 M/N:**49.4%(378/765)落"无方案占位"**;K=0 全部有方案。原 issue 允许"标注无方案"作为最低要求(已达成),但建议后续补 AIV 单缓冲或 Cube 兜底方案消除空洞,或至少在文档(00_总纲 / 04_特殊分支)中明确该限制。
|
||||||
|
|
||||||
|
### #5 【P1】推荐模式不做约束自检 → ✅ 基本闭环(6.0%/18.0% → 0.2%/0.0%)
|
||||||
|
|
||||||
|
架构修复正确:新增 `constraints.py` 为**唯一事实来源**,`router._wrap_checked` 在每次生成后调用同一校验函数,违规时在 arbitration/advice 中显式标注"[自检违规: …]";`evaluator._check_constraints` 改为委托同一函数。同时生成侧收敛辅助(`clamp_base_k` / `clamp_base_mn_l0c`)消除降核 base_k 硬编码等缺陷。
|
||||||
|
|
||||||
|
复测(与 v1 同种子同分布):
|
||||||
|
- 上轮全部最小违规样例(IterBatch K=8/K=2、MergeBatch fp8 K=32、ASW_降核 fp16/fp32、fp32 大 MN 等 12 个)→ **全部 violations=OK**;
|
||||||
|
- 宽范围 8000 例:480 → **16**(0.2%);典型 6000 例:1079 → **0**。
|
||||||
|
|
||||||
|
**残留(P3)**:16/8000 全部为极端瘦长 MergeBatch(如 `B=811 M=33 N=1 K=2459 fp32`、`B=1024 M=1 N=1024 K=1024 int8`):`make_plan` 的 b0 上限只取 L0C/算存比/b_core 三者的 min,**未纳入 L0A/L0B 容量**,导致 `b0*M`(或 `b0*N`)过大时 `base_k≥16`(fractal 下限)也放不进 L0A/L0B。新条件 2b(merge_batch.py:55-62)只按 MIN_B0=2 检查,未覆盖实际选中 b0 的情形。因生成自检已显式标注,属于"可感知的残余",建议把 L0A/L0B 容量加入 b0_max 求解(如 b0 ≤ L0A/(2·m·16·dt) 与 L0B 侧取 min 后取不超过 b_core 的因子)。
|
||||||
|
|
||||||
|
### #6 【P1】dValue 口径矛盾 → ✅ 闭环
|
||||||
|
|
||||||
|
理论裁定已落地并文档化(constraints.py:7-15、docs/01_软件架构.md §3.5):**dValue 128B 下限只对"K 被切分、K 段成为搬移连续维"(k_l1 < K)的方案生效;K 整驻留(k_l1 ≥ K,形态 a/b、小 K 整搬)时连续维是 M/N,豁免 K 向检查**。实现于 `_k_segment_is_contiguous`(constraints.py:118-134)。复测:IterBatch K=8/K=2 整驻留样例 → 无违规;形态 c/d 与 MergeBatch/ASW/StreamK 的 K 切分路径仍受约束。新增回归测试 test_issue6。
|
||||||
|
|
||||||
|
### #7 【P2】out_nd 无法经 CLI 传入 → ✅ 闭环
|
||||||
|
|
||||||
|
`io_csv.load_cases` 现解析 `out_nd`(缺省 True,大小写不敏感)。CLI 复测:同一 StreamK case `out_nd=0` → ASW_Basic_降核(StreamK 条件 4 生效);`out_nd=1` → StreamK。csv 头注释补充说明。新增回归测试 test_out_nd_disables_streamk。
|
||||||
|
|
||||||
|
### #8 【P2】A/B dtype 不一致无告警 → ✅ 闭环
|
||||||
|
|
||||||
|
`io_csv.load_cases` 加载后对 `dtype_a ≠ dtype_b` 打印 `[warn]`(stderr,含 case_id 与建模口径说明)。CLI 复测:`fp8 vs bf16` 输入正确输出告警。
|
||||||
|
|
||||||
|
### #9 【P2】StreamK 归约计账口径 → ⚠️ 部分闭环,引入新疑点(重点跟进)
|
||||||
|
|
||||||
|
修复部分正确:REDUCE 不再进稳态 max()(`assemble_timing` 增加 `reduce_serial=True` 约定,timing.py:96-132),归约只作为 drain 串行追加一次;最终写回只留在 `eval_streamk_reduce` 内;回归测试 test_issue9 验证 `t_total = t_steady + t_drain` 且 `drain = t_reduce`(代数恒等)。
|
||||||
|
|
||||||
|
**但 fixpipe 的新口径有实质问题**(复算证据,demo case:B=4 M=N=128 K=10240 bf16,grid_K=32):
|
||||||
|
|
||||||
|
| 账目 | 字节量 | 带宽分母 | 结果 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `t_reduce` 内 t_write_partial(部分和写 L2) | grid_K×tile×4B = 8.39MB | 整片 5.2TB/s(并发写) | **1.61us** |
|
||||||
|
| 新 `t_fixpipe`(stream_k.py:152-153) | 同一批 8.39MB | **单核份额 162.5GB/s** | **51.62us** |
|
||||||
|
|
||||||
|
1. **同一批部分和字节被计两次账**(一次在 reduce 内按整片带宽、一次在 fixpipe 按单核带宽)——issue#9 原指控的"重复计账"并未消除,只是换了个位置与分母;
|
||||||
|
2. **单核串行化假设与软件自身的带宽模型矛盾**:timing.py 文档与 00_总纲均声明"每核独立 DMA 引擎、带宽按核数配平",即 grid_K 个核的部分和是**并发写出**的,墙钟时间应为 字节量/整片带宽 ≈1.61us;51.62us 相当于把整组部分和压到单核写口串行写,**高估 grid_K=32 倍**;
|
||||||
|
3. 后果已固化进 examples:streamk_demo 总时延 9.96us → **55.03us**,瓶颈从 MTE2_GM → FIXPIPE,且 advice 还给出"fp16/fp8 可减半写出量"的提示——该提示对按 4B 防精度丢失的部分和不成立(advice 生成器未感知 StreamK 例外,evaluator.py 的 _advice FIXPIPE 分支);
|
||||||
|
4. 修复后的回归测试只验证汇总代数式,**验证不了字节量与带宽口径的物理正确性**。
|
||||||
|
|
||||||
|
建议:fixpipe 部分和按"每核自己的部分和(tile×4B)÷ 单核份额"计(=1.61us,与 reduce 内并发口径一致),或与 `t_reduce` 的 t_write_partial **二选一**计账并在文档写明;再校验 8.39MB 是否应为每核 256KB 的并发总和语义。此条建议**重新开启 issue#9 或另开 issue**。
|
||||||
|
|
||||||
|
### #10 【P3】文档-代码漂移 → ✅ 闭环
|
||||||
|
|
||||||
|
README 目录树与分支注释更新为 6 分支全实现;docs/01_软件架构.md 分层图、§3.5(单一约束源)、§4 扩展指南(转Matmul 精切为后续方向、新分支须走 constraints.py)均已同步;Python 版本声明改为"实测 3.12/3.14,建议 3.10+"(3.14 本机复验通过)。轻微残留:`router._wrap`(旧包装函数)成为死代码未删除;README 目录树仍缺 `docs/03_测评报告` 条目(小事)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 回归测试与可复现性
|
||||||
|
|
||||||
|
- 单测:`python -m unittest discover -s tests -v` → **Ran 23 tests, OK**(0.028s)。
|
||||||
|
- recommend 输出 vs 仓库 `examples/result_recommend.csv`:10 行全列 **0 差异**;evaluate 输出 vs `examples/result_evaluate.csv`:**0 差异**(含新的 streamk 55.03us 数值——即作者已用新模型重生成示例)。
|
||||||
|
- 压力回归(与 v1 完全同种子同分布)见 §0 表;两组均 0 崩溃、0 NaN、0 负时延。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 代码质量复查(针对改动)
|
||||||
|
|
||||||
|
- `constraints.py` 抽象正确:校验与生成收敛共用;注释完整记录 dValue 裁定缘由。
|
||||||
|
- 三处发现的小问题:
|
||||||
|
1. `constraints._l1_need_bytes` 对 IterBatch 形态 c 的预算推断(resident = min 侧 + 对侧 k_l1 双缓冲)与生成侧 iter_batch.l1_form 逻辑手写两遍,仍属"双源"(本次未合并),好在口径一致、且有校验兜底;建议未来直接让 make_plan 调用同一 helper。
|
||||||
|
2. `merge_batch.py` 条件 2b 用 MIN_B0 预检但实际 b0 更大时仍可能溢出(§2 #5 残留)——校验兜底已标注,但生成侧仍可产生违规方案(0.2%)。
|
||||||
|
3. 输入范围校验仍缺失:`M=-5` / `M=0` 静默产出 IterBatch 伪方案与无意义时延;非法 dtype 抛裸 ValueError。建议 load_cases 后加维度正数校验。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 遗留问题清单(按优先级)
|
||||||
|
|
||||||
|
| # | 级别 | 问题 | 证据 | 建议 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| N1 | P2 | StreamK fixpipe 部分和写双重计账 + 单核串行化高估 ~32 倍;examples 已固化 55.03us 结果 | stream_k.py:152-153 vs timing.py:77-93;demo 复算 51.62us vs 1.61us | reopen #9 或新开 issue;fixpipe 与 reduce 内部分和写二选一,按并发口径计 |
|
||||||
|
| N2 | P3 | K=1 且 B<128 无方案空洞(占 K=1 域 49.4%) | B=1..255 × K=1 扫描 378/765 占位 | 后续补 AIV 单缓冲/Cube 兜底方案或文档明示 |
|
||||||
|
| N3 | P3 | MergeBatch 瘦长 case b0 未含 L0A/L0B 上限 → 0.2% 自检违规(已标注不静默) | 8000 例中 16 例,样例见 §2 #5 | b0_max 纳入 L0A/L0B 容量约束 |
|
||||||
|
| N4 | P3 | FIXPIPE 建议文案对 StreamK 误导(fp16/fp8 减半对 4B 部分和不成立) | evaluator.py _advice | advice 生成感知分支例外 |
|
||||||
|
| N5 | P3 | M≤0 等非法输入无校验,静默输出伪方案 | M=-5/M=0 探针 | load_cases/route 前参数校验 |
|
||||||
|
| N6 | P3 | 无 .gitignore、整改期误提交 11 个临时文件 commit;死代码 router._wrap;README 目录树缺 03_测评报告 | git log 0e644ac..99a25a6 | 补 .gitignore、删死代码 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 总体评分(第二轮)
|
||||||
|
|
||||||
|
| 维度 | v1 | v2 | 说明 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 设计架构 | 9.0 | 9.0 | 单一约束源使架构更完整 |
|
||||||
|
| 理论-代码映射 | 8.0 | 8.5 | dValue 裁定、归约约定均有文档 |
|
||||||
|
| 功能完成度 | 8.0 | 8.5 | 双模式 + 自检 + 告警 |
|
||||||
|
| 内部一致性 | 5.0 | 7.0 | 0.2%/0% 自洽率;StreamK 口径遗留 |
|
||||||
|
| 鲁棒性/错误处理 | 4.0 | 7.5 | 崩溃消除、占位兜底、告警落地;输入校验仍缺 |
|
||||||
|
| 测试 | 5.0 | 7.0 | 23 条含 6 条 issue 回归;仍无 io_csv/CLI 单测 |
|
||||||
|
| 工程化/交付 | 5.0 | 6.5 | 文档同步、模块化;无 CI/打包/.gitignore、误提交噪音 |
|
||||||
|
| **综合** | **6.2** | **7.6** | POC → 可用工具雏形 |
|
||||||
|
|
||||||
|
**定位更新**:已从"理论文档的代码化 POC"升级为"分支归属与方案生成可自检、结果自洽率 99.8% 的辅助分析工具"。仍不建议把绝对时延数值直接当硬件实测使用(T_cmd=50ns 等参数未标定、StreamK 口径待复核),但**方案归属、tile 取值与相对瓶颈分析已经具备可信度**。
|
||||||
|
|
||||||
|
**优先建议**:① 复核 StreamK fixpipe 口径(N1);② 补输入校验(N5);③ 清理工程噪音并考虑 CI(N6);④ 中远期做硬件标定与 ops-nn 源码对照验证。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 附录 A:本轮复测命令
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git fetch origin && git log --oneline d36e224..origin/main # 整改提交
|
||||||
|
python -m unittest discover -s tests -v # 23/23
|
||||||
|
python -m bmm_theory recommend examples/cases_demo.csv -o r.csv --plans p.csv
|
||||||
|
python -m bmm_theory evaluate examples/cases_demo.csv p.csv -o e.csv
|
||||||
|
# 与 examples/result_*.csv 比对: 0 差异
|
||||||
|
```
|
||||||
|
|
||||||
|
## 附录 B:残余 MergeBatch 违规样例(16/8000,0.2%)
|
||||||
|
|
||||||
|
```
|
||||||
|
B=811/811 M=33 N=1 K=2459 fp32 -> L0A tile 超容量 (b0=25: base tile 825×…×16×4B×2)
|
||||||
|
B=1024/1024 M=1 N=1024 K=1024 int8 -> L0B 超容量 + dValue=48B
|
||||||
|
B=2048/2048 M=512 N=1 K=16 bf16 -> L0A 超容量 (b0=8)
|
||||||
|
...
|
||||||
|
```
|
||||||
|
特征:M 或 N 极小 × B 巨大 × b0 因子选择偏大 → 合并后 base tile 一维过长,L0A/L0B 在 base_k≥16 下无法驻留。已在 advice 中标注"[自检违规]",不静默。
|
||||||
|
|
||||||
|
## 附录 C:K=1 无方案空洞扫描
|
||||||
|
|
||||||
|
B=1..255 × K=1 × M/N∈{64×64, 256×256, 4096×64}:765 例中 378 例(49.4%)返回 used_core_num=0 占位"[无方案]";K=0 同域 0 例占位。
|
||||||
|
|
||||||
|
## 附录 D:遗留问题闭环确认(2026-09-06,commit 4bedf0a / HEAD 3a6c952)
|
||||||
|
|
||||||
|
本报告 §5 的遗留问题(N1–N6)已全部转为 issue #11–#16 并修复闭环:
|
||||||
|
|
||||||
|
| Issue | 修复内容 | 验证 |
|
||||||
|
|---|---|---|
|
||||||
|
| #11 N1(P2)StreamK fixpipe 双重计账/32×高估 | 部分和写/读回/求和/最终写回全部单次计入串行 drain(稳态 Fixpipe 账目清零) | streamk_demo 55.03→**9.96us**,瓶颈回到 MTE2_GM |
|
||||||
|
| #12 N2(P3)K=1 且 B<128 无方案空洞 | 新增 **AIV 单缓冲**模式(B≥128 仍乒乓),04_特殊分支.md 同步 | B=1..255×K=1 全扫 **765/765 全有方案** |
|
||||||
|
| #13 N3(P3)MergeBatch 瘦长不可行(0.2%) | b0_max 纳入 L0A/L0B 容量上限;路由对胜出方案自检,违规时回退(另一切B候选→StreamK→ASW) | 8000 例宽范围违规 **16→0** |
|
||||||
|
| #14 N4(P3)FIXPIPE advice 对 StreamK 误导 | StreamK 跳过 dtype 减半提示,新增 REDUCE 瓶颈建议 | 文案验证 |
|
||||||
|
| #15 N5(P3)输入校验缺失 | BmmCase.__post_init__ 维度/dtype 校验,非法输入明确抛错 | M≤0/K<1/dtype 非法均正确报错 |
|
||||||
|
| #16 N6(P3)工程噪音 | 根目录 .gitignore、删除 router._wrap 死代码、README 目录树补 03_测评报告 | — |
|
||||||
|
|
||||||
|
**修复后全量验证**:单测 **28/28**(新增 8 条回归);宽范围 8000 例 + 典型形状 6000 例压力测试 **违规 0 / 异常 0 / 无方案 0 / NaN 0**;examples 已按新模型重新生成并与代码 0 差异提交。Gitea issue #4–#16 已全部关闭(附修复注释)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*第二轮复评完成:本地动态测试 + 与仓库 HEAD 99a25a6 比对;全部问题均有可复现证据。修复确认见附录 D(HEAD 3a6c952)。*
|
||||||
176
BMM/BMM_Theory/docs/04_差异对照_bmmv3_vs_BMM_Theory.md
Normal file
176
BMM/BMM_Theory/docs/04_差异对照_bmmv3_vs_BMM_Theory.md
Normal file
@@ -0,0 +1,176 @@
|
|||||||
|
# bmmv3 vs BMM_Theory 差异对照
|
||||||
|
|
||||||
|
> 对照对象:`bmmv3/bmmv3/bmmv3_arch35_branch_checker.py`(下称 **bmmv3**)与本仓库 `bmm_theory/branches/merge_batch.py` + `iter_batch.py`(下称 **BMM_Theory**)。
|
||||||
|
> 写作目的:说清两版"是什么关系、差在哪、各自什么时候用、该互相吸收什么"。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 一句话结论
|
||||||
|
|
||||||
|
**两版不是竞争关系,是互补关系。**
|
||||||
|
|
||||||
|
- **bmmv3** = 现有算子源码行为的**预测器/对拍器**(source-of-truth 是 `batch_matmul_v3_mergebatch_basicapi_tiling.cpp` / `iterbatch_tiling.cpp`);
|
||||||
|
- **BMM_Theory** = 理论最优实现的**推导器**(source-of-truth 是《BMM算子优化分析 v0.98》《MergeBatch_vs_IterBatch分析 v1.1》)。
|
||||||
|
|
||||||
|
要"知道现有 kernel 实际会怎么跑"→ 用 bmmv3;要"知道理论上应该怎么跑最优"→ 用 BMM_Theory。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 根本定位差异
|
||||||
|
|
||||||
|
| 维度 | bmmv3 | BMM_Theory |
|
||||||
|
|---|---|---|
|
||||||
|
| 目的 | 判断 case 能否进**现有源码**的 `MERGE_BATCH_BASICAPI` / `ITER_BATCH` | 推导 case 的**理论最优**实现方案 |
|
||||||
|
| 依据 | ops-nn 源码逐条件还原 | 理论文档推导 + 硬件容量/带宽约束 |
|
||||||
|
| tile 语义 | 按源码硬编码(ping_pong 开关、step 幂次搜索) | 按理论容量极限 + 生成后自检 |
|
||||||
|
| 时延模型 | 三行带宽公式(CUBE/FIXP/MTE2 各一行) | 分阶段 max(MMAD,MTE2,FIXPIPE)+drain,对齐 v1.1 |
|
||||||
|
| 分支覆盖 | 仅 MergeBatch + IterBatch | 六分支全(转Matmul/特殊/MergeBatch/IterBatch/StreamK/ASW_Basic) |
|
||||||
|
| 输入校验 | 无(静默接受非法输入) | `__post_init__` 明确报错(issue#15) |
|
||||||
|
| 生成后自检 | 无 | constraints.py 同源校验(issue#5) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. MergeBatch 分支逐项对照
|
||||||
|
|
||||||
|
### 2.1 准入条件
|
||||||
|
|
||||||
|
| 条件 | bmmv3 | BMM_Theory | 差异说明 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| batch 关系 | A/B 4 个 batch 轴完全相等 | BatchA==BatchB | bmmv3 更严(源码实现细节) |
|
||||||
|
| 每核 batch 下限 | `min_batch_l0=4`(每核≥4 batch) | `b_core ≥ 2*b0`(b0≥2 即每核≥4) | 等价,bmmv3 硬编码 4,我们参数化 |
|
||||||
|
| L0C 容量 | `l0cSize = tempAlignM×tempAlignN×4×l0c_factor ≤ L0C` | `2×(b0·M)×(b0·N)×4 ≤ L0C` | 一致(l0c_factor=2 即双缓冲) |
|
||||||
|
| L0A/L0B 容量 | 准入条件 21/22 查(`al0Size/bl0Size`) | **b0 计算时显式收敛(issue#13)** | **我们更完整**——bmmv3 只在准入查,没在 b0 计算里收敛 |
|
||||||
|
| K 对齐下限 | `alignK ≥ merge_min_align_k=64` | 无 | bmmv3 源码特有 |
|
||||||
|
| M≤N 约束 | 有(条件 13) | 无 | bmmv3 源码特有(实现限制,非理论必要) |
|
||||||
|
| 转置 view | 非连续转置 view 排除 | 不建模 | bmmv3 源码特有 |
|
||||||
|
| bias | `has_bias=false` | 不建模 | bmmv3 源码特有 |
|
||||||
|
| 访存 Bound | `2MN/(M+N) < 607/2`(经验常数) | `2MN/(M+N) < R16/b0`(R16=算存比) | **我们更通用**——bmmv3 的 607 对应旧总算力口径 R16=607.5;现行 Cube-only 口径 R16=540(issue#40),fp32 时 R16 再减半但 bmmv3 不会 |
|
||||||
|
|
||||||
|
### 2.2 b0 计算
|
||||||
|
|
||||||
|
```
|
||||||
|
bmmv3: b0 = min(L0C/(2·M·N·4), ceil(B/C), 607.5·(M+N)/(2·M·N))
|
||||||
|
我们: b0 = min(√(L0C/(8MN)), R16·(M+N)/(2MN), √(L0A/(2·M·16·dt)), √(L0B/(2·N·16·dt)), b_core) 取因子
|
||||||
|
```
|
||||||
|
|
||||||
|
**关键差异**:我们多了 L0A/L0B 上限(issue#13),bmmv3 靠 `min_batch_l0=4` 间接保证 L0A/L0B 不溢出,但不如显式约束通用(b0=2 时 bmmv3 的间接保证失效)。
|
||||||
|
|
||||||
|
### 2.3 KL0/KL1 计算
|
||||||
|
|
||||||
|
| | bmmv3 | BMM_Theory |
|
||||||
|
|---|---|---|
|
||||||
|
| KL0 | `floor_align16(min(L0A/(2·b0·M·dt), L0B/(2·b0·N·dt)))` | 同(`align_down(min(...), fractal)`) |
|
||||||
|
| KL1 | `ceil_align16(min(K, 512/dt, L1/(2·b0·(M+N)·dt)))`,不满足 L1 则减 16 循环 | `align_down(min(k_l1_star, K, dvalue_cap), fractal)` |
|
||||||
|
| 实现风格 | 源码试探式(ceil 后逐步减) | 理论反推式(直接 align_down) |
|
||||||
|
|
||||||
|
**差异说明**:bmmv3 的"ceil 后减 16 循环"是源码实现细节(为了向上对齐后再试探),理论反推直接用 `align_down` 即可,结果等价但我们更简洁。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. IterBatch 分支逐项对照
|
||||||
|
|
||||||
|
### 3.1 准入条件
|
||||||
|
|
||||||
|
| 条件 | bmmv3 | BMM_Theory |
|
||||||
|
|---|---|---|
|
||||||
|
| 每核 batch | `floor(B/C) ≥ 1` | `b_core ≥ 1`(ceil) | 一致 |
|
||||||
|
| 负载均衡 | `B mod C == 0` 或 `≥ 26`(硬编码) | `B mod C == 0` 或 `≥ min_core_num`(参数化) | **我们更通用** |
|
||||||
|
| L1 形态 | a/b/c1/c2/d 五种 | a/b/c/d 四种 | **bmmv3 更细**(c 拆 c1/c2) |
|
||||||
|
|
||||||
|
### 3.2 c 形态处理
|
||||||
|
|
||||||
|
**bmmv3**:
|
||||||
|
- c1:MK 整体驻留 + KN/step 分块
|
||||||
|
- c2:KN 整体驻留 + MK/step 分块
|
||||||
|
- step 按 2/4/8/16… 幂次递增遍历取最大
|
||||||
|
|
||||||
|
**BMM_Theory**:
|
||||||
|
- c 形态统一:"驻留较小侧 + 对侧切 K"
|
||||||
|
- k_l1 连续反推:`k_l1 = (budget - resident) / (other·dt) / 2`
|
||||||
|
|
||||||
|
**差异说明**:bmmv3 的 step 幂次搜索是源码为了 tiling 生成方便做的离散化;理论极限下 k_l1 连续取最优即可。**做理论推导用我们的,做源码对拍用 bmmv3 的。**
|
||||||
|
|
||||||
|
### 3.3 d 形态处理
|
||||||
|
|
||||||
|
**bmmv3**:`step_d` 由 16KB 和 128B 两条规则推导,`KL1 = ceil_align16(K/step_d)`,不满足 L1 则减 16 循环。
|
||||||
|
|
||||||
|
**BMM_Theory**:`k_l1 = align_down(L1/(2·(M+N)·dt), fractal)`,直接反推。
|
||||||
|
|
||||||
|
**差异说明**:同上,bmmv3 是源码试探式实现,我们是理论反推。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 时延模型对照
|
||||||
|
|
||||||
|
| | bmmv3 | BMM_Theory |
|
||||||
|
|---|---|---|
|
||||||
|
| CUBE | `2·B·M·K·N / (MAC_TFLOPS·1e6)` | `2·b_core·M·N·K / q16`(单核) |
|
||||||
|
| MTE2 | `(2·M·K + 2·K·N)·B / (MTE2_BW·1e6)` | 分 GM/L2 两段 + T_cmd + drain |
|
||||||
|
| FIXP | `2·M·N·B / (FIXP_BW·1e6)` | `b_core·M·N·out_dtype / bw_pc` + unitflag |
|
||||||
|
| 流水掩盖 | 无 | max(MMAD,MTE2,FIXPIPE) + drain |
|
||||||
|
| 分支仲裁 | 无 | MergeBatch vs IterBatch 谁优 |
|
||||||
|
|
||||||
|
**差异说明**:bmmv3 的时延是纯带宽粗估(三行公式),我们做分阶段 + drain + 仲裁,能定位瓶颈。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 各自适用场景
|
||||||
|
|
||||||
|
| 场景 | 推荐 |
|
||||||
|
|---|---|
|
||||||
|
| 预测现有 kernel 实际行为(对拍) | **bmmv3** |
|
||||||
|
| 推导理论最优方案(设计) | **BMM_Theory** |
|
||||||
|
| 分析瓶颈(MTE2/Cube/Fixpipe 谁卡) | **BMM_Theory**(bmmv3 无瓶颈分析) |
|
||||||
|
| 验证源码准入条件是否正确 | **bmmv3**(BMM_Theory 不管源码现状) |
|
||||||
|
| 大批量 case 筛选(XLSX 原表追加) | **bmmv3**(我们暂无 XLSX) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 可吸收点清单
|
||||||
|
|
||||||
|
### 6.1 bmmv3 → BMM_Theory(我们该补的)
|
||||||
|
|
||||||
|
| 项 | 优先级 | 状态 |
|
||||||
|
|---|---|---|
|
||||||
|
| fp4(0.5B)dtype 支持 | 高 | ✅ **已补(本期)**——`DTYPE_BYTES` 加 `"fp4": 0.5` / `"fp4_e2m1": 0.5`,`dtype_bytes()` 返回 float |
|
||||||
|
| 转置对 dValue/base_k 的影响建模 | 高 | ✅ **已补(本期)**——MergeBatch 加条件 6(A 转置对齐),IterBatch dValue 判定按转置调整连续维 |
|
||||||
|
| 转置判据三处同源 | 中 | ✅ **已补(issue#19)**——`models.dvalue_contig_dims` 统一连续维 dValue,`l1_form` c/d 生成守卫 / IterBatch 条件 4 / `constraints` 校验共用同一判据,非转置行为不变 |
|
||||||
|
| XLSX 原表追加输出 | 中 | 待做 |
|
||||||
|
| c1/c2 的 step 幂次搜索 | 低 | 不做(理论极限不需要离散化) |
|
||||||
|
|
||||||
|
**fp4 已补细节**:
|
||||||
|
- `models.py`: `DTYPE_BYTES` 加 `"fp4": 0.5` / `"fp4_e2m1": 0.5`
|
||||||
|
- `dtype_bytes()` 返回类型改为 `float`(兼容 0.5)
|
||||||
|
- `dtype_in_bytes`/`dtype_out_bytes` 属性返回类型改为 `float`
|
||||||
|
- 测试:`TestFp4Support` 3 个用例
|
||||||
|
|
||||||
|
**转置建模已补细节**:
|
||||||
|
- **MergeBatch**(`merge_batch.py`):新增条件 6——A 转置且 M>1 时 `tempAlignM = b0 * alignM`,校验 L0A 容量
|
||||||
|
- **IterBatch**(`iter_batch.py`):dValue 判定按转置调整连续维——A 不转置判 `K*dt`,A 转置判 `M*dt`;B 不转置判 `N*dt`,B 转置判 `K*dt`
|
||||||
|
- 测试:`TestTransposeModeling` 3 个用例(含 A 转置大 M 小 K 通过、A 不转置小 K 失败的边界 case)
|
||||||
|
|
||||||
|
### 6.2 BMM_Theory → bmmv3(他们可参考的)
|
||||||
|
|
||||||
|
| 项 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| 分支仲裁 | MergeBatch vs IterBatch 谁优的判断 |
|
||||||
|
| 时延 drain/流水掩盖 | 三行公式 → 分阶段 max + drain |
|
||||||
|
| 生成后自检 | 生成即输出 → 生成后跑约束校验 |
|
||||||
|
| L0A/L0B 的 b0 显式收敛 | issue#13:b0 计算时直接纳入 L0A/L0B 上限 |
|
||||||
|
| 输入校验 | issue#15:非法维度/dtype 明确报错 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 结论
|
||||||
|
|
||||||
|
两版**互补不替代**。bmmv3 是"源码行为预测器"(对拍用),BMM_Theory 是"理论最优推导器"(设计用)。
|
||||||
|
|
||||||
|
**已吸收**:fp4 支持 + 转置对 dValue 的建模(本期补入)。
|
||||||
|
|
||||||
|
**不建议吸收**:c1/c2 的 step 幂次搜索(源码离散化细节,理论推导不需要)。
|
||||||
|
|
||||||
|
**待考虑**:XLSX 输出(如用户需要)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*版本:v1.1 | 2026-09-06 | 基于 BMM_Theory main(含 issue#11-#22 修复:含 #17 恢复 StreamK 单次计账/K=1 单缓冲/#14 advice,#19 转置判据三处同源)与 bmmv3 最新版对照*
|
||||||
220
BMM/BMM_Theory/docs/05_L2驻留GM读写与dtype算力口径_设计分析.md
Normal file
220
BMM/BMM_Theory/docs/05_L2驻留GM读写与dtype算力口径_设计分析.md
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
# 05 L2 驻留、GM 读写与 dtype 算力口径 — 设计分析
|
||||||
|
|
||||||
|
> 关联 issue: #27 (MergeBatch Cube 复核) / #28 (dtype 感知算力) /
|
||||||
|
> #29 (GM 读取量下限 + L2 驻留工作集) / #30 (Fixpipe 输出落点)
|
||||||
|
>
|
||||||
|
> 状态: 设计文档先行, 代码按本文档落地 (2026-06 评审轮)。
|
||||||
|
> 本文档是 GM/L2/输出计账与算力口径的**单一语义来源**; 各分支 evaluate/生成注释以本文档为准。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 本轮的四个问题与结论摘要
|
||||||
|
|
||||||
|
| # | 问题 | 结论 | 落地 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| #27 | MergeBatch Cube 时延 "每次 b0 个 batch, flops=2·b0M·b0N·K, 共 b_core/b0 次" | **公式与代码一致** (总 flops = b_core·b0·2MNK, 证明见 §6); 无计算改动 | 注释显式化; 暴露的"字节列每核/芯片口径混用"随 #29 统一 |
|
||||||
|
| #28 | 估算时延恒用 BF16 Cube / fp32 AIV 算力 | **确认错误** | §2 dtype 感知速率表 + 全链路替换 |
|
||||||
|
| #29 | GM 读取量"小于输入数据量" | 计算层面各分支 GM ≥ 输入一次 (逐分支表见 §5); **真问题**: ① 字节列每核/芯片口径混用导致比对失真; ② 缺整 case 驻留边界与统一驻留判定 | §3 公理化 + §4 场景模型 + 芯片口径列统一 + GM≥输入不变量测试 |
|
||||||
|
| #30 | Fixpipe 输出落点 (默认写 L2, 容量不足才写 GM) | **确认错误** (Iter/Merge/特殊恒直写 GM; ASW 场景 A 按单 batch 判定, 整 case 输出累积必逐出) | §4.2 输出落点规则 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 硬件事实 (Ascend950PR, 与 hardware/ascend950pr.py 对应)
|
||||||
|
|
||||||
|
- GM (HBM): **1.6TB/s, 读写共享总线** —— 同一时刻读与写累加计时 (issue#23 已落地)。
|
||||||
|
- L2: 128MB, **5.2TB/s, 读口/写口各自独享** —— L2 重复读(读口)与 Fixpipe→L2 写(写口)互不竞争 (issue#23 已落地)。
|
||||||
|
- 输入首访一律走 GM 带宽计一次, 不叠加 L2 (issue#24 已落地); L2 只吸收"驻留后的再次读取"。
|
||||||
|
- Cube 算力分精度 (白皮书: FP8/MXFP8/HiF8 = 2×FP16, MXFP4 = 4×FP16; 16bit 档 = 基准):
|
||||||
|
BF16 **432 TFLOPS/芯片 (=13.5 TFLOPS/核, Cube-only 口径 = 白皮书表3-1 "Cube算力" 单行,
|
||||||
|
issue#40 用户裁决; 486 为 Cube+Vector 总算力, 不作为 Cube 时延基准)**。
|
||||||
|
FP32/TF32 同代比值白皮书未给 → **假设 ½, 待实测标定** (见 §2 假设表)。
|
||||||
|
- AIV (Vector): 64 核 × 128 fp32 lane/拍 × 1.65GHz = 13.5T 元素/s (fp32 档)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. dtype 感知算力 (issue #28)
|
||||||
|
|
||||||
|
### 2.1 速率表 (因子, 相对基准档)
|
||||||
|
|
||||||
|
Cube 因子 (相对 BF16, 同倍率作用于整芯片与单核):
|
||||||
|
|
||||||
|
| dtype | 因子 | 依据 |
|
||||||
|
|---|---|---|
|
||||||
|
| bf16 / fp16 | 1.0 | 基准档 (950 同速) |
|
||||||
|
| fp32 / tf32 | 0.5 | **假设**: 白皮书仅述 FP16/FP32 单核较上代均 +100%, 未给同代比值; 按 DaVinci 惯例 FP32 累加通量减半。待 msProf 实测标定 |
|
||||||
|
| fp8 / fp8_e4m3 / fp8_e5m2 | 2.0 | 白皮书: FP8 = 2×FP16 |
|
||||||
|
| int8 | 2.0 | **假设** 同 FP8 (8bit 整数张量档), 待实测 |
|
||||||
|
| fp4 / fp4_e2m1 | 4.0 | 白皮书: MXFP4 = 4×FP16; 普通 fp4 按 MXFP4 假设, 待实测 |
|
||||||
|
|
||||||
|
AIV (Vector) 逐元素通量因子 (相对 fp32 lane 基准):
|
||||||
|
|
||||||
|
| dtype | 因子 | 依据 |
|
||||||
|
|---|---|---|
|
||||||
|
| fp32 / tf32 | 1.0 | 基准 (128 lane/拍/核) |
|
||||||
|
| fp16 / bf16 | 2.0 | **假设** 16bit 双元素/lane, 待实测 |
|
||||||
|
| fp8 / int8 | 4.0 | **假设** 8bit 四元素/lane, 待实测 |
|
||||||
|
| fp4 | 8.0 | **假设** 待实测 |
|
||||||
|
|
||||||
|
A/B dtype 不一致 (混精度): Cube 取**较慢一侧**的因子 (max(字节数) 侧, 即 min(因子))。
|
||||||
|
StreamK 归约是对 **fp32 部分和**求和 → AIV 归约恒按 fp32 档 (因子 1.0), 不随输入 dtype 变。
|
||||||
|
|
||||||
|
### 2.2 替换点 (全部时延估算)
|
||||||
|
|
||||||
|
| 位置 | 现状 | 改为 |
|
||||||
|
|---|---|---|
|
||||||
|
| 各分支 t_mmad / t_comp_chunk / drain / 尾轮决策主导项 | flops / spec.q16 | flops / spec.q_cube(dtype_a, dtype_b) |
|
||||||
|
| MergeBatch beats_iterbatch 阈值中的 T_comp | q16 | 同上 (该分支 A/B 同 dtype, 简化为 dtype_a) |
|
||||||
|
| special K=1 t_compute (逐元素乘) | b·m·n / spec.q_aiv (fp32) | b·m·n / spec.aiv_elem_rate(dtype_in) |
|
||||||
|
| StreamK θ_c (归约代价系数) | q16(bf16) | q_cube(输入 dtype) (AIV 侧仍 fp32) |
|
||||||
|
| R16 平衡点 (MergeBatch 进入条件 5 / b0 算存比上限) | r16(bf16 基准) | r16_for(dtype) |
|
||||||
|
| 入口条件/生成侧 | — | 同步按 §2.1 因子 (dtype 只影响时延, 不改变分支结构) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 存储口径公理 (issue #29)
|
||||||
|
|
||||||
|
记号:
|
||||||
|
|
||||||
|
- V_in = 输入存储总量 = batch_a·M·K·dt + batch_b·K·N·dt (广播前实际存储), V_out = batch_c·M·N·out_dt;
|
||||||
|
- a_b = M·K·dt (单 batch A 字节), b_b = K·N·dt (单 batch B 字节)。
|
||||||
|
|
||||||
|
- **R1 (GM 首读下限)**: 每个输入字节最初在 GM, 至少从 GM 读一次 → **GM_read ≥ V_in**。
|
||||||
|
违反即模型 bug (统一用测试锁死)。
|
||||||
|
- **R2 (L2 只吸收驻留后的重复读)**: 同一字节的第 2..n 次访问, 仅当两次访问之间该字节仍驻留 L2
|
||||||
|
(所在"复用窗口"的工作集 ≤ L2 减去其它占用) 才按 L2 读口计时; 否则按 GM 重读计时。
|
||||||
|
- **R3 (整 case 全驻留边界)**: V_in + V_out (StreamK 另加部分和 workspace) ≤ L2 时,
|
||||||
|
全程零逐出: GM_read = V_in (每字节一次), 全部重复读走 L2, 输出驻留 L2 (见 §4.2)。
|
||||||
|
- **R4 (输出落点)**: 见 §4.2。
|
||||||
|
- **R5 (输入工作集场景)**: batch 内 tile 共享块 (A 行块被 n_cnt 个列 tile 复用 →
|
||||||
|
(n_cnt−1) 次重复; B 列块被 m_cnt 个行 tile 复用 → (m_cnt−1) 次) 的流量归属由 §4.1
|
||||||
|
场景判定: 可驻留时按 L2 读口计 (含**单侧全驻留+对侧滑窗**调度, issue#32);
|
||||||
|
双侧都放不下时按最小替换分组 (§4.3)。
|
||||||
|
- **R6 (已知简化, 文档标注)**: ① BatchA≠BatchB (均 ≥2, 如 2 vs 64) 的广播/混合 batch 结构
|
||||||
|
按"每 batch 独立矩阵"计 GM = b·(a_b+b_b), 是 V_in 的**保守放大**
|
||||||
|
(真实为 V_in + 共享矩阵跨 batch 重复读 ≤ 该值), 不做特殊建模;
|
||||||
|
② K 切分 (k_L1<K) 各段数据互不重叠: GM 按实际字节精确计 (每字节恰从 GM 读一次,
|
||||||
|
无 padding 上取; 切段数只影响 DMA 命令数/T_cmd 与双缓冲调度, 见 §5 与 issue#31);
|
||||||
|
③ 场景按单 batch 工作集判定, 多核波次并发窗口按 batch 粒度近似。
|
||||||
|
|
||||||
|
### 3.1 逐分支 GM/L2 流量归属表 (现行模型正确性核查结果)
|
||||||
|
|
||||||
|
| 分支 | 结构 | GM 直读 | L2 重复读 | 核查 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| IterBatch (切 B) | 逐 batch 整驻留/切 K; 每 (batch, K段) 数据互不重叠 | **V_in** (每字节恰一次, 无 padding 上取, issue#31) | 0 (L1 形态吸收核内复用) | ✓ GM=V_in |
|
||||||
|
| MergeBatch | 合并组 (b0 batch) 一次搬入; K 段/整 K 均不跨段重读同一字节 | **V_in** (issue#31) | 0 | ✓ GM=V_in |
|
||||||
|
| ASW (切 M/N) | batch 内 tile 共享行/列块 | S_A/S_B: **V_in** (含单侧全驻留调度, issue#32); S_C: 最小替换分组 GM (≥V_in, 仅双侧超L2 时 >1) | S_A/S_B: b·[(n_cnt−1)·a_b + (m_cnt−1)·b_b]; S_C: 组内窗口计 L2 (issue#32) | ✓ GM≥V_in |
|
||||||
|
| StreamK | 组内 K 段零重复读; 组间共享块同 ASW | V_in (命中时) | 组间共享块 (命中时) | ✓ |
|
||||||
|
| 特殊分支 K=1 | AIV 通路, 每元素一次 | V_in | 0 | ✓ |
|
||||||
|
| 转Matmul | 折叠单次 GEMM | V_in | 0 (Matmul 精切未展开) | ✓ |
|
||||||
|
|
||||||
|
→ **整芯片口径统一后 (issue#29): GM 均 = V_in (无重复读结构) 或 ≥ V_in (S_C 双侧超 L2 的最小替换分组)**, 不变量 GM ≥ V_in 由测试锁死 (违反即 bug)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 场景模型 (输入侧) 与输出落点 (issue #30 / #32)
|
||||||
|
|
||||||
|
### 4.1 场景判定 (对切 M/N 及切 B 分支共用; L2 为整芯片 128MB)
|
||||||
|
|
||||||
|
- **S_A 整 case 全驻留**: V_in + V_out [+ workspace] ≤ L2 → 输出驻留 L2 (R4);
|
||||||
|
- **S_B 单 batch 可驻留**: 不满足 S_A, 但单 batch 输入可全驻留 (a_b + b_b ≤ L2)
|
||||||
|
**或单侧全驻留 + 对侧滑窗**: b_b + 2·block_a ≤ L2 或 a_b + 2·block_b ≤ L2
|
||||||
|
(block_a = a_b/m_cnt = A 行块字节, block_b = b_b/n_cnt = B 列块字节)
|
||||||
|
→ GM = V_in 一次, 共享块重复读全部命中 L2 读口; 输出直写 GM;
|
||||||
|
- **S_C 双侧均超 L2**: a_b、b_b 及其滑窗组合都放不下 → 最小替换 2D 分组 (§4.3)。
|
||||||
|
|
||||||
|
判定顺序 S_A → S_B → S_C。S_B 从原"双侧全驻留"扩展出"单侧全驻留+对侧滑窗"调度
|
||||||
|
(issue#32): 大矩阵行块流式、小矩阵全驻留时, GM 仍 = V_in, 不再被分组模型成倍放大。
|
||||||
|
(原 #24 场景 A 的"单 batch 输入+输出可驻留"判定被 S_A 取代: 整 case 输出跨 batch
|
||||||
|
累积时单 batch 判定不可靠, 见 #30。)
|
||||||
|
|
||||||
|
### 4.2 R4 输出落点规则 (取代一切"恒直写 GM / 单 batch 判定驻留")
|
||||||
|
|
||||||
|
> to_l2(输出写 L2 写口 5.2TB/s, GM 写流量=0, 异步回写不占算子时延 — #23 口径延续)
|
||||||
|
> ⟺ **V_in + V_out [+ StreamK workspace] ≤ L2** (整 case 判定)
|
||||||
|
> 否则输出**直写 GM**: 计入 GM 读写共享总线, 与读累加进 MTE2 搬移链 (#23 口径)。
|
||||||
|
|
||||||
|
理由: 输出字节只写一次无复用价值, L2 应优先保输入工作集 (输入存在重复读);
|
||||||
|
只有当"整 case 输入+输出"可同时全驻留时才值得让输出占 L2 (省掉整条 GM 写时延)。
|
||||||
|
各分支应用:
|
||||||
|
|
||||||
|
| 分支 | 原行为 | 新行为 |
|
||||||
|
|---|---|---|
|
||||||
|
| IterBatch / MergeBatch / 特殊分支 / 转Matmul | 恒 direct_gm | S_A 时 to_l2; 否则直写 GM |
|
||||||
|
| ASW | 场景 A (单 batch in+out 驻留) 时 to_l2 | S_A (整 case) 时 to_l2; S_B/S_C 直写 GM |
|
||||||
|
| StreamK | 归约内最终写回恒走 L2 | 最终写回按 S_A(+workspace); 不满足时写 GM (drain 内按 GM 带宽) |
|
||||||
|
|
||||||
|
### 4.3 场景 C: 最小替换 2D 分组 (issue#32)
|
||||||
|
|
||||||
|
S_C (双侧均不可全驻留) 时, 在**整 L2 容量约束**下搜索最小 GM 的分组
|
||||||
|
(取代旧版 "L2/2 对半预算 + 组内窗口流量零计" 口径; 每 batch 计算, 全 case ×b):
|
||||||
|
|
||||||
|
- 容量约束: m_grp·block_a + n_grp·block_b ≤ L2 (组工作集可占满整 L2);
|
||||||
|
- 目标: GM = ⌈n_cnt/n_grp⌉·a_b + ⌈m_cnt/m_grp⌉·b_b **最小**
|
||||||
|
(A 行块在其列组内一次 GM 首读、B 列块在其行组内一次 GM 首读; GM ≥ V_in);
|
||||||
|
- 窗口 L2: (n_cnt − ⌈n_cnt/n_grp⌉)·a_b + (m_cnt − ⌈m_cnt/m_grp⌉)·b_b
|
||||||
|
(组内共享块其余次复用走 L2 读口, 与 S_B 口径一致);
|
||||||
|
- 无可行分组 (单块对都超 L2) 时保守回落: 每共享块独立落 GM (⌈·⌉ = n_cnt/m_cnt), 窗口不计。
|
||||||
|
|
||||||
|
分组追求"最小 L2 替换": 一次只让一组的工作集占 L2, 组间共享块落空回 GM。
|
||||||
|
|
||||||
|
### 4.4 字节列与计数列语义 (修正 ①, 整芯片口径)
|
||||||
|
|
||||||
|
输出 CSV 中数据量列一律为**整芯片**口径: gm_read_bytes / l2_read_bytes /
|
||||||
|
fixpipe_bytes / cube_flops = 整芯片量; 时延列 t_* 为墙钟时延 (已含核数折算);
|
||||||
|
dma_cmd_count 为**单核**命令数 (各核 DMA 引擎并行执行, 墙钟 T_cmd = 单核命令数 × t_cmd),
|
||||||
|
与字节列口径不同属, 单独标注。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 各分支 GM/输出计账公式落地 (与代码对应)
|
||||||
|
|
||||||
|
统一由 case 属性提供 V_in / V_out; 输出落点判断函数 `output_to_l2(case, spec, workspace)` (S_A)。
|
||||||
|
|
||||||
|
| 分支 | GM 读 (芯片) | L2 读 (芯片) | 输出 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| MergeBatch | V_in (每字节恰一次; K 截断/L1 绑定均无重复读, issue#31) | 0 | S_A→L2 写口; else GM |
|
||||||
|
| IterBatch | V_in (a/b/c/d 各形态均每字节恰一次, issue#31) | 0 | 同 MergeBatch |
|
||||||
|
| ASW 切M/N | S_A/S_B: V_in; S_C: b·[⌈n_cnt/n_grp⌉·a_b + ⌈m_cnt/m_grp⌉·b_b] (最小替换分组) | S_A/S_B: b·[(n_cnt−1)·a_b + (m_cnt−1)·b_b]; S_C: b·[(n_cnt−⌈n_cnt/n_grp⌉)·a_b + (m_cnt−⌈m_cnt/m_grp⌉)·b_b] | S_A→L2; else GM |
|
||||||
|
| StreamK | V_in (组间共享命中) | b·[(n_cnt−1)·a_b + (m_cnt−1)·b_b] (命中时) | 最终写回: S_A(+ws)→L2; else GM |
|
||||||
|
| 特殊分支 | K=1: V_in; K=0: 0 | 0 | S_A→L2 写口; else GM (K=0 只有输出) |
|
||||||
|
| 转Matmul | V_in (折叠后) | 0 | S_A→L2; else GM |
|
||||||
|
|
||||||
|
> **搬移效率 (时间列口径, issue#36)**: 上表为**字节量**口径 (不受效率模型影响);
|
||||||
|
> 切B 两分支的 GM→L1 **时间**按单命令 tile 效率加权 t = (V_A/eff_A + V_B/eff_B)/W_GM,
|
||||||
|
> eff = min(1, tile/min_TileSize), tile = nValue×dValue×dt。MergeBatch 合并使 tile
|
||||||
|
> 放大 b0 倍 -> 效率项是 T_cmd=0 时 MergeBatch 的主要收益来源 (详见
|
||||||
|
> docs/02_分支理论/01_MergeBatch分支.md §4~5)。ASW/StreamK 单命令 tile 大、效率
|
||||||
|
> 恒饱和, 不接入 (极端小 tile 形状走 issue#34 "效率降级"标注通道)。
|
||||||
|
|
||||||
|
时延计算: 全核并发时 t = 芯片字节 / 芯片带宽; 降核 (used < C) 时按 used×单核份额
|
||||||
|
(线性假设, issue#26 标注), Cube t = 芯片 flops / (used × q_cube(dtype))。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. MergeBatch Cube 复核 (issue #27) — 证明
|
||||||
|
|
||||||
|
合并语义: A'[b0·M, K] @ B'[K, b0·N] 作为单次 GEMM, Cube 硬件计算**全网格**
|
||||||
|
(b0·M)×(b0·N) 输出 (含交叉项, 冗余比例 (b0²−b0)/b0² = (b0−1)/b0), 只保留 b0 个对角块。
|
||||||
|
|
||||||
|
- 每次合并计算 (步): flops_step = 2·(b0·M)·(b0·N)·K;
|
||||||
|
- 每核合并组数: b_core/b0;
|
||||||
|
- 总: flops = (b_core/b0)·2·(b0·M)·(b0·N)·K = **b_core·b0·2MNK** —— 与现行代码
|
||||||
|
`flops_pc = b_core·b0·2·m·n·k` 逐项相等, t_mmad = flops/Q16 不变。
|
||||||
|
|
||||||
|
CSV 证据 (merge_demo_k_trunc, B=2048 M=N=32 K=256, b0=4, b_core=64):
|
||||||
|
flops_step = 2·128·128·256 = 8,388,608; 步数 16; 合计 134,217,728 = 代码值 = CSV cube_flops 列;
|
||||||
|
t_mmad = 134,217,728 / 13.5e12 = 9.942us = CSV t_mmad (Cube-only 口径 issue#40;
|
||||||
|
旧总算力口径 15.1875T 时为 8.837us)。**结论: 无需数值修改** (指 flops 公式);
|
||||||
|
若意图是"只算对角块"则与全网格 GEMM 语义冲突 (冗余消失, v0.98 §五 理论前提需另行裁决)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 变更影响与验证口径
|
||||||
|
|
||||||
|
1. 字节列整芯片化 (IterBatch/MergeBatch/special 数值 ×C 或按 b 直接计算);
|
||||||
|
2. S_A 判定下输出写 L2: 小 case (整 case 可全驻留) GM 写流量清零、输出时延降为 L2 写口;
|
||||||
|
3. S_A 之外输出 GM 写流量如实计入共享总线 → 修正单 batch 驻留判定导致的漏计;
|
||||||
|
4. dtype 感知算力: bf16 案例数值零回退; fp32/fp8/fp4/… 案例时延按 §2.1 表修正;
|
||||||
|
5. 新增回归: GM≥V_in 不变量 (全分支随机集)、S_A 输出落点开关、dtype 速率比例、
|
||||||
|
fp32 Cube 时延 = bf16 的 2 倍、字节列整芯片口径等;
|
||||||
|
6. examples 三件套重生成, 与模型改动前 diff 隔离后入库; 压力回归 0 违规/0 NaN。
|
||||||
391
BMM/BMM_Theory/docs/06_落地验证计划_理论方案_vs_bmmv3源码.md
Normal file
391
BMM/BMM_Theory/docs/06_落地验证计划_理论方案_vs_bmmv3源码.md
Normal file
@@ -0,0 +1,391 @@
|
|||||||
|
# 06 理论方案落地 batch_mat_mul_v3 源码 — 比对分析与验证计划
|
||||||
|
|
||||||
|
> 目的:把 BMM_Theory 的理论最优方案落到真实算子源码 `batch_mat_mul_v3`(ops-nn),
|
||||||
|
> 上板(**Ascend950PR 主 bin,arch35 / DAV_3510**)实测验证收益,达标后向真实算子源码仓提 PR。
|
||||||
|
> 比对基线源码:`\\HwFs\HW_WorkBuddy\昇腾NPU\昇腾NPU知识库\代码仓\ops-nn\matmul\batch_mat_mul_v3`
|
||||||
|
> (下称"源码",行号为该仓当前快照);理论侧:本仓 `bmm_theory/` + `docs/02_分支理论/`。
|
||||||
|
> 文中所有"预期收益"数字均为**理论模型测算**(含 T_cmd=0 等未标定假设),仅用于排优先级,
|
||||||
|
> 最终以 msProf 上板实测为准。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 结论摘要(TL;DR)
|
||||||
|
|
||||||
|
| 优先级 | 分支/主题 | 源码现状一句话 | 理论差异一句话 | 模型预估收益(定向 case) | 改动面 | 建议批次 |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| P0 | **StreamK 准入放宽** | 固定 K≥8192 门槛,拒绝后落 ≤4 核 ASW | 动态 grid_K + θ_c≈10.9 归约代价判据(K>~360 起可进) | **总时延 −73%~−82%**(K=4096~6144 细长 case) | **纯 host tiling**(kernel 已有完整 StreamK 实现) | 批次 1 |
|
||||||
|
| P1 | **ASW 尾轮重切** | 尾轮 `index≥totalCnt` 空转(A0);ASW_BASIC 仅做单套 tile 的均衡率枚举 | 方案B(全局一套 tile 重切)/A1b(尾轮第二套 tile),A0 在 r>0 时严格劣 | **总时延 −25%~−43%**(2 波 + 尾轮 ρ=0.125~0.5 的 case) | 方案B 纯 host;A1b 需 kernel 读第二套尾轮参数(共享库有现成范式) | 批次 2 |
|
||||||
|
| P2 | **MergeBatch 准入补齐 + 仲裁接线** | 准入偏宽(缺搬移量/tile/算存比三条),且静态优先级恒先于 IterBatch | 理论 5 条件 + 与 IterBatch 的净收益仲裁(L1 绑定时 MergeBatch 恒劣) | 防误捕获(小搬移量 case 冗余计算/小 tile 低效);仲裁差异 T_cmd=0 下 ≤0.5% | **纯 host tiling** | 批次 3 |
|
||||||
|
| P3 | **IterBatch 形态补全(a/c/d)** | 仅"整 K 双侧驻留 + 双 batch 乒乓"(形态 b),放不下 L1 直接拒绝 → 落 ASW | 理论四形态(a 单 batch 驻留 / c 一侧驻留+对侧切K / d 两侧切K),k_l1 反推不受 512B 上限 | T_cmd=0 下 k_l1 差异本身 ≈0(<0.6%);收益=命令数 4× 减少 ×T_cmd + 消除 ASW 承接时的 L2 重复读 | **kernel 重写 mmad 流水(最难)** + host 准入 | 批次 4 |
|
||||||
|
| P4 | 转Matmul / 特殊分支 / 广播 | 已有 TO_MUL、K_EQ_ZERO、ITER_BATCH_BROADCAST、MergeBatchAndMAxis 折叠 | 理论基本一致(BatchA=1 大 A 的"重排转 Matmul"源码无,低优先) | 对齐核查即可 | 核查 | 随批附带 |
|
||||||
|
|
||||||
|
**总体判断**:收益大头在 **StreamK 准入**与 **ASW 尾轮** 两处,且都可先做 **host-only** 改动拿到大部分收益;
|
||||||
|
IterBatch 形态补全改动最大、收益依赖 T_cmd 标定结果,放最后。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 源码现状架构速览(arch35 / DAV_3510)
|
||||||
|
|
||||||
|
### 1.1 host 侧:策略注册表 + 静态优先级
|
||||||
|
|
||||||
|
`batch_mat_mul_v3_tiling.cpp`:`IsAdvancedSocVersion` → arch35 advanced 路径
|
||||||
|
(`arch35/batch_matmul_v3_tiling_advanced.*`),按 `batch_matmul_v3_tiling_strategy.h`
|
||||||
|
的优先级表**逐策略试 `IsCapable()`,首个通过即中,无回退、无时延仲裁**:
|
||||||
|
|
||||||
|
```
|
||||||
|
K_EQUAL_ZERO(0) → TO_MUL(1) → STREAM_K(2) → MERGE_BATCH_BASICAPI(3)
|
||||||
|
→ ITER_BATCH_BROADCAST_BASICAPI(4) → ITER_BATCH_BASICAPI(5) → ITER_BATCH(6)
|
||||||
|
→ AL1_FULL_LOAD_BASIC(7) → BL1_FULL_LOAD_BASIC(8) → ASW_BASIC(9) → BASE(999)
|
||||||
|
```
|
||||||
|
|
||||||
|
与理论决策树(`router.py`)的结构差异:
|
||||||
|
|
||||||
|
| 项 | 源码 | 理论 |
|
||||||
|
|---|---|---|
|
||||||
|
| 路由方式 | 静态优先级 + IsCapable 一票否决/通过 | 决策树 + 重叠区端到端时延模型仲裁 |
|
||||||
|
| StreamK vs 切B | StreamK **优先于** MergeBatch/IterBatch | 切B(B≥C)优先,StreamK 仅在 P≤C/2 |
|
||||||
|
| Merge vs Iter | MergeBatch 恒优先(无仲裁) | 净收益仲裁 + 时延模型终审 |
|
||||||
|
| 兜底 | ASW_BASIC(等 batch)/ BASE(广义,固定 256×256 起步) | ASW_Basic(含降核)恒出方案 |
|
||||||
|
|
||||||
|
### 1.2 kernel 侧:7 参模板分发
|
||||||
|
|
||||||
|
`op_kernel/arch35/batch_mat_mul_v3.cpp:171` 起:模板参数
|
||||||
|
`<API_LEVEL, A_TRANS, B_TRANS, ITER_MODEL, BMODEL, FULL_LOAD, L0C2OUT>`,`if constexpr` 链分发:
|
||||||
|
IterBatch(cmct `KernelMatMulIterBatch` / 旧库 `MultiBatchKernel` 两世代并存)、
|
||||||
|
MergeBatch(cmct `KernelMatMulMergeBatch`)、ASW(`BatchMatMulAswKernel/AswBlock`)、
|
||||||
|
StreamK(Blaze `MatMulStreamKKernel` 或内置 `MatMulStreamKActKernel`)等。
|
||||||
|
**关键事实**:StreamK 的 kernel(动态 kCnt 调度、fp32 部分和、AIV 归约)**已完整存在**;
|
||||||
|
ASW 尾轮重切所需的 `MatMulV3TailInfo{mCnt,nCnt,kCnt,mTailMain,nTailMain}` 字段在
|
||||||
|
`MatMulV3TilingData` 已存在(共享 mat_mul 的 `block_scheduler_aswt.h`/`mat_mul_asw_block.h`
|
||||||
|
有现成"尾 tile 再切子块"范式),**batch 版 AswBlock 只是没有读这些字段**。
|
||||||
|
|
||||||
|
### 1.3 口径天然对齐(好消息)
|
||||||
|
|
||||||
|
源码 tiling 内的平台公式与理论硬件口径一致:
|
||||||
|
`GetHbmBW = freq×32×31/1024 ≈ 1.6TB/s`;`GetL2BW = freq×32×100/1024 ≈ 5.16TB/s ≈ 理论 5.2TB/s`;
|
||||||
|
`singleCoreComputePower = freq×8(K) ≈ 13.2T ≈ 理论 Cube-only 13.5T`(差 2%,经验取整)。
|
||||||
|
→ 理论模型与源码 tiling 的"带宽/算力世界观"兼容,参数级差异小,差异集中在**策略结构**。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 分支逐项比对与修改重点
|
||||||
|
|
||||||
|
### 2.1 StreamK(P0,最大收益,host-only 可落地大半)
|
||||||
|
|
||||||
|
**理论方案**(`docs/02_分支理论/05_StreamK分支.md`):
|
||||||
|
|
||||||
|
- 进入:P = B·M·N·4B/L0C ≤ C/2;K/grid_K ≥ 256B/dtype;**K > grid_K²/(grid_K−1)·θ_c,θ_c≈10.9**
|
||||||
|
(grid_K=32 → K>360);确定性等级 ≤1 且 ND。
|
||||||
|
- grid_K 动态:blocksPerBatch=⌊C/B⌋,grid_K=blocksPerBatch/(mCnt·nCnt),mCnt/nCnt 收拢为因子。
|
||||||
|
- 部分和 fp32 写 workspace(防精度丢失),AIV 归约后按 C dtype 写出;workspace 应尽量驻留 L2
|
||||||
|
(落 GM 时归约带宽 5.2→~0.64TB/s,θ_c 升至 ~86)。
|
||||||
|
|
||||||
|
**源码现状**(`arch35/batch_matmul_v3_basic_streamk_tiling.cpp`):
|
||||||
|
|
||||||
|
- 准入 `CheckStreamKSKTiling`:**固定门槛 `CeilAlign(K,256) ≥ max(8192, aicNum×256B/dtype)`**
|
||||||
|
(bf16 即 8192);`batch·mCnt·nCnt ≤ aicNum/2`(mCnt/nCnt 按 256 估算);fp32 非 HF32 且 K>2e6 拒绝。
|
||||||
|
- grid_K 已有动态雏形:`tailInfo.kCnt = ⌊aicNum/(batch·mCnt·nCnt)⌋`,`singleCoreK=⌈K/kCnt⌉`
|
||||||
|
(落盘 `skSingleCoreK`);mCnt/nCnt 有">blocksPerBatch/3 且 <blocksPerBatch/2 则收拢到 /2"的启发式。
|
||||||
|
- kernel 侧(`BlockSchedulerStreamKBuiltIn` / `BlockEpilogueStreamK`)完整:AIC 写 fp32 部分和到
|
||||||
|
**GM workspace**(host 分配 `aicNum×256×256×4B + RPC`),AIV `CrossCoreWaitFlag + SyncAll`
|
||||||
|
后读回 UB 级联加、Cast 写 C。**workspace 名义 GM,经 L2 缓存;容量够时实际命中 L2**(待实测确认)。
|
||||||
|
|
||||||
|
**差异清单**:
|
||||||
|
|
||||||
|
| # | 差异 | 影响 |
|
||||||
|
|---|---|---|
|
||||||
|
| S1 | 固定 K≥8192 vs 理论动态 K>360(grid_K=32) | **K∈(360, 8192) 的细长 case 全部被拒绝**,落 ASW 只用 batch·mCore·nCore ≤ C/2 核 |
|
||||||
|
| S2 | 准入无"归约代价 vs 不切 K"比较(θ_c 判据) | 门槛同时承担了 dValue 与归约代价两职,过保守 |
|
||||||
|
| S3 | workspace 大小固定 `C×256×256×4B`(8MB+RPC) | 与 mCnt/nCnt/kCnt 实际解耦,偏大但不阻断 |
|
||||||
|
| S4 | mCnt/nCnt 启发式收拢 vs 理论"收拢为 blocksPerBatch 因子最大化 grid_K" | 参数级,影响 K 并行度 |
|
||||||
|
| S5 | workspace 显式 L2 驻留无法由 op 侧表达(只有 SetL2CacheHint) | 架构级,暂不做,靠 L2 自然缓存 |
|
||||||
|
|
||||||
|
**修改重点(host-only,批次 1)**:
|
||||||
|
|
||||||
|
1. `CheckStreamKSKTiling`:把固定 8192 门槛替换为**理论两条判据**——
|
||||||
|
先按现行逻辑试算 mCnt/nCnt/kCnt,然后要求 `K/kCnt ≥ 256B/dtype`(条件 2,dValue)且
|
||||||
|
`K > kCnt²/(kCnt−1)·θ_c'`(条件 3,归约代价)。θ_c' 取 **GM 口径 ~86**(保守,workspace 名义 GM)
|
||||||
|
或按 workspace ≤ L2 余量切换 L2 口径 10.9——建议第一版用 GM 口径(grid_K=32 时 K>~2841),
|
||||||
|
上板测得归约段实际命中 L2 后再放宽。
|
||||||
|
2. mCnt/nCnt 收拢规则替换为"blocksPerBatch 因子 + grid_K 最大化"(理论 §4)。
|
||||||
|
3. 保持 kernel 不动(tiling 字段 `skSingleCoreK`/`tailInfo.kCnt` 语义不变)。
|
||||||
|
|
||||||
|
**预期时延变化**:减少的是 **MTE2 稳态段**(并行度 4→32 核)与 MMAD 段同比例;
|
||||||
|
新增有限的 **归约尾(t_reduce)**。模型测算(bf16):
|
||||||
|
|
||||||
|
| case | 源码现状(ASW 降核,核数=batch·mCore·nCore) | 理论 StreamK | 总时延节省 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| B=4, M=N=128, K=4096 | 41.9us(4 核) | 8.65us(steady 5.24 + reduce 3.41) | **−79.4%** |
|
||||||
|
| B=4, M=N=128, K=6144 | 62.9us(4 核) | 11.27us | **−82.1%** |
|
||||||
|
| B=8, M=256, N=128, K=4096 | 62.9us(8 核) | 16.60us | **−73.6%** |
|
||||||
|
|
||||||
|
敏感性:若 workspace 实际走 GM 带宽(不命中 L2),K=4096 例 reduce 升至 ~10us,总 ~15.4us,
|
||||||
|
仍省 ~63%。**反向保护**:θ_c 判据保证 K 太小(归约吃不下)时不进——理论 K=2048 例即被判不可行。
|
||||||
|
|
||||||
|
**验证重点**:
|
||||||
|
- 定向 case 集:B∈{2,4,8},M/N∈{64,128,256},K∈{2048, 2843±, 4096, 6144, 8000, 8192±, 10240},
|
||||||
|
覆盖"源码拒绝→修改后进入"的跃迁区间与 θ_c 边界两侧(K∈{2841±} 附近加边界 case,验证不错进/不漏进);
|
||||||
|
- msProf 拆解:MTE2/MMAD 稳态是否如预期缩短,**归约段实测时延 vs 模型 3.4us(L2 口径)/~10us(GM 口径)**
|
||||||
|
—— 据此裁决 workspace 口径与 θ_c' 取值;
|
||||||
|
- 正确性:fp32 部分和 + AIV 归约的数值精度(对比 golden,deterministic_level=0/1);
|
||||||
|
- 回归:K≥8192 老 case 时延不回退(grid_K 推导变化的影响面)。
|
||||||
|
|
||||||
|
### 2.2 ASW_Basic 尾轮重切(P1,方案B 可 host-only)
|
||||||
|
|
||||||
|
**理论方案**(`docs/02_分支理论/06_ASW_Basic分支.md` §6 + `07_尾轮处理策略.md`):
|
||||||
|
总块数 N_blk = B·mCnt·nCnt 不能整除 C 时,尾轮 r = N_blk mod C 个核干活、C−r 核空转一个整块。
|
||||||
|
**A0(不重切)在 r>0 时严格劣**;面积型主导下 A1b(尾轮第二套小 tile 凑满核)与方案B(全局一套
|
||||||
|
tile 均匀重切到 n_wave·C 块)理论时延严格相等 = (n_wave−1+ρ)·T_block;周长型且 ρ≥ρ_dv 时 A1b 恒优;
|
||||||
|
r 小翻出时方案B 微优 ~6%。软件默认输出方案B,周长型 ρ≥ρ_dv 输出 A1b。
|
||||||
|
|
||||||
|
**源码现状**:
|
||||||
|
- kernel `asw_kernel_advanced.h` Process:`if (index < totalCnt)` 才干活,**尾轮 C−r 核真空转(A0)**;
|
||||||
|
- host `ASW_BASIC` 的 `GetRebalanceBlock`(`mat_mul_v3_tiling_helper.cpp:387`)枚举 baseM/baseN
|
||||||
|
最大化"均衡率",**但 `GetBalanceRateWithTail` 里尾轮切分项在 `batchInfo != nullptr` 时被禁用**
|
||||||
|
(:240 早退),即 batch 场景的均衡率按 A0 模型算——枚举只隐式接近"广义方案B",且无
|
||||||
|
"搬入最小化"目标与尾轮翻出概念;
|
||||||
|
- `BASE` 策略(广播/非等 batch 兜底)连 GetRebalanceBlock 都没有,固定 baseM=baseN=256 + A0。
|
||||||
|
|
||||||
|
**差异清单**:
|
||||||
|
|
||||||
|
| # | 差异 | 影响 |
|
||||||
|
|---|---|---|
|
||||||
|
| T1 | 尾轮空转(A0) | r>0 时白丢 (1−ρ)/n_wave 比例的总时延 |
|
||||||
|
| T2 | 首轮 tile 枚举目标=均衡率/cubeBound 启发式 vs 理论"每 batch 搬入 K·dt·(nCnt·M+mCnt·N) 最小 + 尾轮 r 最大 tie-break" | L2 重复读量、dValue 效率差异 |
|
||||||
|
| T3 | 无 A1b(尾轮第二套 tile) | 周长型主导区少拿 A1b vs 方案B 的差 |
|
||||||
|
| T4 | 无降核口径对齐:源码 usedCoreNum=min(batch·mCore·nCore, C) vs 理论 P=L0C 满载粒度降核 | 并行度估计口径不同,需上板对齐 |
|
||||||
|
|
||||||
|
**修改重点**:
|
||||||
|
1. **方案B(host-only,先做)**:`GetRebalanceBlock` 之后追加"整轮均匀重切"——由 N_blk、n_wave、r
|
||||||
|
计算 g = n_wave·C/N_blk,把 baseM/baseN 在候选集内按 1/√g 收缩重选(或直接在枚举目标函数里
|
||||||
|
把 A0 均衡率换成 `w×T_block` 估计),使尾轮满载。kernel 零改动(一套 tile,totalCnt 变为
|
||||||
|
n_wave·C,天然无空转)。
|
||||||
|
2. **A1b(kernel 配合,后做)**:host 计算第二套尾轮 tile 参数写入 `MatMulV3TailInfo` 现有字段;
|
||||||
|
`asw_block_advanced.h` 的 Init/UpdateBasicIndex/UpdateBlockParams/CalcGMOffset 按 roundIdx
|
||||||
|
切换主/尾相位——**直接移植共享库 `block_scheduler_aswt.h` 的尾 tile 子块范式**(mTailCnt/nTailCnt
|
||||||
|
现成字段),保持每子片单核单写(不引入跨核归约)。纯运行期数据驱动,**不增编译变体**。
|
||||||
|
3. 尾轮策略选择逻辑(A0/A1b/方案B 的五步闭式判定)放在 host,输出到 tiling 备注字段便于验证对照。
|
||||||
|
|
||||||
|
**预期时延变化**:减少的是 **drain/尾轮段**((1−ρ)·T_block 量级),即 max 稳态之后的空转暴露。
|
||||||
|
模型测算(理论 ASW evaluate,强制 A0 vs 最优尾轮):
|
||||||
|
|
||||||
|
| case | N_blk / 波次 / ρ | A0(源码现状) | 最优尾轮 | 总时延节省 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| B=2, M=1024, N=1536, K=1024 | 48 / 2 / 0.50 | 19.88us | 14.91us(方案B) | **−25.0%** |
|
||||||
|
| B=2, M=1024, N=1280, K=1024 | 40 / 2 / 0.25 | 19.88us | 12.43us(方案B) | **−37.5%** |
|
||||||
|
| B=2, M=768, N=1536, K=1024 | 36 / 2 / 0.125 | 20.04us | 11.34us(方案B) | **−43.4%** |
|
||||||
|
|
||||||
|
(同为 MMAD/MTE2 主导的面积型 case;周长型边角区 A1b 另有收益,量级待 case 库扫描。)
|
||||||
|
注意:源码 ASW_BASIC 的均衡率枚举已能自发吸收其中一部分(相当于不完整的方案B),
|
||||||
|
**实测基线必须先跑**,预期净收益 = 表值 − 源码枚举已拿到的部分;BASE 兜底命中的 case
|
||||||
|
(广播、非等 batch)则全额可期。
|
||||||
|
|
||||||
|
**验证重点**:
|
||||||
|
- 定向构造 N_blk mod C ≠ 0 且 n_wave ∈ {1,2,3,≥4} 的 case 矩阵(B∈{2,3,5,6} × M/N 组合扫描),
|
||||||
|
确认尾轮核不再空转(msProf 核间时间线/各核结束时刻拉齐);
|
||||||
|
- 对比维度:总时延、尾轮起止时刻、GM/L2 流量(方案B tile 变小 → 搬入周长和 √g 放大,
|
||||||
|
理论已标注 P4 流量放大不入模型,**需实测是否吞掉收益**——这是该改动的头号验证风险);
|
||||||
|
- 正确性:尾轮重切后边界块(mBaseTail/nBaseTail)数值正确性;
|
||||||
|
- 回归:r=0 的 case 零变化(应自动保持 A0)。
|
||||||
|
|
||||||
|
### 2.3 IterBatch 形态补全(P2/P3,kernel 重写流水,收益依赖 T_cmd 标定)
|
||||||
|
|
||||||
|
**理论方案**(`docs/02_分支理论/02_IterBatch分支.md`):核间切 B 后,核内按 L1 容量四形态选一——
|
||||||
|
a) 单 batch 全驻留(b_core=1);b) 双 batch 乒乓(2(MK+KN)·dt ≤ L1,命中最多);
|
||||||
|
c) 一侧驻留+对侧切 K(驻留侧 ≤ L1/min(b_core,2),b_core≥2 时另一半 L1 预取下一 batch 驻留侧);
|
||||||
|
d) 两侧都切 K(兜底,K 段成对流水)。四形态共同保证**单 batch 计算核内零重复读、GM=V_in**;
|
||||||
|
k_l1 按容量反推(不受 512B 硬上限,dValue≥128B 下限约束)。
|
||||||
|
|
||||||
|
**源码现状**:
|
||||||
|
- `iterbatch_basicapi_tiling.cpp` IsCapable:`batchC > aicNum`(严格 B>C)+
|
||||||
|
**`2×((M·K+K·N)·dt+bias) ≤ L1`**——即只有形态 b;放不下直接 false(或 L0 放不下单 batch 时走
|
||||||
|
0.8 均衡率兜底),**无 c/d 形态的退化承接**;
|
||||||
|
- kernel(cmct `BlockMmad` IterBatch 偏特化)实证:L1 布局=整 K 的 A/B 各 2 缓冲按 batch 叠放,
|
||||||
|
L1→L0 才按 baseK 切 K,mmad 循环 iter1(batch)→N→M→K;**代码中不存在"一侧驻留+对侧切K"或
|
||||||
|
"两侧切K"的显式模式**;`mmadParams.unitFlag` 恒 0(理论假设的 UnitFlag 16-granule 细粒度流水
|
||||||
|
在该路径未启用);写回 L0C→GM 直写(ON_THE_FLY)或经 UB/AIV(ND_FIXPIPE_1_2),不经 L2。
|
||||||
|
|
||||||
|
**形态 c/d 的 case 在源码里的实际去向**:落 ASW_BASIC/BASE——ASW 把 M/N 切到 ≤256 的 base 块、
|
||||||
|
K 由 L1 tiling 切,共享块重复读走 L2。**不是灾难,但偏离理论最优**:
|
||||||
|
1. M/N>256 时被切块 → 引入 (nCnt−1)·a_b+(mCnt−1)·b_b 的 L2 重复读(理论 IterBatch 为 0);
|
||||||
|
2. K 粒度受 512B 内轴对齐与 stepK≤8(issue queue)限制,单命令 tile 更小、命令数更多;
|
||||||
|
3. 无 batch 边界预取(理论 c 形态半预算预取驻留侧,边界无气泡)。
|
||||||
|
|
||||||
|
**模型测算**(形态 d 例:B=64, M=N=64, K=8192, bf16):
|
||||||
|
|
||||||
|
| 口径 | k_l1 | 单核 DMA 命令数 | 总时延 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 理论 IterBatch(k_l1 反推=1024) | 1024 | **16** | 85.0us |
|
||||||
|
| 源码式承接(k_l1 上限 512B/dt=256,ASW 模拟) | 256 | **64** | 84.5us |
|
||||||
|
|
||||||
|
**T_cmd=0 假设下时延差 <0.6%**——形态 d 的收益几乎全部押在 **T_cmd > 0**(每核命令少 4×)
|
||||||
|
与 L2 重复读消除上。→ **T_cmd 标定是该批次的入场券**(见 §4.3):若标定出 T_cmd 可观
|
||||||
|
(如 ≥100ns),form_d 例命令差 48 次 → ~5us 级收益;若 T_cmd≈0,则本批次降级为
|
||||||
|
"框架统一 + 大 M/N case 的 L2 重复读消除"(form_c 型例:ASW 承接的 L2 重读 ~96MB ≈ 18.5us 级,
|
||||||
|
需实测扣除其流水掩盖)。
|
||||||
|
|
||||||
|
**修改重点**(若标定结果支持做):
|
||||||
|
1. host `iterbatch_basicapi_tiling.cpp`:IsCapable 放宽——L1 放不下双 batch 整 K 时进入形态
|
||||||
|
选择(a/c/d),按理论公式算驻留侧/k_l1;DoOpTiling 输出形态字段;
|
||||||
|
2. kernel:cmct `BlockMmad`(IterBatch) 的 L1 布局与循环次序重写(c 形态:驻留侧一次 CopyIn、
|
||||||
|
对侧 K 分块双缓冲,batch 边界半预算预取;d 形态:两侧 K 分块成对流水)——
|
||||||
|
**这是全计划最难的 kernel 改动**(L1 flag 4 组 + L0 乒乓 + L0C 半区 + AIC↔AIV 握手耦合极密);
|
||||||
|
tilingData `BatchMatMulV3IterBatchBasicTilingData` 加形态/分侧参数字段(pack(8) 对齐,
|
||||||
|
host/kernel 同步改);尽量纯运行期字段表达以**避免新增模板组合/二进制**;
|
||||||
|
3. 同步考虑启用 unitflag(细粒度排空)以压低 drain——理论 drain 模型(末 batch T_comp+T_write)
|
||||||
|
在 unitflag=0 的源码上可能**低估**真实排空,需实测校准(这也影响理论模型自身的 drain 闭式)。
|
||||||
|
|
||||||
|
**验证重点**:T_cmd 标定先行;形态 c/d 定向 case(M/N>256 且 2(MK+KN)>L1、以及双侧超 L1)的
|
||||||
|
总时延与 L2 流量对比;batch 边界气泡(核内时间线);正确性(K 分段累加顺序变化)。
|
||||||
|
|
||||||
|
### 2.4 MergeBatch 准入补齐 + 仲裁接线(P2,纯 host)
|
||||||
|
|
||||||
|
**理论方案**(`01_MergeBatch分支.md`;与 bmmv3 对拍器口径的逐项对照另见
|
||||||
|
`04_差异对照_bmmv3_vs_BMM_Theory.md` §2):5 条件准入(含 ③ 单核搬移量 ≥480KB、④ 单 tile ≥16KB、
|
||||||
|
⑤ AI < R16/b0 访存 Bound 守卫),b0 = min(L0C/L0A/L0B 容量上限, R16 算存比上限, b_core) 取因子;
|
||||||
|
k_l1 = min(L1 反推, K, 512B/dt);与 IterBatch 重叠区按净收益仲裁
|
||||||
|
(命令节省×T_cmd + 搬移效率节省 − drain 惩罚;**L1 绑定(k_l1<K 且两侧 tile 饱和)时恒劣**)。
|
||||||
|
|
||||||
|
**源码现状**(`mergebatch_basicapi_tiling.cpp` + kernel 实证):
|
||||||
|
- 准入:batch 四轴全等、**batchC ≥ 4·C**(对应理论 b_core≥2b0@b0=2)、K 对齐 ≥64、**M≤N**、
|
||||||
|
L0A/L0B/L0C 容量查(min 4 batch)——**缺理论条件 ③④⑤**;
|
||||||
|
- b0(mergeBatchL0)= min(L0C 多项式解, L0B 上限, ceil(B/C))——无 L0A 显式收敛(靠 min4 间接保证)、
|
||||||
|
无算存比上限;
|
||||||
|
- k 粒度:baseK=min(L0B 反推, 64),L1 四缓冲 stepKa ≤ min(⌈b_core/b0⌉, L1/4/(b0·M·baseK·dt))——
|
||||||
|
无 512B 推荐值概念,但 L1 约束下与理论值经常重合(见下);
|
||||||
|
- kernel 语义与理论一致:batch 并入空间维做稠密 (b0·M)×(b0·N) MMAD,对角片写回
|
||||||
|
(Fixpipe ndNum=b0、对角 stride),冗余比例 (b0−1)/b0 与理论 §6 证明一致;bias 不支持
|
||||||
|
(wrapper 直接 return,理论同样不建模 bias);unitflag 恒 0。
|
||||||
|
- **仲裁缺失**:优先级表 MERGE_BATCH(3) 在 ITER_BATCH(5/6) 之前,IsCapable 通过即中,
|
||||||
|
L1 绑定情形无法回退 IterBatch。
|
||||||
|
|
||||||
|
**差异清单与修改重点**:
|
||||||
|
|
||||||
|
| # | 差异 | 修改 |
|
||||||
|
|---|---|---|
|
||||||
|
| M1 | 缺条件③(单核搬移量 ≥480KB) | IsCapable 追加:小搬移量 case 防误捕获(此类 case 时延绝对值小、启动开销占比高,合并收益不真实) |
|
||||||
|
| M2 | 缺条件④(tile ≥16KB) | IsCapable 追加:tile 不足时有效带宽线性退化(eff=tile/16KB),合并红利不成立 |
|
||||||
|
| M3 | 缺条件⑤(AI < R16/b0) | IsCapable 追加:防把计算 Bound case 抓进来让冗余计算变瓶颈 |
|
||||||
|
| M4 | b0 无 L0A 显式收敛/无算存比上限 | DoOpTiling b0 计算并入两上限(理论 issue#13 口径) |
|
||||||
|
| M5 | 无 Merge vs Iter 仲裁 | 两条路:(a) 简单——IsCapable 里并入"L1 绑定判据"(k_l1 反推 <K 且 tile 饱和 → false,放给 IterBatch);(b) 完整——引入轻量时延估计择优。**建议先 (a)** |
|
||||||
|
| M6 | M≤N 限制、K≥64 限制 | 源码实现限制(B 沿 N 合并+对齐),保留并写入文档说明 |
|
||||||
|
|
||||||
|
**预期时延变化**:参数层(b0/k_l1)在已命中 case 上模型测算差异为 **0%**(两者经常收敛到同值);
|
||||||
|
收益全部在**防止误捕获**与**仲裁防劣化**:L1 绑定区 MergeBatch 相对 IterBatch 的劣势在
|
||||||
|
T_cmd=0 下 ≤0.5%(drain 放大项),T_cmd>0 时放大 b0 倍——**同样依赖 T_cmd 标定**。
|
||||||
|
条件③④⑤ 误捕获区的典型代价:tile=4KB → 有效带宽 ×0.25,或计算 Bound 下 b0 倍冗余 MMAD。
|
||||||
|
→ 该批次定位为**健壮性/正确性加固**,收益以上板实测误捕获 case 为准。
|
||||||
|
|
||||||
|
**验证重点**:构造条件③④⑤ 各自的越界 case(小搬移量、小 tile、高 AI),确认修改后不再进
|
||||||
|
MergeBatch 且落点分支时延更优;L1 绑定仲裁区 case(merge_iter_arbitrate 类:B=128, M=N=64,
|
||||||
|
K=512);已命中 case 全量回归零变化。
|
||||||
|
|
||||||
|
### 2.5 转Matmul / 特殊分支 / 广播(P4,对齐核查)
|
||||||
|
|
||||||
|
| 理论 | 源码对应 | 核查结论 |
|
||||||
|
|---|---|---|
|
||||||
|
| K=0 纯 AIV 写值 | `K_EQUAL_ZERO(0)`(AIV 清输出 kernel) | ✅ 已覆盖,对齐即可 |
|
||||||
|
| K=1 逐元素乘走 AIV | `TO_MUL(1)`(matmul2mul tiling + vector kernel) | ✅ 已覆盖;理论 B<128 的 AIV 单缓冲退化需核查源码对应行为 |
|
||||||
|
| BatchB=1 零代价折叠 | `MergeBatchAndMAxis`(batchB==1 且 !isATrans → 折进 M) | ✅ 已覆盖 |
|
||||||
|
| BatchA=1:A 小留 BMM 广播 / A 大比较重排 | ITER_BATCH_BROADCAST(4)(单广播轴)/ ASW 广播模型 | ⚠️ "A 大则重排转 Matmul"源码无,低优先 |
|
||||||
|
| 交叉广播(双轴/多轴) | 落 ASW_Basic/BASE | ✅ 与理论一致(理论也由 ASW 承接) |
|
||||||
|
|
||||||
|
### 2.6 路由层差异(贯穿项)
|
||||||
|
|
||||||
|
1. **优先级顺序**:源码 StreamK(2) 先于 MergeBatch(3)/IterBatch(5)。重叠区(B≥C 且 P≤C/2 且
|
||||||
|
K≥门槛)源码必选 StreamK,理论先走切 B。模型测算该区域两者接近(切 K 并行度收益 ≈ 切 B 免
|
||||||
|
共享收益),**维持源码顺序不改**,用定向 case 实测确认无显著回退即可。
|
||||||
|
2. **仲裁机制**:源码无"时延模型终审"。短期用各分支 IsCapable 的判据细化(M5(a)、S1/S2)逼近
|
||||||
|
理论裁决;长期若要引入端到端时延估计进 tiling,需单独评审(tiling 耗时预算、常量标定维护)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 横切差异(时延模型与口径)
|
||||||
|
|
||||||
|
| 项 | 理论模型 | 源码 tiling | 处置 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 算力口径 | Cube-only 13.5T/核(432T/32 核,issue#40) | freq×8K≈13.2T(经验式) | 一致(差 2%),无需改 |
|
||||||
|
| 带宽口径 | GM 1.6TB/s 读写共享;L2 5.2TB/s 读写独享 | GetHbmBW≈1.6T、GetL2BW≈5.16T | 一致,无需改 |
|
||||||
|
| 输出落点 | S_A(整 case ≤L2)时写 L2 写口,否则直写 GM | 全部直写 GM,C 从不显式写 L2 | **观察项**:S_A 小区 case 理论上有 GM 写时延节省;硬件 L2 写回自然缓存可能已近似,msProf 实测后再决定是否建模差异 |
|
||||||
|
| T_cmd(DMA 命令固定开销) | 默认 0(未标定) | 源码无显式建模,但 stepK≤8、dValue 512B 等限制隐含"命令有代价"的工程经验 | **标定项**(§4.3),结果决定 IterBatch/MergeBatch 仲裁类收益是否成立 |
|
||||||
|
| 搬移效率 | eff=min(1, tile/16KB)(issue#36) | 源码用 16KB/64KB 硬编码阈值(fullCopySize=64KB、mmadCount=8) | 同源不同形,验证 tile 饱和点 |
|
||||||
|
| drain | 闭式(issue#37) | kernel unitflag=0,tile 级双缓冲排空 | 实测校准理论 drain |
|
||||||
|
| fp32 大 K | 不建模 | 核内 splitK(8192 阈值串行累加) | 理论补标注即可,不动源码 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 上板验证方案(Ascend950PR 主 bin / arch35)
|
||||||
|
|
||||||
|
### 4.1 环境与工具
|
||||||
|
|
||||||
|
- 芯片:Ascend950PR 主 bin(32 AIC / 64 AIV,HBM 1.6TB/s,L2 128MB);arch=DAV_3510。
|
||||||
|
- 入口:`aclnnBatchMatMul`(`examples/arch35/test_aclnn_batchmatmul.cpp` 为模板自建 harness),
|
||||||
|
dtype 以 bf16 为主测面,fp16/fp32 抽查。
|
||||||
|
- 测量:**msProf** 算子级时延(总时延 = case 执行完的墙钟)+ 核间时间线(尾轮空转可视化);
|
||||||
|
正确性用 `tests/assets/bmmv3_aclnn_golden.py` 对拍(bf16 容差按现有 ST 标准)。
|
||||||
|
- 对照:同一 case 跑**修改前源码 build**(基线)与**修改后 build**,各 ≥20 次取中位数;
|
||||||
|
tiling 日志(OP_LOGI/D)落盘确认实际命中策略与理论预测分支一致。
|
||||||
|
|
||||||
|
### 4.2 case 集设计(三层)
|
||||||
|
|
||||||
|
1. **定向差异 case**(每批次一张表,见 §2 各节"验证重点"):围绕每个差异点的进入/退出边界构造,
|
||||||
|
含边界两侧 ±1 档;
|
||||||
|
2. **理论 demo 全量**:本仓 `examples/cases_demo.csv` 44 例全跑,与理论 recommend 分支预测比对
|
||||||
|
"预测分支 == 实际命中策略"命中率(目标 100%,除已知口径差异清单);
|
||||||
|
3. **回归面**:源码自带 ST(`tests/st/arch35/ttk_*.csv` + atk json)全量,保证零回退;
|
||||||
|
另加理论压力回归同分布随机 case(seed7/6000 量级采样到可编译子集)抽查时延不回退。
|
||||||
|
|
||||||
|
### 4.3 标定项(先行,阻塞批次 3/4 的裁决)
|
||||||
|
|
||||||
|
| 标定 | 方法 | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| **T_cmd** | 固定 case,变 k_l1 分段数(命令数)扫 MTE2 时延,线性拟合斜率=带宽、截距/命令数=T_cmd | 决定 IterBatch 形态化与 Merge 仲裁的真实收益(批次 3/4 立项裁决) |
|
||||||
|
| 归约段落点 | StreamK case 扫 K,拆 reduce 段时延,对比 L2(5.2T)/GM(0.64~1.6T) 两口径模型 | 决定 θ_c' 取值与 workspace 策略(批次 1 内完成) |
|
||||||
|
| tile 效率曲线 | 变单命令 tile 大小(4/8/16/32KB)测有效带宽 | 验证 eff=min(1,tile/16KB) 模型与 16KB 饱和点(issue#36 口径上板确认) |
|
||||||
|
| drain 量级 | 单波 case 变 N_blk 测尾轮/排空暴露 | 校准理论 drain 闭式(unitflag=0 下) |
|
||||||
|
| fp32 Cube 算力 | fp32 大 K 计算 Bound case | 确认 ½ 假设(docs/05 §2 待标定项) |
|
||||||
|
|
||||||
|
### 4.4 通过标准(提 PR 门槛)
|
||||||
|
|
||||||
|
1. 定向差异 case:修改后总时延 **严格更短**(中位数,超过噪声带 ~2%);
|
||||||
|
2. 回归面:零精度问题、零 crash、时延回退 case 比例 = 0(个别 <2% 且可解释除外);
|
||||||
|
3. 理论预测分支与实际上板命中策略一致率 100%(不一致项必须有文档化解释);
|
||||||
|
4. tiling 耗时不显著增长(host 枚举量增加控制在毫秒级)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 实施批次与 PR 计划
|
||||||
|
|
||||||
|
| 批次 | 内容 | 改动文件(源码侧) | 依赖 | 风险 |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 0 | 标定(§4.3)+ 基线库建立(44 例 + 定向 case 的修改前时延基线) | 无(测试基建) | 板子/harness | 低 |
|
||||||
|
| 1 | StreamK 准入放宽 + grid_K 推导(§2.1) | `op_host/op_tiling/arch35/batch_matmul_v3_basic_streamk_tiling.cpp`(host-only) | 批次 0 归约口径 | 低(kernel 不动);注意 θ_c 边界 |
|
||||||
|
| 2 | ASW 方案B 尾轮(host-only)→ 视收益决定是否续作 A1b(kernel 第二套 tile,移植 aswt 范式) | `batch_matmul_v3_asw_basic_tiling.cpp` +(A1b 时)`asw_block_advanced.h` / `asw_kernel_advanced.h` | 批次 0 | 中(方案B 低;A1b 的 index 重映射需细致 review) |
|
||||||
|
| 3 | MergeBatch 准入补齐 + L1 绑定仲裁(§2.4) | `batch_matmul_v3_mergebatch_basicapi_tiling.cpp`(host-only) | T_cmd 标定 | 低 |
|
||||||
|
| 4 | IterBatch 形态补全(§2.3),含 unitflag/ drain 校准 | `iterbatch_basicapi_tiling.cpp` + cmct `block_mmad_iterbatch.h` / scheduler + tilingData | T_cmd 标定结果支持 | **高**(kernel 流水重写);可拆子 PR(先 host 形态 d → 再 kernel c/d) |
|
||||||
|
|
||||||
|
每个批次独立 PR:附理论文档链接(本仓 docs/02 对应分支文档 + 本文)、基线/修改后时延对照表、
|
||||||
|
ST/UT 通过证据、定向 case 清单。PR 顺序即批次顺序(host-only 先行,kernel 改动殿后)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 风险与开放问题
|
||||||
|
|
||||||
|
1. **T_cmd 若 ≈0**:批次 3 仲裁价值缩水为"防误捕获",批次 4 失去主要收益支撑 → 届时重新裁决
|
||||||
|
是否只做 host 侧准入/路由对齐。
|
||||||
|
2. **方案B 的流量放大**(tile 缩 1/√g → 搬入周长和放大)理论标注不入模型(issue#37 P4),
|
||||||
|
周长型主导角区可能吞掉尾轮收益——批次 2 必须含 GM/L2 流量实测。
|
||||||
|
3. **理论 ASW 降核口径**(P=L0C 满载粒度)与源码 usedCoreNum 口径不同,"理论落点分支预测"
|
||||||
|
在降核区需要对齐后再比对。
|
||||||
|
4. **Blaze/旧库栈在仓外**:StreamK 的 TENSOR 级与 IterBatch 高级版主体实现不在当前源码包内,
|
||||||
|
批次 1/4 若需触及要单独申请对应源码包;优先走仓内可读的 cmct 内置栈。
|
||||||
|
5. **理论模型自身的已知简化**(docs/05 §3 R6、issue#37 P3/P4)在上板数据回来后需要一轮
|
||||||
|
模型校准,再反哺后续批次的预期值。
|
||||||
|
6. **确定性等级**:StreamK 相关改动保持 deterministic_level>1 禁用不变;批次 1 PR 需带
|
||||||
|
level=0/1 的数值稳定性证据。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*版本:v1.0 | 2026-09-07 | 基于 BMM_Theory main(issue#40 Cube-only 口径)与
|
||||||
|
ops-nn batch_mat_mul_v3 arch35 源码快照比对;量化数字由 `bmm_theory` 测算脚本产出
|
||||||
|
(ASCEND950PR 规格,T_cmd=0 未标定假设),仅供排期参考。*
|
||||||
@@ -1,11 +1,45 @@
|
|||||||
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,deterministic_level
|
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,out_nd,deterministic_level
|
||||||
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,0,0,0,0
|
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,0,0,0,,0
|
||||||
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,0,0,0,0
|
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,0,0,0,,0
|
||||||
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,0,0,0,0
|
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,0,0,0,,0
|
||||||
merge_demo_k_trunc,2048,2048,32,32,256,bf16,bf16,bf16,0,0,0,0
|
merge_demo_k_trunc,2048,2048,16,64,128,bf16,bf16,bf16,0,0,0,,0
|
||||||
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,0,0,0,0
|
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,0,0,0,,0
|
||||||
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,0,0,0,0
|
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,0,0,0,,0
|
||||||
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,0,0,0,0
|
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,0,0,0,,0
|
||||||
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,0,0,0,0
|
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,0,0,0,,0
|
||||||
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,0,0,0,0
|
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,0,0,0,,0
|
||||||
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,0,0,0,0
|
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,0,0,0,,0
|
||||||
|
b4_m1_n128_k256,4,4,1,128,256,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m2_n192_k128,8,8,2,192,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b16_m4_n256_k192,16,16,4,256,192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m8_n128_k256,32,32,8,128,256,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m16_n256_k512,64,64,16,256,512,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b128_m8_n192_k256,128,128,8,192,256,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m16_n8192_k7168,32,32,16,8192,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b4_m1_n8192_k8192,4,4,1,8192,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b16_m2_n4096_k7168,16,16,2,4096,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m64_n64_k7168,32,32,64,64,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m1024_n1024_k7168,64,64,1024,1024,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b128_m2048_n2048_k1536,128,128,2048,2048,1536,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m1024_n8192_k2048,64,64,1024,8192,2048,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m1024_n1024_k512,32,32,1024,1024,512,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b128_m4096_n4096_k8192,128,128,4096,4096,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m8192_n4096_k7168,32,32,8192,4096,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m2048_n2048_k8192,32,32,2048,2048,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m4096_n2048_k128,64,64,4096,2048,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b16_m1024_n1024_k8192,16,16,1024,1024,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b4_m32768_n128_k128,4,4,32768,128,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m32768_n2048_k512,8,8,32768,2048,512,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b4_m131072_n128_k128,4,4,131072,128,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m131072_n1024_k256,8,8,131072,1024,256,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b16_m32768_n8192_k7168,16,16,32768,8192,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b4_m32768_n128_k8192,4,4,32768,128,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m131072_n8192_k128,32,32,131072,8192,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m32768_n8192_k1536,64,64,32768,8192,1536,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m131072_n8192_k8192,8,8,131072,8192,8192,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m16_n7168_k1536,8,8,16,7168,1536,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b64_m1024_n7168_k7168,64,64,1024,7168,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b32_m8192_n8192_k7168,32,32,8192,8192,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b8_m4096_n4096_k128,8,8,4096,4096,128,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
b128_m8192_n8192_k7168,128,128,8192,8192,7168,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
special_k1_b64,64,64,8192,512,1,bf16,bf16,bf16,0,0,0,1,0
|
||||||
|
|||||||
|
@@ -1,11 +1,45 @@
|
|||||||
case_id,branch,npu,op,used_core_num,split_b,m_cnt,n_cnt,grid_k,core_map,b_core,merge_b0,single_core_m,single_core_n,single_core_k,k_l1,b_l1,l1_form,base_m,base_n,base_k,l2_policy_in,l2_policy_out,swizzle_w,workspace_bytes,tail_strategy,tail_m_cnt,tail_n_cnt,tail_k_cnt,tail_m_main,tail_n_main,tail_block_cnt,tail_wave_num,fixpipe_unitflag,out_dtype_bytes,note
|
case_id,branch,npu,op,used_core_num,split_b,m_cnt,n_cnt,grid_k,core_map,b_core,merge_b0,single_core_m,single_core_n,single_core_k,k_l1,b_l1,l1_form,base_m,base_n,base_k,l2_policy_in,l2_policy_out,swizzle_w,workspace_bytes,tail_strategy,tail_m_cnt,tail_n_cnt,tail_k_cnt,tail_m_main,tail_n_main,tail_block_cnt,tail_wave_num,fixpipe_unitflag,out_dtype_bytes,note
|
||||||
to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split"
|
to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split"
|
||||||
special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核"
|
special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核"
|
||||||
special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB 乒乓"
|
special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) UB乒乓,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB乒乓 (B>=2*AIV 双batch乒乓流水)"
|
||||||
merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,128,128,256,256,8,合并驻留,128,128,128,allocate(GM->L1随路驻留L2),direct_gm,0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限19.0/b_core=64); K截断; 合并后单次DMA搬入 A'[128,256]+B'[256,128]"
|
merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,64,256,128,128,12,合并驻留,64,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限21.1/b_core=64); K截断; 合并后单次DMA搬入 A'[64,128]+B'[128,256]; 输出落点: L2驻留 (整case V_in+V_out=40.0MB vs L2=128MB)"
|
||||||
merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1
|
merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1; 输出落点: L2驻留
|
||||||
iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1
|
iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1; 输出落点: L2驻留
|
||||||
iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝"
|
iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: 直写GM"
|
||||||
streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终"
|
streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终"
|
||||||
asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,47,47,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,176,176,1024,256,1,双缓冲驻留当前tile输入,176,176,80,allocate(输入驻留L2吸收重复读),direct_gm(输出直写GM不占L2),4,0,方案B,52,52,1,52,52,2,139,True,2,"L2场景B_输入驻留输出直写GM, r_in=1.00; 尾轮: 周长型主导, rho=0.06<rho_dv=0.53, A1b被dValue卡死, 方案B反超"
|
asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1024,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=16, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入512.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=64.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,direct_gm,0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)"
|
asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)"
|
||||||
|
b4_m1_n128_k256,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,1,128,256,256,1,标准核内流水,1,128,128,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.01<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)"
|
||||||
|
b8_m2_n192_k128,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,2,192,128,128,1,标准核内流水,2,192,80,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.05<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)"
|
||||||
|
b16_m4_n256_k192,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,4,256,192,192,1,标准核内流水,4,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.25<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)"
|
||||||
|
b32_m8_n128_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,8,128,256,256,1,a_单batch全驻留,8,128,128,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,单batch全驻留: (MK+KN)*dtype=68KB <= L1; 输出落点: L2驻留
|
||||||
|
b64_m16_n256_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,16,256,512,240,1,c_一侧驻留+对侧切K,16,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"一侧驻留(A)+对侧切K: A驻留16KB, 预算L1/2, k_L1=240, dValueA=480B/dValueB=512B; 输出落点: L2驻留"
|
||||||
|
b128_m8_n192_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,8,192,256,256,2,b_双batch乒乓,8,192,80,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=200KB <= L1; 输出落点: L2驻留
|
||||||
|
b32_m16_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,1,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,16,1024,7168,112,1,双缓冲驻留当前tile输入,16,1024,16,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: 效率降级(放开约束4, dValue 按 128B 硬下限, 搬移效率低于模型假设, 时延可能低估): P=1, mCnt=1 x nCnt=8 (tile 16x1024, 每batch搬入113.8MB, r=0); Base tile 16x1024 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=3591.0MB, V_out=8.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b4_m1_n8192_k8192,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,1,8192,256,256,1,K段标准分块流水,1,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,4194304,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=0.50, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终"
|
||||||
|
b16_m2_n4096_k7168,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,16,"B/M/N切出16块, 每块16核切K归约 (归约组内核c负责K段[c*K/16,(c+1)*K/16))",1,1,2,4096,448,256,1,K段标准分块流水,2,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=16路切K+归约,1,1,16,0,0,0,0,True,4,"P=2.00, grid_K=16, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终"
|
||||||
|
b32_m64_n64_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,64,64,7168,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: L2驻留"
|
||||||
|
b64_m1024_n1024_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,1024,1024,7168,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: 直写GM"
|
||||||
|
b128_m2048_n2048_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入48.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1536.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b64_m1024_n8192_k2048,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,2048,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=16 (tile 512x512, 每batch搬入128.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2304.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b32_m1024_n1024_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,1024,1024,512,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: L2驻留"
|
||||||
|
b128_m4096_n4096_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16384.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b32_m8192_n4096_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=8 (tile 512x512, 每batch搬入1792.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=5376.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b32_m2048_n2048_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2048.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b64_m4096_n2048_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=4 (tile 512x512, 每batch搬入8.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=96.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b16_m1024_n1024_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=2 x nCnt=2 (tile 512x512, 每batch搬入64.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=512.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b4_m32768_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"resident(整case全驻留S_A: 输出驻留L2异步回写, GM写=0)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入10.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: A_整case全驻留(输入+输出<=L2) (V_in=32.1MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b8_m32768_n2048_k512,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,512,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=64 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=272.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b4_m131072_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,32,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=256 x nCnt=1 (tile 512x128, 每batch搬入40.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=128.1MB, V_out=128.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b8_m131072_n1024_k256,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,256,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=2 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=516.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b16_m32768_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,512,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入14336.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=8960.0MB, V_out=8192.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b4_m32768_n128_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,8192,192,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入640.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2056.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b32_m131072_n8192_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,4096,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1088.0MB, V_out=65536.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b64_m32768_n8192_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2048,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入3072.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7680.0MB, V_out=32768.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b8_m131072_n8192_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入65536.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: C_双侧超L2: 最小替换2D分组(组间落空GM, 窗口L2) (V_in=17408.0MB, V_out=16384.0MB, L2=128MB); 最小替换分组 m_grp=8x n_grp=8 (GM倍率3.76, 窗口L2/batch=57344.0MB); 尾轮: r=0 无尾轮"
|
||||||
|
b8_m16_n7168_k1536,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,2,2,"B/M/N切出16块, 每块2核切K归约 (归约组内核c负责K段[c*K/2,(c+1)*K/2))",1,1,16,3584,768,256,1,K段标准分块流水,16,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,3670016,grid_K=2路切K+归约,1,1,2,0,0,0,0,True,4,"P=14.00, grid_K=2, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终"
|
||||||
|
b64_m1024_n7168_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,14,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,56,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=14 (tile 512x512, 每batch搬入392.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=896.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b32_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b8_m4096_n4096_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入16.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
b128_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=28672.0MB, V_out=16384.0MB, L2=128MB); 尾轮: r=0 无尾轮"
|
||||||
|
special_k1_b64,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) AIV单缓冲,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, AIV单缓冲 (B<2*AIV 逐batch单缓冲串行)"
|
||||||
|
|||||||
|
@@ -1,11 +1,45 @@
|
|||||||
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,out_nd,deterministic_level,plan_case_id,plan_branch,plan_npu,plan_op,plan_used_core_num,plan_split_b,plan_m_cnt,plan_n_cnt,plan_grid_k,plan_core_map,plan_b_core,plan_merge_b0,plan_single_core_m,plan_single_core_n,plan_single_core_k,plan_k_l1,plan_b_l1,plan_l1_form,plan_base_m,plan_base_n,plan_base_k,plan_l2_policy_in,plan_l2_policy_out,plan_swizzle_w,plan_workspace_bytes,plan_tail_strategy,plan_tail_m_cnt,plan_tail_n_cnt,plan_tail_k_cnt,plan_tail_m_main,plan_tail_n_main,plan_tail_block_cnt,plan_tail_wave_num,plan_fixpipe_unitflag,plan_out_dtype_bytes,plan_note,gm_read_bytes,l2_read_bytes,t_mte2_gm,t_mte2_l2,t_mte2,dma_cmd_count,t_dma_cmd,cube_flops,t_mmad,fixpipe_bytes,t_fixpipe,t_reduce,t_steady,t_drain,t_total,bottleneck,feasible,violations,bound_type,advice
|
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,out_nd,deterministic_level,plan_case_id,plan_branch,plan_npu,plan_op,plan_used_core_num,plan_split_b,plan_m_cnt,plan_n_cnt,plan_grid_k,plan_core_map,plan_b_core,plan_merge_b0,plan_single_core_m,plan_single_core_n,plan_single_core_k,plan_k_l1,plan_b_l1,plan_l1_form,plan_base_m,plan_base_n,plan_base_k,plan_l2_policy_in,plan_l2_policy_out,plan_swizzle_w,plan_workspace_bytes,plan_tail_strategy,plan_tail_m_cnt,plan_tail_n_cnt,plan_tail_k_cnt,plan_tail_m_main,plan_tail_n_main,plan_tail_block_cnt,plan_tail_wave_num,plan_fixpipe_unitflag,plan_out_dtype_bytes,plan_note,gm_read_bytes,l2_read_bytes,t_mte2_gm,t_mte2_l2,t_mte2,dma_cmd_count,t_dma_cmd,cube_flops,t_mmad,fixpipe_bytes,t_fixpipe,t_reduce,t_steady,t_drain,t_total,bottleneck,feasible,violations,bound_type,advice
|
||||||
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,False,False,False,True,0,to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split",16777216,0.0,1.048576e-05,0.0,1.048576e-05,0.0,0.0,17179869184.0,3.534952506995885e-05,8388608,5.24288e-06,0.0,3.534952506995885e-05,0.0,3.534952506995885e-05,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,False,False,False,True,0,to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split",16777216,0.0,1.048576e-05,0.0,1.048576e-05,0.0,0.0,17179869184.0,3.97682157037037e-05,8388608,1.6131938461538462e-06,0.0,3.97682157037037e-05,0.0,3.97682157037037e-05,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,False,False,False,True,0,special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核",0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,16777216,1.048576e-05,0.0,1.048576e-05,0.0,1.048576e-05,FIXPIPE,True,,写出Bound,"瓶颈在 Fixpipe 写出: 检查输出 dtype (fp16/fp8 可减半写出量), 或评估输出驻留 L2 异步回写策略"
|
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,False,False,False,True,0,special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核",0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,16777216,3.2263876923076923e-06,0.0,3.2263876923076923e-06,0.0,3.2263876923076923e-06,FIXPIPE,True,,写出Bound(L2写口),"瓶颈在 Fixpipe 写出: 检查输出 dtype (fp16/fp8 可减半写出量), 或评估输出驻留 L2 异步回写策略"
|
||||||
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) UB乒乓,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB乒乓 (B>=2*AIV 双batch乒乓流水)",131072,0.0,8.192e-08,0.0,8.192e-08,0.0,0.0,8388608.0,6.206060606060606e-07,16777216,1.048576e-05,0.0,1.048576e-05,0.0,1.048576e-05,FIXPIPE,True,,写出Bound,"瓶颈在 Fixpipe 写出: 检查输出 dtype (fp16/fp8 可减半写出量), 或评估输出驻留 L2 异步回写策略"
|
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) UB乒乓,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB乒乓 (B>=2*AIV 双batch乒乓流水)",131072,0.0,8.192e-08,0.0,8.192e-08,0.0,0.0,8388608.0,3.103030303030303e-07,16777216,3.2263876923076923e-06,0.0,3.2263876923076923e-06,0.0,3.2263876923076923e-06,FIXPIPE,True,,写出Bound(L2写口),"瓶颈在 Fixpipe 写出: 检查输出 dtype (fp16/fp8 可减半写出量), 或评估输出驻留 L2 异步回写策略"
|
||||||
merge_demo_k_trunc,2048,2048,32,32,256,bf16,bf16,bf16,False,False,False,True,0,merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,128,128,256,256,8,合并驻留,128,128,128,allocate(GM->L1随路驻留L2),direct_gm,0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限19.0/b_core=64); K截断; 合并后单次DMA搬入 A'[128,256]+B'[256,128]",2097152,0.0,4.194304e-05,0.0,4.2743039999999997e-05,16.0,8.000000000000001e-07,134217728.0,8.837381267489712e-06,131072,2.62144e-06,0.0,4.2743039999999997e-05,3.0192408230452674e-07,4.3044964082304525e-05,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
merge_demo_k_trunc,2048,2048,16,64,128,bf16,bf16,bf16,False,False,False,True,0,merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,64,256,128,128,12,合并驻留,64,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限21.1/b_core=64); K截断; 合并后单次DMA搬入 A'[64,128]+B'[128,256]; 输出落点: L2驻留 (整case V_in+V_out=40.0MB vs L2=128MB)",41943040,0.0,2.62144e-05,0.0,2.62144e-05,16,0.0,2147483648.0,4.971026962962963e-06,4194304,8.065969230769231e-07,0.0,2.62144e-05,1.2808460398860398e-07,2.6342484603988603e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,False,False,False,True,0,merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1,524288,0.0,1.048576e-05,0.0,1.0685759999999999e-05,4,2.0000000000000002e-07,16777216.0,1.104672658436214e-06,32768,6.5536e-07,0.0,1.0685759999999999e-05,4.400081646090535e-07,1.1125768164609053e-05,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,False,False,False,True,0,merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1; 输出落点: L2驻留,16777216,0.0,1.048576e-05,0.0,1.048576e-05,4,0.0,536870912.0,1.2427567407407407e-06,1048576,2.0164923076923077e-07,0.0,1.048576e-05,3.6110149287749287e-07,1.0846861492877492e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1,262144,0.0,5.24288e-06,0.0,5.4428799999999995e-06,4,2.0000000000000002e-07,8388608.0,5.52336329218107e-07,32768,6.5536e-07,0.0,5.4428799999999995e-06,3.0192408230452674e-07,5.744804082304526e-06,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1; 输出落点: L2驻留,8388608,0.0,5.24288e-06,0.0,5.24288e-06,4,0.0,268435456.0,6.213783703703703e-07,1048576,2.0164923076923077e-07,0.0,5.24288e-06,2.0575690028490026e-07,5.4486369002849e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝",4194304,0.0,8.388608e-05,0.0,8.468608e-05,16,8.000000000000001e-07,134217728.0,8.837381267489712e-06,16384,3.2768e-07,0.0,8.468608e-05,7.16176329218107e-07,8.540225632921811e-05,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: 直写GM",134217728,0.0,8.388608e-05,0.0,8.388608e-05,16,0.0,4294967296.0,9.942053925925925e-06,524288,3.2768e-07,0.0,8.421376e-05,7.852183703703703e-07,8.499897837037037e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,False,False,False,True,0,streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",327680.0,0.0,6.5536e-06,0.0,6.5536e-06,0.0,0.0,41943040.0,2.761681646090535e-06,0.0,0.0,3.4067453613053613e-06,6.5536e-06,3.4067453613053613e-06,9.960345361305361e-06,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,False,False,False,True,0,streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",20971520,0,1.31072e-05,0.0,1.31072e-05,0.0,0.0,1342177280.0,3.1068918518518518e-06,0.0,0.0,3.4067453613053613e-06,1.31072e-05,3.4067453613053613e-06,1.651394536130536e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,False,False,False,True,0,asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,47,47,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,176,176,1024,256,1,双缓冲驻留当前tile输入,176,176,80,allocate(输入驻留L2吸收重复读),direct_gm(输出直写GM不占L2),4,0,方案B,52,52,1,52,52,2,139,True,2,"L2场景B_输入驻留输出直写GM, r_in=1.00; 尾轮: 周长型主导, rho=0.06<rho_dv=0.53, A1b被dValue卡死, 方案B反超",67108864.0,0.0,4.194304e-05,0.0,4.194304e-05,0.0,0.0,274877906944.0,0.0005655924011193416,268435456,0.00016777216,0.0,0.0005655924011193416,0.0,0.0005655924011193416,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,False,False,False,True,0,asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1024,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=16, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入512.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=64.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",67108864,1006632960,4.194304e-05,0.00019358326153846154,0.00023552630153846153,0.0,0.0,274877906944.0,0.0006362914512592592,268435456,0.00016777216,0.0,0.0006362914512592592,0.0,0.0006362914512592592,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,False,False,False,True,0,asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,direct_gm,0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",2097152.0,0.0,2.62144e-06,0.0,2.62144e-06,0.0,0.0,268435456.0,1.104672658436214e-06,2097152,2.62144e-06,0.0,2.62144e-06,0.0,2.62144e-06,MTE2_GM,True,,访存Bound(GM),"瓶颈在 GM 搬入: 可考虑增大 tile 提升 dValue/单核搬移量, 或利用 L2 驻留吸收重复读 (MergeBatch/ASW swizzle 方向)"
|
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,False,False,False,True,0,asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",2097152,0,2.62144e-06,0.0,2.62144e-06,0.0,0.0,268435456.0,1.2427567407407407e-06,2097152,8.065969230769231e-07,0.0,2.62144e-06,0.0,2.62144e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b4_m1_n128_k256,4,4,1,128,256,bf16,bf16,bf16,False,False,False,True,0,b4_m1_n128_k256,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,1,128,256,256,1,标准核内流水,1,128,128,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.01<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",264192,0,5.28384e-06,0.0,5.28384e-06,0.0,0.0,262144.0,1.9418074074074073e-08,1024,6.3015384615384615e-09,0.0,5.28384e-06,0.0,5.28384e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b8_m2_n192_k128,8,8,2,192,128,bf16,bf16,bf16,False,False,False,True,0,b8_m2_n192_k128,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,2,192,128,128,1,标准核内流水,2,192,80,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.05<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",397312,0,7.94624e-06,0.0,7.94624e-06,0.0,0.0,786432.0,5.825422222222222e-08,6144,3.7809230769230766e-08,0.0,7.94624e-06,0.0,7.94624e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b16_m4_n256_k192,16,16,4,256,192,bf16,bf16,bf16,False,False,False,True,0,b16_m4_n256_k192,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,4,256,192,192,1,标准核内流水,4,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.25<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",1597440,0,3.19488e-05,0.0,3.19488e-05,0.0,0.0,6291456.0,4.660337777777778e-07,32768,2.0164923076923077e-07,0.0,3.19488e-05,0.0,3.19488e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b32_m8_n128_k256,32,32,8,128,256,bf16,bf16,bf16,False,False,False,True,0,b32_m8_n128_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,8,128,256,256,1,a_单batch全驻留,8,128,128,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,单batch全驻留: (MK+KN)*dtype=68KB <= L1; 输出落点: L2驻留,2228224,0.0,1.6384e-06,0.0,1.6384e-06,1,0.0,16777216.0,3.8836148148148146e-08,65536,1.2603076923076923e-08,0.0,1.6384e-06,5.1439225071225065e-08,1.689839225071225e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b64_m16_n256_k512,64,64,16,256,512,bf16,bf16,bf16,False,False,False,True,0,b64_m16_n256_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,16,256,512,240,1,c_一侧驻留+对侧切K,16,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"一侧驻留(A)+对侧切K: A驻留16KB, 预算L1/2, k_L1=240, dValueA=480B/dValueB=512B; 输出落点: L2驻留",17825792,0.0,1.114112e-05,0.0,1.114112e-05,6,0.0,268435456.0,6.213783703703703e-07,524288,1.0082461538461538e-07,0.0,1.114112e-05,1.9604786324786327e-07,1.1337167863247863e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b128_m8_n192_k256,128,128,8,192,256,bf16,bf16,bf16,False,False,False,True,0,b128_m8_n192_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,8,192,256,256,2,b_双batch乒乓,8,192,80,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=200KB <= L1; 输出落点: L2驻留,13107200,0.0,9.17504e-06,0.0,9.17504e-06,4,0.0,100663296.0,2.330168888888889e-07,393216,7.561846153846153e-08,0.0,9.17504e-06,7.71588376068376e-08,9.252198837606838e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b32_m16_n8192_k7168,32,32,16,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m16_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,1,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,16,1024,7168,112,1,双缓冲驻留当前tile输入,16,1024,16,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: 效率降级(放开约束4, dValue 按 128B 硬下限, 搬移效率低于模型假设, 时延可能低估): P=1, mCnt=1 x nCnt=8 (tile 16x1024, 每batch搬入113.8MB, r=0); Base tile 16x1024 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=3591.0MB, V_out=8.0MB, L2=128MB); 尾轮: r=0 无尾轮",3765436416,51380224,0.00235339776,9.880812307692307e-06,0.0023632785723076925,0.0,0.0,60129542144.0,0.00013918875496296296,8388608,5.24288e-06,0.0,0.0023685214523076923,0.0,0.0023685214523076923,MTE2,True,,访存Bound(GM读写共享+L2重复读),"效率降级标注 (plan.note): 搬移效率下限不满足 —— 方案照常给出 (兜底), 但实际效率低于模型假设, 时延可能低估; 建议调整 dtype/布局 | 瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b4_m1_n8192_k8192,4,4,1,8192,8192,bf16,bf16,bf16,False,False,False,True,0,b4_m1_n8192_k8192,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,1,8192,256,256,1,K段标准分块流水,1,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,4194304,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=0.50, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",536936448,0.0,0.00033558528,0.0,0.00033558528,0.0,0.0,536870912.0,1.2427567407407407e-06,0.0,0.0,1.731729603729604e-06,0.00033558528,1.731729603729604e-06,0.0003373170096037296,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b16_m2_n4096_k7168,16,16,2,4096,7168,bf16,bf16,bf16,False,False,False,True,0,b16_m2_n4096_k7168,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,16,"B/M/N切出16块, 每块16核切K归约 (归约组内核c负责K段[c*K/16,(c+1)*K/16))",1,1,2,4096,448,256,1,K段标准分块流水,2,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=16路切K+归约,1,1,16,0,0,0,0,True,4,"P=2.00, grid_K=16, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",939982848,0,0.00058748928,0.0,0.00058748928,0.0,0.0,1879048192.0,4.349648592592593e-06,0.0,0.0,1.7726896037296038e-06,0.00058748928,1.7726896037296038e-06,0.0005892619696037297,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b32_m64_n64_k7168,32,32,64,64,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m64_n64_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,64,64,7168,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: L2驻留",58720256,0.0,3.670016e-05,0.0,3.670016e-05,7,0.0,1879048192.0,4.349648592592593e-06,262144,5.041230769230769e-08,0.0,3.670016e-05,6.71790678062678e-07,3.737195067806268e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b64_m1024_n1024_k7168,64,64,1024,1024,7168,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n1024_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,1024,1024,7168,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: 直写GM",1879048192,0.0,0.00117440512,0.0,0.00117440512,224,0.0,962072674304.0,0.0022270200794074074,134217728,8.388608e-05,0.0,0.0022270200794074074,5.1885093925925924e-05,0.0022789051733333333,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b128_m2048_n2048_k1536,128,128,2048,2048,1536,bf16,bf16,bf16,False,False,False,True,0,b128_m2048_n2048_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入48.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1536.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",1610612736,4831838208,0.00100663296,0.0009291996553846154,0.0019358326153846154,0.0,0.0,1649267441664.0,0.0038177487075555555,1073741824,0.00067108864,0.0,0.0038177487075555555,0.0,0.0038177487075555555,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b64_m1024_n8192_k2048,64,64,1024,8192,2048,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n8192_k2048,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,2048,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=16 (tile 512x512, 每batch搬入128.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2304.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",2415919104,6174015488,0.00150994944,0.0011873106707692308,0.0026972601107692305,0.0,0.0,2199023255552.0,0.005090331610074074,1073741824,0.00067108864,0.0,0.005090331610074074,0.0,0.005090331610074074,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b32_m1024_n1024_k512,32,32,1024,1024,512,bf16,bf16,bf16,False,False,False,True,0,b32_m1024_n1024_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,1024,1024,512,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: L2驻留",67108864,0.0,4.194304e-05,0.0,4.194304e-05,8,0.0,34359738368.0,7.95364314074074e-05,67108864,1.290555076923077e-05,0.0,7.95364314074074e-05,2.2847604695156693e-05,0.0001023840361025641,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b128_m4096_n4096_k8192,128,128,4096,4096,8192,bf16,bf16,bf16,False,False,False,True,0,b128_m4096_n4096_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16384.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮",17179869184,120259084288,0.01073741824,0.023126746978461538,0.033864165218461535,0.0,0.0,35184372088832.0,0.08144530576118518,4294967296,0.00268435456,0.0,0.08144530576118518,0.0,0.08144530576118518,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b32_m8192_n4096_k7168,32,32,8192,4096,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m8192_n4096_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=8 (tile 512x512, 每batch搬入1792.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=5376.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮",5637144576,54492397568,0.00352321536,0.010479307224615384,0.014002522584615384,0.0,0.0,15393162788864.0,0.03563232127051852,2147483648,0.00134217728,0.0,0.03563232127051852,0.0,0.03563232127051852,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b32_m2048_n2048_k8192,32,32,2048,2048,8192,bf16,bf16,bf16,False,False,False,True,0,b32_m2048_n2048_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2048.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",2147483648,6442450944,0.00134217728,0.0012389328738461537,0.002581110153846154,0.0,0.0,2199023255552.0,0.005090331610074074,268435456,0.00016777216,0.0,0.005090331610074074,0.0,0.005090331610074074,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b64_m4096_n2048_k128,64,64,4096,2048,128,bf16,bf16,bf16,False,False,False,True,0,b64_m4096_n2048_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=4 (tile 512x512, 每batch搬入8.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=96.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",100663296,436207616,6.291456e-05,8.388608e-05,0.00014680063999999998,0.0,0.0,137438953472.0,0.0003181457256296296,1073741824,0.00067108864,0.0,0.00081788928,0.0,0.00081788928,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b16_m1024_n1024_k8192,16,16,1024,1024,8192,bf16,bf16,bf16,False,False,False,True,0,b16_m1024_n1024_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=2 x nCnt=2 (tile 512x512, 每batch搬入64.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=512.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",536870912,536870912,0.00033554432,0.00010324440615384615,0.0004387887261538461,0.0,0.0,274877906944.0,0.0006362914512592592,33554432,2.097152e-05,0.0,0.0006362914512592592,0.0,0.0006362914512592592,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b4_m32768_n128_k128,4,4,32768,128,128,bf16,bf16,bf16,False,False,False,True,0,b4_m32768_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"resident(整case全驻留S_A: 输出驻留L2异步回写, GM写=0)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入10.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: A_整case全驻留(输入+输出<=L2) (V_in=32.1MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",33685504,8257536,2.105344e-05,1.5879876923076922e-06,2.2641427692307692e-05,0.0,0.0,4294967296.0,9.942053925925925e-06,33554432,6.452775384615385e-06,0.0,2.2641427692307692e-05,0.0,2.2641427692307692e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b8_m32768_n2048_k512,8,8,32768,2048,512,bf16,bf16,bf16,False,False,False,True,0,b8_m32768_n2048_k512,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,512,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=64 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=272.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",285212672,1862270976,0.00017825792,0.00035812903384615385,0.0005363869538461539,0.0,0.0,549755813888.0,0.0012725829025185184,1073741824,0.00067108864,0.0,0.0012725829025185184,0.0,0.0012725829025185184,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b4_m131072_n128_k128,4,4,131072,128,128,bf16,bf16,bf16,False,False,False,True,0,b4_m131072_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,32,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=256 x nCnt=1 (tile 512x128, 每batch搬入40.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=128.1MB, V_out=128.0MB, L2=128MB); 尾轮: r=0 无尾轮",134348800,33423360,8.3968e-05,6.427569230769231e-06,9.039556923076924e-05,0.0,0.0,17179869184.0,3.97682157037037e-05,134217728,8.388608e-05,0.0,0.00017428164923076922,0.0,0.00017428164923076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b8_m131072_n1024_k256,8,8,131072,1024,256,bf16,bf16,bf16,False,False,False,True,0,b8_m131072_n1024_k256,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,256,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=2 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=516.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮",541065216,1606418432,0.00033816576,0.00030892662153846154,0.0006470923815384616,0.0,0.0,549755813888.0,0.0012725829025185184,2147483648,0.00134217728,0.0,0.0019892696615384617,0.0,0.0019892696615384617,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b16_m32768_n8192_k7168,16,16,32768,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b16_m32768_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,512,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入14336.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=8960.0MB, V_out=8192.0MB, L2=128MB); 尾轮: r=0 无尾轮",9395240960,231122927616,0.0058720256,0.044446716849230766,0.05031874244923076,0.0,0.0,61572651155456.0,0.14252928508207408,8589934592,0.00536870912,0.0,0.14252928508207408,0.0,0.14252928508207408,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b4_m32768_n128_k8192,4,4,32768,128,8192,bf16,bf16,bf16,False,False,False,True,0,b4_m32768_n128_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,8192,192,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入640.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2056.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",2155872256,528482304,0.00134742016,0.0001016312123076923,0.0014490513723076923,0.0,0.0,274877906944.0,0.0006362914512592592,33554432,2.097152e-05,0.0,0.0014700228923076922,0.0,0.0014700228923076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b32_m131072_n8192_k128,32,32,131072,8192,128,bf16,bf16,bf16,False,False,False,True,0,b32_m131072_n8192_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,4096,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1088.0MB, V_out=65536.0MB, L2=128MB); 尾轮: r=0 无尾轮",1140850688,33218887680,0.00071303168,0.006388247630769231,0.007101279310769231,0.0,0.0,8796093022208.0,0.020361326440296295,68719476736,0.04294967296,0.0,0.05005095227076922,0.0,0.05005095227076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b64_m32768_n8192_k1536,64,64,32768,8192,1536,bf16,bf16,bf16,False,False,False,True,0,b64_m32768_n8192_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2048,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入3072.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7680.0MB, V_out=32768.0MB, L2=128MB); 尾轮: r=0 无尾轮",8053063680,198105366528,0.0050331648,0.03809718587076923,0.04313035067076923,0.0,0.0,52776558133248.0,0.12216795864177778,34359738368,0.02147483648,0.0,0.12216795864177778,0.0,0.12216795864177778,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b8_m131072_n8192_k8192,8,8,131072,8192,8192,bf16,bf16,bf16,False,False,False,True,0,b8_m131072_n8192_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入65536.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: C_双侧超L2: 最小替换2D分组(组间落空GM, 窗口L2) (V_in=17408.0MB, V_out=16384.0MB, L2=128MB); 最小替换分组 m_grp=8x n_grp=8 (GM倍率3.76, 窗口L2/batch=57344.0MB); 尾轮: r=0 无尾轮",68719476736,481036337152,0.04294967296,0.09250698791384615,0.13545666087384614,0.0,0.0,140737488355328.0,0.3257812230447407,17179869184,0.01073741824,0.0,0.3257812230447407,0.0,0.3257812230447407,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b8_m16_n7168_k1536,8,8,16,7168,1536,bf16,bf16,bf16,False,False,False,True,0,b8_m16_n7168_k1536,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,2,2,"B/M/N切出16块, 每块2核切K归约 (归约组内核c负责K段[c*K/2,(c+1)*K/2))",1,1,16,3584,768,256,1,K段标准分块流水,16,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,3670016,grid_K=2路切K+归约,1,1,2,0,0,0,0,True,4,"P=14.00, grid_K=2, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",176553984,393216,0.00011034624,7.561846153846153e-08,0.00011042185846153846,0.0,0.0,2818572288.0,6.524472888888889e-06,0.0,0.0,2.566079254079254e-07,0.00011042185846153846,2.566079254079254e-07,0.00011067846638694638,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b64_m1024_n7168_k7168,64,64,1024,7168,7168,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n7168_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,14,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,56,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=14 (tile 512x512, 每batch搬入392.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=896.0MB, L2=128MB); 尾轮: r=0 无尾轮",7516192768,18790481920,0.00469762048,0.0036135542153846152,0.008311174695384616,0.0,0.0,6734508720128.0,0.015589140555851852,939524096,0.00058720256,0.0,0.015589140555851852,0.0,0.015589140555851852,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b32_m8192_n8192_k7168,32,32,8192,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮",7516192768,112742891520,0.00469762048,0.021681325292307693,0.026378945772307694,0.0,0.0,30786325577728.0,0.07126464254103704,4294967296,0.00268435456,0.0,0.07126464254103704,0.0,0.07126464254103704,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
b8_m4096_n4096_k128,8,8,4096,4096,128,bf16,bf16,bf16,False,False,False,True,0,b8_m4096_n4096_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入16.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",16777216,117440512,1.048576e-05,2.2584713846153845e-05,3.307047384615384e-05,0.0,0.0,34359738368.0,7.95364314074074e-05,268435456,0.00016777216,0.0,0.00020084263384615383,0.0,0.00020084263384615383,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
b128_m8192_n8192_k7168,128,128,8192,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b128_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=28672.0MB, V_out=16384.0MB, L2=128MB); 尾轮: r=0 无尾轮",30064771072,450971566080,0.01879048192,0.08672530116923077,0.10551578308923078,0.0,0.0,123145302310912.0,0.28505857016414815,17179869184,0.01073741824,0.0,0.28505857016414815,0.0,0.28505857016414815,MMAD,True,,计算Bound,"瓶颈在 Cube 计算: 已接近理论算力上限, 检查是否有冗余计算 (MergeBatch 交叉项) 可消除"
|
||||||
|
special_k1_b64,64,64,8192,512,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_b64,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) AIV单缓冲,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, AIV单缓冲 (B<2*AIV 逐batch单缓冲串行)",1114112,0.0,6.9632e-07,0.0,6.9632e-07,0.0,0.0,268435456.0,9.92969696969697e-06,536870912,0.00033554432,0.0,0.00033624063999999996,0.0,0.00033624063999999996,MTE2,True,,访存Bound(GM读写共享+L2重复读),"瓶颈在 MTE2 搬移链 (GM 读写共享总线 + L2 重复读): 可增大 tile 提升 dValue/单核搬移量、利用 L2 驻留吸收重复读 (ASW swizzle/分组方向), 或评估输出驻留 L2 以减少 GM 直写与读竞争"
|
||||||
|
|||||||
|
@@ -1,11 +1,45 @@
|
|||||||
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,out_nd,deterministic_level,plan_case_id,plan_branch,plan_npu,plan_op,plan_used_core_num,plan_split_b,plan_m_cnt,plan_n_cnt,plan_grid_k,plan_core_map,plan_b_core,plan_merge_b0,plan_single_core_m,plan_single_core_n,plan_single_core_k,plan_k_l1,plan_b_l1,plan_l1_form,plan_base_m,plan_base_n,plan_base_k,plan_l2_policy_in,plan_l2_policy_out,plan_swizzle_w,plan_workspace_bytes,plan_tail_strategy,plan_tail_m_cnt,plan_tail_n_cnt,plan_tail_k_cnt,plan_tail_m_main,plan_tail_n_main,plan_tail_block_cnt,plan_tail_wave_num,plan_fixpipe_unitflag,plan_out_dtype_bytes,plan_note,gm_read_bytes,l2_read_bytes,t_mte2_gm,t_mte2_l2,t_mte2,dma_cmd_count,t_dma_cmd,cube_flops,t_mmad,fixpipe_bytes,t_fixpipe,t_reduce,t_steady,t_drain,t_total,bottleneck,feasible,violations,bound_type,advice
|
case_id,batch_a,batch_b,m,n,k,dtype_a,dtype_b,dtype_c,trans_a,trans_b,has_bias,out_nd,deterministic_level,plan_case_id,plan_branch,plan_npu,plan_op,plan_used_core_num,plan_split_b,plan_m_cnt,plan_n_cnt,plan_grid_k,plan_core_map,plan_b_core,plan_merge_b0,plan_single_core_m,plan_single_core_n,plan_single_core_k,plan_k_l1,plan_b_l1,plan_l1_form,plan_base_m,plan_base_n,plan_base_k,plan_l2_policy_in,plan_l2_policy_out,plan_swizzle_w,plan_workspace_bytes,plan_tail_strategy,plan_tail_m_cnt,plan_tail_n_cnt,plan_tail_k_cnt,plan_tail_m_main,plan_tail_n_main,plan_tail_block_cnt,plan_tail_wave_num,plan_fixpipe_unitflag,plan_out_dtype_bytes,plan_note,gm_read_bytes,l2_read_bytes,t_mte2_gm,t_mte2_l2,t_mte2,dma_cmd_count,t_dma_cmd,cube_flops,t_mmad,fixpipe_bytes,t_fixpipe,t_reduce,t_steady,t_drain,t_total,bottleneck,feasible,violations,bound_type,advice
|
||||||
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,False,False,False,True,0,to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split",16777216,0.0,1.048576e-05,0.0,1.048576e-05,0.0,0.0,17179869184.0,3.534952506995885e-05,8388608,5.24288e-06,0.0,3.534952506995885e-05,0.0,3.534952506995885e-05,MMAD,True,,计算Bound,"BatchA=1或BatchB=1, 折叠转普通Matmul"
|
to_matmul_demo,1,1,2048,2048,2048,bf16,bf16,bf16,False,False,False,True,0,to_matmul_demo,转Matmul,Ascend950PR,batch_mat_mul_v3,32,1,0,0,1,"折叠为 Matmul [2048,2048]x[2048,2048], 复用 Matmul 切分体系",0,1,0,0,2048,0,1,,0,0,0,,,0,0,转Matmul后由 Matmul 体系决定,1,1,1,0,0,0,0,True,2,"BatchB=1免费折叠: 左矩阵 [1,2048,2048] 视图折叠为 [2048,2048], 零重排零 split",16777216,0.0,1.048576e-05,0.0,1.048576e-05,0.0,0.0,17179869184.0,3.97682157037037e-05,8388608,1.6131938461538462e-06,0.0,3.97682157037037e-05,0.0,3.97682157037037e-05,MMAD,True,,计算Bound,"BatchA=1或BatchB=1, 折叠转普通Matmul"
|
||||||
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,False,False,False,True,0,special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核",0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,16777216,1.048576e-05,0.0,1.048576e-05,0.0,1.048576e-05,FIXPIPE,True,,写出Bound,K=0纯写值
|
special_k0_demo,128,128,256,256,0,bf16,bf16,bf16,False,False,False,True,0,special_k0_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,0,0,1,UB驻留(AIV),0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=0纯写值: 无任何计算, C=bias 或 0, 纯 AIV 写值; 按行均分到 AIV 核",0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,16777216,3.2263876923076923e-06,0.0,3.2263876923076923e-06,0.0,3.2263876923076923e-06,FIXPIPE,True,,写出Bound(L2写口),K=0纯写值
|
||||||
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) UB乒乓,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB乒乓 (B>=2*AIV 双batch乒乓流水)",131072,0.0,8.192e-08,0.0,8.192e-08,0.0,0.0,8388608.0,6.206060606060606e-07,16777216,1.048576e-05,0.0,1.048576e-05,0.0,1.048576e-05,FIXPIPE,True,,写出Bound,"K=1逐元素乘, 走AIV向量通路"
|
special_k1_demo,128,128,256,256,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_demo,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) UB乒乓,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, UB乒乓 (B>=2*AIV 双batch乒乓流水)",131072,0.0,8.192e-08,0.0,8.192e-08,0.0,0.0,8388608.0,3.103030303030303e-07,16777216,3.2263876923076923e-06,0.0,3.2263876923076923e-06,0.0,3.2263876923076923e-06,FIXPIPE,True,,写出Bound(L2写口),"K=1逐元素乘, 走AIV向量通路"
|
||||||
merge_demo_k_trunc,2048,2048,32,32,256,bf16,bf16,bf16,False,False,False,True,0,merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,128,128,256,256,8,合并驻留,128,128,128,allocate(GM->L1随路驻留L2),direct_gm,0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限19.0/b_core=64); K截断; 合并后单次DMA搬入 A'[128,256]+B'[256,128]",2097152,0.0,4.194304e-05,0.0,4.2743039999999997e-05,16.0,8.000000000000001e-07,134217728.0,8.837381267489712e-06,131072,2.62144e-06,0.0,4.2743039999999997e-05,3.0192408230452674e-07,4.3044964082304525e-05,MTE2_GM,True,,访存Bound(GM),"两分支均合法, 仲裁: [分界条件] MergeBatch最优=True (k_L1=K(截断); b_core=64 vs 阈值 b0*(T_comp+T_write)/T_cmd=6.0; drain惩罚=(b0-1)*(T_comp+T_write)=0.23us, 搬移节省=b_core*(1-1/b0)*T_cmd=2.40us); [时延模型] T_MergeBatch=43.04us vs T_IterBatch=45.22us -> MergeBatch更优; [裁决] MergeBatch"
|
merge_demo_k_trunc,2048,2048,16,64,128,bf16,bf16,bf16,False,False,False,True,0,merge_demo_k_trunc,MergeBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B均分(核间零重复读零依赖),64,4,64,256,128,128,12,合并驻留,64,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"b0=4 (L0C上限5.7/算存比上限21.1/b_core=64); K截断; 合并后单次DMA搬入 A'[64,128]+B'[128,256]; 输出落点: L2驻留 (整case V_in+V_out=40.0MB vs L2=128MB)",41943040,0.0,2.62144e-05,0.0,2.62144e-05,16,0.0,2147483648.0,4.971026962962963e-06,4194304,8.065969230769231e-07,0.0,2.62144e-05,1.2808460398860398e-07,2.6342484603988603e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"两分支均合法, 仲裁: [分界条件] MergeBatch最优=True (K截断(k_l1^m=128>=K); 每核命令数 MergeBatch=16 vs IterBatch=64 (命令节省=0.00us) + 搬移效率节省=15.73us vs drain惩罚=(b0-1)*(T_comp+T_write)=0.18us -> MergeBatch优); [时延模型] T_MergeBatch=26.34us vs T_IterBatch=41.98us -> MergeBatch更优; [裁决] MergeBatch"
|
||||||
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,False,False,False,True,0,merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1,524288,0.0,1.048576e-05,0.0,1.0685759999999999e-05,4,2.0000000000000002e-07,16777216.0,1.104672658436214e-06,32768,6.5536e-07,0.0,1.0685759999999999e-05,4.400081646090535e-07,1.1125768164609053e-05,MTE2_GM,True,,访存Bound(GM),"两分支均合法, 仲裁: [分界条件] MergeBatch最优=False (k_L1=K(截断); b_core=4 vs 阈值 b0*(T_comp+T_write)/T_cmd=17.6; drain惩罚=(b0-1)*(T_comp+T_write)=0.44us, 搬移节省=b_core*(1-1/b0)*T_cmd=0.10us); [时延模型] T_MergeBatch=11.49us vs T_IterBatch=11.13us -> IterBatch更优; [裁决] IterBatch"
|
merge_iter_arbitrate,128,128,64,64,512,bf16,bf16,bf16,False,False,False,True,0,merge_iter_arbitrate,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,512,512,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=256KB <= L1; 输出落点: L2驻留,16777216,0.0,1.048576e-05,0.0,1.048576e-05,4,0.0,536870912.0,1.2427567407407407e-06,1048576,2.0164923076923077e-07,0.0,1.048576e-05,3.6110149287749287e-07,1.0846861492877492e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"两分支均合法, 仲裁: [分界条件] MergeBatch最优=False (L1绑定(k_l1^m=256<K=512); 每核命令数 MergeBatch=4 vs IterBatch=4 (命令节省=0.00us) + 搬移效率节省=0.00us vs drain惩罚=(b0-1)*(T_comp+T_write)=0.47us -> IterBatch优); [时延模型] T_MergeBatch=10.90us vs T_IterBatch=10.85us -> IterBatch更优; [裁决] IterBatch"
|
||||||
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1,262144,0.0,5.24288e-06,0.0,5.4428799999999995e-06,4,2.0000000000000002e-07,8388608.0,5.52336329218107e-07,32768,6.5536e-07,0.0,5.4428799999999995e-06,3.0192408230452674e-07,5.744804082304526e-06,MTE2_GM,True,,访存Bound(GM),仅 IterBatch 条件满足
|
iter_demo_form_b,128,128,64,64,256,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_b,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,64,64,256,256,2,b_双batch乒乓,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=128KB <= L1; 输出落点: L2驻留,8388608,0.0,5.24288e-06,0.0,5.24288e-06,4,0.0,268435456.0,6.213783703703703e-07,1048576,2.0164923076923077e-07,0.0,5.24288e-06,2.0575690028490026e-07,5.4486369002849e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(输出仅写一次,直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝",4194304,0.0,8.388608e-05,0.0,8.468608e-05,16,8.000000000000001e-07,134217728.0,8.837381267489712e-06,16384,3.2768e-07,0.0,8.468608e-05,7.16176329218107e-07,8.540225632921811e-05,MTE2_GM,True,,访存Bound(GM),仅 IterBatch 条件满足
|
iter_demo_form_d,64,64,64,64,8192,bf16,bf16,bf16,False,False,False,True,0,iter_demo_form_d,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,64,64,8192,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: 直写GM",134217728,0.0,8.388608e-05,0.0,8.388608e-05,16,0.0,4294967296.0,9.942053925925925e-06,524288,3.2768e-07,0.0,8.421376e-05,7.852183703703703e-07,8.499897837037037e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,False,False,False,True,0,streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",327680.0,0.0,6.5536e-06,0.0,6.5536e-06,0.0,0.0,41943040.0,2.761681646090535e-06,0.0,0.0,3.4067453613053613e-06,6.5536e-06,3.4067453613053613e-06,9.960345361305361e-06,MTE2_GM,True,,访存Bound(GM),"P<=C/2, B/M/N并行度买不满, 切K (grid_K=32)"
|
streamk_demo,4,4,128,128,10240,bf16,bf16,bf16,False,False,False,True,0,streamk_demo,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,128,128,320,256,1,K段标准分块流水,128,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=1.00, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",20971520,0,1.31072e-05,0.0,1.31072e-05,0.0,0.0,1342177280.0,3.1068918518518518e-06,0.0,0.0,3.4067453613053613e-06,1.31072e-05,3.4067453613053613e-06,1.651394536130536e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),"P<=C/2, B/M/N并行度买不满, 切K (grid_K=32)"
|
||||||
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,False,False,False,True,0,asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,47,47,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,176,176,1024,256,1,双缓冲驻留当前tile输入,176,176,80,allocate(输入驻留L2吸收重复读),direct_gm(输出直写GM不占L2),4,0,方案B,52,52,1,52,52,2,139,True,2,"L2场景B_输入驻留输出直写GM, r_in=1.00; 尾轮: 周长型主导, rho=0.06<rho_dv=0.53, A1b被dValue卡死, 方案B反超",67108864.0,0.0,4.194304e-05,0.0,4.194304e-05,0.0,0.0,274877906944.0,0.0005655924011193416,268435456,0.00016777216,0.0,0.0005655924011193416,0.0,0.0005655924011193416,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
asw_demo_full,2,2,8192,8192,1024,bf16,bf16,bf16,False,False,False,True,0,asw_demo_full,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1024,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=16, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入512.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=64.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",67108864,1006632960,4.194304e-05,0.00019358326153846154,0.00023552630153846153,0.0,0.0,274877906944.0,0.0006362914512592592,268435456,0.00016777216,0.0,0.0006362914512592592,0.0,0.0006362914512592592,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,False,False,False,True,0,asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,direct_gm,0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",2097152.0,0.0,2.62144e-06,0.0,2.62144e-06,0.0,0.0,268435456.0,1.104672658436214e-06,2097152,2.62144e-06,0.0,2.62144e-06,0.0,2.62144e-06,MTE2_GM,True,,访存Bound(GM),ASW_Basic兜底 (StreamK未过: 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2)
|
asw_demo_reduce_core,16,16,256,256,128,bf16,bf16,bf16,False,False,False,True,0,asw_demo_reduce_core,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,16,1,1,1,1,"降核: 只用16核, 每核一个L0C满载输出块, 其余核闲置",0,1,256,256,128,128,1,标准核内流水,256,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=16.00<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",2097152,0,2.62144e-06,0.0,2.62144e-06,0.0,0.0,268435456.0,1.2427567407407407e-06,2097152,8.065969230769231e-07,0.0,2.62144e-06,0.0,2.62144e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2)
|
||||||
|
b4_m1_n128_k256,4,4,1,128,256,bf16,bf16,bf16,False,False,False,True,0,b4_m1_n128_k256,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,1,128,256,256,1,标准核内流水,1,128,128,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.01<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",264192,0,5.28384e-06,0.0,5.28384e-06,0.0,0.0,262144.0,1.9418074074074073e-08,1024,6.3015384615384615e-09,0.0,5.28384e-06,0.0,5.28384e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受: K > grid_K^2/(grid_K-1)*theta_c)
|
||||||
|
b8_m2_n192_k128,8,8,2,192,128,bf16,bf16,bf16,False,False,False,True,0,b8_m2_n192_k128,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,2,192,128,128,1,标准核内流水,2,192,80,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.05<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",397312,0,7.94624e-06,0.0,7.94624e-06,0.0,0.0,786432.0,5.825422222222222e-08,6144,3.7809230769230766e-08,0.0,7.94624e-06,0.0,7.94624e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受: K > grid_K^2/(grid_K-1)*theta_c)
|
||||||
|
b16_m4_n256_k192,16,16,4,256,192,bf16,bf16,bf16,False,False,False,True,0,b16_m4_n256_k192,ASW_Basic_降核,Ascend950PR,batch_mat_mul_v3,1,1,1,1,1,"降核: 只用1核, 每核一个L0C满载输出块, 其余核闲置",0,1,4,256,192,192,1,标准核内流水,4,256,64,allocate,resident(整case全驻留S_A),0,0,不涉及(每核一块无尾轮),1,1,1,0,0,0,0,True,2,"P=0.25<C, 降核是理性选择 (强切则 tile 跌破搬移效率下限反而更慢)",1597440,0,3.19488e-05,0.0,3.19488e-05,0.0,0.0,6291456.0,4.660337777777778e-07,32768,2.0164923076923077e-07,0.0,3.19488e-05,0.0,3.19488e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受: K > grid_K^2/(grid_K-1)*theta_c)
|
||||||
|
b32_m8_n128_k256,32,32,8,128,256,bf16,bf16,bf16,False,False,False,True,0,b32_m8_n128_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,8,128,256,256,1,a_单batch全驻留,8,128,128,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,单batch全驻留: (MK+KN)*dtype=68KB <= L1; 输出落点: L2驻留,2228224,0.0,1.6384e-06,0.0,1.6384e-06,1,0.0,16777216.0,3.8836148148148146e-08,65536,1.2603076923076923e-08,0.0,1.6384e-06,5.1439225071225065e-08,1.689839225071225e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
|
b64_m16_n256_k512,64,64,16,256,512,bf16,bf16,bf16,False,False,False,True,0,b64_m16_n256_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,16,256,512,240,1,c_一侧驻留+对侧切K,16,256,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"一侧驻留(A)+对侧切K: A驻留16KB, 预算L1/2, k_L1=240, dValueA=480B/dValueB=512B; 输出落点: L2驻留",17825792,0.0,1.114112e-05,0.0,1.114112e-05,6,0.0,268435456.0,6.213783703703703e-07,524288,1.0082461538461538e-07,0.0,1.114112e-05,1.9604786324786327e-07,1.1337167863247863e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
|
b128_m8_n192_k256,128,128,8,192,256,bf16,bf16,bf16,False,False,False,True,0,b128_m8_n192_k256,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),4,1,8,192,256,256,2,b_双batch乒乓,8,192,80,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,双batch乒乓: 2*(MK+KN)*dtype=200KB <= L1; 输出落点: L2驻留,13107200,0.0,9.17504e-06,0.0,9.17504e-06,4,0.0,100663296.0,2.330168888888889e-07,393216,7.561846153846153e-08,0.0,9.17504e-06,7.71588376068376e-08,9.252198837606838e-06,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
|
b32_m16_n8192_k7168,32,32,16,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m16_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,1,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,16,1024,7168,112,1,双缓冲驻留当前tile输入,16,1024,16,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: 效率降级(放开约束4, dValue 按 128B 硬下限, 搬移效率低于模型假设, 时延可能低估): P=1, mCnt=1 x nCnt=8 (tile 16x1024, 每batch搬入113.8MB, r=0); Base tile 16x1024 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=3591.0MB, V_out=8.0MB, L2=128MB); 尾轮: r=0 无尾轮",3765436416,51380224,0.00235339776,9.880812307692307e-06,0.0023632785723076925,0.0,0.0,60129542144.0,0.00013918875496296296,8388608,5.24288e-06,0.0,0.0023685214523076923,0.0,0.0023685214523076923,MTE2,True,,访存Bound(GM读写共享+L2重复读),"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B [效率降级标注: 搬移效率低于模型假设, 时延可能低估, 见 plan.note]"
|
||||||
|
b4_m1_n8192_k8192,4,4,1,8192,8192,bf16,bf16,bf16,False,False,False,True,0,b4_m1_n8192_k8192,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,32,"B/M/N切出4块, 每块32核切K归约 (归约组内核c负责K段[c*K/32,(c+1)*K/32))",1,1,1,8192,256,256,1,K段标准分块流水,1,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,4194304,grid_K=32路切K+归约,1,1,32,0,0,0,0,True,4,"P=0.50, grid_K=32, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",536936448,0.0,0.00033558528,0.0,0.00033558528,0.0,0.0,536870912.0,1.2427567407407407e-06,0.0,0.0,1.731729603729604e-06,0.00033558528,1.731729603729604e-06,0.0003373170096037296,MTE2,True,,访存Bound(GM读写共享+L2重复读),"P<=C/2, B/M/N并行度买不满, 切K (grid_K=32)"
|
||||||
|
b16_m2_n4096_k7168,16,16,2,4096,7168,bf16,bf16,bf16,False,False,False,True,0,b16_m2_n4096_k7168,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,1,16,"B/M/N切出16块, 每块16核切K归约 (归约组内核c负责K段[c*K/16,(c+1)*K/16))",1,1,2,4096,448,256,1,K段标准分块流水,2,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,8388608,grid_K=16路切K+归约,1,1,16,0,0,0,0,True,4,"P=2.00, grid_K=16, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",939982848,0,0.00058748928,0.0,0.00058748928,0.0,0.0,1879048192.0,4.349648592592593e-06,0.0,0.0,1.7726896037296038e-06,0.00058748928,1.7726896037296038e-06,0.0005892619696037297,MTE2,True,,访存Bound(GM读写共享+L2重复读),"P<=C/2, B/M/N并行度买不满, 切K (grid_K=16)"
|
||||||
|
b32_m64_n64_k7168,32,32,64,64,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m64_n64_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,64,64,7168,1024,1,d_两侧都切K,64,64,256,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=1024, K段成对流水, batch边界天然无缝; dValueA=2048B/dValueB=128B; 输出落点: L2驻留",58720256,0.0,3.670016e-05,0.0,3.670016e-05,7,0.0,1879048192.0,4.349648592592593e-06,262144,5.041230769230769e-08,0.0,3.670016e-05,6.71790678062678e-07,3.737195067806268e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),仅 IterBatch 条件满足
|
||||||
|
b64_m1024_n1024_k7168,64,64,1024,1024,7168,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n1024_k7168,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),2,1,1024,1024,7168,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"direct_gm(整case超L2: 输入优先驻留L2, 输出直写GM不占L2)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: 直写GM",1879048192,0.0,0.00117440512,0.0,0.00117440512,224,0.0,962072674304.0,0.0022270200794074074,134217728,8.388608e-05,0.0,0.0022270200794074074,5.1885093925925924e-05,0.0022789051733333333,MMAD,True,,计算Bound,仅 IterBatch 条件满足
|
||||||
|
b128_m2048_n2048_k1536,128,128,2048,2048,1536,bf16,bf16,bf16,False,False,False,True,0,b128_m2048_n2048_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入48.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1536.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",1610612736,4831838208,0.00100663296,0.0009291996553846154,0.0019358326153846154,0.0,0.0,1649267441664.0,0.0038177487075555555,1073741824,0.00067108864,0.0,0.0038177487075555555,0.0,0.0038177487075555555,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b64_m1024_n8192_k2048,64,64,1024,8192,2048,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n8192_k2048,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,2048,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=16 (tile 512x512, 每batch搬入128.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2304.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",2415919104,6174015488,0.00150994944,0.0011873106707692308,0.0026972601107692305,0.0,0.0,2199023255552.0,0.005090331610074074,1073741824,0.00067108864,0.0,0.005090331610074074,0.0,0.005090331610074074,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b32_m1024_n1024_k512,32,32,1024,1024,512,bf16,bf16,bf16,False,False,False,True,0,b32_m1024_n1024_k512,IterBatch,Ascend950PR,batch_mat_mul_v3,32,32,1,1,1,切B轮转分配(核间零重复读零依赖),1,1,1024,1024,512,64,1,d_两侧都切K,176,176,64,allocate(GM->L1随路驻留L2),"resident(整case输入+输出<=L2: 输出驻留L2异步回写, GM写=0)",0,0,不涉及(核内不切M/N),1,1,1,0,0,0,0,True,2,"两侧都切K: k_L1=64, K段成对流水, batch边界天然无缝; dValueA=128B/dValueB=2048B; 输出落点: L2驻留",67108864,0.0,4.194304e-05,0.0,4.194304e-05,8,0.0,34359738368.0,7.95364314074074e-05,67108864,1.290555076923077e-05,0.0,7.95364314074074e-05,2.2847604695156693e-05,0.0001023840361025641,MMAD,True,,计算Bound,仅 IterBatch 条件满足
|
||||||
|
b128_m4096_n4096_k8192,128,128,4096,4096,8192,bf16,bf16,bf16,False,False,False,True,0,b128_m4096_n4096_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16384.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮",17179869184,120259084288,0.01073741824,0.023126746978461538,0.033864165218461535,0.0,0.0,35184372088832.0,0.08144530576118518,4294967296,0.00268435456,0.0,0.08144530576118518,0.0,0.08144530576118518,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b32_m8192_n4096_k7168,32,32,8192,4096,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m8192_n4096_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=8 (tile 512x512, 每batch搬入1792.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=5376.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮",5637144576,54492397568,0.00352321536,0.010479307224615384,0.014002522584615384,0.0,0.0,15393162788864.0,0.03563232127051852,2147483648,0.00134217728,0.0,0.03563232127051852,0.0,0.03563232127051852,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b32_m2048_n2048_k8192,32,32,2048,2048,8192,bf16,bf16,bf16,False,False,False,True,0,b32_m2048_n2048_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,4,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=4 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2048.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",2147483648,6442450944,0.00134217728,0.0012389328738461537,0.002581110153846154,0.0,0.0,2199023255552.0,0.005090331610074074,268435456,0.00016777216,0.0,0.005090331610074074,0.0,0.005090331610074074,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b64_m4096_n2048_k128,64,64,4096,2048,128,bf16,bf16,bf16,False,False,False,True,0,b64_m4096_n2048_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=8 x nCnt=4 (tile 512x512, 每batch搬入8.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=96.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",100663296,436207616,6.291456e-05,8.388608e-05,0.00014680063999999998,0.0,0.0,137438953472.0,0.0003181457256296296,1073741824,0.00067108864,0.0,0.00081788928,0.0,0.00081788928,MTE2,True,,访存Bound(GM读写共享+L2重复读),"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b16_m1024_n1024_k8192,16,16,1024,1024,8192,bf16,bf16,bf16,False,False,False,True,0,b16_m1024_n1024_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=2 x nCnt=2 (tile 512x512, 每batch搬入64.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=512.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",536870912,536870912,0.00033554432,0.00010324440615384615,0.0004387887261538461,0.0,0.0,274877906944.0,0.0006362914512592592,33554432,2.097152e-05,0.0,0.0006362914512592592,0.0,0.0006362914512592592,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b4_m32768_n128_k128,4,4,32768,128,128,bf16,bf16,bf16,False,False,False,True,0,b4_m32768_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"resident(整case全驻留S_A: 输出驻留L2异步回写, GM写=0)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入10.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: A_整case全驻留(输入+输出<=L2) (V_in=32.1MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",33685504,8257536,2.105344e-05,1.5879876923076922e-06,2.2641427692307692e-05,0.0,0.0,4294967296.0,9.942053925925925e-06,33554432,6.452775384615385e-06,0.0,2.2641427692307692e-05,0.0,2.2641427692307692e-05,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b8_m32768_n2048_k512,8,8,32768,2048,512,bf16,bf16,bf16,False,False,False,True,0,b8_m32768_n2048_k512,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,4,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,512,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,64,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=64 x nCnt=4 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=272.0MB, V_out=1024.0MB, L2=128MB); 尾轮: r=0 无尾轮",285212672,1862270976,0.00017825792,0.00035812903384615385,0.0005363869538461539,0.0,0.0,549755813888.0,0.0012725829025185184,1073741824,0.00067108864,0.0,0.0012725829025185184,0.0,0.0012725829025185184,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b4_m131072_n128_k128,4,4,131072,128,128,bf16,bf16,bf16,False,False,False,True,0,b4_m131072_n128_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,128,128,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,32,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=256 x nCnt=1 (tile 512x128, 每batch搬入40.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=128.1MB, V_out=128.0MB, L2=128MB); 尾轮: r=0 无尾轮",134348800,33423360,8.3968e-05,6.427569230769231e-06,9.039556923076924e-05,0.0,0.0,17179869184.0,3.97682157037037e-05,134217728,8.388608e-05,0.0,0.00017428164923076922,0.0,0.00017428164923076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b8_m131072_n1024_k256,8,8,131072,1024,256,bf16,bf16,bf16,False,False,False,True,0,b8_m131072_n1024_k256,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,2,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,256,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,128,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=2 (tile 512x512, 每batch搬入256.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=516.0MB, V_out=2048.0MB, L2=128MB); 尾轮: r=0 无尾轮",541065216,1606418432,0.00033816576,0.00030892662153846154,0.0006470923815384616,0.0,0.0,549755813888.0,0.0012725829025185184,2147483648,0.00134217728,0.0,0.0019892696615384617,0.0,0.0019892696615384617,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b16_m32768_n8192_k7168,16,16,32768,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b16_m32768_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,512,True,2,"tile枚举: P=2, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入14336.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=8960.0MB, V_out=8192.0MB, L2=128MB); 尾轮: r=0 无尾轮",9395240960,231122927616,0.0058720256,0.044446716849230766,0.05031874244923076,0.0,0.0,61572651155456.0,0.14252928508207408,8589934592,0.00536870912,0.0,0.14252928508207408,0.0,0.14252928508207408,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b4_m32768_n128_k8192,4,4,32768,128,8192,bf16,bf16,bf16,False,False,False,True,0,b4_m32768_n128_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,1,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,128,8192,192,1,双缓冲驻留当前tile输入,512,128,32,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,8,True,2,"tile枚举: P=8, 有界枚举最优 mCnt=64 x nCnt=1 (tile 512x128, 每batch搬入640.0MB, r=0); Base tile 512x128 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=2056.0MB, V_out=32.0MB, L2=128MB); 尾轮: r=0 无尾轮",2155872256,528482304,0.00134742016,0.0001016312123076923,0.0014490513723076923,0.0,0.0,274877906944.0,0.0006362914512592592,33554432,2.097152e-05,0.0,0.0014700228923076922,0.0,0.0014700228923076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b32_m131072_n8192_k128,32,32,131072,8192,128,bf16,bf16,bf16,False,False,False,True,0,b32_m131072_n8192_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,4096,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入1024.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=1088.0MB, V_out=65536.0MB, L2=128MB); 尾轮: r=0 无尾轮",1140850688,33218887680,0.00071303168,0.006388247630769231,0.007101279310769231,0.0,0.0,8796093022208.0,0.020361326440296295,68719476736,0.04294967296,0.0,0.05005095227076922,0.0,0.05005095227076922,MTE2,True,,访存Bound(GM读写共享+L2重复读),"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b64_m32768_n8192_k1536,64,64,32768,8192,1536,bf16,bf16,bf16,False,False,False,True,0,b64_m32768_n8192_k1536,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,64,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,1536,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,2048,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=64 x nCnt=16 (tile 512x512, 每batch搬入3072.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7680.0MB, V_out=32768.0MB, L2=128MB); 尾轮: r=0 无尾轮",8053063680,198105366528,0.0050331648,0.03809718587076923,0.04313035067076923,0.0,0.0,52776558133248.0,0.12216795864177778,34359738368,0.02147483648,0.0,0.12216795864177778,0.0,0.12216795864177778,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b8_m131072_n8192_k8192,8,8,131072,8192,8192,bf16,bf16,bf16,False,False,False,True,0,b8_m131072_n8192_k8192,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,256,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,8192,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=256 x nCnt=16 (tile 512x512, 每batch搬入65536.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: C_双侧超L2: 最小替换2D分组(组间落空GM, 窗口L2) (V_in=17408.0MB, V_out=16384.0MB, L2=128MB); 最小替换分组 m_grp=8x n_grp=8 (GM倍率3.76, 窗口L2/batch=57344.0MB); 尾轮: r=0 无尾轮",68719476736,481036337152,0.04294967296,0.09250698791384615,0.13545666087384614,0.0,0.0,140737488355328.0,0.3257812230447407,17179869184,0.01073741824,0.0,0.3257812230447407,0.0,0.3257812230447407,MMAD,True,,计算Bound,ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b8_m16_n7168_k1536,8,8,16,7168,1536,bf16,bf16,bf16,False,False,False,True,0,b8_m16_n7168_k1536,StreamK,Ascend950PR,batch_mat_mul_v3,32,1,1,2,2,"B/M/N切出16块, 每块2核切K归约 (归约组内核c负责K段[c*K/2,(c+1)*K/2))",1,1,16,3584,768,256,1,K段标准分块流水,16,128,64,allocate(部分和驻留L2),"resident(部分和4B驻留L2, 防精度丢失不随C的fp16/fp8转换)",0,3670016,grid_K=2路切K+归约,1,1,2,0,0,0,0,True,4,"P=14.00, grid_K=2, 部分和驻留L2按4B写出, AIV归约后按C dtype=2B写最终",176553984,393216,0.00011034624,7.561846153846153e-08,0.00011042185846153846,0.0,0.0,2818572288.0,6.524472888888889e-06,0.0,0.0,2.566079254079254e-07,0.00011042185846153846,2.566079254079254e-07,0.00011067846638694638,MTE2,True,,访存Bound(GM读写共享+L2重复读),"P<=C/2, B/M/N并行度买不满, 切K (grid_K=2)"
|
||||||
|
b64_m1024_n7168_k7168,64,64,1024,7168,7168,bf16,bf16,bf16,False,False,False,True,0,b64_m1024_n7168_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,2,14,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,56,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=2 x nCnt=14 (tile 512x512, 每batch搬入392.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=896.0MB, L2=128MB); 尾轮: r=0 无尾轮",7516192768,18790481920,0.00469762048,0.0036135542153846152,0.008311174695384616,0.0,0.0,6734508720128.0,0.015589140555851852,939524096,0.00058720256,0.0,0.015589140555851852,0.0,0.015589140555851852,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b32_m8192_n8192_k7168,32,32,8192,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b32_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,256,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=7168.0MB, V_out=4096.0MB, L2=128MB); 尾轮: r=0 无尾轮",7516192768,112742891520,0.00469762048,0.021681325292307693,0.026378945772307694,0.0,0.0,30786325577728.0,0.07126464254103704,4294967296,0.00268435456,0.0,0.07126464254103704,0.0,0.07126464254103704,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 1_batch关系与每核份额: BatchA==BatchB 且 b_core=B/C>=2*b0; 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
b8_m4096_n4096_k128,8,8,4096,4096,128,bf16,bf16,bf16,False,False,False,True,0,b8_m4096_n4096_k128,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,8,8,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,128,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,16,True,2,"tile枚举: P=4, 有界枚举最优 mCnt=8 x nCnt=8 (tile 512x512, 每batch搬入16.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=16.0MB, V_out=256.0MB, L2=128MB); 尾轮: r=0 无尾轮",16777216,117440512,1.048576e-05,2.2584713846153845e-05,3.307047384615384e-05,0.0,0.0,34359738368.0,7.95364314074074e-05,268435456,0.00016777216,0.0,0.00020084263384615383,0.0,0.00020084263384615383,MTE2,True,,访存Bound(GM读写共享+L2重复读),ASW_Basic兜底 (StreamK未过: 1_并行缺口: P=B*MN*4B/L0C <= C/2; 2_单核K段下限: K/grid_K >= 256B/dtype 且 grid_K>=2; 3_归约代价可接受)
|
||||||
|
b128_m8192_n8192_k7168,128,128,8192,8192,7168,bf16,bf16,bf16,False,False,False,True,0,b128_m8192_n8192_k7168,ASW_Basic,Ascend950PR,batch_mat_mul_v3,32,1,16,16,1,B->M->N线性映射+ASW滑窗蛇形(W=4),0,1,512,512,7168,128,1,双缓冲驻留当前tile输入,256,256,64,allocate(输入驻留L2吸收重复读),"direct_gm(输出直写GM, 输入优先驻留L2)",4,0,A0,1,1,1,0,0,0,1024,True,2,"tile枚举: P=1, 有界枚举最优 mCnt=16 x nCnt=16 (tile 512x512, 每batch搬入3584.0MB, r=0); Base tile 256x256 (L0C 单缓冲方形用满); L2场景: B_单batch可驻留(全驻留或单侧驻留+对侧滑窗) (V_in=28672.0MB, V_out=16384.0MB, L2=128MB); 尾轮: r=0 无尾轮",30064771072,450971566080,0.01879048192,0.08672530116923077,0.10551578308923078,0.0,0.0,123145302310912.0,0.28505857016414815,17179869184,0.01073741824,0.0,0.28505857016414815,0.0,0.28505857016414815,MMAD,True,,计算Bound,"IterBatch/MergeBatch 进入条件均不满足, 回落 ASW_Basic; IterBatch未过: 3_L1驻留形态(四选一, 核心要求: 单batch核内零重复读); MergeBatch未过: 2_L0C容量: 2*(b0*M)*(b0*N)*4B <= L0C; 2b_L0AB容量: 合并后 b0*M/b0*N 在最小K粒度下可驻留 L0A/L0B; 5_访存Bound: 2MN/(M+N) < R16/b0"
|
||||||
|
special_k1_b64,64,64,8192,512,1,bf16,bf16,bf16,False,False,False,True,0,special_k1_b64,特殊分支,Ascend950PR,batch_mat_mul_v3,64,1,1,1,1,AIV 核间按行均分 (无 Cube tile 概念),0,1,0,0,1,0,1,UB驻留(AIV) AIV单缓冲,0,0,0,allocate,direct_gm,0,0,不涉及(AIV逐元素),1,1,1,0,0,0,0,False,2,"K=1逐元素乘: 退化为 C=A⊙B 无累加深度, Cube 16x16x16 粒度浪费 15/16; 走 AIV 通路 GM->UB->Mul->GM, AIV单缓冲 (B<2*AIV 逐batch单缓冲串行)",1114112,0.0,6.9632e-07,0.0,6.9632e-07,0.0,0.0,268435456.0,9.92969696969697e-06,536870912,0.00033554432,0.0,0.00033624063999999996,0.0,0.00033624063999999996,MTE2,True,,访存Bound(GM读写共享+L2重复读),"K=1逐元素乘, 走AIV向量通路"
|
||||||
|
|||||||
|
@@ -12,6 +12,7 @@
|
|||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
from bmm_theory.models import BmmCase
|
from bmm_theory.models import BmmCase
|
||||||
|
from bmm_theory.hardware import ASCEND950PR
|
||||||
from bmm_theory.router import BranchRouter
|
from bmm_theory.router import BranchRouter
|
||||||
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
from bmm_theory.branches.iter_batch import IterBatchBranch
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
@@ -108,12 +109,22 @@ class TestArbitration(unittest.TestCase):
|
|||||||
self.assertIn("L1绑定", detail)
|
self.assertIn("L1绑定", detail)
|
||||||
|
|
||||||
def test_large_batch_mergebatch_wins(self):
|
def test_large_batch_mergebatch_wins(self):
|
||||||
# 大 B + 小 MN + K 截断: MergeBatch 应胜 (v1.1 §4.5)
|
# 大 B + 小 MN + K 截断 + 合并 tile 效率差: MergeBatch 应胜
|
||||||
case = mkcase(2048, 32, 32, 256)
|
# (issue#36: (2048,16,64,128) IterBatch A tile=4KB eff=0.25 vs 合并
|
||||||
|
# 16KB eff=1.0, 效率节省 15.7us >> drain 0.17us; T_cmd=0 默认下仍胜)
|
||||||
|
case = mkcase(2048, 16, 64, 128)
|
||||||
self.assertTrue(MergeBatchBranch().analyze(case).capable)
|
self.assertTrue(MergeBatchBranch().analyze(case).capable)
|
||||||
win, detail = self.mb.beats_iterbatch(case)
|
win, detail = self.mb.beats_iterbatch(case)
|
||||||
self.assertTrue(win, detail)
|
self.assertTrue(win, detail)
|
||||||
|
|
||||||
|
def test_no_eff_diff_mergebatch_loses(self):
|
||||||
|
# issue#36: K 截断但 IterBatch 单命令 tile 已达 16KB 饱和 (效率打平),
|
||||||
|
# T_cmd=0 下命令节省为 0, 合并只剩 drain 惩罚 -> IterBatch 优
|
||||||
|
# (原 t_cmd=50ns 时代的 MergeBatch 胜例 (2048,32,32,256), 语义迁移)
|
||||||
|
case = mkcase(2048, 32, 32, 256)
|
||||||
|
win, detail = self.mb.beats_iterbatch(case)
|
||||||
|
self.assertFalse(win, detail)
|
||||||
|
|
||||||
|
|
||||||
class TestTimingSanity(unittest.TestCase):
|
class TestTimingSanity(unittest.TestCase):
|
||||||
"""时延模型自洽性."""
|
"""时延模型自洽性."""
|
||||||
@@ -134,14 +145,16 @@ class TestTimingSanity(unittest.TestCase):
|
|||||||
# MergeBatch case 必为访存 Bound (进入条件 5)
|
# MergeBatch case 必为访存 Bound (进入条件 5)
|
||||||
case = mkcase(128, 64, 64, 512)
|
case = mkcase(128, 64, 64, 512)
|
||||||
mb = MergeBatchBranch().analyze(case)
|
mb = MergeBatchBranch().analyze(case)
|
||||||
self.assertIn(mb.timing.bottleneck, ("MTE2_GM", "MTE2_L2"))
|
self.assertIn(mb.timing.bottleneck, ("MTE2", "MMAD"))
|
||||||
|
|
||||||
def test_fixpipe_dtype_conversion(self):
|
def test_fixpipe_dtype_conversion(self):
|
||||||
# C 指定 fp16 输出时, 写出量按 2B 而非 L0C 的 4B
|
# C 指定 fp16 输出时, 写出量按 2B 而非 L0C 的 4B;
|
||||||
|
# 字节列为整芯片口径 (issue#29): fixpipe_bytes = B*MN*outB
|
||||||
case = mkcase(128, 64, 64, 512, dtype_c="fp16")
|
case = mkcase(128, 64, 64, 512, dtype_c="fp16")
|
||||||
ib = IterBatchBranch().analyze(case)
|
ib = IterBatchBranch().analyze(case)
|
||||||
expect = ib.plan.b_core * 64 * 64 * 2
|
expect = case.batch_c * 64 * 64 * 2
|
||||||
self.assertAlmostEqual(ib.timing.fixpipe_bytes, expect)
|
self.assertAlmostEqual(ib.timing.fixpipe_bytes, expect)
|
||||||
|
self.assertAlmostEqual(ib.timing.fixpipe_bytes, case.output_bytes)
|
||||||
|
|
||||||
|
|
||||||
class TestIssueRegression(unittest.TestCase):
|
class TestIssueRegression(unittest.TestCase):
|
||||||
@@ -151,7 +164,7 @@ class TestIssueRegression(unittest.TestCase):
|
|||||||
self.router = BranchRouter()
|
self.router = BranchRouter()
|
||||||
|
|
||||||
def test_issue4_k1_small_batch_real_plan(self):
|
def test_issue4_k1_small_batch_real_plan(self):
|
||||||
# issue#4/#12: K=1 且 B<128 不崩溃, 且给出 AIV 单缓冲真实方案 (不再是无方案占位)
|
# issue#4/#12/#17: K=1 且 B<128 不崩溃, 且给出 AIV 单缓冲真实方案 (不再是无方案占位)
|
||||||
case = mkcase(64, 8192, 32, 1, dtype_a="int8", dtype_b="int8")
|
case = mkcase(64, 8192, 32, 1, dtype_a="int8", dtype_b="int8")
|
||||||
r = self.router.route(case)
|
r = self.router.route(case)
|
||||||
self.assertIsNotNone(r["plan"])
|
self.assertIsNotNone(r["plan"])
|
||||||
@@ -210,26 +223,26 @@ class TestIssueRegression(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestIssueRegression2(unittest.TestCase):
|
class TestIssueRegression2(unittest.TestCase):
|
||||||
"""第二轮复评问题 (#11-#16) 回归."""
|
"""第三轮复评问题 (#17-#20) + 恢复 #11-#15 回归."""
|
||||||
|
|
||||||
def setUp(self):
|
def setUp(self):
|
||||||
self.router = BranchRouter()
|
self.router = BranchRouter()
|
||||||
|
|
||||||
def test_issue11_streamk_fixpipe_no_double_count(self):
|
def test_issue11_streamk_fixpipe_no_double_count(self):
|
||||||
# issue#11: 部分和写出只经 t_reduce 计账一次; 稳态 fixpipe 不得再计
|
# issue#11/#17: 部分和写出只经 t_reduce 计账一次; 稳态 fixpipe 不得再计
|
||||||
from bmm_theory.branches.stream_k import StreamKBranch
|
from bmm_theory.branches.stream_k import StreamKBranch
|
||||||
case = mkcase(4, 128, 128, 10240)
|
case = mkcase(4, 128, 128, 10240)
|
||||||
sk = StreamKBranch().analyze(case)
|
sk = StreamKBranch().analyze(case)
|
||||||
|
self.assertTrue(sk.capable)
|
||||||
t = sk.timing
|
t = sk.timing
|
||||||
self.assertAlmostEqual(t.t_fixpipe, 0.0) # 归约串行口径下无稳态 fixpipe 账
|
self.assertAlmostEqual(t.t_fixpipe, 0.0) # 归约串行口径下无稳态 fixpipe 账
|
||||||
self.assertAlmostEqual(t.fixpipe_bytes, 0.0)
|
self.assertAlmostEqual(t.fixpipe_bytes, 0.0)
|
||||||
# 端到端 = max(MTE2, MMAD) + 归约, 不再虚高到 55us/FIXPIPE
|
# 端到端 = 稳态(MTE2搬移链) + 归约, fixpipe 无重复账
|
||||||
expect = max(t.t_mte2, t.t_mmad) + t.t_reduce
|
self.assertEqual(t.bottleneck, "MTE2")
|
||||||
self.assertAlmostEqual(t.t_total, expect)
|
self.assertAlmostEqual(t.t_total, t.t_steady + t.t_drain)
|
||||||
self.assertEqual(t.bottleneck, "MTE2_GM")
|
|
||||||
|
|
||||||
def test_issue12_k1_pingpong_still_ok(self):
|
def test_issue12_k1_pingpong_still_ok(self):
|
||||||
# issue#12: K=1 且 B>=128 仍走 UB 乒乓 (原行为不变)
|
# issue#12/#17: K=1 且 B>=128 仍走 UB 乒乓 (原行为不变)
|
||||||
r = self.router.route(mkcase(128, 256, 256, 1))
|
r = self.router.route(mkcase(128, 256, 256, 1))
|
||||||
self.assertEqual(r["branch"], "特殊分支")
|
self.assertEqual(r["branch"], "特殊分支")
|
||||||
self.assertIn("乒乓", r["plan"].l1_form)
|
self.assertIn("乒乓", r["plan"].l1_form)
|
||||||
@@ -248,9 +261,23 @@ class TestIssueRegression2(unittest.TestCase):
|
|||||||
|
|
||||||
def test_issue13_router_fallback_when_winner_infeasible(self):
|
def test_issue13_router_fallback_when_winner_infeasible(self):
|
||||||
# issue#13: 仲裁胜出的 MergeBatch 自检违规时, 回退到可行候选 IterBatch
|
# issue#13: 仲裁胜出的 MergeBatch 自检违规时, 回退到可行候选 IterBatch
|
||||||
|
# (issue#31 后该样例 MergeBatch 时延已不再胜出, 用 mock 压低其时延以固定
|
||||||
|
# "胜者违规 -> 回退" 机制路径; MergeBatch 方案 dValue=112B<128 仍违规)
|
||||||
|
from unittest import mock
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
from bmm_theory.constraints import check_plan_constraints
|
from bmm_theory.constraints import check_plan_constraints
|
||||||
case = mkcase(256, 1, 256, 4096, dtype_a="int8", dtype_b="int8") # 原 0.2% 违规样例
|
case = mkcase(256, 1, 256, 4096, dtype_a="int8", dtype_b="int8")
|
||||||
r = self.router.route(case)
|
mb_plan = MergeBatchBranch().analyze(case).plan
|
||||||
|
self.assertTrue(check_plan_constraints(case, mb_plan),
|
||||||
|
"该样例 MergeBatch 方案应仍自检违规 (dValue)")
|
||||||
|
real_eval = MergeBatchBranch.evaluate
|
||||||
|
|
||||||
|
def fake_win(self, c, p):
|
||||||
|
t = real_eval(self, c, p)
|
||||||
|
t.t_total = 1e-9 # 强制 MergeBatch 时延胜出 -> 仲裁选它 -> 走自检/回退
|
||||||
|
return t
|
||||||
|
with mock.patch.object(MergeBatchBranch, "evaluate", fake_win):
|
||||||
|
r = self.router.route(case)
|
||||||
self.assertEqual(r["branch"], "IterBatch") # 回退
|
self.assertEqual(r["branch"], "IterBatch") # 回退
|
||||||
self.assertIn("自检违规", r["arbitration"])
|
self.assertIn("自检违规", r["arbitration"])
|
||||||
self.assertIn("回退", r["arbitration"])
|
self.assertIn("回退", r["arbitration"])
|
||||||
@@ -265,6 +292,832 @@ class TestIssueRegression2(unittest.TestCase):
|
|||||||
with self.assertRaises(ValueError):
|
with self.assertRaises(ValueError):
|
||||||
BmmCase(case_id="bad", m=64, n=64, k=1, dtype_a="xxx")
|
BmmCase(case_id="bad", m=64, n=64, k=1, dtype_a="xxx")
|
||||||
|
|
||||||
|
def test_issue18_placeholder_plan_infeasible_in_evaluate(self):
|
||||||
|
# issue#18: 占位方案(used_core_num=0)在 evaluate 中必须不可行,
|
||||||
|
# 不得被当作可行方案给出正常时延
|
||||||
|
from bmm_theory.evaluator import PlanEvaluator
|
||||||
|
from bmm_theory.models import ImplPlan
|
||||||
|
# K=1 且 B<128 已恢复真实单缓冲方案 (#17), 故直接构造占位 plan 验证约束层
|
||||||
|
case = mkcase(64, 8192, 32, 1, dtype_a="int8", dtype_b="int8")
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertGreater(r["plan"].used_core_num, 0) # 真实方案
|
||||||
|
ph = ImplPlan(case_id="ph", branch="特殊分支", used_core_num=0,
|
||||||
|
note="该区域暂无理论方案(进入条件不满足)")
|
||||||
|
er = PlanEvaluator().evaluate(case, ph)
|
||||||
|
self.assertFalse(er.feasible, "占位方案应判不可行")
|
||||||
|
self.assertIn("used_core_num", er.violations)
|
||||||
|
|
||||||
|
def test_issue19_transpose_dvalue_guard_effective(self):
|
||||||
|
# issue#19: 转置感知 dValue 判据在生成守卫/条件4/约束三处同源后真正生效.
|
||||||
|
# 判别形状: B=64 M=4096 N=64 K=4096 bf16 —— d 形态 k_l1=16,
|
||||||
|
# 均不转置时 dv_a=k_l1*2=32B <128 挡下 (B 侧 dv_b=N*2=128B 恰好达标也不放行,
|
||||||
|
# 因为两侧切 K 两侧都要高效);
|
||||||
|
# A 转置后 dv_a=M*2=8192B, 应能走 IterBatch 形态 d.
|
||||||
|
from bmm_theory.constraints import check_plan_constraints
|
||||||
|
c_not = BmmCase(case_id="x", batch_a=64, batch_b=64, m=4096, n=64, k=4096,
|
||||||
|
trans_a=False, trans_b=False)
|
||||||
|
r_not = self.router.route(c_not)
|
||||||
|
self.assertNotEqual(r_not["branch"], "IterBatch") # 非转置被 dValue 守卫挡下
|
||||||
|
c_tr = BmmCase(case_id="x", batch_a=64, batch_b=64, m=4096, n=64, k=4096,
|
||||||
|
trans_a=True, trans_b=False)
|
||||||
|
r_tr = self.router.route(c_tr)
|
||||||
|
self.assertEqual(r_tr["branch"], "IterBatch") # A 转置 M 向连续, 守卫放行
|
||||||
|
self.assertIn("d_", r_tr["plan"].l1_form)
|
||||||
|
self.assertEqual(check_plan_constraints(c_tr, r_tr["plan"]), [])
|
||||||
|
|
||||||
|
|
||||||
|
class TestFp4Support(unittest.TestCase):
|
||||||
|
"""fp4 (0.5B) dtype 支持 (对齐 bmmv3, 2026-09-03)."""
|
||||||
|
|
||||||
|
def test_fp4_dtype_bytes(self):
|
||||||
|
from bmm_theory.models import dtype_bytes
|
||||||
|
self.assertEqual(dtype_bytes("fp4"), 0.5)
|
||||||
|
self.assertEqual(dtype_bytes("fp4_e2m1"), 0.5)
|
||||||
|
|
||||||
|
def test_fp4_case_creation(self):
|
||||||
|
case = mkcase(32, 64, 64, 256, dtype_a="fp4", dtype_b="fp4", dtype_c="fp16")
|
||||||
|
self.assertEqual(case.dtype_in_bytes, 0.5)
|
||||||
|
# fp4 输入 + fp16 输出: 输入 0.5B, 输出 2B
|
||||||
|
self.assertEqual(case.dtype_out_bytes, 2)
|
||||||
|
|
||||||
|
def test_fp4_dvalue_threshold(self):
|
||||||
|
# fp4 (0.5B) 时 dValue 128B 需要 k_l1 >= 256
|
||||||
|
from bmm_theory.constraints import check_plan_constraints
|
||||||
|
case = mkcase(128, 64, 64, 128, dtype_a="fp4", dtype_b="fp4", dtype_c="fp16")
|
||||||
|
r = BranchRouter().route(case)
|
||||||
|
v = check_plan_constraints(case, r["plan"])
|
||||||
|
# fp4 小 K 场景应能正常路由且不报 dValue 违规 (k_l1 连续维是 M/N)
|
||||||
|
self.assertEqual(v, [], f"fp4 case 不应报违规: {v}")
|
||||||
|
|
||||||
|
|
||||||
|
class TestTransposeModeling(unittest.TestCase):
|
||||||
|
"""转置对 dValue 连续维的影响建模 (对齐 bmmv3, 2026-09-03)."""
|
||||||
|
|
||||||
|
def test_transpose_affects_dvalue_judgment(self):
|
||||||
|
# A 不转置: K 向连续, dValue 判 K*dt
|
||||||
|
# A 转置: M 向连续, dValue 判 M*dt
|
||||||
|
# B 不转置: N 向连续, dValue 判 N*dt
|
||||||
|
# B 转置: K 向连续, dValue 判 K*dt
|
||||||
|
from bmm_theory.constraints import _k_segment_is_contiguous
|
||||||
|
# 当前版本不建模转置, 默认按不转置处理
|
||||||
|
case = mkcase(128, 64, 64, 512)
|
||||||
|
case.trans_a = False
|
||||||
|
case.trans_b = False
|
||||||
|
# 验证 trans 字段存在且可读写 (为后续建模做准备)
|
||||||
|
self.assertFalse(case.trans_a)
|
||||||
|
self.assertFalse(case.trans_b)
|
||||||
|
case.trans_a = True
|
||||||
|
self.assertTrue(case.trans_a)
|
||||||
|
|
||||||
|
def test_transpose_a_large_m_small_k(self):
|
||||||
|
# A 转置 + 大 M 小 K: dValue 应判 M*dt (M 向连续), 不受 K 小影响
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
# M=1024 (M*dt=2048B >= 128B), K=8 (K*dt=16B < 128B)
|
||||||
|
case = mkcase(128, 1024, 64, 8, dtype_a="bf16", dtype_b="bf16")
|
||||||
|
case.trans_a = True
|
||||||
|
case.trans_b = False
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
# A 转置时 dValue 判 M*dt=2048B >= 128B, 应通过
|
||||||
|
c4 = [c for c in ib.checks if "搬移效率" in c.name][0]
|
||||||
|
self.assertTrue(c4.passed, f"A 转置时应判 M 向连续: {c4.detail}")
|
||||||
|
|
||||||
|
def test_no_transpose_small_k_fails(self):
|
||||||
|
# A 不转置 + 小 K + c/d 形态: dValue 判 K*dt, K=8 时 16B < 128B 应失败
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
# 大 M/N 让 L1 放不下整 K, 走 c/d 形态
|
||||||
|
case = mkcase(128, 1024, 1024, 8, dtype_a="bf16", dtype_b="bf16")
|
||||||
|
case.trans_a = False
|
||||||
|
case.trans_b = False
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
c4 = [c for c in ib.checks if "搬移效率" in c.name][0]
|
||||||
|
# c/d 形态下 A 不转置时 K=8 应报 dValue 违规
|
||||||
|
if ib.plan.l1_form.startswith(("c_", "d_")):
|
||||||
|
self.assertFalse(c4.passed, f"A 不转置时 K=8 应报 dValue 违规: {c4.detail}")
|
||||||
|
|
||||||
|
|
||||||
|
class TestZeroCmdHandling(unittest.TestCase):
|
||||||
|
"""950PR 默认 T_cmd=0 (未标定, issue#36): 整链路不得除零/崩溃; 合并收益由
|
||||||
|
搬移效率模型 (move_eff, 合并 tile 放大 b0 倍) 刻画, 不再设策略覆盖."""
|
||||||
|
|
||||||
|
def test_beats_iterbatch_zero_cmd(self):
|
||||||
|
# T_cmd=0: 命令节省项为 0, 由效率节省 vs drain 惩罚决定 (issue#36)
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
|
mb = MergeBatchBranch() # 默认 spec 即 t_cmd_ns=0
|
||||||
|
# (b, m, n, k, MergeBatch应胜与否=效率节省>drain惩罚)
|
||||||
|
# (2048,16,64,128): iter A tile=4KB eff=0.25 vs 合并 16KB eff=1.0 -> 大胜
|
||||||
|
# (2048,32,32,256): iter A tile=16KB 已饱和, 效率打平 -> drain 惩罚 -> 恒劣
|
||||||
|
# (128,64,64,512): issue#35 第三情形 (dValue cap 截断, 命令/效率均打平) -> 恒劣
|
||||||
|
# (256,128,128,4096): L1 绑定, tile 均 >=128KB 饱和 -> 恒劣
|
||||||
|
cases = [(2048, 16, 64, 128, True), (2048, 32, 32, 256, False),
|
||||||
|
(128, 64, 64, 512, False), (256, 128, 128, 4096, False)]
|
||||||
|
for b, m, n, k, mb_wins in cases:
|
||||||
|
win, detail = mb.beats_iterbatch(mkcase(b, m, n, k))
|
||||||
|
self.assertEqual(win, mb_wins, f"{b},{m},{n},{k}: {detail}")
|
||||||
|
self.assertIn("效率节省", detail)
|
||||||
|
|
||||||
|
def test_route_with_zero_cmd_efficiency_decides(self):
|
||||||
|
# 默认 t_cmd=0: 有效率低下的 IterBatch 小 tile case 由 MergeBatch 胜;
|
||||||
|
# tile 均饱和的 case 由 IterBatch 胜 (drain 惩罚, 无策略覆盖)
|
||||||
|
router = BranchRouter()
|
||||||
|
r = router.route(mkcase(2048, 16, 64, 128)) # 效率差显著 -> MergeBatch
|
||||||
|
self.assertEqual(r["branch"], "MergeBatch")
|
||||||
|
self.assertIsNotNone(r["timing"])
|
||||||
|
r2 = router.route(mkcase(2048, 32, 32, 256)) # tile 均饱和 -> IterBatch
|
||||||
|
self.assertEqual(r2["branch"], "IterBatch")
|
||||||
|
shapes = [(128, 64, 64, 512), (64, 64, 64, 8192), (512, 128, 128, 128),
|
||||||
|
(128, 128, 128, 1024), (32, 4096, 4096, 4096)]
|
||||||
|
for b, m, n, k in shapes:
|
||||||
|
rr = router.route(mkcase(b, m, n, k))
|
||||||
|
t = rr["timing"]
|
||||||
|
self.assertIsNotNone(t, f"{b},{m},{n},{k} 应有 timing")
|
||||||
|
self.assertGreater(t.t_total, 0.0)
|
||||||
|
self.assertEqual(t.t_total, t.t_total) # 非 NaN
|
||||||
|
|
||||||
|
def test_zero_cmd_random_smoke(self):
|
||||||
|
# 随机小样本冒烟: 无异常/无 NaN
|
||||||
|
import random
|
||||||
|
from bmm_theory.hardware import NpuSpec
|
||||||
|
router = BranchRouter(NpuSpec(t_cmd_ns=0.0))
|
||||||
|
rng = random.Random(9)
|
||||||
|
for _ in range(300):
|
||||||
|
b = rng.choice([2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048])
|
||||||
|
m = rng.choice([8, 16, 32, 64, 128, 256, 512, 1024, 4096])
|
||||||
|
n = rng.choice([8, 16, 32, 64, 128, 256, 512, 1024, 4096])
|
||||||
|
k = rng.choice([16, 32, 64, 128, 256, 512, 1024, 4096, 16384])
|
||||||
|
c = BmmCase(case_id="z", batch_a=b, batch_b=b, m=m, n=n, k=k)
|
||||||
|
rr = router.route(c)
|
||||||
|
self.assertIsNotNone(rr["plan"])
|
||||||
|
t = rr["timing"]
|
||||||
|
self.assertIsNotNone(t)
|
||||||
|
self.assertGreater(t.t_total, 0.0)
|
||||||
|
self.assertEqual(t.t_total, t.t_total)
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue27to30(unittest.TestCase):
|
||||||
|
"""第四轮评审 (issue#27-#30, 设计文档 docs/05).
|
||||||
|
|
||||||
|
#27 MergeBatch Cube 公式复核 (每步 2*(b0M)(b0N)K x b_core/b0 步);
|
||||||
|
#28 dtype 感知算力 (Cube/AIV 速率表);
|
||||||
|
#29 GM 首读下限 GM>=V_in + 字节列整芯片口径;
|
||||||
|
#30 Fixpipe 输出落点 (整case驻留 -> L2, 否则直写 GM).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
from bmm_theory.hardware import ASCEND950PR
|
||||||
|
self.s = ASCEND950PR
|
||||||
|
self.router = BranchRouter()
|
||||||
|
|
||||||
|
# ---------------- #27 ----------------
|
||||||
|
def test_issue27_merge_cube_flops_matches_formula(self):
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
|
case = mkcase(2048, 32, 32, 256)
|
||||||
|
mb = MergeBatchBranch().analyze(case)
|
||||||
|
self.assertTrue(mb.capable)
|
||||||
|
p, t = mb.plan, mb.timing
|
||||||
|
b0, bc = p.merge_b0, p.b_core
|
||||||
|
# 整芯片列 = B*b0*2MNK; 等价 (b_core/b0) 步 x 每步 2*(b0M)(b0N)K x C 核
|
||||||
|
step_flops = 2.0 * (b0 * 32) * (b0 * 32) * 256
|
||||||
|
self.assertAlmostEqual(t.cube_flops,
|
||||||
|
step_flops * (bc / b0) * self.s.aic_num)
|
||||||
|
self.assertAlmostEqual(t.cube_flops,
|
||||||
|
case.batch_c * b0 * 2.0 * 32 * 32 * 256)
|
||||||
|
# t_mmad = 每核 flops / 单核算力
|
||||||
|
self.assertAlmostEqual(t.t_mmad,
|
||||||
|
(bc * b0 * 2.0 * 32 * 32 * 256) / self.s.q16)
|
||||||
|
|
||||||
|
def test_issue27_merge_gm_chip_col_equals_input(self):
|
||||||
|
# K 截断合并: GM 每字节一次 -> gm 整芯片列 = V_in (issue#29 口径)
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
|
case = mkcase(2048, 32, 32, 256)
|
||||||
|
t = MergeBatchBranch().analyze(case).timing
|
||||||
|
self.assertAlmostEqual(t.gm_read_bytes, case.input_bytes)
|
||||||
|
self.assertAlmostEqual(t.l2_read_bytes, 0.0)
|
||||||
|
|
||||||
|
# ---------------- #28 ----------------
|
||||||
|
def test_issue28_cube_dtype_rate(self):
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
base = mkcase(128, 64, 64, 512) # IterBatch 形态 b, 各 dtype 均可行
|
||||||
|
t16 = IterBatchBranch().analyze(base).timing
|
||||||
|
t32 = IterBatchBranch().analyze(
|
||||||
|
mkcase(128, 64, 64, 512, dtype_a="fp32", dtype_b="fp32")).timing
|
||||||
|
t8 = IterBatchBranch().analyze(
|
||||||
|
mkcase(128, 64, 64, 512, dtype_a="fp8", dtype_b="fp8")).timing
|
||||||
|
# 同计算量, 时延反比于 dtype 算力: fp32=1/2, fp8=2x (相对 bf16)
|
||||||
|
self.assertAlmostEqual(t32.t_mmad / t16.t_mmad, 2.0, places=6)
|
||||||
|
self.assertAlmostEqual(t8.t_mmad / t16.t_mmad, 0.5, places=6)
|
||||||
|
|
||||||
|
def test_issue28_aiv_dtype_rate_k1(self):
|
||||||
|
# K=1 AIV 逐元素: bf16 通量 2x fp32 -> fp32 时延为 bf16 的 2 倍
|
||||||
|
base = mkcase(64, 8192, 512, 1)
|
||||||
|
t_bf16 = BranchRouter().route(base)["timing"]
|
||||||
|
t_fp32 = BranchRouter().route(
|
||||||
|
mkcase(64, 8192, 512, 1, dtype_a="fp32", dtype_b="fp32"))["timing"]
|
||||||
|
self.assertAlmostEqual(t_fp32.t_mmad / t_bf16.t_mmad, 2.0, places=6)
|
||||||
|
|
||||||
|
# ---------------- #29 ----------------
|
||||||
|
def test_issue29_gm_floor_and_chip_columns(self):
|
||||||
|
# 各分支代表 case: GM 整芯片列 >= V_in (R1), 且迭代/合并/转matmul/special
|
||||||
|
# 等于 V_in (无重复读结构)
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
|
from bmm_theory.branches.stream_k import StreamKBranch
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
checks = [
|
||||||
|
(IterBatchBranch().analyze(mkcase(128, 64, 64, 512)).timing,
|
||||||
|
mkcase(128, 64, 64, 512)),
|
||||||
|
(MergeBatchBranch().analyze(mkcase(2048, 32, 32, 256)).timing,
|
||||||
|
mkcase(2048, 32, 32, 256)),
|
||||||
|
(StreamKBranch().analyze(mkcase(4, 128, 128, 10240)).timing,
|
||||||
|
mkcase(4, 128, 128, 10240)),
|
||||||
|
(AswBasicBranch().analyze(mkcase(8, 4096, 4096, 1024)).timing,
|
||||||
|
mkcase(8, 4096, 4096, 1024)),
|
||||||
|
]
|
||||||
|
for t, case in checks:
|
||||||
|
self.assertGreaterEqual(t.gm_read_bytes + 1e-6, case.input_bytes,
|
||||||
|
f"{case.case_id} GM < V_in")
|
||||||
|
# 转Matmul / 特殊分支 (K=1) 亦等于 V_in
|
||||||
|
r = self.router.route(mkcase(1, 2048, 2048, 2048))
|
||||||
|
t = r["timing"]
|
||||||
|
self.assertGreaterEqual(t.gm_read_bytes + 1e-6,
|
||||||
|
BmmCase(case_id="x", batch_a=1, batch_b=1,
|
||||||
|
m=2048, n=2048, k=2048).input_bytes)
|
||||||
|
r = self.router.route(mkcase(128, 256, 256, 1))
|
||||||
|
self.assertAlmostEqual(r["timing"].gm_read_bytes, 128 * (256 + 256) * 2)
|
||||||
|
|
||||||
|
def test_issue29_gm_floor_random_smoke(self):
|
||||||
|
# 随机多 dtype 冒烟: 所有可行方案 GM 整芯片列 >= V_in, 无 NaN
|
||||||
|
import random
|
||||||
|
rng = random.Random(20260609)
|
||||||
|
dtypes = ["bf16", "fp16", "fp32", "int8", "fp8", "fp4"]
|
||||||
|
for _ in range(400):
|
||||||
|
dt = rng.choice(dtypes)
|
||||||
|
kw = dict(dtype_a=dt, dtype_b=dt, dtype_c=rng.choice(["bf16", "fp16"]))
|
||||||
|
b = rng.choice([1, 2, 4, 8, 16, 32, 64, 128, 256, 2048])
|
||||||
|
m = rng.choice([1, 8, 32, 64, 128, 256, 1024, 4096])
|
||||||
|
n = rng.choice([1, 8, 32, 64, 128, 256, 1024, 4096])
|
||||||
|
k = rng.choice([0, 1, 32, 64, 128, 256, 512, 1024, 4096])
|
||||||
|
c = BmmCase(case_id="gm", batch_a=b, batch_b=b, m=m, n=n, k=k, **kw)
|
||||||
|
r = self.router.route(c)
|
||||||
|
t = r["timing"]
|
||||||
|
self.assertIsNotNone(t, c.case_id)
|
||||||
|
self.assertEqual(t.t_total, t.t_total) # 非 NaN
|
||||||
|
self.assertGreaterEqual(t.gm_read_bytes + 1e-6, c.input_bytes,
|
||||||
|
f"{dt} b={b} m={m} n={n} k={k} gm={t.gm_read_bytes}")
|
||||||
|
|
||||||
|
# ---------------- #30 ----------------
|
||||||
|
def test_issue30_iter_output_l2_when_whole_case_fits(self):
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
# 整 case 输入+输出 16.8MB+1.0MB <= 128MB -> 输出写 L2 写口, GM 写=0
|
||||||
|
case = mkcase(128, 64, 64, 512)
|
||||||
|
self.assertLess(case.input_bytes + case.output_bytes, self.s.l2_bytes)
|
||||||
|
t = IterBatchBranch().analyze(case).timing
|
||||||
|
self.assertAlmostEqual(t.fixpipe_bytes, case.output_bytes)
|
||||||
|
self.assertAlmostEqual(t.t_fixpipe, case.output_bytes / self.s.bw_l2)
|
||||||
|
|
||||||
|
def test_issue30_iter_output_gm_when_case_overflows(self):
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
# 输出 268MB > L2 余量 -> 直写 GM (输入优先驻留)
|
||||||
|
case = mkcase(128, 1024, 1024, 64)
|
||||||
|
self.assertGreater(case.input_bytes + case.output_bytes, self.s.l2_bytes)
|
||||||
|
t = IterBatchBranch().analyze(case).timing
|
||||||
|
self.assertAlmostEqual(t.fixpipe_bytes, case.output_bytes)
|
||||||
|
self.assertAlmostEqual(t.t_fixpipe, case.output_bytes / self.s.bw_gm)
|
||||||
|
|
||||||
|
def test_issue30_asw_scene_whole_case_labels(self):
|
||||||
|
# S_A (整case全驻留): l2_policy_out=resident, t_fixpipe 走 L2 写口
|
||||||
|
case_sa = mkcase(2, 4096, 4096, 512)
|
||||||
|
self.assertLess(case_sa.input_bytes + case_sa.output_bytes, self.s.l2_bytes)
|
||||||
|
r = self.router.route(case_sa)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
self.assertTrue(r["plan"].l2_policy_out.startswith("resident"), r["plan"].l2_policy_out)
|
||||||
|
self.assertIn("A_整case全驻留", r["plan"].note)
|
||||||
|
self.assertAlmostEqual(r["timing"].t_fixpipe,
|
||||||
|
case_sa.output_bytes / self.s.bw_l2)
|
||||||
|
# S_B (整case超L2, 单batch输入可驻留): 输出直写 GM
|
||||||
|
case_sb = mkcase(64, 4096, 4096, 1024)
|
||||||
|
self.assertGreater(case_sb.input_bytes + case_sb.output_bytes, self.s.l2_bytes)
|
||||||
|
self.assertLess(4096 * 1024 * 2 * 2, self.s.l2_bytes) # 单batch输入 16.8MB
|
||||||
|
r = self.router.route(case_sb)
|
||||||
|
self.assertIn("ASW", r["branch"])
|
||||||
|
self.assertTrue(r["plan"].l2_policy_out.startswith("direct_gm"),
|
||||||
|
r["plan"].l2_policy_out)
|
||||||
|
self.assertAlmostEqual(r["timing"].t_fixpipe,
|
||||||
|
case_sb.output_bytes / self.s.bw_gm)
|
||||||
|
|
||||||
|
def test_issue30_to_matmul_output_l2_toggle(self):
|
||||||
|
# 转Matmul 粗估: 整case可驻留 -> L2; 大 case -> GM
|
||||||
|
small = mkcase(1, 2048, 2048, 2048)
|
||||||
|
self.assertLess(small.input_bytes + small.output_bytes, self.s.l2_bytes)
|
||||||
|
r = self.router.route(small)
|
||||||
|
self.assertEqual(r["branch"], "转Matmul")
|
||||||
|
self.assertAlmostEqual(r["timing"].t_fixpipe,
|
||||||
|
small.output_bytes / self.s.bw_l2)
|
||||||
|
big = mkcase(1, 8192, 8192, 4096)
|
||||||
|
self.assertGreater(big.input_bytes + big.output_bytes, self.s.l2_bytes)
|
||||||
|
r = self.router.route(big)
|
||||||
|
self.assertEqual(r["branch"], "转Matmul")
|
||||||
|
self.assertAlmostEqual(r["timing"].t_fixpipe,
|
||||||
|
big.output_bytes / self.s.bw_gm)
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue31to32(unittest.TestCase):
|
||||||
|
"""第五轮评审: #31 切B类 GM 每字节恰读一次 = V_in; #32 ASW 场景升级
|
||||||
|
(单侧全驻留 -> S_B 使 GM=V_in; 双侧超 L2 -> S_C 最小替换 2D 分组 + 窗口 L2 计账)."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
from bmm_theory.hardware import ASCEND950PR
|
||||||
|
self.s = ASCEND950PR
|
||||||
|
self.router = BranchRouter()
|
||||||
|
|
||||||
|
# ---------------- #31 ----------------
|
||||||
|
def test_issue31_iter_form_c_gm_equals_vin(self):
|
||||||
|
# b64_m16_n256_k512: 形态 c, K=512 切 n_K=3 (k_L1=240) 非整除 -> GM 仍 = V_in
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
case = mkcase(64, 16, 256, 512)
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
self.assertIn("c_", ib.plan.l1_form)
|
||||||
|
self.assertLess(ib.plan.k_l1, case.k)
|
||||||
|
t = ib.timing
|
||||||
|
self.assertAlmostEqual(t.gm_read_bytes, case.input_bytes)
|
||||||
|
self.assertAlmostEqual(t.l2_read_bytes, 0.0)
|
||||||
|
self.assertAlmostEqual(t.t_mte2_gm, case.input_bytes / self.s.bw_gm)
|
||||||
|
|
||||||
|
def test_issue31_iter_form_d_gm_equals_vin(self):
|
||||||
|
# K=5000 非整除切段 (k_L1=1024, n_K=5, 末段剩 904): GM 仍恰一次 = V_in
|
||||||
|
from bmm_theory.branches.iter_batch import IterBatchBranch
|
||||||
|
case = mkcase(128, 64, 64, 5000)
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
self.assertIn("d_", ib.plan.l1_form)
|
||||||
|
self.assertLess(ib.plan.k_l1, case.k)
|
||||||
|
self.assertAlmostEqual(ib.timing.gm_read_bytes, case.input_bytes)
|
||||||
|
self.assertAlmostEqual(ib.timing.t_mte2_gm,
|
||||||
|
case.input_bytes / self.s.bw_gm)
|
||||||
|
|
||||||
|
def test_issue31_merge_l1_bound_gm_equals_vin(self):
|
||||||
|
# MergeBatch L1 绑定 (k_L1<K): 切 K 段互不重叠, GM = V_in
|
||||||
|
from bmm_theory.branches.merge_batch import MergeBatchBranch
|
||||||
|
case = mkcase(811, 33, 1, 2459, dtype_a="fp32", dtype_b="fp32")
|
||||||
|
mb = MergeBatchBranch().analyze(case)
|
||||||
|
self.assertTrue(mb.capable)
|
||||||
|
self.assertLess(mb.plan.k_l1, case.k) # L1 绑定
|
||||||
|
self.assertAlmostEqual(mb.timing.gm_read_bytes, case.input_bytes)
|
||||||
|
self.assertAlmostEqual(mb.timing.l2_read_bytes, 0.0)
|
||||||
|
|
||||||
|
# ---------------- #32 ----------------
|
||||||
|
def test_issue32_asw_single_side_resident_gm_equals_vin(self):
|
||||||
|
# b32_m8192_n4096_k7168: 单batch输入 176MB > L2, 但 B=58.7MB 可全驻留
|
||||||
|
# + A 行块滑窗 (58.7+2x2.5 <= 128) -> S_B: GM = V_in, 重复读全命中 L2
|
||||||
|
case = mkcase(32, 8192, 4096, 7168)
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
p, t = r["plan"], r["timing"]
|
||||||
|
a_b = case.m * case.k * case.dtype_in_bytes
|
||||||
|
bb_b = case.k * case.n * case.dtype_in_bytes
|
||||||
|
m_cnt, n_cnt = p.m_cnt, p.n_cnt
|
||||||
|
self.assertGreater(a_b + bb_b, self.s.l2_bytes) # 单 batch 确实超 L2
|
||||||
|
self.assertLessEqual(bb_b + 2 * (a_b / m_cnt),
|
||||||
|
self.s.l2_bytes) # 单侧全驻留成立
|
||||||
|
self.assertAlmostEqual(t.gm_read_bytes, case.input_bytes)
|
||||||
|
exp_l2 = case.batch_c * ((n_cnt - 1) * a_b + (m_cnt - 1) * bb_b)
|
||||||
|
self.assertAlmostEqual(t.l2_read_bytes, exp_l2)
|
||||||
|
|
||||||
|
def test_issue32_asw_scene_c_min_gm(self):
|
||||||
|
# b8_m131072_n8192_k8192: 双侧均不可全驻留 -> S_C 最小替换分组;
|
||||||
|
# gm == 容量约束最小解 (测试内复算), 且 >= V_in
|
||||||
|
case = mkcase(8, 131072, 8192, 8192)
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
p, t = r["plan"], r["timing"]
|
||||||
|
self.assertIn("C_", p.note)
|
||||||
|
m_cnt, n_cnt = p.m_cnt, p.n_cnt
|
||||||
|
dt = case.dtype_in_bytes
|
||||||
|
a_b = case.m * case.k * dt
|
||||||
|
bb_b = case.k * case.n * dt
|
||||||
|
blk_a = a_b / m_cnt
|
||||||
|
blk_b = bb_b / n_cnt
|
||||||
|
best = None
|
||||||
|
for mg in range(1, m_cnt + 1):
|
||||||
|
for ng in range(1, n_cnt + 1):
|
||||||
|
if mg * blk_a + ng * blk_b > self.s.l2_bytes:
|
||||||
|
continue
|
||||||
|
gm = (n_cnt + ng - 1) // ng * a_b + (m_cnt + mg - 1) // mg * bb_b
|
||||||
|
if best is None or gm < best:
|
||||||
|
best = gm
|
||||||
|
self.assertIsNotNone(best)
|
||||||
|
self.assertAlmostEqual(t.gm_read_bytes, case.batch_c * best)
|
||||||
|
self.assertGreaterEqual(t.gm_read_bytes, case.input_bytes)
|
||||||
|
# 最小替换解应严格优于"每块独立落 GM"的保守上界
|
||||||
|
self.assertLessEqual(t.gm_read_bytes,
|
||||||
|
case.batch_c * (n_cnt * a_b + m_cnt * bb_b))
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue33(unittest.TestCase):
|
||||||
|
"""ASW_Basic tile 选择 (issue#33, v1.91 §5.1/§5.2 + 尾轮 v1.5 §2.1 修正口径):
|
||||||
|
BaseM/N = 256 方形 (UnitFlag 单缓冲 L0C 用满 65536 元素), SingleCoreM/N 有界枚举."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
from bmm_theory.hardware import ASCEND950PR
|
||||||
|
self.s = ASCEND950PR
|
||||||
|
self.router = BranchRouter()
|
||||||
|
|
||||||
|
def test_base_tile_square_single_buffer(self):
|
||||||
|
# 默认 BaseM=BaseN=256 (非 176 双缓冲口径), L0C 恰用满 (256*256*4=256KB)
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
p = AswBasicBranch().make_plan(mkcase(32, 4096, 4096, 4096))
|
||||||
|
self.assertEqual((p.base_m, p.base_n), (256, 256))
|
||||||
|
self.assertEqual(p.base_m * p.base_n * 4, self.s.l0c_bytes)
|
||||||
|
self.assertEqual(p.base_k, 64) # 64KB/(2*256*2) = 64
|
||||||
|
self.assertTrue(p.fixpipe_unitflag) # UnitFlag 单缓冲
|
||||||
|
|
||||||
|
def test_base_tile_exception_small_m(self):
|
||||||
|
# M=128 < 256: BaseM=128 (被迫跟随), BaseN = min(65536/128=512, N)=512
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
p = AswBasicBranch().make_plan(mkcase(64, 128, 4096, 1024))
|
||||||
|
self.assertEqual((p.base_m, p.base_n), (128, 512))
|
||||||
|
self.assertEqual(p.base_k, 32) # min(64K/(2*128*2)=128, 64K/(2*512*2)=32)
|
||||||
|
|
||||||
|
def test_v191_example_singlecore(self):
|
||||||
|
# v1.91 §5.2 完整实例: B=8 M=N=2048 K=1024 bf16 -> (4,4) 512x512, k_l1=128, r=0
|
||||||
|
case = mkcase(8, 2048, 2048, 1024)
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
p = r["plan"]
|
||||||
|
self.assertEqual((p.single_core_m, p.single_core_n), (512, 512))
|
||||||
|
self.assertEqual((p.m_cnt, p.n_cnt), (4, 4))
|
||||||
|
self.assertEqual(p.k_l1, 128)
|
||||||
|
self.assertEqual(p.tail_block_cnt, 0) # 8*4*4=128 % 32 = 0 完美整除
|
||||||
|
|
||||||
|
def test_singlecore_not_constant_176(self):
|
||||||
|
# 回归原 bug: SingleCoreM/N 恒为 176 (双缓冲 Base 兜底); 现在自适应
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
p = AswBasicBranch().make_plan(mkcase(8, 4096, 4096, 1024))
|
||||||
|
self.assertNotEqual((p.single_core_m, p.single_core_n), (176, 176))
|
||||||
|
self.assertEqual((p.single_core_m, p.single_core_n), (512, 512))
|
||||||
|
# 且为 Base 的整数倍 (约束 4)
|
||||||
|
self.assertEqual(p.single_core_m % p.base_m, 0)
|
||||||
|
self.assertEqual(p.single_core_n % p.base_n, 0)
|
||||||
|
|
||||||
|
def test_l0c_single_buffer_constraint_pass(self):
|
||||||
|
# ASW 单缓冲: base 256x256 通过约束 (无违规)
|
||||||
|
from bmm_theory.constraints import check_plan_constraints
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
case = mkcase(8, 4096, 4096, 1024)
|
||||||
|
p = AswBasicBranch().make_plan(case)
|
||||||
|
self.assertEqual(check_plan_constraints(case, p), [])
|
||||||
|
|
||||||
|
def test_p1_fallback_to_enum_when_no_split_infeasible(self):
|
||||||
|
# B>=C 时不切分 k_l1 过 dValue 下限失败 -> 强制切分枚举 -> 512x512 (8,8)
|
||||||
|
case = mkcase(128, 4096, 4096, 8192)
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
p = r["plan"]
|
||||||
|
self.assertEqual((p.single_core_m, p.single_core_n), (512, 512))
|
||||||
|
self.assertEqual((p.m_cnt, p.n_cnt), (8, 8))
|
||||||
|
|
||||||
|
def test_issue34_k_l1_decomposition_derived(self):
|
||||||
|
# b32_m16_n8192_k7168: 分解 = Base 16x1024 (例外: M=16<256; N 侧受 L0B 单边
|
||||||
|
# 上限收敛), SingleCore 16x1024 (mCnt=1,nCnt=8), k_l1 = L1 双缓冲反推
|
||||||
|
# ⌊L1/(2·(sM+sN)·dt)⌋16 = ⌊524288/(2·1040·2)⌋16 = 112 (v1.91 §5.2)
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
case = mkcase(32, 16, 8192, 7168)
|
||||||
|
p = AswBasicBranch().make_plan(case)
|
||||||
|
self.assertEqual((p.base_m, p.base_n), (16, 1024))
|
||||||
|
self.assertEqual((p.single_core_m, p.single_core_n), (16, 1024))
|
||||||
|
self.assertEqual(p.k_l1, 112)
|
||||||
|
|
||||||
|
def test_pathological_shape_degraded_warning_always_plan(self):
|
||||||
|
# issue#34: 兜底分支恒出方案 —— 极端形状 (N=8 int8 大K, B 侧 dValue=8B
|
||||||
|
# 物理不可满足) 照常给方案 + 标注效率降级 (warning), 不判违规/不判不可行
|
||||||
|
from bmm_theory.constraints import check_plan_constraints
|
||||||
|
from bmm_theory.evaluator import PlanEvaluator
|
||||||
|
case = mkcase(4096, 2048, 8, 1024, dtype_a="int8", dtype_b="int8")
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
self.assertEqual(check_plan_constraints(case, r["plan"]), [])
|
||||||
|
self.assertIn("效率降级", r["plan"].note)
|
||||||
|
er = PlanEvaluator().evaluate(case, r["plan"])
|
||||||
|
self.assertTrue(er.feasible)
|
||||||
|
self.assertIn("效率降级", er.advice)
|
||||||
|
self.assertGreater(er.timing.t_total, 0.0)
|
||||||
|
|
||||||
|
def test_pathological_n8_hard_floor_fallback(self):
|
||||||
|
# 更极端: N=8 int8 连 128B 硬下限也不满足 -> 仍恒出方案, 效率降级标注
|
||||||
|
from bmm_theory.evaluator import PlanEvaluator
|
||||||
|
case = mkcase(512, 4096, 1, 8192, dtype_a="int8", dtype_b="int8")
|
||||||
|
r = self.router.route(case)
|
||||||
|
self.assertEqual(r["branch"], "ASW_Basic")
|
||||||
|
self.assertIsNotNone(r["plan"])
|
||||||
|
self.assertIn("效率降级", r["plan"].note)
|
||||||
|
er = PlanEvaluator().evaluate(case, r["plan"])
|
||||||
|
self.assertTrue(er.feasible)
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue35(unittest.TestCase):
|
||||||
|
"""issue#35: MergeBatch L1 绑定情形 DMA 命令数多计 b0 倍修复 + 分界泛化口径.
|
||||||
|
|
||||||
|
用户 case 家族: B=128, M=1~16, N=128, K=512, bf16. b_core=4, b0=4,
|
||||||
|
合并后 k_l1^m=240/224 < K=512 (L1 绑定区), 真实每核命令数 = 1x3=3 条
|
||||||
|
(< IterBatch 的 4 条), 修复前被多计为 12 条导致仲裁翻错方向.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self.router = BranchRouter()
|
||||||
|
self.mb = MergeBatchBranch()
|
||||||
|
self.ib = IterBatchBranch()
|
||||||
|
|
||||||
|
def test_merged_cmd_count_formula(self):
|
||||||
|
# 每核命令数 = ceil(b_core/b0) * ceil(K/k_l1^m) (K截断时 = b_core/b0)
|
||||||
|
case = mkcase(128, 16, 128, 512)
|
||||||
|
r = self.mb.analyze(case)
|
||||||
|
self.assertTrue(r.capable)
|
||||||
|
p = r.plan
|
||||||
|
expect = -(-p.b_core // p.merge_b0) * (-(-case.k // p.k_l1))
|
||||||
|
self.assertEqual(r.timing.dma_cmd_count, expect)
|
||||||
|
# 本 case: b0=4, k_l1=224 -> 1*3 = 3 条 (修复前 12 条)
|
||||||
|
self.assertEqual((p.merge_b0, p.k_l1), (4, 224))
|
||||||
|
self.assertEqual(r.timing.dma_cmd_count, 3)
|
||||||
|
|
||||||
|
def test_cmd_count_truncated_unchanged(self):
|
||||||
|
# K 截断情形数值不变: cmds = b_core/b0 = IterBatch 的 1/b0
|
||||||
|
case = mkcase(2048, 32, 32, 256)
|
||||||
|
mb = self.mb.analyze(case)
|
||||||
|
ib = self.ib.analyze(case)
|
||||||
|
self.assertGreaterEqual(mb.plan.k_l1, case.k) # 合并后仍截断
|
||||||
|
self.assertEqual(mb.timing.dma_cmd_count,
|
||||||
|
-(-mb.plan.b_core // mb.plan.merge_b0))
|
||||||
|
self.assertAlmostEqual(
|
||||||
|
mb.timing.dma_cmd_count / ib.timing.dma_cmd_count,
|
||||||
|
1.0 / mb.plan.merge_b0, places=6)
|
||||||
|
|
||||||
|
def test_boundary_uses_merged_k_l1(self):
|
||||||
|
# 截断判定与 plan.k_l1 口径一致: k_l1^m < K 时不得声称 K截断
|
||||||
|
case = mkcase(128, 1, 128, 512)
|
||||||
|
_, detail = self.mb.beats_iterbatch(case)
|
||||||
|
p = self.mb.make_plan(case)
|
||||||
|
self.assertLess(p.k_l1, case.k)
|
||||||
|
self.assertIn("L1绑定", detail)
|
||||||
|
self.assertNotIn("K截断", detail)
|
||||||
|
|
||||||
|
def test_user_case_family_routing(self):
|
||||||
|
# B=128,M=1~16,N=128,K=512: issue#36 效率模型后, iter A tile=m*1KB 未饱和
|
||||||
|
# (m<16), 合并 tile 4 倍大 -> 效率节省 ~0.61us 恒定, drain 随 M 线性增长;
|
||||||
|
# m<=8 效率节省 > drain -> MergeBatch; m=16 iter tile 恰达 16KB 饱和 ->
|
||||||
|
# 效率打平, drain 0.66us 决定 -> IterBatch
|
||||||
|
expect = {1: "MergeBatch", 2: "MergeBatch", 4: "MergeBatch",
|
||||||
|
8: "MergeBatch", 16: "IterBatch"}
|
||||||
|
for m, branch in expect.items():
|
||||||
|
r = self.router.route(mkcase(128, m, 128, 512))
|
||||||
|
self.assertEqual(r["branch"], branch, f"m={m}: {r['arbitration']}")
|
||||||
|
self.assertEqual(r["candidates"], {"MergeBatch": True, "IterBatch": True})
|
||||||
|
self.assertEqual(r["self_check_violations"], [])
|
||||||
|
|
||||||
|
def test_arbitration_text_final_winner_consistent(self):
|
||||||
|
# 仲裁文本 [裁决] 位必须是最终胜者 (分界与时延不一致时括注说明)
|
||||||
|
import re
|
||||||
|
r = self.router.route(mkcase(128, 2, 128, 512))
|
||||||
|
m = re.search(r"\[裁决\] (\w+)", r["arbitration"])
|
||||||
|
self.assertIsNotNone(m)
|
||||||
|
self.assertEqual(m.group(1), r["branch"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue36(unittest.TestCase):
|
||||||
|
"""issue#36: 合并搬移效率建模 (tile=nValue*dValue*dt 放大 b0 倍) + t_cmd_ns=0.
|
||||||
|
|
||||||
|
用户澄清: MergeBatch 合并多 batch 左/右矩阵一起搬移, 单块 tile 放大 ->
|
||||||
|
搬移效率更高, 即便 T_cmd=0 也有效益; 堆叠方向视转置 (A ND 非转置沿
|
||||||
|
M(nValue), B ND 非转置沿 N(dValue)), 乘积口径不变。
|
||||||
|
"""
|
||||||
|
|
||||||
|
def test_t_cmd_default_zero(self):
|
||||||
|
self.assertEqual(ASCEND950PR.t_cmd_ns, 0.0)
|
||||||
|
self.assertEqual(ASCEND950PR.t_cmd, 0.0)
|
||||||
|
|
||||||
|
def test_move_eff_curve(self):
|
||||||
|
from bmm_theory.models import move_eff
|
||||||
|
cap = ASCEND950PR.min_tile_size # 16KB 饱和点
|
||||||
|
self.assertEqual(move_eff(cap, cap), 1.0) # 饱和
|
||||||
|
self.assertEqual(move_eff(2 * cap, cap), 1.0) # 超出仍饱和
|
||||||
|
self.assertAlmostEqual(move_eff(cap / 4, cap), 0.25) # 之下线性
|
||||||
|
self.assertEqual(move_eff(0, cap), 1.0) # 零值守卫
|
||||||
|
|
||||||
|
def test_merge_eff_gain_beats_iterbatch(self):
|
||||||
|
# (2048,16,64,128): iter A tile=4KB eff=0.25, 合并 A'=16KB eff=1.0
|
||||||
|
# -> 效率节省 ~15.7us >> drain 0.17us, T_cmd=0 下 MergeBatch 大胜
|
||||||
|
case = mkcase(2048, 16, 64, 128)
|
||||||
|
mb = MergeBatchBranch().analyze(case)
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
self.assertTrue(mb.capable and ib.capable)
|
||||||
|
win, detail = MergeBatchBranch().beats_iterbatch(case)
|
||||||
|
self.assertTrue(win, detail)
|
||||||
|
self.assertLess(mb.timing.t_total, ib.timing.t_total)
|
||||||
|
|
||||||
|
def test_gm_bytes_unchanged_by_eff(self):
|
||||||
|
# 效率模型只影响时间列: GM 字节量仍 = V_in (issue#31 口径不被破坏)
|
||||||
|
for shp in [(128, 1, 128, 512), (2048, 16, 64, 128), (128, 64, 64, 512)]:
|
||||||
|
case = mkcase(*shp)
|
||||||
|
for br in (MergeBatchBranch(), IterBatchBranch()):
|
||||||
|
r = br.analyze(case)
|
||||||
|
if r.capable:
|
||||||
|
self.assertAlmostEqual(r.timing.gm_read_bytes,
|
||||||
|
case.input_bytes, places=3)
|
||||||
|
|
||||||
|
def test_iter_small_tile_slower_than_merge(self):
|
||||||
|
# 用户 case m=1: IterBatch A tile=1KB eff=1/16 -> t_mte2_gm 显著高于
|
||||||
|
# MergeBatch (A'=1.9KB eff=0.117), 且两者 GM 字节相同
|
||||||
|
case = mkcase(128, 1, 128, 512)
|
||||||
|
mb = MergeBatchBranch().analyze(case)
|
||||||
|
ib = IterBatchBranch().analyze(case)
|
||||||
|
self.assertGreater(ib.timing.t_mte2_gm, mb.timing.t_mte2_gm)
|
||||||
|
self.assertEqual(mb.timing.gm_read_bytes, ib.timing.gm_read_bytes)
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue37(unittest.TestCase):
|
||||||
|
"""issue#37: ASW_Basic evaluate 尾轮残余 drain 闭式化 (方案甲).
|
||||||
|
|
||||||
|
- P1: A0 + r>0 的 drain 由"全 case 三级 max"(≈2x 稳态) 修为 (1-ρ)·T_block;
|
||||||
|
- P2: 周长型 A1b/方案B 补残余 (√ρ−ρ) / (√(n_wave(n_wave−1+ρ))−(n_wave−1+ρ))·T_load;
|
||||||
|
- 面积型 A1b/方案B 与 r=0 残余恒 0 (v1.5 §4.3 严格相等);
|
||||||
|
- 块级三段时延与 _decide_tail 同源 (_block_times, L2 命中口径).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
from bmm_theory.branches.asw_basic import AswBasicBranch
|
||||||
|
self.br = AswBasicBranch()
|
||||||
|
self.s = ASCEND950PR
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _plan(case, sm, sn, m_cnt, n_cnt, strategy, r, n_wave):
|
||||||
|
from bmm_theory.models import ImplPlan
|
||||||
|
return ImplPlan(case_id=case.case_id, branch="ASW_Basic",
|
||||||
|
used_core_num=32, m_cnt=m_cnt, n_cnt=n_cnt,
|
||||||
|
single_core_m=sm, single_core_n=sn,
|
||||||
|
single_core_k=case.k, k_l1=128,
|
||||||
|
base_m=256, base_n=256, base_k=64,
|
||||||
|
tail_strategy=strategy, tail_block_cnt=r,
|
||||||
|
tail_wave_num=n_wave, fixpipe_unitflag=True,
|
||||||
|
out_dtype_bytes=case.dtype_out_bytes)
|
||||||
|
|
||||||
|
def test_block_times_formula(self):
|
||||||
|
# 块级三段 = v1.5 §2.1 口径 (k_L1 约掉, L2 命中带宽)
|
||||||
|
case = mkcase(3, 1024, 1024, 1024)
|
||||||
|
t_mm, t_mv, t_fx = self.br._block_times(case, 256, 256)
|
||||||
|
self.assertAlmostEqual(t_mm, 2 * 256 * 256 * 1024
|
||||||
|
/ self.s.q_cube("bf16", "bf16"))
|
||||||
|
self.assertAlmostEqual(t_mv, 1024 * (256 + 256) * 2 / self.s.bw_l2_pc)
|
||||||
|
self.assertAlmostEqual(t_fx, 256 * 256 * 2 / self.s.bw_pc)
|
||||||
|
|
||||||
|
def test_a0_drain_is_block_level_residual(self):
|
||||||
|
# P1 修复: A0 + r>0 的 drain = (1-ρ)·T_block (修复前误用全 case 三级
|
||||||
|
# max -> t_total ≈ 2x 稳态)
|
||||||
|
case = mkcase(3, 1024, 1024, 1024)
|
||||||
|
p = self._plan(case, 512, 128, 2, 8, "A0", r=16, n_wave=2)
|
||||||
|
t = self.br.evaluate(case, p)
|
||||||
|
t_block = max(self.br._block_times(case, 512, 128))
|
||||||
|
self.assertAlmostEqual(t.t_drain, (1 - 16 / 32) * t_block)
|
||||||
|
self.assertAlmostEqual(t.t_total, t.t_steady + t.t_drain)
|
||||||
|
self.assertLess(t.t_drain, t.t_steady) # 不再 ~2x 稳态
|
||||||
|
|
||||||
|
def test_area_dominated_tail_residual_zero(self):
|
||||||
|
# 面积型 (MMAD/FIX 主导): A1b/方案B 残余恒 0 (v1.5 §4.3 严格相等)
|
||||||
|
case = mkcase(3, 1024, 1024, 1024)
|
||||||
|
for strat in ("A1b", "方案B"):
|
||||||
|
p = self._plan(case, 512, 128, 2, 8, strat, r=16, n_wave=2)
|
||||||
|
t = self.br.evaluate(case, p)
|
||||||
|
self.assertEqual(t.t_drain, 0.0, strat)
|
||||||
|
self.assertAlmostEqual(t.t_total, t.t_steady)
|
||||||
|
|
||||||
|
def test_perimeter_a1b_residual(self):
|
||||||
|
# P2: 周长型 (块级 MTE2 主导) + A1b: drain = (√ρ−ρ)·T_load
|
||||||
|
case = mkcase(1, 640, 1408, 4096)
|
||||||
|
t_mm, t_mv, t_fx = self.br._block_times(case, 64, 64)
|
||||||
|
self.assertGreater(t_mv, max(t_mm, t_fx)) # 确认为周长型前提
|
||||||
|
p = self._plan(case, 64, 64, 10, 22, "A1b", r=28, n_wave=7)
|
||||||
|
t = self.br.evaluate(case, p)
|
||||||
|
rho = 28 / 32
|
||||||
|
self.assertAlmostEqual(t.t_drain, (rho ** 0.5 - rho) * t_mv)
|
||||||
|
self.assertAlmostEqual(t.t_total, t.t_steady + t.t_drain)
|
||||||
|
|
||||||
|
def test_perimeter_planb_residual(self):
|
||||||
|
# P2: 周长型 + 方案B: drain = (√(n_wave(n_wave−1+ρ))−(n_wave−1+ρ))·T_load
|
||||||
|
case = mkcase(1, 640, 1408, 4096)
|
||||||
|
p = self._plan(case, 64, 64, 10, 22, "方案B", r=28, n_wave=7)
|
||||||
|
t = self.br.evaluate(case, p)
|
||||||
|
t_mv = self.br._block_times(case, 64, 64)[1]
|
||||||
|
rho, x = 28 / 32, 7 - 1 + 28 / 32
|
||||||
|
self.assertAlmostEqual(t.t_drain, ((7 * x) ** 0.5 - x) * t_mv)
|
||||||
|
self.assertGreater(t.t_drain, 0.0)
|
||||||
|
|
||||||
|
def test_r0_drain_zero(self):
|
||||||
|
case = mkcase(3, 1024, 1024, 1024)
|
||||||
|
p = self._plan(case, 512, 128, 2, 8, "A0", r=0, n_wave=2)
|
||||||
|
self.assertEqual(self.br.evaluate(case, p).t_drain, 0.0)
|
||||||
|
|
||||||
|
def test_make_plan_perimeter_a1b_end_to_end(self):
|
||||||
|
# 端到端 (make_plan 自产方案): 瘦长 case 周长型 + ρ≥ρ_dv -> A1b,
|
||||||
|
# drain 与闭式一致 (_decide_tail 与 evaluate 同源)
|
||||||
|
case = mkcase(33, 16, 8192, 7168)
|
||||||
|
p = self.br.make_plan(case)
|
||||||
|
self.assertGreater(p.tail_block_cnt, 0)
|
||||||
|
self.assertEqual(p.tail_strategy, "A1b", p.note)
|
||||||
|
t_mm, t_mv, t_fx = self.br._block_times(
|
||||||
|
case, p.single_core_m, p.single_core_n)
|
||||||
|
self.assertGreater(t_mv, max(t_mm, t_fx)) # 周长型前提
|
||||||
|
t = self.br.evaluate(case, p)
|
||||||
|
rho = p.tail_block_cnt / 32
|
||||||
|
self.assertAlmostEqual(t.t_drain, (rho ** 0.5 - rho) * t_mv)
|
||||||
|
self.assertAlmostEqual(t.t_total, t.t_steady + t.t_drain)
|
||||||
|
|
||||||
|
|
||||||
|
class TestIssue38(unittest.TestCase):
|
||||||
|
"""issue#38: 昇腾950 系列多 SKU 硬件规格 (950PR 32/28核 + 950DT 36/32/28核,
|
||||||
|
白皮书表3-1/表4-2), 默认加载调用不变."""
|
||||||
|
|
||||||
|
def test_default_spec_unchanged(self):
|
||||||
|
# 默认加载调用不受影响: ASCEND950PR 各项与 NpuSpec() 关键字默认构造等价
|
||||||
|
from bmm_theory.hardware import NpuSpec, get_spec
|
||||||
|
s = ASCEND950PR
|
||||||
|
self.assertEqual((s.aic_num, s.aiv_num), (32, 64))
|
||||||
|
self.assertEqual(s.bw_gm, 1.6e12)
|
||||||
|
self.assertEqual(s.bw_l2, 5.2e12) # PR 保持 5.2TB/s
|
||||||
|
self.assertEqual(s.l2_bytes, 128 * 1024 * 1024)
|
||||||
|
self.assertAlmostEqual(s.q16, 432e12 / 32) # 13.5T (issue#40 Cube-only)
|
||||||
|
self.assertEqual(s.cube_peak_tflops, 432.0)
|
||||||
|
self.assertAlmostEqual(s.r16, 540.0) # 432e12/(1.6e12/2)
|
||||||
|
self.assertEqual(s.gm_capacity_gb, 128.0)
|
||||||
|
self.assertIs(get_spec(), ASCEND950PR) # 默认 = 950PR 主bin
|
||||||
|
self.assertIs(get_spec("Ascend950PR"), ASCEND950PR)
|
||||||
|
self.assertEqual(NpuSpec(), ASCEND950PR) # 无参构造 == 主bin
|
||||||
|
|
||||||
|
def test_spec_registry_five_skus(self):
|
||||||
|
from bmm_theory.hardware import SPECS
|
||||||
|
self.assertEqual(sorted(SPECS), [
|
||||||
|
"Ascend950DT", "Ascend950DT_C28", "Ascend950DT_C32",
|
||||||
|
"Ascend950PR", "Ascend950PR_C28"])
|
||||||
|
for s in SPECS.values():
|
||||||
|
# 共架构 + Cube-only 口径 (issue#40): 单核 Cube 全系列精确 13.5T
|
||||||
|
# (432/32=378/28=486/36); AIV 恒为 AIC 两倍
|
||||||
|
self.assertAlmostEqual(s.q16, 13.5e12, places=6)
|
||||||
|
self.assertEqual(s.aiv_num, 2 * s.aic_num)
|
||||||
|
self.assertEqual(s.l1_bytes, 512 * 1024) # 表4-2 各档一致
|
||||||
|
self.assertEqual(s.l0c_bytes, 256 * 1024)
|
||||||
|
|
||||||
|
def test_950dt_specs(self):
|
||||||
|
from bmm_theory.hardware import (ASCEND950DT, ASCEND950DT_C28,
|
||||||
|
ASCEND950DT_C32)
|
||||||
|
self.assertEqual((ASCEND950DT.aic_num, ASCEND950DT.aiv_num), (36, 72))
|
||||||
|
self.assertEqual(ASCEND950DT.bw_gm, 4.0e12) # 4TB/s HBM
|
||||||
|
self.assertEqual(ASCEND950DT.l2_bytes, 128 * 1024 * 1024)
|
||||||
|
self.assertEqual(ASCEND950DT.gm_capacity_gb, 144.0)
|
||||||
|
self.assertEqual(ASCEND950DT.cube_peak_tflops, 486.0) # Cube-only (issue#40)
|
||||||
|
# L2 带宽: DT 三档 7.5TB/s 读写各自独享 (issue#39 用户澄清)
|
||||||
|
for dt_spec in (ASCEND950DT, ASCEND950DT_C32, ASCEND950DT_C28):
|
||||||
|
self.assertEqual(dt_spec.bw_l2, 7.5e12)
|
||||||
|
self.assertAlmostEqual(ASCEND950DT.bw_l2_pc, 7.5e12 / 36)
|
||||||
|
self.assertEqual((ASCEND950DT_C32.aic_num,
|
||||||
|
ASCEND950DT_C32.cube_peak_tflops), (32, 432.0))
|
||||||
|
self.assertEqual((ASCEND950DT_C28.aic_num,
|
||||||
|
ASCEND950DT_C28.cube_peak_tflops), (28, 378.0))
|
||||||
|
self.assertEqual(ASCEND950DT_C28.gm_capacity_gb, 96.0)
|
||||||
|
# 派生量: 单核 GM 份额 4TB/36; r16 平衡点随带宽升至 4TB/s 而减半
|
||||||
|
self.assertAlmostEqual(ASCEND950DT.bw_pc, 4.0e12 / 36)
|
||||||
|
self.assertAlmostEqual(ASCEND950DT.r16, 486e12 / (4.0e12 / 2)) # 243
|
||||||
|
self.assertAlmostEqual(ASCEND950DT_C32.r16, 432e12 / (4.0e12 / 2)) # 216
|
||||||
|
# AIV 白皮书交叉验证: fp32 通量 72 核 x 128 lane x 1.65GHz x 2 ≈ 30T
|
||||||
|
self.assertAlmostEqual(ASCEND950DT.aiv_elem_rate_fp32 * 2 / 1e12,
|
||||||
|
30.0, places=0)
|
||||||
|
|
||||||
|
def test_950pr_c28(self):
|
||||||
|
from bmm_theory.hardware import ASCEND950PR_C28
|
||||||
|
self.assertEqual((ASCEND950PR_C28.aic_num,
|
||||||
|
ASCEND950PR_C28.aiv_num), (28, 56))
|
||||||
|
self.assertEqual(ASCEND950PR_C28.bw_gm, 1.4e12)
|
||||||
|
self.assertEqual(ASCEND950PR_C28.bw_l2, 5.2e12) # PR 系列保持 5.2TB/s
|
||||||
|
self.assertEqual(ASCEND950PR_C28.l2_bytes, 112 * 1024 * 1024)
|
||||||
|
self.assertEqual(ASCEND950PR_C28.gm_capacity_gb, 112.0)
|
||||||
|
|
||||||
|
def test_get_spec_unknown_raises(self):
|
||||||
|
from bmm_theory.hardware import get_spec
|
||||||
|
with self.assertRaises(KeyError):
|
||||||
|
get_spec("Ascend910")
|
||||||
|
|
||||||
|
def test_dt_router_smoke(self):
|
||||||
|
# 换芯片只换 spec: DT 路由/时延正常; 同一 case GM 段 4TB/s 快于 1.6TB/s
|
||||||
|
from bmm_theory.hardware import ASCEND950DT
|
||||||
|
case = mkcase(8, 4096, 4096, 1024)
|
||||||
|
r_dt = BranchRouter(ASCEND950DT).route(case)
|
||||||
|
r_pr = BranchRouter().route(case)
|
||||||
|
self.assertGreater(r_dt["timing"].t_total, 0.0)
|
||||||
|
self.assertLess(r_dt["timing"].t_mte2_gm, r_pr["timing"].t_mte2_gm)
|
||||||
|
def test_dt_l2_bw_effective_in_fixpipe(self):
|
||||||
|
# issue#39: S_A (整case驻留L2) 下 DT 的 fixpipe 走 7.5TB/s 写口
|
||||||
|
from bmm_theory.hardware import ASCEND950DT
|
||||||
|
case = mkcase(2, 4096, 4096, 512) # 输入+输出 < 128MB -> S_A
|
||||||
|
self.assertLess(case.input_bytes + case.output_bytes,
|
||||||
|
ASCEND950DT.l2_bytes)
|
||||||
|
r = BranchRouter(ASCEND950DT).route(case)
|
||||||
|
self.assertAlmostEqual(r["timing"].t_fixpipe,
|
||||||
|
case.output_bytes / 7.5e12)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
Reference in New Issue
Block a user