In short重點
Most pixels in a street scene are easy to classify — road, sky, building. FoR-Net learns an importance map, hard-selects the top K% of locations, and spends its multi-scale reasoning there. 80.5% mIoU on Cityscapes with 55.7M parameters and no attention module anywhere.
街景影像中大多數的像素都容易分類——道路、天空、建築。FoR-Net 學習一張重要性圖，以硬選擇取出前 K% 的位置，並把多尺度推理集中在那裡。在 Cityscapes 上以 55.7M 參數達到 80.5% mIoU，且架構中沒有任何注意力模組。
The problem問題
Segmentation frameworks generally process every spatial location with the same effort. In a high-resolution street scene that is a poor allocation: only a small fraction of pixels sit on boundaries or thin structures, and those are precisely the ones that are underrepresented in training and most likely to be lost to over-smoothing.
Soft attention is the usual answer, but soft weights still distribute some capacity everywhere. This work asks what happens when the selection is made hard instead.
分割框架通常對每一個空間位置投入相同的運算。在高解析度的街景中，這是很差的資源配置： 只有一小部分的像素落在邊界或細長結構上，而這些恰好是訓練中樣本最少、也最容易被過度平滑抹掉的部分。
常見的解法是柔性注意力，但柔性權重仍然會把一部分算力分配到每個地方。 本研究要問的是：若改為硬性選擇，結果會如何。
The idea作法
A selector module runs on the high-level feature map and produces an importance map S ∈ [0,1]. A Top-K operation converts it into a binary mask, so only the selected locations survive into the reasoning stage. Unlike soft attention, this enforces genuine sparsity — unselected regions contribute nothing.
Four parallel convolution branches then reason over the selected features, each with a different receptive field and, importantly, a different selection ratio:
- 1×1 — boundary-sensitive detail, top 10%
- 3×3 — small semantic objects, top 20%
- 5×5 — medium contextual aggregation, top 30%
- 7×7 dilated — large semantic context, top 40%
The pairing matters. Narrow receptive fields get the strictest selection because their job is precise boundary work; wide branches need broader coverage or contextual propagation gets starved. An auxiliary boundary loss supervises the selector directly, so the importance map learns to align with real edges rather than drifting.
選擇器模組作用於高階特徵圖，輸出一張重要性圖 S ∈ [0,1]。 接著由 Top-K 運算將其轉為二值遮罩，只有被選中的位置能進入後續的推理階段。 與柔性注意力不同，這種作法強制產生真正的稀疏性——未被選中的區域完全不貢獻。
隨後由四個平行的卷積分支對選中的特徵進行推理，各自具有不同的感受野，以及——這點很關鍵——不同的選取比例：
- 1×1 — 邊界敏感細節，取前 10%
- 3×3 — 小型語義物件，取前 20%
- 5×5 — 中等尺度的上下文聚合，取前 30%
- 7×7 空洞卷積 — 大尺度語義上下文，取前 40%
這個搭配是有意義的。窄感受野搭配最嚴格的選取，因為它的任務是精細的邊界處理； 寬分支則需要更大的覆蓋率，否則上下文傳播會被餓死。 此外，一個輔助的邊界損失直接監督選擇器，讓重要性圖學會對齊真實的邊緣，而不會漂移。
Results實驗結果
| Method | Params | GFLOPs | mIoU |
|---|---|---|---|
| DeepLabV3 | 60.22 | 254.0 | 77.23 |
| PSPNet | 65.60 | 256.0 | 78.40 |
| DeepLabV3+ | 84.74 | 348.0 | 79.55 |
| CCNet | 66.13 | 276.0 | 80.5 |
| DNL | 71.49 | 343.0 | 80.5 |
| FoR-Net (ours) | 55.72 | 243.6 | 80.5 |
| 方法 | 參數量 | GFLOPs | mIoU |
|---|---|---|---|
| DeepLabV3 | 60.22 | 254.0 | 77.23 |
| PSPNet | 65.60 | 256.0 | 78.40 |
| DeepLabV3+ | 84.74 | 348.0 | 79.55 |
| CCNet | 66.13 | 276.0 | 80.5 |
| DNL | 71.49 | 343.0 | 80.5 |
| FoR-Net (ours) | 55.72 | 243.6 | 80.5 |
FoR-Net matches CCNet and DNL while using 16% and 22% fewer parameters respectively, and roughly 30% fewer GFLOPs than DNL. It trains with a plain combination of cross-entropy, Dice, and the selector loss — no multi-stage schedule, no attention refinement, no elaborate regularisation.
FoR-Net 在準確度上與 CCNet、DNL 持平，但參數量分別少了 16% 與 22%，GFLOPs 也比 DNL 少約 30%。它的訓練只用了交叉熵、Dice 與選擇器損失的單純組合——沒有多階段排程、沒有注意力修復模組，也沒有複雜的正則化設計。
| Dilation | mIoU |
|---|---|
| 1 (none) | 77.42 |
| 2 | 78.59 |
| 4 | 79.44 |
| 8 | 80.16 |
| 16 | 80.49 |
| 空洞率 | mIoU |
|---|---|
| 1 (none) | 77.42 |
| 2 | 78.59 |
| 4 | 79.44 |
| 8 | 80.16 |
| 16 | 80.49 |
Larger receptive fields help consistently, but the marginal gain shrinks — from +1.17 at the first step down to +0.33 at the last. There is a saturation point where extra context stops paying for itself, which is the same balance between local detail and global reach that motivates the whole design.
更大的感受野持續帶來幫助，但邊際增益逐步縮小——從第一步的 +1.17 降到最後一步的 +0.33。確實存在一個飽和點，越過之後額外的上下文就不再划算，而這正是整個設計背後所要平衡的：局部細節與全域視野之間的取捨。
Limitations限制
The Top-K ratios are fixed rather than adaptive, so a scene with an unusual proportion of hard regions is not served optimally. Evaluation is confined to Cityscapes under a standard training setup, and comparison against the strongest recent architectures is limited — the goal here was to test whether hard region selection works as an inductive bias under practical constraints, not to top a leaderboard.
Top-K 比例是固定的，並非自適應，因此當場景中困難區域的比例異常時，配置就不會是最佳的。評估僅限於 Cityscapes 且採用標準訓練設定，與最新最強架構的比較也有限——這篇論文的目標是在實務限制下檢驗「困難區域硬選擇」是否能作為有效的歸納偏置，而不是刷排行榜。