《MMO 服务器开发》2.3 热区隔离与背压

无界队列把「慢」变成「OOM」,热区不隔离让一段慢拖垮整帧。本节把 AOI、战斗等 CPU 热点拆到各自的有界队列与独立 worker 上,用三种溢出策略——拒绝、丢弃最旧、降级——显式决定「满了怎么办」,并用可跑的测试断言慢 AOI 不会阻塞移动、战斗事件永不被静默丢弃。

2.3 热区隔离与背压

2.1 量出 AOI 广播是相位大头,2.2 量出分配与回收才是时间黑洞。两节合起来指向同一个工程事实:一帧里总有几个「热点」比其他部分重得多。热点本身不可怕,可怕的是热点把别的东西一起拖下水——AOI 广播一旦因为某个玩家装备特效爆炸而变慢,同一帧里的移动、战斗、定时器全部被卡在后面,玩家看到的是「整个世界卡住」。这一节解决两件事:怎么把热点关起来(热区隔离),以及关起来之后队列满了怎么办(背压)。

无界队列的陷阱

最直觉的「解耦」是给热点接一个 chan 或 list,让生产者无脑往里塞、消费者慢慢取。这在演示里很优雅,在线上是灾难:队列无界,等于把「处理不过来」这个事实藏起来。 生产者继续以 30Hz 塞入,消费者以 20Hz 取出,队列以每帧 10 个的速度无限增长——半小时后它吃光内存,进程被 OOM Killer 干掉。故障从「服务变慢」升级成「服务消失」,还顺手把同一台机器上的其他进程一起带走。

所以正确的做法只有一条:队列必须有界,且「满了怎么办」必须是一个被写下来的策略,不是运行时随机撞出来的行为。一个没有显式溢出策略的队列,等于一个没定义行为的系统。

热区隔离:把每个热点关进自己的队列

隔离的核心是一个热点一个队列、一个 worker,让慢只慢它自己。aegis 把 tick 里的三个热点——移动、AOI、战斗——各配一条独立的有界队列,Drain 在各自的 goroutine 上跑。这样某个热点的 worker 卡住,只会把自己的队列填满,其他热点的 worker 照常排水。下面这段是 L1:

// file: internal/profile/isolation.go
// tier: L1
// verify: go test ./internal/profile/ -run TestHotspotIsolation
package profile

// Hotspot names a CPU-heavy stage of a tick that aegis runs on its own worker.
type Hotspot string

const (
	HotspotMovement Hotspot = "movement"
	HotspotAOI      Hotspot = "aoi"
	HotspotCombat   Hotspot = "combat"
)

// IsolatedLoop runs each hotspot on its own goroutine with its own bounded
// queue. A stalled hotspot can only fill its own queue; the others keep
// draining, so a slow AOI broadcast cannot stall movement. This is the
// structural fix for the "one slow phase eats the whole tick" failure — the
// per-hotspot queue is the blast radius of that hotspot's slowness.
type IsolatedLoop struct {
	queues map[Hotspot]*BoundedQueue
}

// NewIsolatedLoop builds a loop whose hotspots each get a queue of the given
// capacity. Movement and AOI drop the oldest under pressure (latest position
// wins); combat rejects, because dropping a damage event corrupts the fight.
func NewIsolatedLoop(capacity int) *IsolatedLoop {
	return &IsolatedLoop{queues: map[Hotspot]*BoundedQueue{
		HotspotMovement: NewBoundedQueue(capacity, OverflowDropOldest),
		HotspotAOI:      NewBoundedQueue(capacity, OverflowDropOldest),
		HotspotCombat:   NewBoundedQueue(capacity, OverflowReject),
	}}
}

// Submit enqueues work for one hotspot.
func (l *IsolatedLoop) Submit(h Hotspot, v any) any { return l.queues[h].Push(v) }

// Queue returns one hotspot's queue, e.g. for metrics.
func (l *IsolatedLoop) Queue(h Hotspot) *BoundedQueue { return l.queues[h] }

// Drain runs fn over up to max items of one hotspot and returns how many it
// processed. Running this on a per-hotspot worker goroutine is what isolates
// the hotspots from each other: a slow fn only slows its own hotspot.
func (l *IsolatedLoop) Drain(h Hotspot, max int, fn func(any)) int {
	q := l.queues[h]
	n := 0
	for n < max {
		v, ok := q.Pop()
		if !ok {
			break
		}
		fn(v)
		n++
	}
	return n
}

注意 NewIsolatedLoop 给不同热点配了不同的溢出策略:移动和 AOI 用「丢弃最旧」,战斗用「拒绝」。这不是随手写的,是业务语义决定的——见下一节。

有界队列与三种背压策略

队列满时只有三种诚实的选择,各自对应一类业务语义:

  • 拒绝(Reject):新来的不要,返回失败,让上游感知到压力。适合每个事件都重要的流——战斗伤害、扣钱、任务完成。丢掉一次伤害,战斗结果就错了,宁可让上游重试或报错。
  • 丢弃最旧(DropOldest):踢掉队头最老的,腾位子给最新的。适合最新状态即真理的流——AOI 位置快照。玩家 200ms 前的位置早没意义了,最新的才重要。
  • 降级(Degrade):踢掉最旧的,但存进去的是更便宜的形态。适合丢细节比丢事件好的流——AOI 更新保留坐标、丢掉特效列表。

下面这段是 L1,把三种策略连同计数器做成一个可断言的环形有界队列:

// file: internal/profile/backpressure.go
// tier: L1
// verify: go test ./internal/profile/ -run TestBoundedQueue
package profile

import "sync/atomic"

// Overflow is the policy a bounded queue applies when it is full. The whole
// point of naming the policy is that "what happens when we are full" is a
// decision, not an accident.
type Overflow int

const (
	// OverflowReject refuses the newest item and signals the producer to slow
	// down. Correct when every item matters and the producer can shed load.
	OverflowReject Overflow = iota
	// OverflowDropOldest evicts the oldest queued item to make room. Correct
	// for "latest state wins" streams such as AOI snapshots.
	OverflowDropOldest
	// OverflowDegrade evicts the oldest but stores the new item in a cheaper
	// form. Correct when losing detail beats losing the event.
	OverflowDegrade
)

// Degradable is a payload that can be reduced to a cheaper form under pressure,
// e.g. an AOI update that keeps position but drops effects.
type Degradable interface {
	Degrade() any
}

// Degrade reduces v to a cheaper form if it implements Degradable, else returns
// v unchanged.
func Degrade(v any) any {
	if d, ok := v.(Degradable); ok {
		return d.Degrade()
	}
	return v
}

// Counters records what a bounded queue did under pressure. Every field is
// atomic so a metrics exporter can scrape it while the queue is live.
type Counters struct {
	Enqueued atomic.Uint64
	Dequeued atomic.Uint64
	Rejected atomic.Uint64
	Dropped  atomic.Uint64
	Degraded atomic.Uint64
}

// BoundedQueue is a fixed-capacity queue with an explicit overflow policy. An
// unbounded queue does not remove backpressure — it converts a throughput
// problem into a memory-exhaustion one, and moves the failure from "slow" to
// "OOM-killed". Capacity is therefore a required constructor argument.
//
// A BoundedQueue is not internally synchronized: it is owned by a single
// goroutine (the hotspot's worker), which is exactly how section 2.3 wires it.
// The Counters are atomic so a metrics scraper on another goroutine can read
// them without a lock, but Push and Pop themselves must not run concurrently.
type BoundedQueue struct {
	items  []any
	head   int
	size   int
	policy Overflow
	count  Counters
}

// NewBoundedQueue returns a queue with the given capacity and overflow policy.
func NewBoundedQueue(capacity int, policy Overflow) *BoundedQueue {
	if capacity <= 0 {
		panic("profile: queue capacity must be positive")
	}
	return &BoundedQueue{items: make([]any, capacity), policy: policy}
}

// Len is the number of queued items.
func (q *BoundedQueue) Len() int { return q.size }

// Cap is the fixed capacity.
func (q *BoundedQueue) Cap() int { return len(q.items) }

// Counters exposes the queue's live counters.
func (q *BoundedQueue) Counters() *Counters { return &q.count }

// Push enqueues v, applying the overflow policy when full. It returns the value
// actually stored, or nil when the item was rejected.
func (q *BoundedQueue) Push(v any) any {
	if q.size < len(q.items) {
		q.items[(q.head+q.size)%len(q.items)] = v
		q.size++
		q.count.Enqueued.Add(1)
		return v
	}
	switch q.policy {
	case OverflowDropOldest:
		q.items[q.head] = v // overwrite the oldest slot with the newest value
		q.head = (q.head + 1) % len(q.items)
		q.count.Dropped.Add(1)
		q.count.Enqueued.Add(1)
		return v
	case OverflowDegrade:
		q.items[q.head] = Degrade(v)
		q.head = (q.head + 1) % len(q.items)
		q.count.Degraded.Add(1)
		q.count.Enqueued.Add(1)
		return q.items[(q.head+q.size-1)%len(q.items)]
	default: // OverflowReject
		q.count.Rejected.Add(1)
		return nil
	}
}

// Pop removes and returns the oldest item.
func (q *BoundedQueue) Pop() (any, bool) {
	if q.size == 0 {
		return nil, false
	}
	v := q.items[q.head]
	q.items[q.head] = nil
	q.head = (q.head + 1) % len(q.items)
	q.size--
	q.count.Dequeued.Add(1)
	return v, true
}

NewBoundedQueue 对 capacity <= 0 直接 panic——容量不是可选项。这是刻意的设计:无界队列是一个应该被禁止的默认值,逼调用方每次都想清楚「这条队列最多积压多少」。

实测断言

隔离和背压最容易写成「看起来对」,所以要用测试把它们钉住。四条关键断言:

  • 慢 AOI 不阻塞移动:AOI 的 worker 阻塞在 <-release 上,移动的 worker 照常在 100ms 内排空自己 8 个任务——若两者共享队列,移动会一起卡住。
  • 溢出影响是局部的:往 AOI 塞 100 个(容量 4)后,移动队列的 Dropped 仍为 0。
  • 战斗拒绝而非丢弃:容量 2 的战斗队列塞 3 个,Dropped == 0 且 Rejected == 1。
  • 计数器守恒:Enqueued == Dequeued + Dropped + Len。

本机实跑(go test ./internal/profile/ -run 'TestHotspot|TestCombatRejects' -v):

=== RUN   TestHotspotIsolation
--- PASS: TestHotspotIsolation (0.00s)
=== RUN   TestHotspotQueueOverflowIsLocal
--- PASS: TestHotspotQueueOverflowIsLocal (0.00s)
=== RUN   TestCombatRejectsRatherThanDrops
--- PASS: TestCombatRejectsRatherThanDrops (0.00s)
PASS
ok  	aegis/internal/profile	0.569s
=== RUN   TestBoundedQueueReject
--- PASS: TestBoundedQueueReject (0.00s)
=== RUN   TestBoundedQueueDropOldest
--- PASS: TestBoundedQueueDropOldest (0.00s)
=== RUN   TestBoundedQueueDegrade
--- PASS: TestBoundedQueueDegrade (0.00s)
=== RUN   TestBoundedQueueCapacityIsMandatory
--- PASS: TestBoundedQueueCapacityIsMandatory (0.00s)
=== RUN   TestCountersAccumulate
--- PASS: TestCountersAccumulate (0.00s)
PASS
ok  	aegis/internal/profile	0.550s

TestHotspotIsolation 是这一节的灵魂:它把「AOI 卡住不能拖垮移动」从一句口号变成一条会变红的测试。将来有人为了「省 goroutine」把三个热点合并回一个队列,这条测试会立刻失败。TestCombatRejectsRatherThanDrops 守的是另一条底线——战斗事件永远不被静默丢弃,只能被拒绝,让上游看见。

本节小结 / 实测结论

  1. 无界队列是隐藏的故障:它不消除背压,只把「处理不过来」变成「内存耗尽」,把故障从「慢」升级成「OOM 被杀」。所以容量必须是构造参数,NewBoundedQueue(0) 直接 panic。
  2. 热区隔离 = 一热点一队列一 worker:慢只慢它自己。IsolatedLoop 让 AOI 的阻塞无法传导到移动,测试断言了这一点。
  3. 背压是策略,不是行为:拒绝(每个事件都重要)、丢弃最旧(最新状态即真理)、降级(丢细节不丢事件)三种,按业务语义选;计数器暴露在指标里,运维才能看见队列在何时开始丢东西。

至此第 2 章的调优三件套齐了:先量(帧预算 + 剖析),再对照(Go vs JVM GC),最后结构性兜底(隔离 + 背压)。下一章把视角从「单帧内」抬到「全服务」——可观测性。

阅读导航:上一节:2.2 双栈对照:Go GC vs JVM GC · 下一节:3.1 日志该记什么 。

继续阅读

探索更多技术文章

浏览归档,发现更多关于系统设计、工具链和工程实践的内容。

全部文章 返回首页

「mmo」更多文章

  1. 《MMO 服务器开发》工程化与运营 · 目录
  2. 《MMO 服务器开发》9.3 上线检查清单
  3. 《MMO 服务器开发》9.2 性能预算达标