《MMO 服务器开发》1.1 分区分服与集群拓扑

卷一造出了单机的网络内核,卷二装上了单服的玩法系统,可一旦在线人数上来,一台机器就扛不住了。这一节先把集群的三条路线——分区、分服、全球同服——摊开对比,再把网关、场景、战斗、世界、目录五类节点的职责边界定清楚,最后用一份能跑的 Go 代码把「哪个节点持有什么状态」钉成断言。

1.1 分区分服与集群拓扑

前两卷造出的 aegis 一直有个隐含前提:所有玩家都在一台机器上。卷一《网络与并发内核》把 socket、编解码、KCP、时间轮、Actor 都写完了,卷二《单服架构与玩法系统》把背包、任务、公会、交易、匹配、战斗管线都装上了——但这些统统是「一个进程里」的架构。真正的 MMO 从上线第一天起就会撞上三堵墙:一台机器的 CPU 撑不住全部 CCU、一次重启会把所有在线玩家踢下线、两个不同服的朋友想一起打副本却根本不在同一个世界里。

卷三《分布式与跨服》要解决的就是这三堵墙。而解决的第一步不是写代码,是先把「拆成什么样」想清楚。拓扑没定下来,后面每一章都会各写各的:服务发现按一套假设注册节点、RPC 按另一套假设定超时、网关又按第三套假设做粘性,最后拼不到一起。所以这一节不讲协议、不写中间件,只做两件事:把「分区、分服、全球同服」三种形态摊开对比,再把集群里五类节点的职责边界定死。定完这两件事,后面十章的每一块零件才有地方安。

三种形态:分区、分服、全球同服

行业里把集群形态讲得很乱,其实落到「玩家能不能任意相遇」这一条上,只有三种:

  • 分区(zone):世界被切成若干分区,玩家在进入时就绑定到一个区,不同区的玩家永远不会在同一个世界里出现。区的边界是硬隔离——这也是它的优点:一个区出故障,只影响这个区的人。
  • 分服(server):每个服是一个自包含的完整世界,玩家进入时就选定了服。跨服玩法(跨服战场、跨服拍卖)要单独做一条跨服通道,普通玩法不跨服。它比分区更彻底,运维也更简单:一个服就是一套独立的进程组。
  • 全球同服(global single shard):只有一个逻辑世界,任意两个玩家都可能被放进同一场对局。它必须用动态分片 + 路由层把「谁在哪」实时算出来,代价是路由层成了全局单点,必须高可用。

三种形态的对照:

形态玩家能否任意相遇是否必须路由层故障隔离典型代价
分区 zone否,同区才能否一个分区宕机只影响本区热门区与鬼区冷热不均
分服 server否,跨服玩法才相遇是(仅跨服时)一个服宕机只影响本服玩家被锁在服内,跨服要单独做
全球同服 global是是(始终)无硬边界,靠背压与排空兜底路由层是全局单点,必须高可用

这张表要读出的关键,是**「能否任意相遇」与「是否需要路由层」是绑定的**:只要玩家不能任意相遇,状态归属就能在登录时静态定下来,节点不需要随时查表;一旦要求任意相遇,归属就必须在运行时动态决定,于是目录层从「可有可无」变成「必须存在且不能挂」。大部分团队其实是从分区或分服起步的——它简单、隔离好、运维省事,只有当产品明确需要「全区全服一起打」时,才值得付全球同服的复杂度。

五类节点的职责边界

定完形态,下一步是把集群拆成几类进程。aegis 的集群有五类节点,每类节点的边界由它持有的状态决定,不由代码目录决定——这条原则在卷二 1.1 已经立过,这里把它从「单服的四个逻辑层」升级成「集群的五类节点」:

节点持有状态归属粒度单写者横向扩展
网关 gateway连接:心跳、重连票据、限速窗口每连接一份否(每连接各持副本)是
场景 scene地图实例:坐标、AOI、刷怪每实例一份是是
战斗 battle一场对局:队伍、冷却、比分每对局一份是是
世界 world玩家:背包、任务、公会、社交每玩家一份是是
目录 directory路由表:id → 归属节点复制控制面否(Leader 写、多副本读)是

逐行看它们的边界:

  • 网关持连接状态——心跳时间、重连票据、限速窗口。这些状态每个连接一份,所以网关天然可复制:三台网关各持自己那份连接副本,永远不会分叉。网关不认识「玩家」也不认识「地图」,所以它可以无脑加机器扛连接。
  • 场景持地图实例状态——坐标、AOI、刷怪。以实例为键,一个实例只能有一个写者,这是不可复制的硬约束。同一个人在同一块地图上的位置,只能由一台场景进程说了算。
  • 战斗持一场对局的状态——队伍、冷却、比分。它和场景一样是单写者,区别在于生命周期:场景实例常驻,战斗实例打完就销毁。战斗节点因此是「用完即走」的,调度器可以按需起停。
  • 世界持玩家状态——背包、任务、公会、社交。以 PlayerID 为键,同一玩家只有一个写者,但不同玩家可以分到不同的世界节点。它是唯一持「跨场景持久状态」的节点。
  • 目录不持任何玩家状态,只持路由表——「玩家 P 现在在哪个世界节点、实例 I 现在在哪个场景节点」。它是唯一被刻意复制的节点:多副本读、Leader 写,因为一旦目录挂了,整个集群就找不到彼此。

这五类节点里,前四类都是「自己状态的单写者」,只有目录是复制控制面。这个不对称很关键:单写者保证了状态不分叉,而复制控制面保证了路由永远查得到。 两类节点用完全不同的高可用策略——前者靠「一个实例一个进程」,后者靠「Leader 选举 + 多副本」。第 2 章讲 Leader 选举时,会专门回来处理目录这一类的所有权问题。

可运行产物:把拓扑写成表

光画图不够。节点边界如果不落成代码,半年后一定会有人把「移动」的处理挪进世界节点,因为它「看起来也是玩家发来的」。本节的可运行产物把两条判断——五类节点各持有什么、能不能复制、三种形态能不能任意相遇、要不要路由层——编码成两张常量表,并用测试锁住它们。

下面这段是 L1,go test 可以直接跑。先看 internal/topology/topology.go:

// file: internal/topology/topology.go
// tier: L1
// verify: go test ./internal/topology/
// Package topology encodes the two questions chapter one asks about a cluster:
// which node roles exist, and which of the three deployment shapes (sharded
// zones, split servers, one global shard) a title is running.
//
// The node table is the important one. A role is defined by the state it owns,
// not by the directory it lives in: gateway owns connections, scene owns a map
// instance, battle owns one match, world owns a player, directory owns the
// routing table. Every role except directory is single-writer for the state it
// holds, which is what makes the cluster shardable at all.
package topology

// Role names one kind of process in an aegis cluster.
type Role string

const (
	// RoleGateway holds connection state: handshake, heartbeat, reconnect
	// ticket, rate-limit window. One copy per socket, so gateways replicate
	// freely and scale horizontally.
	RoleGateway Role = "gateway"
	// RoleScene holds map-instance state: authoritative X/Y, AOI, spawns.
	// Exactly one process mutates a given instance.
	RoleScene Role = "scene"
	// RoleBattle holds one match's state: teams, cooldowns, round score. It is
	// created per match and thrown away afterwards.
	RoleBattle Role = "battle"
	// RoleWorld holds a player's persistent state: bag, quest, guild, social.
	// Keyed by PlayerID, so players shard across world nodes.
	RoleWorld Role = "world"
	// RoleDirectory holds no player state: it owns the routing table that maps
	// PlayerID / InstanceID / NodeID to the node that currently owns it.
	RoleDirectory Role = "directory"
)

// Ownership describes how a role's state may be placed across processes.
type Ownership string

const (
	// OwnPerConnection state is copied once per open socket; two gateways each
	// holding a copy cannot disagree.
	OwnPerConnection Ownership = "per-connection"
	// OwnPerInstance state has exactly one writer, keyed by instance ID.
	OwnPerInstance Ownership = "per-instance"
	// OwnPerPlayer state has exactly one writer per player, keyed by PlayerID.
	OwnPerPlayer Ownership = "per-player"
	// OwnReplicated state is a control-plane copy that many nodes read and one
	// leader writes; it is deliberately replicated for availability.
	OwnReplicated Ownership = "replicated-control-plane"
)

// NodeRole is the section 1.1 boundary table in code.
type NodeRole struct {
	Role      Role
	Owns      string
	Ownership Ownership
	// SingleWriter is true when exactly one process may mutate this state.
	// Every role but the directory is single-writer for what it holds.
	SingleWriter bool
	// Horizontal is true when the role scales out by adding identical nodes
	// with no coordination beyond the routing table.
	Horizontal bool
}

// nodeRoles is the five-node boundary table of section 1.1.
var nodeRoles = []NodeRole{
	{RoleGateway, "connection: heartbeat, reconnect ticket, rate limit", OwnPerConnection, false, true},
	{RoleScene, "map instance: position, AOI, spawns", OwnPerInstance, true, true},
	{RoleBattle, "one match: teams, cooldowns, score", OwnPerInstance, true, true},
	{RoleWorld, "player: bag, quest, guild, social", OwnPerPlayer, true, true},
	{RoleDirectory, "routing table: id -> owning node", OwnReplicated, false, true},
}

// RoleOf returns the boundary entry for a role.
func RoleOf(r Role) (NodeRole, bool) {
	for _, n := range nodeRoles {
		if n.Role == r {
			return n, true
		}
	}
	return NodeRole{}, false
}

// AllRoles returns the five roles in table order.
func AllRoles() []Role {
	out := make([]Role, 0, len(nodeRoles))
	for _, n := range nodeRoles {
		out = append(out, n.Role)
	}
	return out
}

// Form names one of the three deployment shapes.
type Form string

const (
	// FormZone is the classic "分区": the world is cut into zones, and players
	// in different zones never meet. The zone boundary is a hard partition.
	FormZone Form = "zone"
	// FormServer is "分服": whole independent servers, each a self-contained
	// world; interaction across servers happens only through explicit
	// cross-server features.
	FormServer Form = "server"
	// FormGlobal is "全球同服": one logical world, scaled with dynamic shards
	// so any two players can meet. It pays for that reach with a routing layer.
	FormGlobal Form = "global"
)

// Shape is the section 1.1 comparison of the three forms.
type Shape struct {
	Form Form
	// MeetAnyPair is true when any two players can be placed in one match.
	MeetAnyPair bool
	// NeedsRouting is true when a directory/routing layer is mandatory, because
	// state ownership is decided at runtime rather than fixed at login.
	NeedsRouting bool
	// Isolation is what the shape buys you when one part of the world fails.
	Isolation string
}

// shapes is the three-form comparison table of section 1.1.
var shapes = []Shape{
	{FormZone, false, false, "a zone outage is contained to that zone"},
	{FormServer, false, true, "a server outage is contained to that server"},
	{FormGlobal, true, true, "no hard boundary; needs backpressure and draining to contain failure"},
}

// ShapeOf returns the comparison entry for a form.
func ShapeOf(f Form) (Shape, bool) {
	for _, s := range shapes {
		if s.Form == f {
			return s, true
		}
	}
	return Shape{}, false
}

再配一个测试 internal/topology/topology_test.go,断言四条不变量:五类节点齐全、只有目录不是单写者、网关与世界持有不同粒度的状态、只有全球同服才能任意相遇:

// file: internal/topology/topology_test.go
// tier: L1
// verify: go test ./internal/topology/
package topology

import "testing"

func TestFiveRolesArePresent(t *testing.T) {
	want := []Role{RoleGateway, RoleScene, RoleBattle, RoleWorld, RoleDirectory}
	got := AllRoles()
	if len(got) != len(want) {
		t.Fatalf("AllRoles() has %d roles, want %d", len(got), len(want))
	}
	for i := range want {
		if got[i] != want[i] {
			t.Errorf("AllRoles()[%d] = %s, want %s", i, got[i], want[i])
		}
	}
}

func TestSingleWriterRoles(t *testing.T) {
	// A role is single-writer when exactly one process may mutate the state it
	// holds. gateway owns per-connection state (each socket has its own copy),
	// and directory is a replicated control plane; both may hold more than one
	// copy. scene, battle and world are the single writers.
	single := map[Role]bool{
		RoleGateway:   false,
		RoleScene:     true,
		RoleBattle:    true,
		RoleWorld:     true,
		RoleDirectory: false,
	}
	for _, r := range AllRoles() {
		n, ok := RoleOf(r)
		if !ok {
			t.Fatalf("RoleOf(%s): not found", r)
		}
		if n.SingleWriter != single[r] {
			t.Errorf("RoleOf(%s).SingleWriter = %v, want %v", r, n.SingleWriter, single[r])
		}
	}
}

func TestGatewayAndWorldOwnDifferentKeys(t *testing.T) {
	gw, _ := RoleOf(RoleGateway)
	world, _ := RoleOf(RoleWorld)
	if gw.Ownership != OwnPerConnection {
		t.Errorf("gateway ownership = %s, want per-connection", gw.Ownership)
	}
	if world.Ownership != OwnPerPlayer {
		t.Errorf("world ownership = %s, want per-player", world.Ownership)
	}
	if gw.Ownership == world.Ownership {
		t.Fatal("gateway and world must not share an ownership scope")
	}
}

func TestEveryRoleScalesHorizontally(t *testing.T) {
	for _, r := range AllRoles() {
		n, _ := RoleOf(r)
		if !n.Horizontal {
			t.Errorf("role %s must scale out by adding nodes", r)
		}
	}
}

func TestOnlyGlobalShapeMeetsAnyPair(t *testing.T) {
	for _, f := range []Form{FormZone, FormServer, FormGlobal} {
		s, ok := ShapeOf(f)
		if !ok {
			t.Fatalf("ShapeOf(%s): not found", f)
		}
		wantMeet := f == FormGlobal
		if s.MeetAnyPair != wantMeet {
			t.Errorf("ShapeOf(%s).MeetAnyPair = %v, want %v", f, s.MeetAnyPair, wantMeet)
		}
	}
}

func TestGlobalShapeNeedsRouting(t *testing.T) {
	// The global single shard is the only shape where ownership is decided at
	// runtime, so it is the only one where a routing layer is mandatory.
	g, _ := ShapeOf(FormGlobal)
	if !g.NeedsRouting {
		t.Fatal("the global shape must require a routing layer")
	}
	z, _ := ShapeOf(FormZone)
	if z.NeedsRouting {
		t.Fatal("a hard zone boundary must not require a runtime routing layer")
	}
}

本机实跑结果(go test ./internal/topology/ -v):

=== RUN   TestFiveRolesArePresent
--- PASS: TestFiveRolesArePresent (0.00s)
=== RUN   TestSingleWriterRoles
--- PASS: TestSingleWriterRoles (0.00s)
=== RUN   TestGatewayAndWorldOwnDifferentKeys
--- PASS: TestGatewayAndWorldOwnDifferentKeys (0.00s)
=== RUN   TestEveryRoleScalesHorizontally
--- PASS: TestEveryRoleScalesHorizontally (0.00s)
=== RUN   TestOnlyGlobalShapeMeetsAnyPair
--- PASS: TestOnlyGlobalShapeMeetsAnyPair (0.00s)
=== RUN   TestGlobalShapeNeedsRouting
--- PASS: TestGlobalShapeNeedsRouting (0.00s)
PASS
ok  	aegis/internal/topology	0.464s

TestGlobalShapeNeedsRouting 这条断言值得单独说一句。它把「全球同服必须配路由层」这条架构判断,变成了一条可以失败的测试:将来有人想把全球同服当成「分区去掉边界」来做,测试会立刻变红。同样,TestSingleWriterRoles 守的是「谁能复制、谁不能」——哪天有人想把场景状态也做成多副本「提高可用性」,测试会先拦住他。架构判断只有变成断言,才不会被下一个人改掉。

本节小结

这一节确立了三件事:

  1. 集群形态只有三种,判据是「玩家能否任意相遇」。分区与分服是硬隔离、归属可在登录时静态定死;全球同服要求任意相遇,因此必须引入运行时的路由层,并承担路由层成为全局单点的代价。
  2. 五类节点各有明确的职责边界:网关持连接状态(可复制)、场景持地图实例状态(单写者、常驻)、战斗持对局状态(单写者、用完即走)、世界持玩家状态(按 PlayerID 分片)、目录持路由表(唯一的复制控制面)。
  3. 边界是可执行的:「哪个节点持有什么」「哪种形态要不要路由层」被编码成两张常量表和一组件是断言,不是注释里的约定。

拓扑定下来了,但「拆」这件事本身有成本。下一节不急着搭集群,先回答一个更前置的问题:一台机器到底能扛到多少人才值得拆,拆开又换来什么、付出什么。

阅读导航:上一节:目录 · 下一节:1.2 什么时候该拆、什么时候不该 。

继续阅读

探索更多技术文章

浏览归档,发现更多关于系统设计、工具链和工程实践的内容。

全部文章 返回首页

「mmo」更多文章

  1. 《MMO 服务器开发》工程化与运营 · 目录
  2. 《MMO 服务器开发》9.3 上线检查清单
  3. 《MMO 服务器开发》9.2 性能预算达标