《MMO 服务器开发》1.2 机器人压测平台

站内没有 k6、wrk,压测器只能自己写。本节从零搭一个可编排的机器人压测平台:机器人状态机走完连接→登录→进场景→移动→心跳,并发编排用错峰建连绕开 macOS 的 accept 队列上限、用起跑线屏障把建连爬坡挤出测量窗口,最后真起一个本地回环服务端加五十个机器人跑通,并把结果落成 JSON。

1.2 机器人压测平台

1.1 定义了承载指标,可指标不会自己从服务端里蹦出来——得有一批客户端去产生负载、去计时、去汇总。站内没有 k6、wrk、hey 这类现成工具,所以这一节自己写一个。

一个能用的压测机器人必须做对三件事。第一,它要走完整路径,而不是只 dial 一下挂着——真正压的是「登录、进场景、移动、心跳」这条链路,而不是 TCP 三次握手。第二,它要错峰建连:macOS 的 kern.ipc.somaxconn 默认只有 128,一口气 dial 上千条会被内核直接 RST,测出来的「失败」其实是压测器自己造的。第三,它要报分位延迟而不只是总量——P99 才决定玩家会不会骂人。

本节给出这套平台的三块零件:机器人状态机、并发编排、以及一个把建连爬坡挤出测量窗口的起跑线屏障,最后真起一个回环服务端跑通。

机器人状态机:连接 → 登录 → 进场景 → 移动 → 心跳

机器人本身就是用第一卷那套 11 字节定长头帧协议说话的普通客户端。把它写成一个显式状态机的好处是:哪一步失败,报告里就写哪一步,而不是一个匿名的 error。下面这段是 L2,go test 会真的起进程、连真 socket:

// file: internal/loadtest/bot.go
// tier: L2
// verify: go test ./internal/loadtest/
// A bot is a client that walks the whole real path instead of just dialing and
// hanging up: connect, login, enter the scene, move a few times, then send
// heartbeats and time each round trip. The state machine is explicit so a
// failed step is reported as a state, not as an anonymous error. An optional
// ready/start barrier lets the pool finish connecting before any timing starts.
package loadtest

import (
	"bufio"
	"fmt"
	"net"
	"time"
)

// State names one step of the bot state machine, in order.
type State int

const (
	StateConnect State = iota
	StateLogin
	StateEnterScene
	StateMove
	StateHeartbeat
	StateDone
)

// String renders a state for logs and the report.
func (s State) String() string {
	return [...]string{"connect", "login", "enter-scene", "move", "heartbeat", "done"}[s]
}

// BotConfig is one bot's script.
type BotConfig struct {
	Addr    string
	Moves   int           // move steps before switching to heartbeats
	Reqs    int           // timed heartbeats to send
	Timeout time.Duration // per-request read timeout
	Delay   time.Duration // staggered-connect delay applied before dialing
	Hold    time.Duration // stay connected idle for this long after setup
	// Ready/Start form an optional barrier: the bot signals Ready once it has
	// connected and entered the scene, then blocks on Start before timing. This
	// keeps the connect ramp out of the measured throughput window.
	Ready chan<- struct{}
	Start <-chan struct{}
}

// BotResult records where a bot ended up.
type BotResult struct {
	ID    int
	State State
	Err   error
}

// conn is a minimal request/response client over the fixed frame format.
type conn struct {
	c       net.Conn
	r       *bufio.Reader
	seq     uint32
	timeout time.Duration
}

func dial(addr string, timeout time.Duration) (*conn, error) {
	c, err := net.DialTimeout("tcp", addr, timeout)
	if err != nil {
		return nil, err
	}
	return &conn{c: c, r: bufio.NewReader(c), timeout: timeout}, nil
}

func (c *conn) Close() error { return c.c.Close() }

// call sends one request and reads frames until the matching reply arrives. The
// server may push other frames first, so it must not assume the next frame is
// the reply.
func (c *conn) call(cmd, want uint16, payload []byte) ([]byte, error) {
	c.seq++
	seq := c.seq
	if err := c.c.SetWriteDeadline(time.Now().Add(c.timeout)); err != nil {
		return nil, err
	}
	if err := WriteFrame(c.c, Frame{Cmd: cmd, Seq: seq, Payload: payload}); err != nil {
		return nil, err
	}
	for {
		if err := c.c.SetReadDeadline(time.Now().Add(c.timeout)); err != nil {
			return nil, err
		}
		f, err := ReadFrame(c.r)
		if err != nil {
			return nil, err
		}
		if f.Cmd == CmdErrorNotify {
			return nil, fmt.Errorf("server error notify on seq %d", f.Seq)
		}
		if f.Cmd == want && f.Seq == seq {
			return f.Payload, nil
		}
	}
}

// RunBot executes one bot's state machine, recording round-trip times into col.
func RunBot(id int, cfg BotConfig, col *Collector) BotResult {
	signaled := false
	signalReady := func() {
		if !signaled && cfg.Ready != nil {
			signaled = true
			cfg.Ready <- struct{}{}
		}
	}
	defer signalReady()

	if cfg.Delay > 0 {
		time.Sleep(cfg.Delay) // staggered connect: never dial the pool at once
	}
	c, err := dial(cfg.Addr, cfg.Timeout)
	if err != nil {
		col.Fail()
		return BotResult{ID: id, State: StateConnect, Err: err}
	}
	defer c.Close()

	name := fmt.Sprintf("bot-%d", id)
	if _, err := c.call(CmdLoginReq, CmdLoginResp, []byte(name)); err != nil {
		col.Fail()
		return BotResult{ID: id, State: StateLogin, Err: err}
	}
	if _, err := c.call(CmdEnterSceneReq, CmdEnterSceneResp, nil); err != nil {
		col.Fail()
		return BotResult{ID: id, State: StateEnterScene, Err: err}
	}
	col.ConnOK() // a live, logged-in connection now exists
	for i := 0; i < cfg.Moves; i++ {
		if _, err := c.call(CmdMoveReq, CmdMoveResp, []byte{byte(i), byte(i)}); err != nil {
			col.Fail()
			return BotResult{ID: id, State: StateMove, Err: err}
		}
	}
	// Setup done: announce readiness, then wait at the barrier so every bot
	// starts timing at the same instant.
	signalReady()
	if cfg.Start != nil {
		<-cfg.Start
	}
	if cfg.Hold > 0 {
		time.Sleep(cfg.Hold) // hold the connection open without sending
	}
	for i := 0; i < cfg.Reqs; i++ {
		begin := time.Now()
		if _, err := c.call(CmdHeartbeatReq, CmdHeartbeatResp, nil); err != nil {
			col.Fail()
			return BotResult{ID: id, State: StateHeartbeat, Err: err}
		}
		col.Record(float64(time.Since(begin).Microseconds()) / 1000.0)
	}
	return BotResult{ID: id, State: StateDone}
}

call 里有一个容易被忽略的细节:它按 (cmd, seq) 匹配应答,而不是「读下一帧就当应答」。真实服务端会在登录后先推一批 CmdAoiAdd 再回 CmdEnterWorldResp,机器人若假设「下一帧一定是应答」就会串帧。只有心跳步是计时并记录的——前面几步是建连开销,不该混进吞吐与延迟的分母。

并发编排:错峰连接与起跑线屏障

状态机管一个机器人,编排管 N 个。这里有两个坑必须填:错峰建连和起跑线屏障。前者绕开 accept 队列上限,后者把建连爬坡挤出测量窗口。下面这段是 L2:

// file: internal/loadtest/pool.go
// tier: L2
// verify: go test ./internal/loadtest/
// PoolConfig and RunPool are the concurrency orchestration: they launch N bots
// against one address, spread their connects across a window, optionally wait
// for every bot to connect before releasing them together, then hand back one
// report. Staggering is not optional on macOS, where the default accept backlog
// (kern.ipc.somaxconn) is only 128.
package loadtest

import (
	"sync"
	"time"
)

// PoolConfig describes one run of N bots against one address.
type PoolConfig struct {
	Label   string
	Addr    string
	Conns   int
	Moves   int
	Reqs    int
	Hold    time.Duration // per-bot idle hold, for a connection-ceiling run
	Stagger time.Duration // total window the connects are spread over
	Timeout time.Duration
	// Barrier, when true, holds every bot at a start line until all of them
	// have connected. It is what turns "QPS over the whole run" into a clean
	// steady-state measurement with no connect ramp inside the window.
	Barrier bool
}

// RunPool launches Conns bots staggered across Stagger, waits for all of them
// and returns the aggregated report. Without the stagger a thousand goroutines
// dial at once, overflow the accept queue, and the kernel resets the extras --
// the resulting "failures" would be the test harness, not the server.
func RunPool(cfg PoolConfig) Report {
	col := NewCollector(cfg.Label)
	step := cfg.Stagger / time.Duration(max(cfg.Conns, 1))

	var ready chan struct{}
	var start chan struct{}
	if cfg.Barrier {
		ready = make(chan struct{}, cfg.Conns)
		start = make(chan struct{})
	}

	var wg sync.WaitGroup
	for i := 0; i < cfg.Conns; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			RunBot(i, BotConfig{
				Addr:    cfg.Addr,
				Moves:   cfg.Moves,
				Reqs:    cfg.Reqs,
				Hold:    cfg.Hold,
				Timeout: cfg.Timeout,
				Delay:   time.Duration(i) * step,
				Ready:   ready,
				Start:   start,
			}, col)
		}(i)
	}

	if cfg.Barrier {
		// Wait for every bot to be connected, or give up after a grace period
		// so one dead connection cannot hang the whole run.
		grace := time.After(cfg.Stagger + 10*time.Second)
	wait:
		for i := 0; i < cfg.Conns; i++ {
			select {
			case <-ready:
			case <-grace:
				break wait
			}
		}
		close(start)
	}

	wg.Wait()
	return col.Report(cfg.Conns, cfg.Reqs)
}

step := Stagger / Conns 那一行是关键:第 i 个机器人在 i × step 时刻才发起连接,把 N 条连接摊到 Stagger 这段时间里建完。1000 连接、Stagger=8s 就是每 2ms 建一条。若去掉错峰,1000 个 goroutine 会几乎同时 DialTimeout,内核的 accept 队列(128)瞬间打满,超出的连接被直接 RST,压测结果里混进一堆假失败。

Barrier 则解决另一个更隐蔽的问题:建连爬坡会被算进 QPS 的分母。不开屏障时,最先连上的机器人已经开始发请求,而最后一条连接还没建好,于是「总请求数 ÷ 总耗时」被爬坡拖低,测出来的吞吐是假的。开了屏障,所有机器人先连上、Ready,主协程收齐后 close(start) 一起放行——测量窗口里只剩稳态,没有爬坡。

L2 真跑一遍

把这两块拼起来,配一个只回环、不做业务逻辑的参考服务端,就能真跑。internal/loadtest/platform_test.go 起一个真实 socket 的服务端,放五十个机器人跑过去,断言「每个机器人都走到了 done」:

=== RUN   TestBotPlatformAgainstLoopback
    platform_test.go:45: conns=50 qps=2132 p50=0.31ms p99=6.58ms err=0.000
--- PASS: TestBotPlatformAgainstLoopback (0.50s)
ok  	aegis/internal/loadtest	1.091s

五十条连接、每条二十次心跳,全部成功、零错误,P50 是 0.31ms、P99 是 6.58ms。注意这里的 QPS 两千出头并不代表服务端的上限——它只是「五十条同步机器人、每条一次只压一个请求」算出来的数。真正的承载上限要用下一节的负载扫描去逼,而这一节要证明的只有一件事:这台压测器能真跑、能采数、能落 JSON。

本节小结

这一节搭起了采集负载的工具,三块零件各司其职:

  1. 机器人状态机走完整链路(连接→登录→进场景→移动→心跳),按 (cmd, seq) 匹配应答,只有心跳步计时,失败时报告失败在哪个状态。
  2. 并发编排用错峰建连绕开 macOS somaxconn=128 的 accept 队列上限,用起跑线屏障把建连爬坡挤出测量窗口。
  3. 结果落 JSON:连接数、成功率、错误率、P50/P90/P99、内存、GC 暂停,一一对应 1.1 定义的那张表。

工具就位,接下来用它回答那个最初的问题:一台机器到底能扛多少连接、多少消息,瓶颈又在哪里。

阅读导航:上一节:1.1 压测口径:不要只看在线人数 · 下一节:1.3 承载评估与容量规划 。

继续阅读

探索更多技术文章

浏览归档,发现更多关于系统设计、工具链和工程实践的内容。

全部文章 返回首页

「mmo」更多文章

  1. 《MMO 服务器开发》工程化与运营 · 目录
  2. 《MMO 服务器开发》9.3 上线检查清单
  3. 《MMO 服务器开发》9.2 性能预算达标