1.2 机器人压测平台
1.1 定义了承载指标,可指标不会自己从服务端里蹦出来——得有一批客户端去产生负载、去计时、去汇总。站内没有 k6、wrk、hey 这类现成工具,所以这一节自己写一个。
一个能用的压测机器人必须做对三件事。第一,它要走完整路径,而不是只 dial 一下挂着——真正压的是「登录、进场景、移动、心跳」这条链路,而不是 TCP 三次握手。第二,它要错峰建连:macOS 的 kern.ipc.somaxconn 默认只有 128,一口气 dial 上千条会被内核直接 RST,测出来的「失败」其实是压测器自己造的。第三,它要报分位延迟而不只是总量——P99 才决定玩家会不会骂人。
本节给出这套平台的三块零件:机器人状态机、并发编排、以及一个把建连爬坡挤出测量窗口的起跑线屏障,最后真起一个回环服务端跑通。
机器人状态机:连接 → 登录 → 进场景 → 移动 → 心跳
机器人本身就是用第一卷那套 11 字节定长头帧协议说话的普通客户端。把它写成一个显式状态机的好处是:哪一步失败,报告里就写哪一步,而不是一个匿名的 error。下面这段是 L2,go test 会真的起进程、连真 socket:
// file: internal/loadtest/bot.go
// tier: L2
// verify: go test ./internal/loadtest/
// A bot is a client that walks the whole real path instead of just dialing and
// hanging up: connect, login, enter the scene, move a few times, then send
// heartbeats and time each round trip. The state machine is explicit so a
// failed step is reported as a state, not as an anonymous error. An optional
// ready/start barrier lets the pool finish connecting before any timing starts.
package loadtest
import (
"bufio"
"fmt"
"net"
"time"
)
// State names one step of the bot state machine, in order.
type State int
const (
StateConnect State = iota
StateLogin
StateEnterScene
StateMove
StateHeartbeat
StateDone
)
// String renders a state for logs and the report.
func (s State) String() string {
return [...]string{"connect", "login", "enter-scene", "move", "heartbeat", "done"}[s]
}
// BotConfig is one bot's script.
type BotConfig struct {
Addr string
Moves int // move steps before switching to heartbeats
Reqs int // timed heartbeats to send
Timeout time.Duration // per-request read timeout
Delay time.Duration // staggered-connect delay applied before dialing
Hold time.Duration // stay connected idle for this long after setup
// Ready/Start form an optional barrier: the bot signals Ready once it has
// connected and entered the scene, then blocks on Start before timing. This
// keeps the connect ramp out of the measured throughput window.
Ready chan<- struct{}
Start <-chan struct{}
}
// BotResult records where a bot ended up.
type BotResult struct {
ID int
State State
Err error
}
// conn is a minimal request/response client over the fixed frame format.
type conn struct {
c net.Conn
r *bufio.Reader
seq uint32
timeout time.Duration
}
func dial(addr string, timeout time.Duration) (*conn, error) {
c, err := net.DialTimeout("tcp", addr, timeout)
if err != nil {
return nil, err
}
return &conn{c: c, r: bufio.NewReader(c), timeout: timeout}, nil
}
func (c *conn) Close() error { return c.c.Close() }
// call sends one request and reads frames until the matching reply arrives. The
// server may push other frames first, so it must not assume the next frame is
// the reply.
func (c *conn) call(cmd, want uint16, payload []byte) ([]byte, error) {
c.seq++
seq := c.seq
if err := c.c.SetWriteDeadline(time.Now().Add(c.timeout)); err != nil {
return nil, err
}
if err := WriteFrame(c.c, Frame{Cmd: cmd, Seq: seq, Payload: payload}); err != nil {
return nil, err
}
for {
if err := c.c.SetReadDeadline(time.Now().Add(c.timeout)); err != nil {
return nil, err
}
f, err := ReadFrame(c.r)
if err != nil {
return nil, err
}
if f.Cmd == CmdErrorNotify {
return nil, fmt.Errorf("server error notify on seq %d", f.Seq)
}
if f.Cmd == want && f.Seq == seq {
return f.Payload, nil
}
}
}
// RunBot executes one bot's state machine, recording round-trip times into col.
func RunBot(id int, cfg BotConfig, col *Collector) BotResult {
signaled := false
signalReady := func() {
if !signaled && cfg.Ready != nil {
signaled = true
cfg.Ready <- struct{}{}
}
}
defer signalReady()
if cfg.Delay > 0 {
time.Sleep(cfg.Delay) // staggered connect: never dial the pool at once
}
c, err := dial(cfg.Addr, cfg.Timeout)
if err != nil {
col.Fail()
return BotResult{ID: id, State: StateConnect, Err: err}
}
defer c.Close()
name := fmt.Sprintf("bot-%d", id)
if _, err := c.call(CmdLoginReq, CmdLoginResp, []byte(name)); err != nil {
col.Fail()
return BotResult{ID: id, State: StateLogin, Err: err}
}
if _, err := c.call(CmdEnterSceneReq, CmdEnterSceneResp, nil); err != nil {
col.Fail()
return BotResult{ID: id, State: StateEnterScene, Err: err}
}
col.ConnOK() // a live, logged-in connection now exists
for i := 0; i < cfg.Moves; i++ {
if _, err := c.call(CmdMoveReq, CmdMoveResp, []byte{byte(i), byte(i)}); err != nil {
col.Fail()
return BotResult{ID: id, State: StateMove, Err: err}
}
}
// Setup done: announce readiness, then wait at the barrier so every bot
// starts timing at the same instant.
signalReady()
if cfg.Start != nil {
<-cfg.Start
}
if cfg.Hold > 0 {
time.Sleep(cfg.Hold) // hold the connection open without sending
}
for i := 0; i < cfg.Reqs; i++ {
begin := time.Now()
if _, err := c.call(CmdHeartbeatReq, CmdHeartbeatResp, nil); err != nil {
col.Fail()
return BotResult{ID: id, State: StateHeartbeat, Err: err}
}
col.Record(float64(time.Since(begin).Microseconds()) / 1000.0)
}
return BotResult{ID: id, State: StateDone}
}
call 里有一个容易被忽略的细节:它按 (cmd, seq) 匹配应答,而不是「读下一帧就当应答」。真实服务端会在登录后先推一批 CmdAoiAdd 再回 CmdEnterWorldResp,机器人若假设「下一帧一定是应答」就会串帧。只有心跳步是计时并记录的——前面几步是建连开销,不该混进吞吐与延迟的分母。
并发编排:错峰连接与起跑线屏障
状态机管一个机器人,编排管 N 个。这里有两个坑必须填:错峰建连和起跑线屏障。前者绕开 accept 队列上限,后者把建连爬坡挤出测量窗口。下面这段是 L2:
// file: internal/loadtest/pool.go
// tier: L2
// verify: go test ./internal/loadtest/
// PoolConfig and RunPool are the concurrency orchestration: they launch N bots
// against one address, spread their connects across a window, optionally wait
// for every bot to connect before releasing them together, then hand back one
// report. Staggering is not optional on macOS, where the default accept backlog
// (kern.ipc.somaxconn) is only 128.
package loadtest
import (
"sync"
"time"
)
// PoolConfig describes one run of N bots against one address.
type PoolConfig struct {
Label string
Addr string
Conns int
Moves int
Reqs int
Hold time.Duration // per-bot idle hold, for a connection-ceiling run
Stagger time.Duration // total window the connects are spread over
Timeout time.Duration
// Barrier, when true, holds every bot at a start line until all of them
// have connected. It is what turns "QPS over the whole run" into a clean
// steady-state measurement with no connect ramp inside the window.
Barrier bool
}
// RunPool launches Conns bots staggered across Stagger, waits for all of them
// and returns the aggregated report. Without the stagger a thousand goroutines
// dial at once, overflow the accept queue, and the kernel resets the extras --
// the resulting "failures" would be the test harness, not the server.
func RunPool(cfg PoolConfig) Report {
col := NewCollector(cfg.Label)
step := cfg.Stagger / time.Duration(max(cfg.Conns, 1))
var ready chan struct{}
var start chan struct{}
if cfg.Barrier {
ready = make(chan struct{}, cfg.Conns)
start = make(chan struct{})
}
var wg sync.WaitGroup
for i := 0; i < cfg.Conns; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
RunBot(i, BotConfig{
Addr: cfg.Addr,
Moves: cfg.Moves,
Reqs: cfg.Reqs,
Hold: cfg.Hold,
Timeout: cfg.Timeout,
Delay: time.Duration(i) * step,
Ready: ready,
Start: start,
}, col)
}(i)
}
if cfg.Barrier {
// Wait for every bot to be connected, or give up after a grace period
// so one dead connection cannot hang the whole run.
grace := time.After(cfg.Stagger + 10*time.Second)
wait:
for i := 0; i < cfg.Conns; i++ {
select {
case <-ready:
case <-grace:
break wait
}
}
close(start)
}
wg.Wait()
return col.Report(cfg.Conns, cfg.Reqs)
}
step := Stagger / Conns 那一行是关键:第 i 个机器人在 i × step 时刻才发起连接,把 N 条连接摊到 Stagger 这段时间里建完。1000 连接、Stagger=8s 就是每 2ms 建一条。若去掉错峰,1000 个 goroutine 会几乎同时 DialTimeout,内核的 accept 队列(128)瞬间打满,超出的连接被直接 RST,压测结果里混进一堆假失败。
Barrier 则解决另一个更隐蔽的问题:建连爬坡会被算进 QPS 的分母。不开屏障时,最先连上的机器人已经开始发请求,而最后一条连接还没建好,于是「总请求数 ÷ 总耗时」被爬坡拖低,测出来的吞吐是假的。开了屏障,所有机器人先连上、Ready,主协程收齐后 close(start) 一起放行——测量窗口里只剩稳态,没有爬坡。
L2 真跑一遍
把这两块拼起来,配一个只回环、不做业务逻辑的参考服务端,就能真跑。internal/loadtest/platform_test.go 起一个真实 socket 的服务端,放五十个机器人跑过去,断言「每个机器人都走到了 done」:
=== RUN TestBotPlatformAgainstLoopback
platform_test.go:45: conns=50 qps=2132 p50=0.31ms p99=6.58ms err=0.000
--- PASS: TestBotPlatformAgainstLoopback (0.50s)
ok aegis/internal/loadtest 1.091s
五十条连接、每条二十次心跳,全部成功、零错误,P50 是 0.31ms、P99 是 6.58ms。注意这里的 QPS 两千出头并不代表服务端的上限——它只是「五十条同步机器人、每条一次只压一个请求」算出来的数。真正的承载上限要用下一节的负载扫描去逼,而这一节要证明的只有一件事:这台压测器能真跑、能采数、能落 JSON。
本节小结
这一节搭起了采集负载的工具,三块零件各司其职:
- 机器人状态机走完整链路(连接→登录→进场景→移动→心跳),按
(cmd, seq)匹配应答,只有心跳步计时,失败时报告失败在哪个状态。 - 并发编排用错峰建连绕开 macOS
somaxconn=128的 accept 队列上限,用起跑线屏障把建连爬坡挤出测量窗口。 - 结果落 JSON:连接数、成功率、错误率、P50/P90/P99、内存、GC 暂停,一一对应 1.1 定义的那张表。
工具就位,接下来用它回答那个最初的问题:一台机器到底能扛多少连接、多少消息,瓶颈又在哪里。
阅读导航:上一节:1.1 压测口径:不要只看在线人数 · 下一节:1.3 承载评估与容量规划 。
继续阅读
探索更多技术文章
浏览归档,发现更多关于系统设计、工具链和工程实践的内容。