Network stack

The client speaks seven network protocols. All live services are Demonware, over IPv4 only. There is no bungie.net REST API in this build. Web-service calls travel over bdLobby instead.

Demonware class names such as bdSocketRouter, bdLobby class names, and peer message names such as membership-update are the game’s own. Other names are descriptive, not the game’s.

The protocols

A telemetry HTTP path also exists. It fails soft and does not block play.

The engine’s socket layer has a kind on every socket. It decides the path a datagram takes.

SignOn

SignOn is one HTTPS POST. The request body is protobuf and carries a Steam session ticket. The build number and platform are in the query string.

The response is protobuf, decoded by the same nanopb runtime bdLobby uses. Field 1 selects the outcome.

A success body carries an opaque session token, the BAP server’s IPv4 address and port, material for the secure channel, and up to eight feature names. The feature list is the only source of feature flags, so every named feature is off unless SignOn names it. Details of the token and channel setup are on From launch to orbit.

bdLobby BAP

BAP is a TCP link to one server. The client opens it after SignOn. It later opens a second BAP link per activity, to an address the server chooses.

The frame

Every frame starts with a 6-byte outer header. All integers are big-endian.

struct BapOuterHeader {         /* 6 bytes */
    uint8_t  magic;             /* +0x00  1 */
    uint8_t  frame_type;        /* +0x01  0 or 2 plaintext, 1 encrypted */
    uint32_t length;            /* +0x02  bytes after this header */
};

An encrypted frame (type 1) then carries a 16-byte tag and AES-128-GCM ciphertext. length counts the tag. A plaintext frame carries the inner message directly.

The inner message starts with a header whose size depends on the service.

struct BapInnerHeader {         /* 6, 8 or 14 bytes */
    uint16_t service_id;        /* +0x00  must be below 308 */
    uint32_t task_id;           /* +0x02  a response echoes the request's value */
    uint16_t status;            /* +0x06  responses only; must be 200 */
};

TCP gives reliability, so BAP has no sequence or acknowledgement of its own. One unanswered request jams the link, so every request needs its response, with a body its decoder accepts. An empty body is not a neutral answer.

Sunrise parses and builds frames in src/middleware/bap/frame.h and bap_frame.cpp.

The service registry

The client registers 71 services. Each is a singleton with a fixed id and a body codec. Ids 308 and above are dropped at the header, so no newer service can reach this build.

The full list is ids 6 to 51, then 100, 102, 110, 112, 121 to 124, 150, 151, 153, 160 to 162, 171, 250, 251 and 300 to 307. Ids 36 to 41 and 48 to 51 have no enum name and print as UNKNOWN. Sunrise keeps its own catalog in src/middleware/bap/service_catalog.h.

The body codec takes one of four shapes.

Nineteen services have a fixed body length, and each rejects any other size.

Pushes

A server-initiated message is a push. The client accepts pushes on exactly 15 ids: 6, 8, 9, 20, 27, 28, 47, 51, 100, 123, 124, 150, 160, 161 and 301.

• Ids 6 and 161 have no handler behind them. A push that decodes on either id crashes the client.
• An id outside the 15 is a null call, not an ignored message.
• That leaves 13 usable ids. Twelve of them do real work.

Protobuf in the client

All protobuf decoding goes through one embedded copy of nanopb. It is decode-only. A separate hand-written encoder walks the same schema tables to write the few protobuf messages the client sends.

The client also encodes seven bdLobby request roots it never decodes. The bdLobby protobuf surface is 33 roots: 26 decoded and 7 encoded.

A nanopb field descriptor is 32 bytes.

struct PbField {                /* 32 bytes; tag 0 ends the array */
    uint32_t tag;               /* +0x00  field number */
    uint32_t flags;             /* +0x04  low 4 bits: LTYPE; next 4 bits: HTYPE */
    uint8_t  data_offset;       /* +0x08  step after the previous field */
    int8_t   size_offset;       /* +0x09  has-bit address, relative to the value */
    uint8_t  pad[2];
    uint32_t data_size;         /* +0x0C  one element's size */
    uint32_t array_size;        /* +0x10  capacity, for arrays */
    uint32_t pad2;
    const void *submsg;         /* +0x18  child schema, or a callback */
};

LTYPE is varint, svarint, fixed32, fixed64, bytes, string or submessage. HTYPE is required, optional, array or callback. Three rules matter to a server.

• A missing required field fails the whole message.
• Strings and byte arrays have fixed capacities. An oversized value is rejected, not cut short.
• A bytes slot stores its length in its first 8 bytes. A slot declared as 96 bytes holds 88.

Sunrise’s protobuf reader and writer are in src/middleware/protobuf/.

The engine peer transport

The peer transport carries group sessions and gameplay replication between machines. It has three layers.

Which path a peer takes is set by byte +93 of the 128-byte join descriptor, the transport method. 6 and 7 select Steam Datagram Relay. Every other value selects Demonware bdNet.

The peer address

Many peer messages carry a serialized bdCommonAddr. All 86 bytes take part in equality, padding included.

struct NetAddr {                /* 86 bytes; ports low byte first */
    uint8_t  local_ipv4_0[4];   /* +0x00  first local endpoint, dotted order */
    uint16_t local_port_0;      /* +0x04 */
    uint8_t  local_more[24];    /* +0x06  four more optional 6-byte endpoints */
    uint8_t  public_ipv4[4];    /* +0x1E  observed public endpoint */
    uint16_t public_port;       /* +0x22 */
    uint32_t address_id;        /* +0x24  identity hash of the endpoint */
    uint8_t  nat_type;          /* +0x28  1 open, 2 moderate, 3 strict */
    uint8_t  reserved[44];      /* +0x29  zero */
    uint8_t  transport_method;  /* +0x55  0 to 5 bdNet, 6 and 7 Steam relay */
};

The association handshake

Every bdNet peer connection opens with four packets. Nothing above runs until they complete.

Every field is a plain byte append. Multi-byte integers go low byte first. There is no length prefix and no checksum. Types 10 to 13 on the same port belong to NAT traversal.

Every packet opens with the same 8-byte header.

struct AssocHeader {            /* 8 bytes */
    uint8_t  type;              /* +0x00  1 to 4, or 6 for data */
    uint8_t  constant_2;        /* +0x01  2, never read back */
    uint16_t addressed_tag;     /* +0x02  the tag this packet is for */
    uint32_t sequence;          /* +0x04  0 on handshake packets */
};

struct AssocInit {              /* 18 bytes */
    AssocHeader hdr;            /* +0x00  type 1, tag 0 */
    uint16_t requester_tag;     /* +0x08  chosen by the requester */
    uint8_t  security_id[8];    /* +0x0A */
};

struct AssocInitAck {           /* 52 bytes */
    AssocHeader hdr;            /* +0x00  type 2, tag = requester_tag */
    uint32_t free_running;      /* +0x08  stored, never checked */
    uint8_t  cookie[16];        /* +0x0C  the responder's own value */
    uint16_t responder_tag;     /* +0x1C */
    uint16_t responder_tag2;    /* +0x1E  same value again */
    uint16_t requester_tag;     /* +0x20 */
    uint16_t old_responder_tag; /* +0x22  0 on a fresh association */
    uint16_t old_requester_tag; /* +0x24  0 on a fresh association */
    uint8_t  observed_addr[6];  /* +0x26  requester address as the responder saw it */
    uint8_t  security_id[8];    /* +0x2C  copied from the init */
};

struct AssocCookieEcho {        /* 209 bytes */
    AssocHeader hdr;            /* +0x00  type 3, tag = responder_tag */
    AssocInitAck echoed;        /* +0x08  the init ack, unchanged */
    uint8_t  requester_addr[41];/* +0x3C  serialized address claim */
    uint8_t  security_id[8];    /* +0x65 */
    uint8_t  public_key[100];   /* +0x6D  exported ECC public key, zero padded */
};

struct AssocCookieAck {         /* 116 bytes */
    AssocHeader hdr;            /* +0x00  type 4, tag = requester_tag */
    uint8_t  public_key[100];   /* +0x08 */
    uint8_t  security_id[8];    /* +0x6C */
};

The security id routes every packet. The socket router keys its association table on it. A packet whose id matches nothing is dropped before any association sees it, and nothing is logged. The requester chooses the id, and every reply repeats it.

The key exchange is ECDH on the curve secp224r1, using LibTomCrypt. Each side exports its public key in the cookie echo or cookie ack. The association then derives its record keys.

After the handshake, data travels in type-6 records.

struct AssocData {              /* 18 bytes + ciphertext */
    AssocHeader hdr;            /* +0x00  type 6; sequence at +0x04 */
    uint8_t  mac[8];            /* +0x08  truncated MAC */
    uint16_t plain_len;         /* +0x10 */
    uint8_t  ciphertext[];      /* +0x12  rounded up to the cipher block */
};

Records use AES-128 in CBC mode. Each record is an independent CBC run. The MAC covers the header and everything from the length on, truncated to 8 bytes. A 32-slot sliding window rejects replays.

Sunrise implements this layer in src/middleware/gameplay/dtls/.

The engine also has its own separate association layer, with a 3-bit opcode set (connect request, key-exchange offer, key-exchange response, reject, keepalive). The join descriptor cannot select it. Its opcode-0 body is unknown.

The peer message channel

One UDP payload carries a chain of peer messages in one bitstream, most significant bit first.

/* Peer message chain, widths in bits */
bit  packet_marker;            /* 1   written once per packet */
/* repeated: */
bit  more;                     /* 1   1 = a message follows, 0 = end */
u6   message_id;               /* 6   0 to 44 */
u18  struct_size;              /* 18  the id's fixed struct size, not the bits used */
...  payload;                  /*     the id's own writer */

struct_size is a constant per id. The receiver uses it only to size a scratch buffer, and it drops a message whose declared size does not match the table. A reader failure ends the whole chain.

There are two paths.

An established datagram opens with a 0 bit, a fragmentation bit, and the connection sequence modulo 4. Then come the handlers in a fixed order: the connection ACK, reliable queue A (32-byte fragments), reliable queue B (6-byte fragments), and then external handlers such as voice. A datagram over the address limit (1,228 bytes on direct IPv4) is split into at most eight pieces.

The receiver accepts a packet sequence at most 128 ahead of its window. A sender further ahead is read as a whole ring behind, and the connection never recovers.

Peer message ids

The channel has 45 ids, 0 to 44.

Clock sync (id 29) computes offset = ((t2 - t1) + (t3 - t4)) / 2. The receive time t4 is never on the wire. A reply is terminal: answering one loops forever.

Sunrise’s peer codecs are in src/middleware/gameplay/peer/ and src/middleware/gameplay/group/.

Session membership, peer message 30

The group-session host sends membership-update to tell members who is in the session. A joining peer waits for it. The body is one 31,104-byte struct in four regions.

On the wire the protobuf region is a 13-bit byte length and that many bytes. The rest is bit-packed. This is the only place the client nests protobuf inside a Bungie bitstream.

A base revision of 0 means a complete snapshot. Otherwise the update is a delta against the revision the receiver holds.

Root fields

The member record

Each protobuf field is copied straight into the receiver’s session state at the same offset. So a wrong value becomes the client’s own state.

struct MembershipMember {       /* 184 bytes, nanopb layout */
    bool     has_addr;          /* +0x00 */
    uint64_t addr_len;          /* +0x08  must be 86 */
    uint8_t  addr[88];          /* +0x10  tag 1: the peer's NetAddr */
    bool     has_machine_id;    /* +0x68 */
    uint64_t machine_id_len;    /* +0x70  must be 8 */
    uint8_t  machine_id[8];     /* +0x78  tag 2 */
    bool     has_join_id;       /* +0x80 */
    uint64_t join_id;           /* +0x88  tag 3: the join attempt's id; 0 when empty */
    bool     has_party_id;      /* +0x90 */
    uint64_t party_id;          /* +0x98  tag 8: 0 and all-ones both mean none */
    bool     has_player_count;  /* +0xA0 */
    uint32_t player_count;      /* +0xA4  tag 9 */
    uint64_t player_slot_count; /* +0xA8  tag 10, at most 1 */
    uint32_t player_slot;       /* +0xB0  index into the player table */
    bool     has_leave_pending; /* +0xB4 */
    uint8_t  host_leave_pending;/* +0xB5  tag 11, a bool */
    bool     has_leave_done;    /* +0xB6 */
    uint8_t  host_leave_done;   /* +0xB7  tag 12, a bool */
};

Tags 8 and 9 group members into parties for matchmaking. Tags 11 and 12 drive a rejoin after the host leaves.

The join id (tag 3) is checked first. The receiver finds the entry whose address equals its own. If that entry’s join id is not the one its current join sent, the update is “from a stale join” and is dropped with no other effect.

Peer and player deltas

A peer delta names a member index (6 bits), a changed flag, a connection state (4 bits), and optional connection data. The member states are named in the client:

A player delta names a player slot, an owning member, a player-add counter, and an optional profile. The profile carries the player’s name and the account and character ids. An activity join reads those ids from this row, so a row without a profile costs the peer its player.

What a bad update costs

The receiver applies the update first and validates after. On most failures it clears its member and player tables and rebuilds a minimal two-member session. The hash at +31096 is the last check. It is a Bob Jenkins lookup3 hash over the receiver’s own 28,768-byte session state after applying. A mismatch ends in no membership information, forcing disconnect, and the join retries.

Sunrise builds a replica of the session state to compute this hash, in src/middleware/gameplay/group/session_state.h.

QoS probe and the session advertisement

Before joining a session found by matchmaking, the client probes it over UDP. An unanswered session is marked unsuitable and never joined. The probe uses Demonware’s bdQoSProbe. Both packets are little-endian and unframed.

struct QosRequest {             /* 18 bytes */
    uint8_t  type;              /* +0x00  40 */
    uint64_t send_time;         /* +0x01  echoed back */
    uint32_t probe_id;          /* +0x09  routes the reply */
    uint32_t security_id;       /* +0x0D  echoed back */
    uint8_t  last_probe;        /* +0x11  1: put the payload in the reply */
};

struct QosReply {               /* 27 bytes + payload */
    uint8_t  type;              /* +0x00  41 */
    uint32_t probe_id;          /* +0x01  from request +0x09 */
    uint64_t send_time;         /* +0x05  from request +0x01 */
    uint8_t  accepted;          /* +0x0D  0 is an explicit refusal */
    uint32_t payload_len;       /* +0x0E  must equal the bytes after +0x1B */
    uint32_t security_id;       /* +0x12  from request +0x0D */
    float    hold_time;         /* +0x16  seconds, subtracted from the round trip */
    uint8_t  has_data;          /* +0x1A */
    uint8_t  payload[];         /* +0x1B  the session advertisement */
};

The payload is required. An empty or undecodable one marks the session unsuitable, with a named reason such as qos-payload-empty or qos-incompatible-versions.

The session advertisement

The payload is a 13,184-byte session advertisement, bit-packed. LAN peer message 3 carries the same block, and both use one decoder. Its header comes first.

Payload type 1 is a full advertisement: topology kind, host mode, join controls, the 128-byte join descriptor, five 6-bit slot counts, and an optional extension with up to 32 member rows. Payload type 2 is a 688-byte group-session descriptor. Matchmaking in this build expects type 2.

Every field comes from live session state. A zero block fails the range checks.

STUN, NAT, UPnP and DNS

• STUN. The client walks a 16-entry STUN host table and needs one valid entry. If none is valid it falls back, but it then needs at least one local address, or UPnP refuses to start.
• NAT traversal. bdNATTravClient shares the association port and uses packet types 10 to 13.
• UPnP. SSDP discovery, one description GET, and four SOAP actions. A failed mapping falls back to a direct or relayed path.
• DNS. c-ares runs its own DNS over its own sockets. It sends one IPv4 A query, UDP first, TCP on truncation, with retries. curl keeps a 60-second host cache and ignores DNS TTL.

Gameplay replication

Replication runs on the reliable peer connection. Its model is the simulation object (sobject) plus network facets and component blocks. Dispatch is computed from reflection data, so there is no numeric opcode switch on the wire.

A replication view opens through peer message 40 in six ordered stages. At stage 2 the initiator publishes its scheduler signature. A mismatch logs view signature mismatch, no replication. Stages 4 and 5 open provisional and then full replication.

The local player’s biped does not need replication. It is built by a local call. Replication is what other entities need.

Open questions

• The body of the engine association’s opcode 0.
• The meaning of service 28’s fields.
• The leaf layout of the 304-byte peer-property block (peer message 31).
• The reason enums of connect-refuse and connect-closed.
• The exact byte partition of the protected formatter under the engine’s direct association.

Related pages

• From launch to orbit: SignOn and the BAP handshake as Sunrise answers them.
• The Activity Host protocol: the activity messages that ride bdLobby.
• Bootflow: the order in which the client uses these links.