SUNRISE

Accounts, characters and saves

Sunrise keeps one account in a local SQLite database. Every answer about the player is read from that database, turned into the game’s own data layout and sent over the BAP link. Every change the player makes is written to the database before the game is told it succeeded. In co-op each player’s account stays on their own install.

The BAP link itself is in From launch to orbit. The layout of the data inside the game is in Items, characters and inventory.

The local database

The database is one file, Sunrise\data\investment.sqlite3, in the folder that holds the Sunrise DLL. The code is in src/state/investment/.

  • On first start Sunrise creates the file. It writes the schema and the default rows. Both are built into the DLL as resources (resources/database/).
  • The defaults hold one account with three level-50 characters: a Hunter, a Titan and a Warlock.
  • On a later start Sunrise checks the schema version. Versions 1 to 7 are upgraded in place, one step at a time, to version 8. A newer version, or a file from another program, is refused.
  • Every write runs inside a SQLite savepoint. A failed write rolls back and leaves no partial rows.
  • Writes are synchronous (PRAGMA synchronous=FULL). No unsaved copy is held in memory, except in a cheat session.

The file is plain SQLite. Back it up before you edit it by hand.

Tables

tablewhat it holds
accountthe account SOID and whether profile setup is done
charactersone row per slot 0 to 2: SOID, race, gender, class, level, title, last orbit destination
itemscharacter items; location 0 is equipped (17 slots), location 1 is inventory (333 rows, shared with stacks)
socketsthe plug in each socket lane of an item
item_objectivesthe objective values of one item instance, such as a bounty’s progress and deadline
profile_itemsaccount-wide currencies and materials, up to 701 rows
character_stacksper-character materials that have no instance
unlocksflag, objective and progression banks, per account or per character
character_objective_valuesper-character objective values outside the unlock banks
character_helmet_tierseach character’s Gambit Prime helmet tier per role
eververse_wallet_syncthe Store wallet generation the account object publishes
mission_flagsvalues a mission script keeps in the save, keyed by mod id and name
family5account-wide unlock and value overrides
entitlementsowned content, sent in the SignOn reply
dismantle_rewardswhat dismantling gear pays out
pending_rewardsrewards earned but not yet granted
parked_itemsitems, stacks and profile rows whose content is not installed; see Content that is no longer installed
starter_character, starter_items, starter_unlockswhat a created character starts with; see Creating and deleting characters
bootstrapone-time setup steps already done
account_*game settings: controls, audio, display, interface, social, key bindings

The account_* tables come from a second schema file, account_settings_schema.sql. The starter_* tables come from character_starter.sql.

Cheat sessions

When a loaded mod sets "cheat": true, or client.cheats unlocks the cheat tools in settings.json, the session never writes to the file.

  1. After the mods load and the save opens, Sunrise copies the database into memory with the SQLite backup API. This runs before account repairs or progression seeding read mod content.
  2. It closes the file and sends every read and write to the copy.
  3. The copy is gone when the game exits.

So every table above is back to its old state after a restart, game settings and key bindings included. Account repairs and progression seeding also stay in the copy. Creating a new database or upgrading its schema happens when the file opens, before the copy.

If the copy fails, Sunrise does not start. The code is use_session_copy in src/state/investment/investment_database.cpp. How a mod sets the flag is in Mods. The settings key is in Settings, logs and in-game tools.

Content that is no longer installed

A mod can add items. When the mod is removed, the save still holds those items, and the game cannot show an item whose definition is gone. Before Sunrise builds any account data for the game, it moves such rows out of the account into parked_items. Nothing is deleted.

  • A row is parked when its own definition is not installed, or, for an item, when one of its socket plugs is not.
  • An equipped item that is parked leaves its slot empty. Sunrise then equips the first unequipped item of that character that fits the slot, if there is one.
  • Each parked row writes a warning to the log: ev=items stage=park, with the definition hash.
  • On a later start, a parked row whose content is installed again goes back. An item returns to its old equipped slot if that slot is empty, else to the end of the inventory. A row stays parked if there is no room, or if the account would not resolve with it.
  • Deleting a character also deletes its parked rows.

The code is reconcile_installed_content in src/state/runtime/state_account_parking_runtime.cpp.

Creating and deleting characters

The game creates a character with ws opcode 501 and deletes one with ws opcode 502. Sunrise saves each change in one save transaction, so a refused change leaves nothing behind. The code is in src/state/runtime/state_account_roster_runtime.cpp.

A new character:

  • takes the slot after the last character. The account holds at most three characters.
  • gets the lowest free SOID after the account SOID: account SOID plus 1, 2 or 3.
  • keeps the class, race, gender and appearance from the creation screen.
  • copies its level and other fields from starter_character, and its per-character unlocks from starter_unlocks.
  • gets the items in starter_items for its class, then the game’s own new-character item set for that class. The first item for a slot is equipped; a later one goes to the inventory.
  • gets weapons and armour at the highest item level any other character holds, and at least 1, so it keeps pace with the account.
  • is not selected. Only a pick selects a character.

The 501 reply carries the new character’s SOID. A refused request gets code 1 and SOID 0.

Deleting a character removes its items and every row keyed by its slot. The characters after it move down one slot and keep their SOIDs. If it was the selected one, no character is selected.

The account model

In memory the account is one AccountState value (src/state/account/account_state.h). It is read from the database each time it is needed. There is no separate cached copy.

/* Simplified from src/state/account/account_state.h. */
struct AccountState {
    uint64_t       primarySoid;          /* the account's object id */
    ProfileItem    profileItems[];       /* currencies and materials */
    CharacterState characters[3];        /* slots 0 to 2 */
    bool           profileSetupCompleted;
    AccountSettings settings;            /* controls, audio, display, ... */
};

struct CharacterState {
    uint64_t soid;                       /* primarySoid + 1 + slot */
    bool     selected;                   /* session only */
    uint8_t  race, gender, characterClass, level;
    uint16_t currentActivityIndex;       /* session only */
    uint16_t equippedTitleRecordIndex;
    uint64_t signInSeconds;              /* session only */
    Equipment equipment;                 /* equipped items by slot */
    CharacterItems inventory;            /* unequipped items */
    CharacterStacks stacks;              /* non-instanced materials */
};

Every object the game tracks has a 64-bit id, a SOID. The account has one, each character has one, and each item instance has one. In the defaults, and after Sunrise adopts a new account SOID, the character SOIDs are the account SOID plus 1, 2 and 3.

Which account SOID is used

The game names the account in its first request, ws opcode 503.

  • If that request carries a nonzero account id, Sunrise adopts it. It stores it as the new account SOID and moves the character SOIDs with it.
  • If the id is zero, Sunrise answers with the SOID already in the database.

This is in src/server/web_service/web_service_runtime.cpp and set_primary_soid in src/state/runtime/state_account_runtime.cpp.

How the data reaches the client

There are three paths. All of them run over the encrypted BAP link.

pathcarrieswhen
ws opcode 503 replythe account SOID, the server clock, the family 5 objectonce, during investment_signin
service 123 pushqueuez family snapshots and updatesafter a subscribe, and after every change
ws reply statusthe family 4 version a change will publishin the reply to each action

ws opcode 503: the account bootstrap

The game sends 503 once, on the main link, during boot step 23. The reply is built in src/middleware/web_service/messages/opcode503_codec.cpp. In order, it holds:

  1. the status pair: code 0, and value -1, which means “no family 4 version to wait for”
  2. the account SOID, 64 bits
  3. two 32-bit fields that nothing reads. Sunrise sends 0.
  4. the family 5 object: a fixed sentinel SOID, the server clock, and the override lists from the family5 table

The clock always moves forward, even for two requests in the same second. The client ignores a family 5 update whose clock is not newer.

Replicated families: queuez

The queuez service is a replicated object store. The client holds a copy of each family it subscribed to. The server keeps it current with pushes on service 123. There is no “get everything” request.

A family is identified by its type and a root SOID. Sunrise builds these types:

typewhat Sunrise sendsbuilder
0the banner pair for one characterpush/snapshot/banner_snapshot.cpp
2the social roster member record, with the selected character’s Power and the Triumph scorepush/snapshot/social_roster_snapshot.cpp
3the character roster; its count picks selection or creationpush/snapshot/roster_snapshot.cpp
4the account object, the selected character object, one object per itempush/snapshot/family4_snapshot_preparer.cpp
5the global override objectpush/queuez/queuez_family5_push.cpp

Paths are under src/server/bap/encrypted/. Any other family gets an empty full snapshot. An empty snapshot still moves the client’s record to “synced”.

The client subscribes in two ways:

  • Service 12. The body is 9 bytes: a u8 family type, then the u64 root SOID. Sunrise answers with an empty service 13, then pushes the first snapshot on service 123.
  • ws opcode 206. The body is bit-packed: 4 bits of family type plus 1, then 64 bits of root SOID. The client uses it for family 3. Sunrise puts the first snapshot in the first trailer blob of the 206 reply. If family 4 is not yet live on that link, Sunrise then pushes the family 4 snapshot and the banner pair on service 123. It sends family 4 once more 400 ms later.

Both paths end in append_queuez_notification in push/queuez/queuez_subscription.cpp.

The service 123 body

The body is big-endian and has no padding. It is encoded by src/middleware/queuez/queuez_update.cpp.

#pragma pack(push, 1)
struct QueuezUpdate {            /* the service 123 body */
    uint32_t family_count;       /* +0x00, never 0 */
    /* family_count QueuezFamily records follow */
};

struct QueuezFamily {            /* 21 bytes */
    uint32_t family_type;        /* +0x00 */
    uint64_t family_root_soid;   /* +0x04 */
    int32_t  version;            /* +0x0C, -1 is refused */
    uint8_t  flags;              /* +0x10, bit 0 = full snapshot */
    uint32_t object_count;       /* +0x11, 0 is legal */
    /* object_count QueuezObject records follow */
};

struct QueuezObject {            /* 20 bytes, then the payload */
    uint32_t object_id;          /* +0x00, a definition id, not a slot */
    uint64_t object_version;     /* +0x04, the object's SOID for family 4 */
    uint32_t payload_len;        /* +0x0C, 0 = delete this object */
    uint32_t encoding;           /* +0x10, see below */
    /* payload_len bytes */
};
#pragma pack(pop)
encodingmeaningSunrise uses it for
0none; goes with a delete–
1tag reflection, a bit-packed schema walkfamily 5, and some small updates
2binary diffnot used
3raw bytes, starting with the object versionnot used
4Oodle-compressed imagethe family 4 objects

Rules:

  • The first update to a family must set flag bit 0, full snapshot. The client refuses it otherwise.
  • A full snapshot replaces the family. Objects it does not name are deleted.
  • After the first snapshot, each change goes out at the next version. Sunrise tracks the version it last sent on each connection.
  • Sunrise caps one notification at 64 KiB and one object payload at 96,280 bytes.

Family 4: the account itself

Family 4 is what boot step 23 waits for. Its root SOID is the account SOID from the 503 reply.

objectsize before compressionholds
account96,280 bytesroster, profile inventory, progressions, flags, settings, key bindings
selected character46,928 bytesthe character’s equipment, inventory and state; only after a pick
item instance416 bytes eachone per owned item, including profile items the game can act on

Each object is a byte-exact image of the game’s own structure. Sunrise fills it from AccountState (src/middleware/datagen/family4/), then compresses it with Oodle. The layouts are described in Items, characters and inventory.

Inventory actions: the ws RPC

Every player action is a web service call on BAP service 10, answered on service 11. The same envelope runs on 110 and 112. The codec is src/middleware/web_service/web_service_envelope.cpp.

struct WsEnvelope {              /* the service 10 or 11 body */
    uint16_t opcode;             /* +0x00, big-endian */
    uint32_t transaction_id;     /* +0x02, big-endian; the reply echoes it */
    /* +0x06: the payload, bit-packed MSB first, laid out by the opcode's definition */
    /* then two trailer blobs, each: 1 presence bit [, 16-bit length, bytes] */
};

The reply echoes the opcode and the transaction id. Most replies then start with a status:

/* The status fields, in order, all bit-packed. */
code  : 5 bits   /* stored as logical code + 1; 0 = success, 1 = refused */
value : 32 bits  /* stored as logical value - INT32_MIN; a family 4 version, or -1 */
flag  : 1 bit    /* only for opcodes 104 and 901 */
shapebitsexample opcodes
status only5206
status pair37402, 403, 404, 504, 701, 903, 1801, 2400
status pair and flag38104, 901
no status0opcodes Sunrise does not know

Sunrise sets both trailer presence bits to 0, except in the 206 reply.

The version in the reply

A reply that changes the account names the family 4 version that will carry the change. The client waits for that version before it treats the action as done.

/* How Sunrise answers one account action, e.g. opcode 403 equip. Simplified. */
begin_database_transaction();
plan = prepare(request);                    /* check the action against the account */
if (!plan.ok) {
    rollback();
    send(reply(code = 1));                  /* refused; nothing changes */
    return;
}
next  = session.family4Version + 1;
reply = reply(code = 0, value = next);
push  = family4_update(plan, next);         /* on service 123 */
if (fits(reply, push) && commit_database_transaction()) {
    send(reply, push);                      /* one socket write */
    session.family4Version = next;
} else {
    rollback();                             /* nothing is sent */
}
  • The value -1 means there is nothing to wait for. Sunrise sends it in 503 and in any reply that changes nothing.
  • A refused action gets code 1 in its own shape. An opcode whose own codec fails gets a bare echo of the header, so the client’s task still completes.

Opcodes Sunrise acts on

The dispatch is in src/server/web_service/web_service_runtime.cpp and src/server/bap/encrypted/body/bap_service_body.cpp.

opcodeaction
104sync the Store wallet; publishes a new wallet generation
205send the family 5 object again
206subscribe to a family
402dismantle an item
403, 404equip, unequip
405claim an item from the Postmaster
406change item state flags
501create a character
502delete a character
503account bootstrap
504pick a character
505change character
601complete a loot pickup
701save game settings
702the character write-back the client sends on its own
801pick a subclass
901buy from a vendor; the seasonal artifact vendor has its own path
903, 1901insert a socket plug
904pick up a quest
1801claim a record
1820acquire one item instance
1821equip a title
2002open a held engram
2400claim a season pass reward

Any other opcode gets a correlated reply and no action. The reply uses the status shape Sunrise has on record for that opcode, or no status at all.

Rewards and progress

Sunrise reads prices, rewards and rules from the installed packages. The account changes land through the same version rule as every other action. The rules are in src/state/runtime/ and src/server/web_service/.

featurewhat Sunrise does
vendors (901)sells a row only while its availability conditions hold for the selected character. A Silver price is charged in Bright Dust, one for one
faction reputation (901)a turn-in pays its reputation; each new rank pays the rank reward the vendor package names
socket actions (903)charges the plug’s cost, scaled by the socket type. Dawning oven recipes, the Gambit Prime Synthesizer and the Chalice of Opulence run here
engrams (2002)draws what the engram holds through the reward resolver, and removes the engram in the same grant
bounties and questsa granted bounty starts with its deadline. Kills advance held bounties; see below
pursuitsa completed pursuit is redeemed through the reward grant and moves its quest to the next step. A pursuit that completes on its own is redeemed by the server
Triumphs (1801)a claim pays the record’s rewards. A Triumph with intervals is claimed one step at a time
Store (104, service 21)Sunrise answers and publishes the account object with a new saved wallet generation. That this is what ends the client’s Store sync is unverified

A kill reaches Sunrise as an activity incident, message 19 on the Activity Host link. Sunrise checks it against the bounties the selected character holds. A bounty rule can name the victim’s race, the weapon or ability, a precision kill, the killer’s subclass element, the destination or a named bubble. A matching kill writes the bounty’s new progress as an update of that one item. In co-op the host sends a guest’s kills to the guest’s own install.

What persists

survives a restartlost at exit
the account SOID and profile setup flagwhich character is selected
characters, their items, sockets and stacksthe activity each character is in
currencies and materialsthe sign-in time
unlocks, objectives and progressionsthe SignOn token and every link key
family 5 overrideseach connection’s queuez versions
game settings and key bindings
pending rewards
bounty progress and deadlines
mission flags

The session-only fields live beside the database, not in it (Session in src/state/investment/store_internal.h). After a restart no character is selected until the player picks one.

In a cheat session nothing in the left column changes on disk.

Open questions

  • The second family 4 push after ws opcode 206 runs on a fixed 400 ms delay. The client report that should trigger it is unknown.