Databases & Editors (1/3) (in Rust & With<Bevy>)

In the last post we briefly covered the in-memory representation of data “registries”. I also mentioned in that post that This doesn’t make sense if we have to add new spells in the code so into the world of databases we dive!

Git commit reading "Add testing SQLite db & load data from it!"

Basically, for each registry, we create a nice table in the database. This database is currently represented by an SQLite file, which isn’t ideal for the future in the event I add other developers to this project but it works for now.

Game database file

Now loading data from this is easy, we simply need a library like SQLx that implements the database driver. SQLx is asynchronous but since we need most registries before we do anything my loading code ignores this.

pub fn load_y_registry(
    db_conn: Res<RegistryConnectionPool>, // A new-type around an sqlx::Pool.
    mut y_registry: ResMut<YRegistry>,
) {
    // get() is a wrapper function for getting the connection from the sqlx::Pool.
    let mut connection = db_conn.get().unwrap();

    let Ok(res) = async_std::task::block_on(sqlx::query("SELECT id,x FROM y").fetch_all(&mut connection)) else {
        error!("Failed to load DB_NAME_HERE!");
        return;
    };

    for row in res {
        let Ok(id) = YId::try_from_i64(row.get(0)) else {
            warn!("Y ID {} couldn't be parsed.", row.get::<i64, usize>(0));
            continue;
        };

        let x = row.get(1);

        y_registry.0.insert(
            id,
            Y {
                x,
            },
        );
    }
}

And ignoring the async properties of SQLx works fine since we only need mutable access to the registry we are filling up. We are perfectly* parallelized.

*Not quite. There are some limitations with SQLite given it's a single file & we aren't read-only.

Is it that simple?

No. A crux appears when we want to make an editor. This editor would need access to all registries but they are spread across three crates (common, client, and server), alongside gameplay code we don’t need for the editor.

An edit of XKCD 927

A topical edit of XKCD 927

So we split the gameplay logic from the registries. We now have six crates:

common (shared components, systems, and types)
common_data (shared registries, Spells & Auras exist here)
client (components, systems, and types unique to the client)
client_data (registries unique to the client, language & visuals go here)
server (components, systems, and types unique to the server)
server_data (registries unique to the server, creature stats, spawns, etc go here)

Data crates also contain the logic for loading their respective registries.

If data crates contain the loading logic, aren’t you locked into using SQLite?

No, SQLx has the any feature allowing us to connect to any database via the same interface. I’m writing the SQL statements to work on most SQL databases. In the future, I’d love to have a dev database running somewhere that another developer could work toward instead of the aforementioned SQLite file.

How does this work out with spells where you have `Vec<>`s and complex enums?

It becomes complex. SQLite does not have support for arrays so instead, we use serde & serde_json to serialize & deserialize properties such as spell effects into strings.

Spells in the database

This is not perfect, it’s a bit verbose in parts for my liking but it’s quite simple to work with. I’d prefer to have it replace the enum names (Damage, ApplyAura) with an integer but support for explicit discriminators for enums with fields didn’t land until Rust 1.66 in December of 2022.

For any other enums we can simply serialize it to the discriminator, we do this with target and global_cooldown above.

Editor Begins

Git commit reading "Work on spell editor!"

Weeks after the crate split, much fiddling with the database structure and manually adding rows using DB Browser for SQLite, I was growing rather tired of the workflow. So, the time had come to make an editor.

For me, the seemingly best way to make this was to add it as a superset of the client. This means we load the server registries on the client as well when in “Editor-mode” (which is simply a crate feature).

Part of why we did the crate split.

Crate editor feature

In previous posts, I’ve shown my expertise making in game UI with Bevy.

In-game UI

This has been overlapping for months. Never fixed it after updating Bevy.

This is built using bevy’s built-in UI crate but we aren’t going to use that for the editor. Why? Because it’s way overkill for that purpose. Instead, we’ll be using egui and bevy_egui. If you’ve used ImGui this style of “immediate mode ui” might be familiar to you, it is for me which is part of why we picked it up for the editor.

What’s the advantage of using immediate mode ui?

Well, to pop up this window:

Registry window

We only need this code:

fn draw_registry_editor_system(mut egui_context: ResMut<EguiContext>) {
    egui::Window::new("Registry Edit")
        .resizable(true)
        .show(egui_context.ctx_mut(), |ui| {
            ui.label("Hellooo?");
        });
}

Adding a button to this that when clicked does something is simply adding ui.button():

Registry window with a button

if ui.button("Clicky").clicked() {
    // Do something when clicked!
}

The cost of this is… Performance? Visuals? Something. I don’t fully understand how it works but it makes it easy to scrawl out an editor and any debug UI. This is also what bevy_inspector_egui was made with, which is an excellent crate I’ve had in my project for a while now.

World Inspector

But that’s not a registry editor.

Indeed. Registries & their entries aren’t represented by entities but rather as HashMaps. We need a solution of our own. For this we’ll need to figure out how to:

List all editable registry entries. (Covered in this post)
Modify entries. (In the next one)
(Ideally) condense it all into reusable components.

Listing Entries.

We’re gonna display a table of registry entries. Does egui have something for that? No, but it’s child egui_extras does. egui_extras has a table type.

Table Demo from EGUI

egui table demo

All we need to do is feed our registry entries in as rows. Wait, hashmaps don’t have an order, so we’ll get all the entries in a non-sequential order which is a bit confusing.

Instead of iterating directly over the registry, let’s get, sort, and store the keys somewhere, and iterate over that instead.

/// Sorted list of Spells.
#[derive(Default, Resource)]
struct Spells(Vec<SpellId>);

Given we will have quite a few of these registry editor tables that look pretty much the same, here’s a quick_table function that sets up a table:

fn quick_table<'a>(
    ui: &'a mut egui::Ui, // UI context.
    columns: &[&str] // Column names.
) -> Table<'a> {
    // Create table.
    let table = TableBuilder::new(ui)
        .striped(true) // Alternating row stripes.
        .cell_layout(egui::Layout::left_to_right(egui::Align::Center)) // Layout for each cell (column x row) 
        .columns(Column::auto().at_least(15.0), columns.len()); // Size & count of columns.

    // Set up the header/column titles of the table.
    table.header(20.0, |mut header| {
        for column in columns.iter() {
            header.col(|ui| {
                // Bold text for every column name.
                ui.strong(*column);
            });
        }
    })
}

This gives us a table with columns.len() columns for us to fill up with information. Additionally, we have a helper function for creating rows.

fn quick_row(
    row: &mut TableRow, // Table row context
    column_values: &[String], // Values to fill columns with.
    selected: bool // Whether this entry is the currently selected one.
) -> bool {
    let mut clicked = false;

    for value in column_values.iter() {
        row.col(|ui| {
            // A selectable label allows us to both interact with & mark the selected entry.
            clicked |= ui.selectable_label(selected, value).clicked();
        });
    }

    clicked
}

Putting this all together, the code to display the following table: Spell Registry Table

Note how entry 0 is highlighted indicating it's selected.

Boils down to this (omitting system boilerplate):

egui::Window::new("Spell Registry").show(egui_context.ctx_mut(), |ui| {
    let text_height = egui::TextStyle::Body.resolve(ui.style()).size * 1.4;

    quick_table(ui, &SPELL_COLUMN_NAMES).body(|body| {
        // Using egui_extras built-in support for creating many rows at once.
        body.rows(
            text_height,
            spells.0.len(),
            |row_index, mut row| {
                let id = &spells.0[row_index];
                // Fetch spell data from registries.
                let Some(((spell, lang), internal_data)) = spell_registry.get(id).zip(lang_registry.get(id)).zip(internal_registry.get(id)) else {
                        return;
                };

                if quick_row(
                    row,
                    &[
                        id.0.to_string(),
                        lang.name.clone(),
                        internal_data.internal_name.clone(),
                        spell.target.to_string(),
                        spell.get_range().to_string(),
                    ],
                    spell_editor.selected == Some(*id), // We'll get to this.
                ) {
                    // Handle changing the selected spell.
                }
            },
        )
    });
});

Wrapping up.

Now that we can display the registry, we need to figure out how to edit entries & push them back into the database but that will be another post. I don’t want to overflow this one.

Whilst writing this Bevy 0.10 came out and it looks fantastic, I’d like to write a post about it eventually. I will have to wait to update though because of dependencies. I also need to update my crate oxidized_navigation and it also probably deserves its own post.

I get very caught up in programming but I’ve posted a bit more on my mastodon page. Spoilers for my editor progress.

Written 07 Mar 2023 by Grim

GRIM