Rust @programming.dev Doods @infosec.pub 4w ago

Debug-time enforcement of borrowing rules, and safety in general.

Another crazy idea I share with this website.

I was developing a game and an engine in Rust, so I was reading many articles, most of which criticize the 'borrow checker'.

I know that Rust is a big agenda language, and the extreme 'borrow checker' shows that, but if it weren't for the checker, Rust would be a straight-up better C++ for Game development, so I thought: "Why not just use unsafe?", but the truth is: unsafe is not ergonomic, and so is Refcell<T> so after thinking for a bit, I came up with this pattern:

let mut enemies = if cfg!(debug_assertions) {
    // We use `expect()` in debug mode as a layer of safety in order
    // to detect any possibility of undefined bahavior.
    enemies.expect("*message*");
    } else {
    // SAFETY: The `if` statement (if self.body.overlaps...) must
    // run only once, and it is the only thing that can make
    // `self.enemies == None`.
    unsafe { enemies.unwrap_unchecked() }
};

You can also use the same pattern to create a RefCell<T> clone that only does its checks in 'debug' mode, but I didn't test that; it's too much of an investment until I get feedback for the idea.

This has several benefits:

1 - No performance drawbacks, the compiler optimizes away the if statement if opt-level is 1 or more. (source: Compiler Explorer)

2 - It's as safe as expect() for all practical use cases, since you'll run the game in debug mode 1000s of times, and you'll know it doesn't produce Undefined Behavior If it doesn't crash.

You can also wrap it in a "safe" API for convenience:

// The 'U' stands for 'unsafe'.
pub trait UnwrapUExt {
    type Target;

    fn unwrap_u(self) -> Self::Target;
}

impl<T> UnwrapUExt for Option<T> {
    type Target = T;

    fn unwrap_u(self) -> Self::Target {
        if cfg!(debug_assertions) {
            self.unwrap()
        } else {
            unsafe { self.unwrap_unchecked() }
        }
    }
}

I imagine you can do many cool things with these probably-safe APIs, an example of which is macroquad's possibly unsound usage of get_context() to acquire a static mut variable.

Game development is a risky business, and while borrow-checking by default is nice, just like immutability-by-default, we shouldn't feel bad about disabling it, as forcing it upon ourselves is like forcing immutability, just like Haskell does, and while it has 100% side-effect safety, you don't use much software that's written in Haskell, do you?

Conclusion: we shouldn't fear unsafe even when it's probably unsafe, and we must remember that we're programming a computer, a machine built upon chaotic mutable state, and that our languages are but an abstraction around assembly.

26 comments

One thing I've noticed with Rust is that if you find yourself fighting with the borrow checker, that's a sign that your codebase isn't well structured.

So I'm curious; what problem have you been trying to solve where the borrow checker has been this much of an obstacle? There might be a cleaner design for it.
- I disagree. It's a sign your code isn't structured in a way that the borrow checker understands, but that is a subset of well-structured code.
  
  In other words, if your code nicely fits with the borrow checker then it's likely well structured, but the inverse is not necessarily true.
  
  One thing I always run into is using lambdas to reduce code duplication within a function. For example writing a RLE encoder:
  
  fn encode(data: &[u8]) -> Vec<u8> { let mut out = Vec::new(); let mut repeat_count = 0; let mut output_repeat = || { out.push(... repeat_count ...); }; for d in data { output_repeat(); ... } output_repeat(); out }
  
  This is a pretty common pattern where you have a "pending" thing and need to resolve it in the loop and after the loop. In C++ you can easily use lambdas like this to avoid duplication.
  
  Doesn't work in Rust though even though it's totally fine, because the borrow checker isn't smart enough. Instead I always end up defining inline functions and explicitly passing the parameters (&mut out) in. It's much less ergonomic.
  
  (If anyone has any better ideas how to solve this problem btw I'm all ears - I've never heard anyone even mention this issue in Rust.)
  
  @FizzyOrange @savvywolf
  I think this is a fine pattern and I use it in rust occasionally.
  how does this not work for you?
  https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0f06fe574e1c69424b4749ef910a1fa7
First of all, unsafe famously doesn't disable the borrow checker, which is something any Rustacean would know, so your intro is a bit weird in that regard.

And if you neither like the borrow checker, nor like unsafe rust as is, then why are you forcing yourself to use Rust at all. If you're bored with C++, there are other of languages out there, a couple of which are even primarily developed by game developers, for game developers.

The fact that you found a pattern that can be alternatively titled "A Generic Method For Introducing Heisenbugs In Rust", and you are somehow excited about it, indicates that you probably should stop this endeavor.

Generally speaking, I think the Rust community would benefit from making an announcement a long the lines of "If you're a game developer, then we strongly advise you to become a Rustacean outside the field of game development first, before considering doing game development in Rust".
Have you actually measured a performance impact from RefCell checks?
I just wanted to advice you against thinking that if there's something in all cases you've tried, there's something every time. When you put something in an optional and then unwrap, it's okay because you can see that the value is there, but even then there are usually better ways to express that. When you expect that since you've run the code thousands of times and it didn't break [in a way that you would notice, e.g. panic in another thread will only affect that thread] means that everything is fine, you may get weird bugs seemingly out of nowhere and will also need to test much more than strictly necessary.

Regarding the borrow checker, it has limitations and there are improvements that I hope will some day find way into upstream, but most of the time it may be better to change the code flow to allow borrow checker to help with bugs, instead of throwing it away completely. The same goes for unsafe, as in most cases it's better to not uphold invariants manually.
Instead of checking if debug assertions are disabled, you should use debug assertions, it would make the code much neater.

If you wanna eliminate the borrow checker this way, I guess you could use raw pointers instead of references, and have debug assertions to check if those pointers are null. At that point you'd have a mix of C and Rust. The memory would be completely unsafe (you'd have to allocate mostly on the heap, and drop it manually), but you'd have rusts's type system. You'd also lose a lot of ergonomics though.

EDIT: just to be clear. I think completely disabling the borrow checker this way is absolutely nuts. Maybe you could have some raw pointers in troublesome locations (where RC/RefCell would have too big of a performance impact for a 144fps game). But most of the code should be borrow checked, since the borrow checker only gets in your way sometimes. It's not like lifetimes go away with the borrow checker, you still have to think about lifetimes if you manually manage your memory.
Two thoughts come to mind for me:

I think people should feel free to use any language however they want for their own needs and projects, but it's also important to understand what exactly "unsound" and "undefined behavior" mean if you're going to dabble with them. If it's a risk you're willing to take, go for it, but don't be surprised if things break in ways that make no sense at all. Realistically a compiler won't delete your root directory if you trigger UB or anything, but subtle bugs can creep in and change behaviors in ways that still run but which make unrelated code break in difficult to debug ways.

The borrow checker is one of Rust's biggest features, so looking for ways around it feels a bit counterproductive. Feature-wise, Rust has a lot of cool constructs around traits and enums and such, but the language and its libraries are built around the assumption that the guarantees the compiler enforces in safe code will always be true. These guarantees extend beyond the borrow checker to things like string representation and thread safety as well. As an alternative, some other languages (like C++, which you mentioned, or maybe even Zig) might be better suited for this approach to "dirty-but-works" development, and especially with C++, there are some excellent tools and libraries available for game development.
- Hello Pers,
  
  I made a mistake when writing the post, it reads like I am against the borrow checker, which I am not, I love the checker, and didn't encounter any - major - problems with it.
  
  I meant that even if we used unsafe everywhere it would still be a good language, which is an attempt at arguing with those saying that Rust isn't fit for gamedev because the of the checker. Which I failed at due to lack of experience, as this is my first time making a game, and Rust is my first language*.
  
  Regarding: "If it doesn't panic in Debug, it won't act weird on Release", even if I got reported a really weird bug related to UB, I should (I am not experienced enough to make a claim) be able to know it's UB since the game's gonna crash when I try to recreate the bug in Debug.
  
  Some would say that shipping the game with runtime checks won't have an effect on performance, which is probably true, since it's so simple the list of entities is an array (not a vector), and the game state is - effectively - global (everything is impl CombatContext { fn x(&mut self) {} })**, and some (most? too early in development to tell) of the game is locked at 5fps (maybe I'll bump it up a bit)***.
  
  I am so concerned about performance because I had to daily drive a computer that most people on this website - and especially on Reddit - would consider garbage E-waste, for 4 years, and was trying hard to play games on it, which was further amplified by my GPU not supporting Vulkan (nor Dx9 for some time), which meant I couldn't use Proton, which taught me some hacks that are... let's not talk about them.
  
  So I find huge pain in leaving any possible performance optimizations, especially that some people I know are stuck on - arguably - worse machines****; accessibility is a big priority.
  
  It also makes me angry to see pixel games come with 70Mib binaries and require Vulkan because:
  
  1 - internet costs money
  
  2 - they claim in the system requirements that their game "Should run on anything".
  
  Memes like: "Oh my game could run on a potato" infuriate me (good thing I don't use social media), NO, your game can't run a potato, DooM can, it was actually optimized properly, your 2D pixels can't even render on a machine a 100x more powerful, you should feel ashamed*(5).
  
  *: I was messing around with C# + Godot not super long ago, nothing serious.
  
  **: I have been refactoring my code lately to limit the scope of most functions, in a way inspired by ECSs, but significantly more primitive.
  
  ***: the game has both a 3D and a 2D part, the 2D part has locked FPS, the 3D part can run at any framerate.
  
  ****: Macroquad supporting OpenGL only down to 2.0ES would be a problem, if I wasn't intending on forking it anyway to reduce the binary size (grim is an extremely bloated dependency, I managed to shove off 10 Mib in a few hours), and unless using 1.x is as hard people on the internet claim it is, which is probably false, as these people are mostly weak and say the same things about using a custom engine.
  
  *(5): this might sound toxic, but that's how people get better.
  
  Do you have any sources about this "unfitness" of Rust for gamedev? From my experience many people have problems with the borrow checker due to habits they obtained coding in other languages. Due to the strict enforcement of ownership rules some things can't be done exactly like in e.g. C++, but the language offers the tools to design and implement the software in a way to work with (not around!) the borrow checker (an example would be the discussion in another comment thread).
  
  So I'd be interested what special behavior occurs in gamedev that makes coding in Rust more difficult than for other software.
  
  Also, I really like that you're considering users with lower spec machines for your game. However, have you run a profiler over your software to see where there is optimization potential? In cases like this, I often use the (maybe over-used) quote from Donald Knuth:
  
  We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.
  
  Many people think that with this quote, no optimization whatsoever should be considered. However, humans are often bad predictors on where the real bottlenecks are.
  
  So yes, have some things in mind like e.g. algorithm performance for the expected input sizes. But in many cases, "optimization" done doesn't impact the runtime at all but makes the code much more difficult to understand and often more "unsafe" due to potentially increasing UB or other issues.
  
  For this, a profiler helps to detect where your "critical 3%" are. And when you know them, then you can start optimization work to get big improvements. This also helps to direct your most likely limited effort to spend to the important parts of your game.
  
  I see now, that you were misunderstood in some parts.
  
  even if I got reported a really weird bug related to UB, I should (I am not experienced enough to make a claim) be able to know it's UB since the game's gonna crash when I try to recreate the bug in Debug.
  
  This may be problematic for several reasons: it may be hard to reproduce, the more complicated the state, the harder; bug may rely on some race condition that may be much rarer in Debug because of speed difference; UB is notorious for causing things that should (seemingly) never happen, like returning from infinite loops, proving false statements true, and such, so it may be hard to understand what at all happened and why.
  
  Regarding optimisations, it might still be better to try to profile the code (I will be honest, I don't do that until the moment when I can't go further without optimisation, and I haven't reached that with Rust) and see what are the real hot spots that require optimisations. I hope that someday you will be able to upgrade your machine, and hope that your game will be a good example of something that really runs anywhere

26 comments