Looking to learn Rust? Curious about Rust? Feel free to ask me

TL;DR:

• Learned basic C++ as my first language (would not recommend) many years ago
• Went to study computer science at university
• Was intrigued by Rust from online blog posts, learned it in my free time as a student
• Have been working professionally with Rust for more than 2 years now

Long version:

I "learned" C++ as my first language many years ago. I say "learned" because I just learned the basic syntax and some of the concepts about computers, like how the stack and the heap works and processes and threads and stuff like that. It was very early and I didn't really understand much, was just experimenting.

I then started studying computer science. We were exposed to many different programming languages at the university. Over the years, you got a feel for the strengths and weaknesses of each one. Especially during my master's I learned a lot as I was exposed to Haskell and in general nicer-working type systems.

Around the same time I started noticing blog posts about Rust online. I became quite intrigued with the language as it had a lot of promise and I was curious if it could really live up to the hype.

I then just started reading The Book and I was very quickly convinced that this language was actually living up to the hype. It's a very nice blend of object-oriented and functional programming, with an algebraic type system and monad-like error handling, which is just miles ahead of the usual exception-based error handling that many other languages use.

Then I basically started using it in all my side-projects in my free time, slowly building up a familiarity with the package ecosystem and the idiomatic way of writing Rust, reading lots of examples and documentation and such.

Nowadays, I would humbly call myself an expert in Rust or at least close to it. I think I've seen nearly all the language has to offer and ran into many of its weirder parts. I have a very good understanding of ownership and the borrow checker and I usually know exactly what is wrong and what to do about it when the compiler yells at me. The only thing I haven't really touched is embedded (no-std) programming in Rust.

Is it possible to become as productive with rust as one is with higher level languages?

Definitely. Rust can be very high level, just as or at least very close to the level of languages like Python. Just as high level as Go or Java I would say.

Rust is quite a wide language, in the sense that it is able to do both low-level and high-level programming. More generally, I'd say the distinction between "high level" and "low level" is just a matter of how much abstraction you use and you can build very powerful abstractions in Rust, and people have done this in many crates.

For instance while using the axum web server framework, you might see code similar to this:

async fn new(db: State<PgPool>, value: Json<MyType>) -> StatusCode {
    ...
}

This function defines a HTTP endpoint that takes a JSON body that deserializes to MyType and internally accesses a PostgreSQL database (for example to save the value in the database) and returns an empty body with a given status code. This is a very "dense" definition in the sense that a lot of semantics is pressed into so few lines of code. This is all possible due to the abstractions built by the axum web framework, and you see similar things in many other crates.

I was kinda hoping to make rust be my go to language for general purpose stuff.

It is a great language for all kinds of stuff - I use it as my general purpose language all the time :)

I would encourage you to stick to it and see how the language feels once you try it a bit more in a real project, and not just in exercises. The book can be a little dry because it has to explain both the high level and low level features of Rust.

If you have more specific questions about what exactly makes it awkward, maybe I can provide more guidance.

C++ knowledge should make it easy to understand Rust. In a lot of ways, the Rust compiler is just enforcing all the stuff you need to do in C++ anyways in order to avoid undefined behaviour.

I think The Book is fine, that's what I used to learn. You can also learn quite a bit by just reading the standard library documentation, which is very nicely structured.

Unfortunately there's not really anything to teach it as the first language. But I think it's still possible if you want

I’ve been wanting to write rust for quite some time, but I can’t get over crates. The system just seems insecure to me.

You're not the only one with this concern but it is essentially how modern package management works, not just for Rust but all modern programming languages.

What happens in 10 years when the servers go down?

While I don't think that would happen, there are ways to avoid this. You can host your own registry and mirror the crates.io crates, if you want.

Is there any sort of mitigation for supply chain attacks?

Whenever you have dependencies, you obviously need to either trust them or vet them. If the package is popular enough and the author is reliable enough, then you can choose to trust it. It really depends on what kind of risk you're willing to take on.

As I understand it anyone can submit code; what’s stopping someone from putting malicious code into a crate I’ve been using?

In principal nothing. Again, if you have dependencies, you need to vet them. This isn't really a Rust problem, it's just a general problem with depending on other people's code. You would still have this problem even if you manually downloaded external pieces of code from other people instead of via cargo.

In practice, there is a team managing crates.io and I believe they do look for malware or malicious crates (like crates with names very similar to popular crates that attempt to trick people into downloading due to a typo in the name).

But yes, this isn't really a problem with Rust specifically. I will say that the popular crates in the Rust ecosystem are generally very high quality and I have a fair bit of trust for them myself. Unless you are a big company that needs to carefully vet your dependencies, I wouldn't worry too much.

Do you have any sort of synthesis of the core ideas or principles or best practices that work for you?

I think it's hard to give some sort of master theorem because it really comes down to what you are doing.

That said, start by considering ownership. As you start, avoid creating types with non-static references inside of them i.e. keep to types that own their contents. Types with references requires you to use lifetimes and you'll be better off getting used to that later. And frankly, it's often not necessary. The vast majority of the types you'll ever make will fully own their contents.

Now when you make functions or methods, think of the three options you can use when taking parameters:

1. Take ownership, i.e. MyType or self for methods. Use this if your function needs to own the data, often because it needs to take that data and put it into another value or otherwise consume it. Honestly this is the least common option! It's quite rare that you need to consume data and not let it be available to anyone else after the function call.
2. Take a shared reference i.e. &MyType or &self for methods. Use this is your function only needs read access to the data. This is probably the most common case. For instance, say you need to parse some text into some structured type; you'd use &str because you just need to read the text, not modify it.
3. Take a unique reference, i.e. &mut MyType or &mut self. You'll need this if you want to refer to some data that is owned elsewhere but that you need temporary exclusive (unique) access to, so that you can modify it. This is often used in methods to be able to modify private fields of self for example. You need to think about the fact that no one else can have a reference to the data at the same time though. Often this is not a problem, but sometimes you need to be able to mutate stuff from multiple different places. For that, you can consider either passing ownership around, for instance via channels and sending messages or you could reach for Arc<Mutex<T>> to allow mutation through shared references with a tiny bit of runtime performance cost.

When you think in terms of ownership, the borrow checker becomes easy to understand. All it's doing is checking who owns what and who is borrowing what owned data from who and are they giving it back when they said they would?

I hope that helps but again, it's a very general question. If you give me a concrete case I could also give more concrete advice.

PS: Abso-fucking-lutely just clone and don't feel bad about it. Cloning is fine if you're not doing it in a hot loop or something. It's not a big deal. The only thing you need to consider is whether cloning is correct - i.e. is it okay for the original and the clone to diverge in the future and not be equal any more? Is it okay for there to be two of this value? If yes, then it's fine.

Unfortunately embedded programming is not something I have first hand experience with, but I've read how it works.

The Rust standard library is actually split into 3 layers:

1. Core: This is core stuff built into the language. No dependencies needed. No memory allocation, no syscalls, no nothing basically. This defines the primitive types like i32 and str and pure functions on those.
2. Alloc: Provides heap memory allocation on top of core, giving access to types like Vec and String. Still no operating system so no file IO or anything like that.
3. Std: The full standard library that generally assumes the presence of an operating system with file systems and networking and all that jazz.

You probably want to check out the Rust embedded book.

The language is obviously still the same even if not using the full standard library. Feel free to ask more specific stuff, though I'm not very familiar with embedded.

Lets start of with, is there a difference between the const value vs const reference?

No. Not any practical difference at least. AFAIK behind the scenes, the const reference is just a const value that Rust automatically creates a reference to. So it's just a matter of preference or what makes sense for your case.

1. Since my functions always take a reference, is there any advantage to any of them. References are read-only, but since it’s const it probably doesn’t matter. What is prefered?

I think I need to see an example of the function (or at least the signature) to give any concrete advice. It really depends on what you're doing and what the function is.

2. I know String does allocations, while &str is a string slice or something which may be on the stack. Do I not end up making any allocations in this case since stucts are on the stack by default, and only hold the pointers to a string “slice”. Especially given how they are made in this case.

Okay so there's multiple things to unpack here. First of all, &str is a string slice, as you say. It is a "fat pointer", as all slices are, with 2 components: A pointer to the data and a length.

However, &str does not say anything about whether or not the data that it points to or even the &str itself (pointer + length) is on the stack or not. The pointer inside the &str may point anywhere and there's no guarantee that even the &str itself is on the stack (e.g. a Box<&str> is a &str on the heap and the pointer inside that &str could point anywhere).

When you write a string literal like "Hello world!", it is a &'static str slice, where the &str itself (pointer + length) exists on the stack and the data that it points to is memory inside your final executable binary. But you can get &str's from elsewhere, like a &str that refers to memory on the heap that doesn't have a 'static lifetime.

So when you write a string literal like you are in your consts there, you are not allocating any memory on the heap (in fact it is impossible to allocate memory on the heap in a const context). All you're doing is adding a bit of string data to your final executable and that's what the &str points to. To allocate, you would need to use a String.

3. Is structs with references like this okay, this Page is constant but I’m going to make many “Pages” later based on the pages my blog has, as well as some other endpoints of course.

If all of your Pages are constant and you will write them out like this with literal strings, it's not like there is any "problem". However, you'll only be able to make Pages that can be defined at compile-time. Maybe that's fine if all your pages are pre-defined ahead of time and doesn't use any dynamic values or templating? If so, you can keep it like this in principle. You could even write the pages in a separate file and use the include_str! macro to import them as a &'static str as if you had written it out directly in your code.

If you need just some Pages to have dynamic content, then you'll need to either use a String (which would make all of them heap-allocated) or you could use a Cow (clone-on-write pointer) which is an enum that can either be borrowed data like a &'static str or owned data like String. Using Cow you could allow the static pages to not allocate while the dynamic ones do.

4. If the struct is cloned, is the entire string as well, or just the pointer to the string slice? I assume it does copy the entire string, since to the best of my knowledge a &str does not do any reference counting, so deleting a &str means deleting it’s content, not just the reference. Since that is the case, a clone will also copy the string.

No, cloning a &str does not clone the underlying text data. You can see this via a simple program:

fn main() {
    let s1 = "Example text";
    let s2 = s1.clone();
    
    println!("{s1:p}");
    println!("{s2:p}");
}

The :p format specifier is the "pointer" specifier and will print the pointer of the string slice. If you run this, you'll see that it prints the exact same pointer twice - i.e. both s1 and s2 are pointing to the exact same data (which only exists once).

No reference counting is needed. Remember, Rust keeps track of reference lifetimes at compile-time. If you clone a &'a str then the clone is also a &'a str. This is fine, it's just another reference to the same string data and of course it has the same lifetime because it will be valid for just as long. Of course it's valid for just as long, it's pointing to the same data after all.

Note that mutable references cannot be cloned, as that would allow multiple mutable references to the same data and that's not allowed.

Note that dropping ("deleting" is not a term used in Rust) a &str does not free the underlying memory. It is just a reference after all, it doesn't own the memory underneath.

5. I am contemplating adding a “body” string to my Page struct. These string will of course be large and vary depending on the page. Naturally, I do not want to end up copying the entire body string every time the Page is cloned. What is the best course here, it kind of depends on the previous question, but is an Arc the solution here? There is only reading to be done from them, so I do not need to worry about any owner holding it.

If the body is a &str, then it will only clone the reference and the string data itself will not be copied, so this shouldn't be an issue. But remember as I said above that this might only be true if your pages are indeed defined ahead-of-time and don't need dynamic memory allocation.

I can give more concrete advice if you give more code and explain your use case more thoroughly (see also XY problem).

I wanted to serve pages in my blog. The blog doesn’t actually exist yet (but works locally, need to find out how I can safely host it later…), but lets assume it becomes viral, and by viral i mean the entire internet has decided to use it. And they are all crazy picky about loading times…

Of course it depends if doing this kind of optimization work is your goal but... if you just want a blog and you want it to be fast (even with many visitors, but perhaps not the entire internet...), I would say make a static web server that just serves the blog pages directly from &'static strs and predefine all blog posts ahead of time. For example, you could write all your blog posts in HTML in separate files and include them into your code at compile time.

You'd need to recompile your code with new blog post entries in order to update your blog... but like how often are you gonna add to your blog? Recompiling and redeploying the blog server wouldn't be an issue I imagine. That's how I would do it if I wanted a fast and simple blog.

Also general software development wisdom says "don't code for the future" aka YAGNI - you aren't gonna need it. I mean, sorry, but chances are the whole internet will not be crazy about visiting your blog so probably don't worry about it that much 😅. But it is a good learning thing to consider I guess.

#[derive(Clone)]
pub struct Page<'a> {
   pub title: &'a str,
   pub endpoint: &'a str,
}

I'm a little confused about the use of the word "endpoint" here - that usually indicates an API endpoint to me but I would think it would be the post contents instead? But maybe I'm just too hung up on the word choice.

I wanted to create a HashMap that held all my pages, and when I updated a source file, the a thread would replace that page in the mapping.

To me, this sounds like you want to dynamically (i.e. at runtime, while the server is running) keep track of which blog entry files exist and keep a shared hashmap of all the blog files.

So there's multiple things with that:

1. You'd need to dynamically allocate the storage for the files on the heap as you load them in memory, so they'd need to be String or an Arc<str> if you only need to load it in once and not change it. Since you don't know at compile-time how big the blog posts are.
2. As you note, you'd need a way to share read-only references to the hashmap while also providing a way to add/remove entries to it at runtime. This requires some kind of lock-syncing like Mutex or RwLock, yes.

why can’t I have a AtomicPointer to my data that always exist?

Does it always exist though? The way you talk about it now sounds like it's loaded at runtime, so it may or may not exist. I think I'd need to see more concrete code to know.

and I’m actually calling unsafe code. I have heard it can produce unexpected error outside it’s block.

Yes, indeed. Safe code must never produce undefined behaviour, but safe code assumes that all unsafe blocks does the correct thing. For instance, safe code will always assume a &str contains UTF-8 encoded data but some unsafe code may have earlier changed the data inside of it to be some random data. That will break the safe code that makes the assumption! But it's not the safe's code fault.

Unsafe in general is a very sharp tool and you should be careful. In the best case, your program crashes. In worse cases, your program continues with garbage data and slowly corrupts more and more. In the even worse case, your program almost always works but rarely produces undefined behaviour that is extremely hard to track down. You could also accidentally introduce security vulnerabilities even if your code works correctly most of the time.

In general, I would advise you to avoid unsafe like the plague unless you really need it. A hypothetical optimization is certainly not such a case. If you really want to use unsafe, you definitely need to carefully peruse the Rustonomicon first.

In your specific case, the problem is (of course) with the unsafe block:

unsafe { self.data.load(Relaxed).read_unaligned() }.clone()

So what is this doing? Well self.data.load(Relaxed) returns a *mut Arc<T> but it is only using safe code so the problem must be with the read_unaligned call. This makes sense, obtaining a raw pointer is fine, it's only using it that may be unsafe.

If we check the docs for the read_unaligned function, it says:

Reads the value from self without moving it. This leaves the memory in self unchanged.

Here "self" is referring to the *mut Arc<T> pointer. So this says that it reads the Arc<T> directly from the memory pointed to by the pointer.

Why is this a problem? It's a problem because Arc<T> is a reference-counted pointer, but you've just made one without increasing the reference count! So the Arc believes there are n references but in fact there are n + 1 references! This is bad! Once the Arc is dropped, it will decrease the reference count by 1. If the reference count is 0, it will drop the underlying data (the T).

So let's say you get into this situation with 2 Arcs but actually the reference count is 1. The first one will drop and will try to free the memory since the reference count is now 0. The second one will drop at some later time and try to update the reference count but it's writing into memory that has been freed so it will probably get a segmentation fault. If it doesn't get the segfault, it will get a problem once it tries to free the memory since it's already been free. Double free is bad!

So yea that's why it probably works once (first arc gets dropped) but not twice (second arc gets a bad experience).

I’m mainly interested in using Rust as my go to back end language for HTTP/TCP servers and developing JSON and HTML APIs. Can you tell me which frameworks and crates/packages would be good for me to be aware of?

Look into axum. It's built on top of tower and tower-http, which is a general server/client framework. Axum makes it super easy to make HTTP servers, using either raw HTML or JSON. It has in-built functionality for JSON and if you want to do HTML you can reach for stuff like maud or askama.

There's lot of crates around axum as well. You can try searching lib.rs which is a bit more nicely categorized than crates.io. For logging for example, look into tracing.

I’m also interested in creating some CLI and TUI applications, so any frameworks/crates/packages I should be aware of in that realm you might recommend would also be greatly appreciated!

For command-line arguments, use the derive functionality from clap. It lets you declare the arguments you want/need as a type and produces all the argument parsing logic for you.

For TUI, I haven't tried it myself but I've heard that ratatui is good.