Rust Design Patterns as a Book
rust-unofficial.github.ioA very low-effort way to learn good Rust patterns is to put
#![warn(clippy::all)]
Thanks a lot for the suggestion, I can't believe I didn't know about this. However I just tried it and I can't get it to work.
I added this to the top of https://github.com/etesync/etebase-rs/blob/master/src/lib.rs and then ran `cargo clippy`
Any idea what's missing? Why is it not failing?#![warn(clippy::all)] // Should fail https://rust-lang.github.io/rust-clippy/master/index.html#float_cmp pub fn bool_test(x: f32, y: f32) -> bool { x == y }Edit: I know the above example is bad code, that's the point. I want clippy to complain about it but it doesn't.
Does it work if you run `cargo clean` and then `cargo clippy` again? Clippy runs it's lints in an early pass of the compiler/checker. Many IDEs/editors will automatically `cargo check` under the hood to grab errors. Then when you run `cargo clippy`, that part of the compilation is already cached and so clippy doesn't give you any output :(
To my knowledge (it's been a while since I looked) fixing this behavior is blocked on cargo stabilizing something and has been for literal years.
That point of frustration aside, it's worth it...Clippy is an absolutely amazing piece of software. Both for pedagogy and normal development.
EDIT: Just dug up the issue. If you're on nightly you can use `cargo clippy -Z unstable-options` to avoid the clean/rebuild. Hopefully stuff gets stabilized soon. Here's the issue for reference: https://github.com/rust-lang/rust-clippy/issues/4612
That fixed it, thanks a lot!
Instead of `cargo clean`, `touch src/main.rs` or `touch src/lib.rs` (or actually touching any source file and thereby changing the `last modified` date to now) will have the same effect. That's what I've been using.
I’m not sure and can’t check at the moment; but maybe the functions not called so it’s being ignored?
It's public in my lib, so it's exported. Though I also tried calling it from my code, it still doesn't trigger a warning. :|
Comparing floats by equality is a dangerous pattern. It's easy for small precision errors to occur. You should instead check that they are close enough to each other, using an epsilon that you find appropriate, perhaps 1e-10. `(x - y).abs() < epsilon` should do the trick
I know, it's an example I was hoping clippy would catch in order for it to fail so I know it works. Read what I wrote...
It's in fact the dangerous pattern used to illustrate the linter under discussion here.
Float comparison is an anti-pattern. Use an epsilon instead. https://stackoverflow.com/questions/4915462/how-should-i-do-...
I know, it's an example I was hoping clippy would catch in order for it to fail so I know it works. Read what I wrote...
Wait what? You can use dyn on the stack (without a Box)?
This has been one of my biggest complaints about Rust: I've been using it for years at this point. I read most of the Book, I've read a few unofficial books. And I do love the language, but it has so many cases like this where things that you're allowed to do (syntactically or otherwise) are somehow so non-obvious that you can miss them entirely. I still get blindsided by something like this every couple months, and I always end up a little mad that I've been doing things the hard way until some obscure unofficial material (or more often, a stack overflow answer) teaches me about an entire feature that I didn't know existed.
Rust is really great at telling you what you can't do, and in many ways its documentation is incredibly thorough, but it has a real problem when it comes to discoverability and establishing a consistent mental-model of what its syntax actually means (and how you can then apply it to other situations). I don't know what the root cause of this problem is. But it's really distressing to me each time I discover a huge blind-spot; it makes me feel like I never fully understood the language concepts that I thought I understood.
I say all of this out of love: I really want Rust to succeed. I still prefer to use it despite this issue. I just believe this is a huge thorn in its side, especially when it comes to adoption, for which it already has an uphill climb.
Honestly, who cares about not knowing that something was possible if you never wanted it? `dyn` working on the stack is more consistent than it not working on the stack.
I'd say one way to a language more completely is to:
a) Read its stackoverflow pages (not sure about Rust but this helped a ton with C++)
b) Get code reviews from others. The rust community is fairly active, or at least it was back when there was just an IRC channel, I don't know as much now since I don't get onto whatever the medium is today. Ask people if there's an easier way to do something.
c) Read others code. I, for example, like to review some of my dependencies just so I understand a bit more about how they work, and I've picked up a lot from that. I used to get on IRC in the morning while I was kinda getting into my work-day and see if I couldn't help others out, this really taught me a lot.
> who cares about not knowing that something was possible if you never wanted it?
I have wanted that feature at times in the past (or something like it), and wasn't able to determine that it would be possible. That's exactly my point.
As for your suggestions: this is indeed one way to gather this knowledge, and I have learned a lot particularly from looking at stack overflow. But I don't think it's ideal when knowledge about fundamental language features has to be acquired by word-of-mouth, because it wasn't conveyed in the course of the normal learning path (official tutorials + discovery through application of learned concepts to new situations where their relevance is self-evident).
I guess when I assume a feature is possible, and I want to use that feature, I try it and see if it works.
I also don't typically learn via books myself, I learned rust pre-book, and basically entirely via IRC. To me, that's the default-path. But I get why it's not ideal - a written record (like the one linked) is definitely important to have, and it is a legitimate weakness that they're so nascent.
dyn just requires the object to be behind some kind of pointer. The vast majority of the time, that pointer is a Box or an Rc/Arc, but any form of indirection can work.
Taking a second look at this particular example, I guess it is a bit of a "trick" because of the ahead-declaration of both possible "holder" variables on the stack. It's just... wildly unintuitive that this should be possible. Even if you showed me this code without saying whether or not it should compile, I wouldn't be sure.
Here's the train of intuition:
1) dyn requires a pointer that may be to one of multiple types of structs
2) a group of multiple types of structs has an undefined memory layout, so the value must either live on the heap or be wrapped up in an enum
That feels like an airtight understanding. But then Rust lets you do this weird juggling maneuver based on control-flow that allows you to do it on the stack.
I'm not saying Rust shouldn't let you do this, and I'm not really sure how it could be made intuitive given the "normal" case. I'm just expressing that subjectively, this feels very weird and non-obvious, and it's far from the first example like this that I've encountered. Here's another example: https://news.ycombinator.com/item?id=25595120
I think where your intuition is leading you wrong is that in #2, you are assuming ownership. That is, you're saying "I need a place to put an arbitrary thing." But you don't! A &dyn T doesn't own T, just like a &T doesn't own T. Trait objects are a (pointer to data, pointer to vtable), (may be in the reverse order we don't guarantee layout) and so that pointer can point to anywhere, heap or stack.
Interestingly enough, this was special in Rust 1.0 to Rust 1.5. In 1.5, it finally became non-special. It's interesting because I totally get what you're saying, but at the same time, this is an example of Rust being orthogonal, not special cased.
(It is really hard to address the string thing without an example, to be honest.)
I think you're right about my intuition, and that's interesting about the syntax change. It could just be that during my learning stage I learned (based on example bias maybe?) that dyn is for owned values, and not just any reference. I'm sure this was never stated explicitly, but somehow that idea got lodged in my brain
For the String thing (really, the Deref thing), further down others weigh in with a case where it doesn't work as expected, and then the workaround &*. The latter is something that feels like it should be a non-op, yet it's required in certain cases like this one to trigger something in the compiler. I'm sure there's some internal reason for this, but from the user's perspective it's, "What does dereferencing and then re-referencing this value have to do with performing what amounts to a cast?"
I want to emphasize that I'm not complaining just to complain, nor placing blame on any specific party. I'm just "reporting a bug" in my learning experience with the language, and trying to provide as much info as possible :)
> I want to emphasize that I'm not complaining just to complain,
Oh yeah totally! It is very helpful.
> It could just be that during my learning stage I learned
I mean, I think this is very reasonable and intentional. Trait objects are a pretty niche feature of Rust already, and non-owned trait objects are even more niche than that. The book does guide you towards Box<dyn Trait> for this reason.
> or the String thing (really, the Deref thing), further down others weigh in with a case where it doesn't work as expected,
Yeah so the trick here is a balance between not wanting coercion willy-nilly, and also making some cases work well. You had cited method calls specifically, and those should work due to auto-ref/deref. The example given isn't about method calls, it's about match not doing Deref coercion. That being said it's really easy to assume that it always does it, because it does do it in the right places most of the time! There's interesting tradeoffs here...
> The book does guide you towards Box<dyn Trait> for this reason.
Yeah. And that makes sense, though an aside that explains the broader concept ("Note: dyn is usually paired with Box, but it can be used to describe any reference") would help establish the more generalized understanding (it is possible this aside already exists and I just missed it)
Re: the deref coercion, I do think many cases of this class of problem I'm describing come down to implicit behavior the compiler does to infer certain commonly-used and onerous syntactical elements, to make code cleaner and easier to write. I understand why this was deemed necessary, and it's not as much a problem as "magical" behavior in other technologies, because (seemingly) everything the compiler does implicitly maps directly to an equivalent explicit version.
But it leads to a lot of confusion when those rails eventually break. In terms of brevity, this can be looked at as "gracefully degrading": it makes the normal cases better, and reverts to the "baseline" behavior when you step outside of those. But in the context of learning, it is not such a strict win, because the implicit cases have colored the user's understanding of the language itself, actively hampering their ability to venture off the golden path (or form generalizations about concepts).
I kind of wish the compiler had a "turn off all implicit behavior" option, so that you could learn how everything is done explicitly before turning the helpers back on for the sake of productivity.
> (it is possible this aside already exists and I just missed it)
Yeah I am not sure if it does or not; I'll make a mental note to go check sometime.
> I kind of wish the compiler had a "turn off all implicit behavior" option,
The problem is that what is "implicit" is different for everyone. Some people would say that Box::new is "implicit" because you don't see the malloc. Some might even say that malloc is implicit because you don't see the sbrk/memmap!
> But in the context of learning, it is not such a strict win,
Yeah so it's tricky! The thing is, without some way to jumpstart knowledge, people may never even learn in the first place. Like, by this argument, everyone should start with physics, because well, C compiles to asm complies to machine code which is actually just code for a chipset. There is no "I know what everything is doing" starting point. In reality, people jump in, start somewhere, and then expand what they know from there. This is true on every topic in every field. We still teach kids Newtonian physics even though on some level it's "wrong," you know?
Sure, but Rust takes implicity a lot further than most. Implementation details of a function call are one thing; applying an operator in the current context, or adding an invisible generic type (lifetime parameter) to the current context, are something else. I think one key difference is that the unknowns are within the actual code that you're writing, not in places where you explicitly delegate to some other code to do something. The other key difference is that they're highly irregular: they apply sometimes and don't apply other times, which makes them hard to predict. The same syntax means one thing in one situation, and something different (or "nothing", when it doesn't compile) in another situation.
I guess I assumed there was some rustc compiler pass that would match certain patterns where stuff (like dereferences or lifetime parameters) has been omitted, and transform it into code where those things have been inferred and explicitly inserted. In which case it would theoretically be easy to skip that pass and turn those omissions into compiler errors.
Maybe it's my Ruby background, but I don't find Rust very implicit at all, haha. We all have our own things :)
A list of design patterns for a language amounts to a list of weaknesses in the language or in its library.
If the pattern could be captured in a library, that pattern would just be using the library. If the core language provided the feature, the pattern would just amount to using the feature.
Generally it is better to improve the language to the point where the feature can be added as a library component, but sometimes that is too hard. Thus, for most languages nowadays a "dictionary" type is provided in the core language, because a useful hash table library cannot be written with language primitives. In C, hash tables are open-coded again in each place where one is needed, because the language provides neither the feature, nor facilities sufficient to capture it in a library. Rust is powerful and expressive enough that hash tables are library components.
Conversely, pattern matching and coroutines are built into Rust. It should never be forgotten that (1) this was because the language was not expressive enough to capture the features satisfactorily in a library; and that (2) it would be better if, someday, the core feature became unnecessary because the language became expressive enough provide it as a library.
One reason it is better for features to be provided in a library is that another library can implement a variation on the feature that might not be as widely useful as the core version, but is better tuned to a less-common but still important use.
Another is that users can invent whole new features by combining powerful primitives that the language designers would not have time, or possibly inclination, to implement themselves.
Thus, in a certain sense, all patterns are anti-patterns.
Is there anything that can be achieved by using the visitor pattern [0], that cannot be done by using pattern matching? I have only used the visitor pattern in languages that do not have pattern matching as a language feature (e.g. Java before it got a Scala-like `switch` construct [1]).
Edit: one limitation of pattern matching is, that all values need a common supertype (e.g. be variants of the same enum in Rust, if we see each variant as a type and the enum as the common supertype. There is an RFC [2] to make enum variants accessible as types), while the visitor pattern could be implemented for any set of independent types. On the other hand, you then cannot have a typed collection/container that contains values of these types, so you'd need some common trait like `Visitable` so you could accept an `Vec<dyn Visitable>`.
[0]: https://rust-unofficial.github.io/patterns/patterns/visitor....
I find that this kind of pattern can be useful if you have a default implementation for each visit_foo method.
For example, suppose that you want to create a traversal that walks through the entire Ast and does something special on just the Name nodes. And another traversal that does something special on just the integer literal nodes. One way to do this is to create a default traversal that walks through the entire tree without doing anything and then create a "subclass" that overrides just the visit_name and another that overrides just the visit_expr method.
One place that I've seen this in the wild is the Ocaml compiler:
* https://github.com/ocaml/ocaml/blob/trunk/parsing/ast_iterat... * https://github.com/ocaml/ocaml/blob/trunk/parsing/ast_mapper...
Or create an iterator and use Iterator::filter, which is probably a bit more idiomatic.
In many languages, creating a pull-based iterator for a tree-shaped data-structure can be tricky. It might require turning the recursion inside out to use an explicit stack... Do you know how well Rust can handle this? I'm not familiar with the details.
In any case, while an iterator might help in the "ast_iterator" example, in the "ast_mapper" example I am not sure if there is a way around it.
The Visitor pattern is great if you want to process a data structure as "stream" without actually instantiating it, in the same way you can read a file line-by-line instead of loading it completely into memory.
For example, serde uses the visitor pattern to encode its intermediate representation. If it used pattern matching instead of the visitor pattern, it would have to instantiate its intermediate representation as an enum, which would add unnecessary overhead.
I guess you are talking about this `Visitor` trait [0]. I have only used serde in combination with the derive macros so please enlighten me, but each `visit_* ` function of the `Visitor` trait already takes a typed value (`bool`, integer types, ...) so these already have too be parsed (instantiated?) from the input string/bytes. Couldn't you have an `SerdeTypes` enum that implements `From` for each `visit_* ` type to remove some boilerplate and then use pattern matching? Could you elaborate where there would be overhead?
Don't get me wrong. I'm sure, the serde developers know better than me and have good reasons to implement it the way they did, but I'd like to understand the rational behind the decision.
Edit: formatting
[0]: https://github.com/serde-rs/serde/blob/master/serde/src/de/m...
In section 2.10, "Privacy for extensibility", are there any pros and cons to this approach over using the #[non_exhaustive] attribute? The latter works on both enums and structs, and doesn't require extra fields to be included.
https://doc.rust-lang.org/reference/attributes/type_system.h...
Using private fields you can more precisely control the “private scope”. #[non_exhaustive] is “crate scoped”, it does not apply limits for use in the same crate.
Private fields are by default module scoped, and can be tweaked. So you can limit the use even in the same crate.
It looks like the book is outdated in this respect
Once you get past writing a language idiomatically, is a list of design patterns a good thing? It is an obvious negative for code readability because it reduces the number of people who can clearly understand your code from Rust users to Rust users who also memorize design patterns.
When are design patterns useful? And how are they useful?
Design patterns are intended to be a description of patterns found in the wild — that is, it’s descriptive, not prescriptive.
Once distilled to their essence, and documented, and named, it becomes possible to efficiently talk about it and reference it.
They are natural things, to be found in wild codebases, that are documented and named here. That they get used and abused, or people come up with terrible names (ClassFactoryFactory) is more a lack of taste than anything — but knowing that such patterns exist and their utility is a requirement of any tradesman seeking to move above novice. The name isn’t as much needed, but it makes it trivial to google.
But again, these are patterns found in the wild — it intends to document problems & solutions that programmers have encountered, and it’s a pattern because it comes up often enough — which implies that you will likely encounter similar problems and (perhaps discovered on your own) implement similar solutions.
Also an addendum — design patterns originate from Christopher Alexander, who was discussing the same fundamental idea but applied to natural architecture (that is, not bounded by strict regulations, laws, complex requirements, etc) in various populations; he noted that communities tend towards certain patterns and trends that work, and apply them repeatedly. But the key difference between Alexander’s pattern architecture and say, a McMansion, is the pattern only acts as a start; depending on its context (including other patterns used) the pattern is modified to fit more appropriately. And everything else is modified in reaction to that.
So the movement of a door changes how the window should be placed, which changes the best position of the sofa, which changes the ideal spot for the door, and so on. So it’s a feedback loop, approaching some optimal state where everything is in harmony with everything else (a state he calls “beautiful”, an inherent property all people recognize even if they cannot produce it themselves) — but because people live in the home, and use it and modify it and change through generations, that harmony is a moving goalpost, always sought but only possible to achieve once abandoned.
The analogy to software development is easy to make, but the key point here is that in usage, patterns are not the end but rather the beginning of a design — they can and should be modified to fit, until it is beautiful
This is a brilliant distinction, that design pattern are meant to be descriptive rather than prescriptive. Because the problems I tend to see in the wild are when developers treat design patterns as prescriptive.
Design patterns are not necessarily less understandable, much the contrary in fact. They basically encode known good ways to do something, so it should be okay even for beginners who haven't learnt the patterns yet.
What's more or less understandable is a bit subjective. There are some things that are obviously less complex than others. Some things that are obviously more complex than others. And then a lot of gray area.
In my experience there is at least a faction of developers, myself among them that have a disdain for "design patterns thinking." Which I would describe as: spending a lot of focus learning various design patterns, then while coding actively looking for places where those patterns could be put to use.
In my opinion this is an anti-pattern similar to overuse of abstraction in simple cases before an abstraction adds to the understanding of the code itself, rather than makes the code more complex.
I've seen lists of common design patterns dozens of times, and occasionally recognize several of them as useful examples of things I've actually done in the past or my colleagues have done in the past. But it seems to me "learning design patterns" as an end is encouraging the destructive side of design patterns where you learn something and eagerly look for a use for it.
Is that really any different from leaning a language, tool, framework, library, service, etc.? You learned the new thing, you're excited to try to out, and you should still be judicious in analyzing whether this is or isn't the place for it. And some developers just want to play with the new shiny.
None of that is an argument against learning new things. Just keep adding tools to your toolbox, so you don't end up with every problem looking like a nail.
That's an interesting comparison. I can understand the argument that design patterns are just like learning another service or tool or library. But aggressive use of design patterns is taking a language we already agree on, say Rust or Javascript, and adding a dialect to the language that can easily divide the existing commonality we have within idiomatic use of the language. Design patterns just strike me as incredibly non-utilitarian, divisive and non-additive in a field that already struggles to find agreement.
I wouldn't say adding a service, framework or external library to your stack is as divisive for code readability as being a big fan of utilizing design patterns.
I'm a bad man, and I have sinful [ooo] thoughts, but...
Is-a inheritance is extremely useful for creating extensible components. "It may be wrong, but it's much too strong."
In rust, how can you make a component that is just like another component, but ever so slightly tweaked without copying the entire external API of that other component? I understand I can wrapper with has-a relationship, intercept the correct API, and then pass through the rest of the entire interface, but how can I avoid copying the entire interface of the object when I only want to tweak something tiny?
With a car, I can swap out the engine with another, I just have to make sure the external interface is the same.
It may be a "bad thing", but it is extremely useful for the scenario where I say, "I want a Chevy smallblock, but I want to only tweak metal alloy on the interior piston."
class MyBlock(ChevySmallBlock):
def get_interior_alloy(self):
return metals.Unobtanium
How can you do this with rust? I know that is-a inheritance is sinful, but show me the better way! I truly want to know it, and I've been trying to find a pattern for this.
Having documents like this is awesome - thanks for building it.
Is there somewhere I can PR? One example - the `new` constructor takes no arguments, so at minimum there should be a note about the (mentioned-next) Default pattern.
Also, I don't think Default is most useful for abstracting construction (I think closures are better for this), they're really just to make construction easier imo ie:
Foo {
prop: override_default,
...Default::default(),
}
Some idiomatic suggestions seemed to replace lacking language features, which is a smell to me; I think it's best to use the language as designed instead of creating something weird and maybe even unstable like a "finally" block using the Drop trait for a dummy struct.
Design patterns were weird too. The builder pattern to hide complex initialization? Not a fan; maybe it would be best to remove the complexity instead? Rust is a functional language, so OOP patterns seem like an anti-pattern.
kinda related topic: im debating if I should learn Rust.
I'm attracted to Go, but at the end of the day I can do everything that Go can with Nodejs minus the speed part.
Rust has just way to many new concepts to simply learn it.
Why did you learn Rust over Go. How are you using it?
Patterns are nice, but dangerous to follow blindly in an immature and fast growing industry operating like a pop culture. As an example the book suggest to use YAGNI, a pattern/principle Jim Coplien suggest we fight. I’m inclined to agree. Plan ahead when you can. And btw pull in decisions to get early feedback in stead of delaying to the last responsible moment. Just some thoughts, but I like that patterns are collected for different contexts. It allows for discussion and learning.