Would have liked to see the machine code generated by the example function, and a deeper dive mapping compiler choices to the unintuitive results.
The article (indeed the point of it) abstracts that all away behind "undefined behavior" and a mental model sitting between your code and its resulting executable. Which is fine, but it leaves a loose end which fails to sate my curiosity.
It depends on whether you generate using rustc 1.28 or rustc 1.36, and whether you're compiling with or without optimzations. This does not crash in unoptimized rust (either 1.36 or 1.28) but it will crash in optimized rust 1.36.
I think though (as your question indicates); that the author misses the point of why people care about "What the hardware does". At the end of the day, assembly code is going to execute, and that assembly code is going to (despite the authors protestations to the contrary) have well defined memory of one value or another. The moment you start saying "Rust has a third value of uninitialized" the question comes up "How is that abstraction enforced by the hardware?" This is valuable information for understanding how the language works.
From the authors discussion, I was expecting some sort of sentinel value being checked; however, instead, the uninitialized memory access is detected by the compiler and it panics uniformly regardless of the actual memory state.
The idea that one should only worry about the abstract virtual machine of rust seems like an encouragement of magical thinking. "Don't worry about how any of this works, the compiler will just make it happen". This will not go over well with many people who are curious about learning Rust.
However, if the author is arguing "Don't let the behavior of a naive enforcement of a Rust safety construct dictate how the optimized version should work" this seems like a more interesting position; but it's not clear that is the argument being made here.
> However, if the author is arguing "Don't let the behavior of a naive enforcement of a Rust safety construct dictate how the optimized version should work" this seems like a more interesting position; but it's not clear that is the argument being made here.
This is exactly the point the author is arguing. The focus of all their work on UB is to make sure safe Rust can do all the optimizations we would like, by careful design of the abstract machine.
The immediately visible outcome of this work is a set of rules for what you can do in unsafe Rust, which taken together amount to this weird-looking abstract machine with its extra "uninitialized" values- something that can be implemented efficiently on real hardware assuming no UB.
The point here is that this abstract machine is a better, simpler, easier way to convince yourself whether or not an unsafe Rust program is well-defined, and that "what the hardware does" is too many layers removed to be a good tool here. You can think about "what the hardware does" another time, for other purposes, but trying to do so in this context is actively unhelpful.
shrug. There's a difference between saying "This is a useful abstraction" and saying "understanding what assembly is generated is irrelevant in understanding what your program does so I will try to end all discussions where it comes up".
I mean, the latter seems quite a bit more extreme, and is what the author explicitly is calling for.
There are most certainly guarantees on the generated assembly. The assembly has to enforce the abstract machine. I want to know how it does that. It can change, that's fine, it can be improved, it can be made worse, but the idea that the rust program doesn't run on physical hardware, as explicitly stated in the article, is pure bullshit.
> The assembly has to enforce the abstract machine.
The assembly has to implement the abstract machine only if your program has no UB. The assembly never has to check if memory is "initialized" or not even though that distinction is real on the abstract machine, because if the difference would matter, your program would have UB.
To determine if your program has UB, looking at the assembly is useless. The only way is to consider the abstract machine.
Personally, I like to bind the compiler to implementing the abstract machine in all cases, and in the face of undefined behavior the abstract machine has no requirements on its behavior. Of course, this is just a semantic quibble: in practice, the results are the same ;)
The standard will not specify anything, so what the compiler outputs is gibberish. You are literally looking at a sequence of bytes on which no constraints whatsoever are imposed. LLVM could have compiled my UB program do `0xDEADBEEF` (which I assume is not valid x86 but I do not know) and there would be no compiler bug. Looking at `0xDEADBEEF` here is not useful.
Trying to interpret the assembly of a UB program is like trying to interpret the noise of a radio when there is no station on the given frequency. It has more to do with fortune telling than anything else. There is no signal in there, or at least not enough of it to be useful.
There is no standard mapping between "your C code" and "what your computer will do" if your code has undefined behavior. Your compiler will produce some assembly, which you cannot rely on, and that will be "what your hardware does". If that's what you're trying to say I think we agree.
> The assembly has to enforce the abstract machine.
Yes, but this only really means anything in the absence of undefined behavior. The compiler's job is generate assembly that produces the results that running the code in the abstract machine would, but the issue is that undefined behavior allows the abstract machine to do arbitrary things, so the compiler is free to generate whatever it likes in this case.
Um... this seems to be a stronger case then to ask what the hardware does. If the compiler can generate arbritray code, then the only recourse to understand what the resulting binary actually does is to look at the actual generated assembly. Understanding what the compiler was trying (and yes, this will change based on which compiler version you used) to do would presumably be helpful in that process. Sure, don't design around this behavior, but if you find yourself deploying an executable with undefined behavior, and you need to figure out the scope of the problem; this seems useful.
I don't get this hostility to understanding the tools you're using.
> the only recourse to understand what the resulting binary actually does is to look at the actual generated assembly
Pretty much, yes.
> I don't get this hostility to understanding the tools you're using.
Don't take me the wrong way: I'm interested in how compilers work, but I accept the concession that I can only really understand their output when my program is free of undefined behavior. It would be nice to have the compiler try its best in the cases where I am violating the rules of the programming language, and often it will do so, but in general I cannot expect this and trying to do so will require making some sort of tradeoff with regards to performance or language power.
> At the end of the day, assembly code is going to execute, and that assembly code is going to (despite the authors protestations to the contrary) have well defined memory of one value or another.
The point is that you may not get the assembly you assume you’re going to get. Like the example shows, it may never even generate something that accesses the value at all.
Guess there's some bigger context that I'm missing here. I wouldn't have expected saying "Understanding how your compiler enforces it's abstract machine is beneficial" would be a controversial position to take.
No, that's when it's least interesting to ask what the hardware does. A program with UB will not reliably compile to any particular hardware behavior, so changing unrelated parts of the program or upgrading your compiler can change which hardware behavior you get.
The actual hardware behavior is useful for other purposes, like understanding why the abstract machine is the way it is, or understanding and improving the performance of well-defined programs, but it is not useful at all once you have UB.
`main` just calls `panic` immediately. The functions returns undef. The reason: `x > 150` is undefined, `undef || ...` is undefined thus the first if-statement with side-effects may be interpret undefined as true.
I wonder why the optimiser didn't choose false and let the assert pass?
> I wonder why the optimiser didn't choose false and let the assert pass?
I don't know exactly, but it's kind of a moot point. I would have negated the statements until I found a way to make it return what I want it to.
The point is that the compiler picks some result, and it does so "locally", so when it picks results for multiple comparisons it makes no attempt to check that these results are all "consistent" and can even arise for a single value. The result that we can observe is that the value is "unstable".
To "fix" this (assuming we wanted to specify that unstable values are not allowed in C/C++/Rust), the compiler would have to keep track of which constant foldings it already did for some uninitialized value, and make sure it remains consistent with that. That's a hard problem and likely undecidable in general. Allowing unstable values frees the optimizer from this burden, letting it optimize more code better.
That's what I expected. And I that is probably due to an optimisation rather than uninitialised variables (can anyone confirm that?).
I am sceptical the author really knows much as some of their statements seem blatantly wrong or just nonsense:
"So, one time we 'look' at x it can be at least 150, and then when we look at it again it is less than 120, even though x did not change."
Is talking about "x < 150 || x > 120", but gets it the wrong way around, ouch!
"Memory remembers if you initialized it. The x that is passed to always_return_true is not the 8-bit representation of some number, it is an uninitialized byte."
Benefit of doubt could be extremely poor metaphors, or referencing the wrong code?
Also stating C is not low-level is a conceited attempt to redefine the word.
> "So, one time we 'look' at x it can be at least 150, and then when we look at it again it is less than 120, even though x did not change."
> Is talking about "x < 150 || x > 120", but gets it the wrong way around, ouch!
It's not the wrong way around. The assertion failure being discussed happens when the function returns false, which happens when both sides of the || are false. Technically he should have said "less than or equal to 120" rather than just "less than", but otherwise it's accurate.
> Technically he should have said "less than or equal to 120" rather than just "less than", but otherwise it's accurate.
I suspect it may be because the author is German. In French at least «inférieur» means “less or equal than” and you need to say «strictement inférieur» to say “less than”, and I wouldn't be surprised if it were the same in German.
Fair call. But the real point is that the function gets compiled to:
xor eax, eax
ret
i.e. the input variable is not compared with 150 or 120. His intuition about his code is wrong - it has been compiled out (unless I am missing something about choosing a different optimisation level, or declaring things volatile, etc).
You are making exactly the mistake the post is all about. :)
Only UB-free programs can be made sense of by looking at their assembly. Whether a program has UB is impossible to tell on that level. For that, you need to think in terms of the abstract machine.
I mean, look at the code I wrote! It literally compares `x` with 150 and 120. That's the program I wrote. This program has a "meaning"/"behavior" that is entirely irrelevant of compilers and optimizations, and determined by the langauge specification. How can you argue that it does compare `x`?
Right. When you are talking about the compiler and the abstract machine, perhaps some sentences could be clearer about that (e.g. the sentences I latched onto - which I admit is my fault for skim reading).
Responding to whether C is "low-level" I like this comment: http://lambda-the-ultimate.org/node/5534#comment-95721 And processors have undefined behaviour so should we say assembly is not "low-level"? e.g. "Grep through the ARM architecture reference manual for 'UNPREDICTABLE' (helpfully typeset in all caps), for example…" - pcwalton
The argument for C not being low-level is not via UB, it is via the fact that a lot happens when C gets translated to assembly, and to explain that you need to consider an abstract machine that is many things, but not low-level.
The author tries to ascribe too much meaning to undefined behavior and gets some parts of this wrong, but they are correct in saying that C is not a low-level language in the context that they're using it.
I talked a bit about it here: https://news.ycombinator.com/item?id=20435309. Basically, the compiler has no need to do things like perform reads in the face of undefined behavior: it could output an empty executable if it wished. Maybe your specific version of the compiler does, but that doesn't mean others (or even a future version of yours) will. Trying to figure out what a compiler might do in the face of undefined behavior is generally not a worthwhile exercise.
> Trying to figure out what a compiler might do in the face of undefined behavior is generally not a worthwhile exercise.
That is exactly the point of my post! If you think I disagree with that statement, we seriously miscommunicated somewhere.
The parts you seem to be concerned about are those where I try to explain why the abstract machine is the way it is. Hardware and compiler concerns do come in at that point, and my feeling is just dogmatically giving an abstract machine won't help convince people of its usefulness.
The article (indeed the point of it) abstracts that all away behind "undefined behavior" and a mental model sitting between your code and its resulting executable. Which is fine, but it leaves a loose end which fails to sate my curiosity.