Cello – High Level C
libcello.orgThe strategy in the GC for determining the stack top for hunting GC roots will not work on all architectures.
On aaarch-64, the address of a local dummy variable may be above a register save area in the stack frame, and thus the scan will miss some GC roots.
In TXR Lisp, I used to use a hacked constant on aarch64: STACK_TOP_EXTRA_WORDS. It wasn't large enough to straddle the area, and so operation on aarch64 was unreliable.
http://www.kylheku.com/cgit/txr/commit/?id=3aa731546c4691fac...
A good stack-top-getting trick occurred to me: call alloca for a small amount of memory and use that address. It has to be below everything; alloca cannot start allocating above some register save area in the frame, because then it would collide with it; alloca has not know the real stack top and work from there.
Since we need to scan registers, we use alloca for the size of the register file (e.g. setjmp jmp_buf), and put that there: kill two birds with one stone.
http://www.kylheku.com/cgit/txr/commit/?id=7d5f0b7e3613f8e8b...
> On aaarch-64, the address of a local dummy variable may be above a register save area in the stack frame
Then use two stack frames! Every problem can be solved by adding an additional level of indirection. ;-)
In this case it won't help, because:
0. We are already in a frame that doesn't take any arguments of the "val" object type; how come that's not good enough?
1. The current stack frame is entered with a bunch of callee-saved registers, some of which contain GC roots.
2. The current stack frame's code saves some of them: those ones that it clobbers locally. It leaves others in their original registers.
3. Thus, if a another stack frame is called, there are still some callee-saved registers, probably containing GC roots, and some of these will go into the area below the locals.
4. You might think that if the save all the necessary registers ourselves into the stack and then make another stack frame, we would be okay. But in fact, no. Because by the time we save registers, the compiler generated function entry has already executed and saved some of those registers into the below-locals save area and clobbered them for its own use! So our snapshot possibly misses GC roots. The compiler generated code always has "first dibs" at the incoming registers, to push them into the below-locals save area, thus kicking the GC roots farther up the stack.
I use argv[0] as stack head.
Seen this posted here years ago. Now as then, my gut feeling is that anyone doing serious work in C would never use something like this-- I feel like the fine grained low level control is exactly the reason they chose C in the first place, and they're not looking to escape from it or they would just choose a different language.
"Why does this exist? I made Cello as a fun experiment to see what C looks like hacked to its limits. As well as being a powerful library and toolkit, it should be interesting to those who want to explore what is possible in C."
"Can it be used in Production? It might be better to try Cello out on a hobby project first. Cello does aim to be production ready, but because it is a hack it has its fair share of oddities"
It sounds like they don't intend for it to be anything other than an interesting case study.
I found their FAQ to be refreshingly honest. This is in no way suited for large projects or where multiple people will be contributing. A case study sounds like a good description.
And the authors seem quite okay with that.
Not only that, but last I looked into this library's code there was a lot of undefined behavior and general sloppiness that goes against good C practices, eg. ignoring errors, casting all types to void * literally all the time or treating char VLAs as structs without regard for alignment.
My sympathy and respect to the author, but they did not appear learn C well before trying to "fix" it. It is kind of irresponsible, I think, to say it "aims to be production ready" and write it up as something other C neophytes may be interested in with some of these issues.
> Not only that, but last I looked into this library's code there was a lot of undefined behavior
Neither GCC's nor Clang's sanitisers pick up any undefined behaviour - and it's been like that for at least the last few years I've looked at it.
As to ignoring errors, and ignoring alignment, I don't think I've ever seen anything like that in the project. I have seen several pull requests delayed so that they will.
Overall, for what it's doing, this is one of the cleaner codebases I've dealt with.
You can't take char[sizeof(foo)] on the stack and cast it to a foo*. malloc implementations for example are cafeul to align the buffer they give you. Windows, for example, aligns on 8 bytes. Probably popular alloca implementations do this too. For example, googling for "alloca alignment" finds some documentation: "... returns a void pointer to the allocated space, which is guaranteed to be suitably aligned for storage of any type of object". I have seen things break in the real world when these expectations are violated. The cello tree does this in a more few places since I last looked, eg. skimming it again I see the same pattern in the Windows stack trace code. It will probably work there but it's a coincidence, and not guaranteed by the standard AFAIK.#define alloc_stack(T) ((struct T*)header_init( \ (char[sizeof(struct Header) + sizeof(struct T)]){0}, T, AllocStack))It looks like they got rid of some undefined things I saw when I looked in 2015. eg. they used to think you can do arithmetic on void pointers, which I think even gcc -Wall would flag for you. [Edit: Trying it out it seems I am wrong, on gcc and clang you need -pedantic to get that warning.]
I'm really surprised that you both came up with opposing results... What tests were done indicating undefined behavior?
The fine-grained control is just a statement away. When you're not using the $<ident>(...) macros, it's just normal C.
It's pretty much exactly the same like the low-level-C techniques being instantly available for you in C++ or Objective C.
The biggest advantage for me in using C is that the syntax never changes.
When I go look at a C code, I don't have to look up some annotation or new syntax that got introduced behind some abstraction that gets compiled in automatically after the library is pulled from the internet by whatever build system the project uses.
> I feel like the fine grained low level control is exactly the reason they chose C in the first place
That's not the only reason, there is also simplicity, static typing and performance. If you favor the later two for whatever reason it can be used in places where you'd normal write a python/shell script or small program without too much extra effort (see https://github.com/RhysU/c99sh or suckless tools). Complexity is where Cello seems to fall down though, it seems like it introduces much more complexity than just using plain C with a decent "standard" library like glib.
> That's not the only reason, there is also simplicity, static typing and performance.
I think the only meaningful benefit here is performance.
Simplicity is at best determined by the nature of the problem and at worst a completely subjective opinion for C.
Similarly, static typing is not usually something the programmer should care about that much. You need to know which paradigm your language uses, of course, but beyond that it does not matter all that much. IMX, you're more concerned with type safety, and C is not fully type safe like, say, Java is.
Yes it's somewhat subjective, but other languages for the use case usually contain a lot more abstractions. They're more complex languages but they might enable less complex solutions to the problem at hand.
> IMX, you're more concerned with type safety, and C is not fully type safe like, say, Java is.
I'm concerned with finding errors, preferably at compile time. There have been very few times being fully type safe runtime like the jvm have done much for me compared to the java compiler. If I wanted more down that road then rust or ada would probably be better.
Measure the complexity of a language as the number of axioms required to define it. In reality, the C standard is incredibly brief, defining the core language in ~150 pages. This makes C simpler than languages like C++ and Java and the definition is not at all subjective and is useful.
You could also just pick something like Nim (https://nim-lang.org) if you wanted to hit the intersection of low effort, script-style programming with the addition of types and performance.
Did Nim compile times improve? I wrote a production project in it which was about 50k loc and I spent most time waiting for compilation.
I just checked out and built the Nim compiler + stdlib itself.
real: 161.26s (including the GCC bootstrap stage).
cloc reports 253500 lines of Nim in the project.
This was on a Ryzen 7 3700x @ 3.6GHz.
Re-running it immediately after (does not build the GCC bootstrap stage)
real 22.55s
This is just an amusing aside: C is the lowest level of an actual cello. ;-)
I would have never gotten the wordplay at hand. Thanks ! Very clever.
Isn't this sort of what Glib is getting at? Bringing higher level data structures and capabilities (extendable arrays, hash tables, heaps, etc.) into C.
https://developer.gnome.org/glib/stable/glib-data-types.html
You don't get Cello's macros, and it uses reference counting instead of invisible garbage collection, but you get a lot of fun high-level capabilities.
And if you really want to go high-level wouldn't Vala be the language?
Vala is actually a really pleasant way to interact with the various bits of the Gnome environment.
It's a shame, in my opinion, that it never really received widespread love.
I think all serious work on it stopped about a decade ago.
Isn't it still used by gnome, elementary, etc?
I suppose it's very gnome specific.
It's used but some of the core gnome developers have advocated for people to stop using it.
What I can find with a quick search this morning: [1] [2]
Then again, there are also several people who still say it's worth using: [3] [4]
So maybe it's just Emmanuele Bassi advocating against it. If you look at the development pace, though, it hasn't really seen any significant development for quite a long time.
[1] https://twitter.com/ebassi/status/827482509982195712
[2] https://www.bassi.io/articles/2017/02/13/on-vala/
[3] https://blogs.gnome.org/mcatanzaro/2017/02/19/on-problems-wi...
[4] https://blogs.gnome.org/despinosa/2017/02/14/vala-is-not-a-p...
There was a snippet I saw a while ago where someone made C look like Java for a joke, using macros. I wish I could find it to share here, it's great.
The original bourne shell source was using mac.h ( https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd... ), a bunch of macros, according to wikipedia, "to give the C source code an ALGOL 68 flavor."
Here's the original source tree, if you're interested: https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd...
Here it is: https://stackoverflow.com/a/653028/981408
Could be worse:
I'm left wondering about the iteration example that is also quoted in the home page:
https://github.com/orangeduck/Cello/blob/master/examples/ite...
Okay, so the vector is garbage-collectable once the function terminates ... but it has references to stack-allocated integers i0, i1 and i2. That leaves me wondering: won't the GC walk these and trample on stack memory that has been deallocated/reused.
(Maybe those integer values have a tag right in the val pointer that gets the GC to avoid dereferencing them.)
Just my opinion, don't mean to be inflammatory, but if the user has to know and manually manage stack vs heap objects, then I wouldn't call it "High Level" language.
Then just allocate everything on the stack and use the optional garbage collector: http://libcello.org/learn/garbage-collection
Allocating on the stack doesn't need GC.
No, but it makes it easier (more “high level”) when you don’t need to explicitly deallocate things, ever.
That raises a question, would you call C# a "High Level" language?
In C# it's more relevant to understand semantics (ref/value type) than allocation details (unless you actually care about low level details for performance/interop)
I'd say the fact that structs are stack-allocated, and you can slip right through years of development without even knowing that fact... yeah, it's pretty high-level. C# doesn't have `malloc`. .Net apps are managed, so all of those low-level things one has to/gets to do are abstracted away.
Didn't C++ start out as a set of hacks on C? Fairly sure it was originally a preprocess stage ahead of an ordinary c compiler.
Raises the question of how usefully far you can make C twist using macros / preprocessor.
Candidates like Forth or Lisp seem possible. A few weekends at most. Might need to take a few liberties.
Python... Perhaps if you implement a less dynamic subset? Duck typing may trip you up. To what extent?
What about Elixir?
I wrote my own Lisp. The virtual machine, which is a stack machine, is written in C. The interpreter, which runs until the system compiles itself, is written in C, but makes heavy use of C macros. The rest of the code, including the compiler, is in Lisp.
Code in the interpreter is directly converted to byte code, e.g. the macro Car generates the virtual machine instruction Car, rather than executing the code for car. The alternative would have been to generate byte code by hand, would have been error-prone. Here's the code for cons and let:
It is actually C, with heavy use of macros. But it can be read as Reverse Polish Lisp. It can also be thought of as a Lispy Forth.Define("cons", 2) Local1 Local2 Cons Ret Termin DefineF("let") Local1 Car Local2 /* initialize new env */ Prog(1) Ret Params(2) Until Local1 Null Do Local1 Caar /* var */ Local1 Cadar Free12 Call("eval") /* val in old env */ Local2 /* env */ ACons SetLocal2 Pop /* update new env */ PopLocal1 Od Free11 Cdr Local2 Call("progn") /* use new env */ Ret TerminInteresting approach. maybe develop it further into a JIT arrangement or a library builder.
Brainf*ck is another classic.
Nim transpiles to C. It's very cool since there are C compilers for almost every processor.
Nim is very good. I'm using it as a glue language right now and it works out well.
Thanks. I didn't know about this library. Interesting, but perhaps I am missing something... about stack "allocation":
var i0 = $(Int, 5);
int i0[5];
Is it using macros to achieve that?
For a different take for "better C", try Zig language, it looks pretty cool.
This gets reposted every couple years and it's still bad for all of the same reasons.
It's not higher level than C in the sense that you get any additional safety guarantees or real beneficial abstractions. If you are fine without the safety but want abstractions, use C++. If you want safety and abstractions, use Rust or Go or Zig. If you really want a transpile-to-C language, you've got Nim.
Finally, it's not good at being C; everything it does is poor practice and should be quickly recognized as such by experienced C developers, IMO. It's got no developer community and no real-world production consumers.
What type of sorcery is this?
The good one, dark, very dark.
Yeah, the foreach macro is particularly interesting to me in that respect. Quite a neat bit of sorcery.
This is great. It made me smile.
Why not just C++?
Speaking for myself, I’ve never found C++’s complexity appealing, and it only seems to be getting worse over the last 20 years.
As the creator says, it’s not for production use. If I’m doing a side project, I’d give this a serious look.
> Speaking for myself, I’ve never found C++’s complexity appealing, and it only seems to be getting worse over the last 20 years.
True, but that's why we've got Rust these days. (Rust is actually more optimized than C, e.g. it will automatically reshuffle your structs to get rid of excess padding, and reference accesses will automatically take advantage of compiler-checked 'restrict' constraints, thus equalizing performance with e.g. FORTRAN.)
How is Rust an answer to not liking C++‘a complexity. With the complexities inherent in the borrow semantics (with the box ref cell stuff that comes with it) and the complexities inherent in the type system and trait system, Rudy seems to be at least as complex as C++. In fact this point gets tossed out a lot, Rust is not a C replacement for C developers, it is at best a C++ replacement for developers who want something more complex than C. Not commenting on the quality of the complexity, just that it seems odd that you would say Rust is an answer to disliking complexity.
But note what those complexities buy you: they bring restrictions and limitations that allow you reason _more_ about the code.
This is in stark contrast to some other complex features that allow more stuff to happen with less code.
Rust is indeed, complex in the sense that it's features are non-trivial, but I find it less complex than C++ in the sense that it has 1) less surface syntax (because of lack of historical baggage) 2) more cohesive, principled feature set (again, hindsight is 20/20) 3) it's inherently more limiting, which helps reading, understanding and reasoning about code.
I have to say, I'm really exhausted with the way every single C/C++/Go discussion on HN ends up having someone chime in to say, "Why not just use ~Rust~?". I'm a huge fan of Rust, as I'm sure many of us are, but it isn't some cure-all. It can't just be dropped into any arbitrary use-case where performance happens to be somewhere on the radar, and magically surpass all other options.
Rust is a compelling alternative to C++ for large scale systems software (browsers, video games, etc) but, as of the last time I checked (~2 months ago) it doesn’t scale down as well as C does, or to as many platforms as C does. A lot of this is that C has an unfair advantage from being well established, but an advantage is an advantage .
> Rust is a compelling alternative to C++ for [...] video games
I think it is unlikely to see rust gain much traction in the game dev world. Everything is currently done in C++, and there is low incentives to move to something "safer" or "more secure", because that's not seen as relevant properties by game developers.
What matters for game dev is mostly (not listed in a specific order):
- raw performances
- low latency
- as low as an overhead as possible when dealing with GPUs
- control over memory management
In that context the borrow checker can be an unnecessary constraint, and the safety concern isn't really something that relevant. In the other hand, the rust package manager is really something that is missing in the C++ world, and would be awesome to have for game devs.
I'd say (as a former RTOS lead) that it scales down as well as C, there just aren't as many backends yet. XTensa, AVR, and 8051 are the notable ones missing.
> and reference accesses will automatically take advantage of compiler-checked 'restrict' constraints,
I think that's temporarily turned off because emitting noalias attributes so much exposed some llvm bug. See: https://github.com/rust-lang/rust/issues/54878
In the long run though, yeah that is an inherent optimizability advantage of Rust vs. C.
C has restrict
also, gcc and llvm can infer restrict much of the time if you don't manually use it
and I fail to see how having explicit control over struct layout is ever a bad thing
'just use rust' is like some 2010s advice everyone's moved on to ATS now
> e.g. it will automatically reshuffle your structs to get rid of excess padding
This is just more complexity as well. It introduces surprising behavior that can burn you when creating interfaces or serializing and forces the developer to know that the compiler will be doing such magic.
> forces the developer to know that the compiler will be doing such magic.
Do you regularly order your stack variables? :D
99.99% of cases there are no expectations about details of a `struct` and having to think about it all the time is a PITA. That's a good example how silly defaults in C/C++ are. Just because I might need manual ordering once in a while, does not mean I want to be bothered by it all the time.
> Rust is actually more optimized than C....
I think what you mean is that current Rust _implementations_ optimize better than current C _implementations_. There's no reason a C99 compiler can't do those optimizations.
>There's no reason a C99 compiler can't do those optimizations.
No, there actually is, the C standard says structs have to be laid out in memory in the same order they're written and C has much weaker aliasing rules.
C99 has the restrict keyword, which guarantees that the pointer won't be aliased. As far as reordering struct members, there's no reason why an implementation can't provide that as an optional optimization you must explicitly turn on. Providing such an optimization and corresponding compiler flag would not disqualify it from being a conforming implementation.
There have been compilers that did this optimization.
179.art, one of the SPEC2000 benchmarks, has some poorly laid out structs. Sun Microsystems was the first company to introduce targeted optimizations for this benchmark.
GCC also had an optimization pass[0] for this. It may have been removed.
[0] https://www.research.ibm.com/haifa/dept/_svt/papers/golovane...
Sure, but Cello's complexity turns me off for the same reasons as C++'s complexity. The question isn't "why not use C++?" in isolation, but instead, "why would you use Cello over C++?"
I agree wholeheartedly with this. You can write high level code in C++, but the cost is a terribly complex language and standard library.
In contrast, the C language is fairly simple, except for a few twisty passages (pointer declaration syntax, anyone?). The standard library does leave something to be desired, but that's not that big of a deal given all the third party libraries out there.
It would be interesting to compare the same program written in straight C vs C++ vs Cello, both for developer experience issues (clarity, simplicity, etc.) and performance. I'll have to have a look at http://libcello.org/learn/benchmarks but this does really seem like something I'd like to use on a personal project someday.
You can't compare C++ and C in complexity in general. Any C program that does the same things as C++ does is bound to be just as complex, and even more so since C lacks many of the C++'s niceties.
This isn't a library, it's a sort-of-a-modification of C, it seems.
Well, for a non-C language with high-level abstractions that lets me use C code relatively seamlessly - I'm content with C++. Many complain about its complexity, but you can actually avoid a lot of that complexity in _your_ code using facilities with complex implementation but relatively easy use.
Looks interesting, but I can't help but notice they're distributing their source tarball via that site, and it doesn't have HTTPS. I don't understand why projects don't have SSL certs these days, especially considering Let's Encrypt has automated it all and made it free.