I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC or NASM is really annoying, and it feels like something so foundational should just exist.
I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs.
I wrote it for the purpose of making it easy to write JIT compilers in C++ and it has been adopted even by projects that are written in C (like Erlang). It's also one of the smallest libraries out there and has a great performance.
I understand why people don't want to use projects such as libgccjit or llvm, but asmjit is just super easy to use compared to others.
The reason why it's written in C++ is to make it practical. I understand why some people would want it in C, but honestly I have never seen a nice JIT assembler library written in C - I saw few incomplete assemblers in C before as part of other projects, but it was usually ugly code full of macros, so no proper C API to interface with anyway.
I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer.
I understand, but I don't think you understand me.
asmjit is great if you're writing a fast JIT and want to write C++, or can tolerate a C++ requirement. If you're writing a compiler in any other language, requiring C++ is a burden that may be insurmountable. What I'm lamenting is the lack of a widely used, decent C library for generating x86 and ARM code, which doesn't seem to exist, nor does asmjit fill that gap.
> I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer.
Not everyone cares about sub-millsecond performance for code generation. It's easier to generate GCC syntax ASM and shell out to `asm` or `cc` than to write bindings to asmjit and interface from a different language, and the performance is fine because lowering your AST into an IR that can be used to generate asm either as text or the input to asmjit is a much bigger bottleneck.
Yeah, I usually write compilers in C++ and it seems we have totally different use-cases :)
High performance assembling is literally what asmjit was designed for and that allows it to be used in interesting projects - for example there are multiple databases that use asmjit to JIT compile queries, and low-latency compilation is the key feature here.
BTW if you need a pure C library to encode x86 there is zydis (yeah it used to be a disassembler only, but it has now also an encoder), but it's only for x86.
> I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC or NASM is really annoying, and it feels like something so foundational should just exist.
It does, it's called C :-)
> I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs.
This is only for language developers who aren't writing in C, right? Because if you're writing in C, what would you use this assembler for?
Anything a cross-platform assembler emits is going to be portable, so it's only going to be as performant as C is, so why not just emit C?
Having a C library that emits C might be a nice thing to have - then all the other languages can emit "code" that is cross-platform.
C is not assembler nor is it close to assembler nor is it desirable to have a C compiler in your toolchain to compile something that needs assembler
For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to.
If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs.
> For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to.
Yes, all trivially bypassed or worked-around if you're emitting C. Considering the initial constraint was "cross-platform":
1. Calling convention - what would be the point of a calling convention different to the rest of the system? You wouldn't be able to do system calls, for one. And if you need a different calling convention within the language itself C will let you push elements onto a stack and jump to a different piece of code[1], optionally saving and restoring registers.
2. The notion of a stack and heap - what's the alternative you have in mind when using assembler? Assemblers also have a notion of a stack and non-stack addresses, after all.
3. Arbitrary control flow - you can emit goto's to your hearts content (see [1] below).
4. Threading - this is platform-specific by nature, but if you're emitting C you can emit calls to the platform's thread primitives (CreateThreadEx, pthread_create, etc) the same way you'd have to do if you were using an assembler. Nothing stops you from proving a wrapper for threads that your emitted C code calls (so that it works on multiple platforms).
5. Process-spawning - once again, this is platform-specific, so even in assembler you're going to have to make a call to the platform's CreateProcess/posix_spawn/exec functions. If you're going to call a platform function from assembly, you may as well call it from C.
6. Memory model - I'll give you this point, even though the memory model is not that much different from assembler. If you're emitting C, you're still free to allocate/define a large array and use that memory in whatever way you wish.
> If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs.
Actually, many languages with non-trivial and advanced features, such as Common Lisp, already have implementations that transpile to C. They implement the entire Common Lisp standard, and don't seem to be missing any features even though they are emitting C and not assembler.
Anonymous functions? Done with emitted C.
Closures, continuations? Done with emitted C.
Garbage collection? Same.
What feature do you have in mind that cannot be implemented by emitting C[2], but can be implemented by emitting assembler[3]?
[1] Some constraints on the jumping exist, but emitting GCC-specific C let's you do calculated expressions for the jump target - you don't need to `goto` a constant address.
[2] I initially thought that something like Goroutines or async functions might be a candidate, but off the top of my head, I can think of at least one way to implement async functions and/or go routines by emitting C. There's probably more.
[3] I'm not being contentious, I'd really rather like to know - my own toy language (everyone has one) currently in the phase of "I'm thinking really hard about what it should look like" is going to be emitting C. I couldn't find any feature for the language that can't be done by emitting C, so I really want to know what you have in mind.
Plenty of languages have language-internal calling conventions nothing like C, and usually for good reasons.
Some try to stay close to make passing the boundary easy, some are so different they need an expensive conversion step anyway and then there's little point.
Sure, you can always map your requirement onto the standard calling convention somehow, but if you can't directly call out or in anyway it's a constraint it's totally reasonable to not want to be forced into.
EDIT: Note that there's nothing wrong with emitting C if you don't have any compelling reasons not to. When we don't it's often for reasons such as wanting a simpler toolchain, wanting the environment to be self-hosted etc. that are tradeoffs that sometimes are frankly ideological (take that as you want - I don't mean anything inherently negative with that other than when the authors ideology differs from mine ;) I like that there's choice and that some deviates from the norms ), sometimes based on requirements that doesn't really have to do with the language itself. I do agree with you that you can fit all languages onto C, and most languages neatly onto C.
One example where it has traditionally been painful without compiler extensions, though, is tail call elimination. You can still do it. Other fun times involve closures, but you can do that too (been there, done that, wrote a blog post years ago) even though it quickly gets ugly.
Often you end up wanting to demand a specific set of extensions.
Overall nothing stops you, but at some point it also doesn't necessarily buy you all that much - you still need to do most of the hard bits.
> Sure, you can always map your requirement onto the standard calling convention somehow, but if you can't directly call out or in anyway it's a constraint it's totally reasonable to not want to be forced into.
I get that, but what I don't get is what about C makes it harder to define your own calling convention than defining it in an assembler language.
In the context of this thread, OP wants a portable assembler. Implementing a custom calling convention by emitting any assembly language is bound to be more work than implementing a custom calling convention by emitting C.
Maybe the situation regarding floating point registers when passing floating point values? That's about the only thing I can think of that makes it harder to emit C code that implements a custom calling convention than doing it in assembler, but even for that I can think of workarounds, just so that other high-level constructs can be used.
> I get that, but what I don't get is what about C makes it harder to define your own calling convention than defining it in an assembler language.
You don't get to control which registers are caller or callee saved, or which registers are free to use.
If your desired calling convention fits within those constraints, you can use it via C all you want. If it doesn't, you usually will need to either resort to compiler extensions or give up on using C. Whether it's wise to change that for your given use case can be a worthwhile question to ask, but that is another issue.
Ironically sometimes this choice is about making C interop easier. E.g. I have a prototype Ruby compiler. It can call out to C from a low level "escape". The same exact code is used to generate calls to Ruby methods and to C. But the Ruby code needs the arity of the code that's being called to be passed along. Putting it in a register that's not used for arguments in the standard calling convention means that when calling out to e.g. glibc the C code just won't be able to access those values.
Now, in my case, if I'd been on the fence about using C as the target, then I'd have handled that in a different way instead, so that didn't really influence my choice to emit assembler. But I can't define that calling convention in ISO C.
> Implementing a custom calling convention by emitting any assembly language is bound to be more work than implementing a custom calling convention by emitting C.
Generally, your calling convention simply involves what goes into registers and what goes on the stack in which order. Storing to a register vs. outputting to an argument list only marginally different in effort. In the compiler mentioned above, preparing arguments is literally:
If I was emitting to a C argument list instead, the only thing that'd change is the `@e.save_to_stack` bit. `save_to_stack` is a one-line method. The non-C compatible parts add up to 2-3 more lines in that case. One doing something more unusual might be longer, but not a lot.
There is plenty of complexity of dealing with assembler directly, and you certainly ought to have a reason for doing so over outputting to a higher-level target, but the complexity cost of assembler is not there.
> OP wants a portable assembler
And as they pointed out, C is not a portable assembler. It is sometimes close enough, but it imposes plenty of restrictions that you can't get around in ISO C. That can be perfectly fine, but it's a tradeoff. Sometimes you can find a halfway house of requiring a specific compiler, but that also removes at least some of the advantage.
You make some very good points, and I agree with your conclusion, viz it's a trade-off.
I'm still wondering if the OP (@duped) meant the same thing as you did, or if he had another use-case in mind when he said "calling conventions".
FWIW ...
> But the Ruby code needs the arity of the code that's being called to be passed along. Putting it in a register that's not used for arguments in the standard calling convention means that when calling out to e.g. glibc the C code just won't be able to access those values.
In general, when a call returns you can't trust the register contents for those registers that are unused in the calling convention, because the callee might clobber it (for example by calling back into your language before returning).
What I did[1] when doing the ffi to the external C functions was emit a `setjmp()`, call the function, and emit `longjmp()` when the callee returned.
That allowed recursive calls no matter how many times that FFI boundary is crossed in a single function call: The language calls into C API, which calls something that invokes a callback in the language, which makes another call to a C function, etc.
[1] IOW, this is not the usual way, as far as I know. All a bit hackier than when I (for a different toy language) used `libffi` when emitting the code to call external C functions, which (IIRC) makes certain gaurantees about some of the registers.
> In general, when a call returns you can't trust the register contents for those registers that are unused in the calling convention, because the callee might clobber it (for example by calling back into your language before returning).
Depends on the register and the calling convention. E.g. for i386 the Intel ABI requires that eax, edx, and ecx are caller-saved, and so can be clobbered by the called function, but the rest are callee-saved, and so you can rely on them surviving just fine if you're calling any code generated by a compiler conforming to the Intel ABI. So calling out to C-code is fine.
(Calling back into the Ruby code requires extra effort, because the C code can't know how to set the arity; I might change this to store the arity of a method so that it can be looked up by the caller, in which case it might be doable to not require setting it at the time of the call, but that increases the amount of code at each call site as you'd need to resolve the method address and look it up relative to the actual method address instead of an indirect call, in order to avoid race conditions if someone redefines a method (we do love redefining methods at runtime in Ruby; it doesn't actually happen that often, but it can in theory happen at any time, and proving it won't for a given piece of code is a pain); it might be worth it at some point, even optionally, as when you can safely cache the method address you'd be able to jump straight past the arity check if you can guarantee the right number of arguments, but it's not simple).
> what would be the point of a calling convention different to the rest of the system
C is actually the exception and not the rule, most languages have different or multiple calling conventions.
> what's the alternative you have in mind when using assembler? Assemblers also have a notion of a stack and non-stack addresses, after all.
Growable segmented stacks.
Now sure, you can do mostly anything in C. But again, it's not assembly. Is it that odd that someone would want to write a compiler without having to write an x86 code generator, or tying themselves to C which again, cannot implement certain things at all?
> you can emit calls to the platform's thread primitives (CreateThreadEx, pthread_create, etc)
pthread_create is not a platform primitive, it's a glibc function, and interestingly cannot be written in C.
> What feature do you have in mind that cannot be implemented by emitting C
You can't implement fibers with growable stacks in C. Now you can use <ucontext.h> (getcontext, setcontext, swapcontext are all libc functions implemented in assembly) but they are very expensive with low hanging fruit for optimizations (for example, not saving the fp status registers or signal mask). Context switching also depends on your calling convention, so you may want to have multiple variants that are emitted for different parts of the program, or evolve over time.
For growable stacks you need to insert a prologue into function signatures to check if the stack needs to grow, however you probably don't want to do this for every function call and can write peephole optimizations in your compiler to avoid doing it.
There are other problems with C that aren't even worth mentioning, notably the linkage/loading model.
I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs.