Llvm abstracts the "backend" which generates the actual assembly for each target machine. You only have to write a "frontend" that generates an llvm intermediate representation.
But in general, yes. To generate assembly you need to know assembly.
LLVM is definitely more complex than a toy assembly you might learn in an intro computer architecture course, but it's generally somewhat less complex than working with real assembly languages. Although the complexity in LLVM is a very different kind of complexity from assembly languages; LLVM is ultimately a higher-level abstraction than machine code, and the semantics of that abstraction can be complex in its own right.
But in general, yes. To generate assembly you need to know assembly.