Lesson learned the hard way: Don't waste time getting this up an running in an Alpine container.
At least I encountered these errors:
Absolutely no output from csharp-lsp and since alpine uses musl libc which weren't compatible with glibc I gave up on using Alpine as a development machine.

But using Debian (mcr.microsoft.com/dotnet/sdk:8.0-bookworm-slim) got me to the finish line.