Sunday, August 30, 2009

Debian squeeze in a solaris BrandZ zone!

I wanted to run some debian squeeze programs on my opensolaris machine in a branded zone but after successfully installing it using some guide I found online I noticed that lots of commands fail due to functions not being implemented!

unconfigured:~# touch foo
unconfigured:~# rm foo
rm: cannot remove `foo': Function not implemented

Oh noes! How come rm isn't implemented, that's pretty basic right?

unconfigured:~# strace rm foo 2>&1 | grep -i "not implemented"
unlinkat(AT_FDCWD, "foo", 0) = -1 ENOSYS (Function not implemented)

Ah, what's this? unlinkat? I bet lx-brand doesn't implement this syscall since it's so new or maybe there's some switch to enable it? But how does this BrandZ voodoo magic work anyway? Turns out the emulation itself is done in userland by one library (/usr/lib/, and after some digging I came across {"unlinkat", NULL, NOSYS_NULL, 0} in some code which pretty much confirms that unlinkat and the other *at functions aren't implemented at all!
Well damn, this sucks, maybe I'll recompile squeeze libraries and programs so that they don't use these syscalls and thus work with BrandZ, but that sounds tedious and not very fun... so clearly it's time to hack some new syscalls into lx-brand!
But how the hell do you compile onnv-gate code anyway? It seems to use a very odd build system which is designed for build robots rather than humans! Well luckily there are other crazy people on the internet who have already found out how, is a very nice tutorial on how to do it and it works great, just instead of going to usr/src/uts goto usr/src/lib/brand/lx/lx_brand!

So a few hours of hacking and reading posix standards later I now have a /usr/lib/ which implements openat(), mkdirat(), mknodat(), fchownat(), futimesat(), fstatat64(), unlinkat(), renameat(), linkat(), symlinkat(), readlinkat(), fchmodat() and faccessat() resulting in a working debian squeeze userland, although there are still a few issues like the debconf flock issue mentioned in the etch-zone tutorial and some syscalls still fail but seem to be innocuous...
Also I'm sure the syscalls implemented aren't fully compatible with the proper linux versions and I'm sure there are lots of error conditions that will freak out user mode programs... but for now it works well enough to run debian squeeze which was the goal!

If you're interested in trying out what I've done you can download my patch and a pre-built library (built against the b118 onnv-gate source, since that's what I happened to be running at the time) from my website, and if you do try it out please let me know how it goes! Although be warned that it's super experimental and not recommended for production systems, I highly recommend you take snapshots before proceeding!

[2009-12-17 Update] The patch has been integrated: e8c975bf2038 should be in snv131 and above


  1. Hey Jbit,

    Any chance you'll sign an SCA so that SUN can include your patches in the next release of OpenSolaris? Everyone is waiting for you to respond in the BrandZ discussion forums.

  2. Dude, srsly, thank you for your work! However what Mr. Mous said above is usefully true. Without going off on a tear about it, I am headed towards doing the same kind of thing, for similar reasons.

    *I* want you to do this so that I can work against this stuff without giving it a lot of thought & so that like not many bytes down the road it'll be easier for me. I have already been mulling over the SCA thing cause I know I am going to have to do it because of a "side project" I have at work. I had no idea other people would want you to do that for this reason until I ran into what you've done & came here to see that mentioned above.

    In terms of usablity & your blog comments, well yea, some of these things are not going to function exactly like their Linux counterparts. That is ok though, and it will only cause lossage where people have had to code around tuxisms. There are a few cases where this might cause some configure "tests" to start working, and others to suddenly "fail" but these cases are few, sparsely distributed, and anyone who runs into them is going to be able to fix the trouble easy enough.

    The bottom line is that most code just will not care at that point, it's just looking at results. Is why this is kewl.