Skip to content

Blog

Stage 3 Progress

Well, it's been a few days since we last spoke, so now it's time for a quick roundup. Long story short, we're approaching the end of the stage3 bootstrap.

Fully functional chroot

In an effort to simplify our bootstrap process, we dropped the newly-introduced stage2.5 and came up with a new strategy for stage3. In order to make it all work nicely, we bind-mount the stage2 resulting runtime at /serpent within the stage3 chroot environment, executing the /serpent/usr/bin/bash shell.

In order to make this work, we build an intermediate musl package for libc.so in the very start of stage3, with all subsequent builds being performed in chroots. Part of the build is done on the host, i.e. extraction and patching, minimising the tool requirements for the chroot environment. The configuration, build and install is performed from within the initially empty chroot environment, replacing all the /serpent/usr/bin tools and /serpent/usr/lib libraries.

Mild Blockers

As we move further through stage3, towards a fully usable chroot environment, we've encountered a small number of blockers. Now, we could solve them by using existing patchwork and workarounds, but most have not and will not be accepted upstream. Additionally it is incredibly hard to track the origin and history of most of these, making security rather more painful.

libc-support

We're going to start working on a project to flesh out the musl runtime with some missing utilities, written with a clean-room approach. These will initially include the getconf and getent tools, which will be written only with Linux in mind, and no legacy/BSD support.

These will be maintained over at our GitHub

libwildebeest

As a project we strive for correctness in the most pragmatic way. Some software, such as systemd, is heavily reliant on GNU GCC/GLibc extensions. In some software there are feasible alternatives when using musl, however in a good number of cases, functionality required to build certain software is missing and has no alternative.

Over time we'll try to work with upstreams to resolve those issues, but we're working on an interim solution called 'libwildebeest'. This will provide source compatibility for a limited number of software packages relying on so-called 'GNUisms'. Binary compatibility is not an aim whatsoever, and will not be implemented. This convenience library will centralise all patchwork on packages that need more work to integrate with musl, until such time as upstreams have resolved the remaining issues.

Additionally it will help us track those packages needing attention in the distribution, as they will have a build-time dependency on libwildebeest. We do not intend to use this project extensively.

This will be maintained over at our GitHub

A Word On Init Systems

Recently we've had many queries regarding the init system, as there is an expectation that due to our use of musl/llvm we also dislike systemd or wish to be a small OS, etc. There is a place in the world for those projects already, and we wish them much success. However from our own stance and goals, systemd has already "won the battle" and actually fits in with our design.

If it is possible in future with package manager considerations and packaging design, then we may make it possible to swap systemd for a similar set of packages. However, we only intend at this time to support systemd/udev/dbus directly in Serpent OS and leave alternatives to the community.

Other News

Just a quick heads up, we've been talking to the cool folks over at fosshost.org and they've agreed to kindly provide us with additional hosting and mirroring. This will allow us to build scale in from the very start, ensuring updates and images are always available. Once the new server is up and running we'll push another blogpost with the details and links.

While initially we intended to avoid public bug trackers, the rate of growth within the project and community have made it highly apparent that proper communication channels need establishing. Therefore we will be setting up a public Phabricator instance for reporting issues, security flaws, and contributing packaging.

Much of our website is in much need of update, but our current priority is with building the OS. Please be patient with us, we'll have it all sorted out in no time.

Where We At?

Well, stage3 completes fully, builds the final compiler, which has also been verified. A usable chroot system is produced, built using musl, libc++, libunwind, clang, etc. Some might say that stage3 is complete, however we wish to avoid circular dependency situations. We'll mark stage3 as complete once we've integrated an initial slimmed down build of systemd and appropriately relinked the build.

As soon as this stage is done, we'll proceed with stage4. This is the final stage where we'll add package management and define the OS itself, with global flags, policies, etc.

With the speed we're moving at, that really isn't too far away.

And finally..

I personally wish to thank the Serpent OS team as a whole for the commitment and work undertaken of late. Additionally I want to thank the growing community around Serpent OS, primarily residing in our IRC channel (#serpentOS on freenode) and our Twitter account. Your input has been amazing, and it's so refreshing to have so many people on the same page. Stay awesome.

Stage2 Complete

Just in case you thought we were sleeping behind the wheel, we've got another blogpost for your viewing pleasure. In a nutshell, we completed stage2 bootstrap.

Complete build-target for ARMv8

In order to simplify life, we greatly reduced the size of the stage2 build component. This decision was taken to better support cross-compilation in the face of software that is distinctly cross-compilation unfriendly.

A support stage, stage2.5 will be added which will chroot into a copy of stage2, and natively compile a small handful of packages required to complete stage3, also within the chroot environment.

For cross-compilation, we'll be relying on qemu-static to complete 2.5 and 3. However, at this point in time, we have the following:

  • Working cross-compilation of the entire bootstrap
  • Complete LLVM based toolchain: clang, llvm, libc++, lib++abi, libunwind
  • Entirety of stage2 built with musl libc.
  • Working, minimal, chroot environment as product of stage2, with working compiler (C & C++)

x86_64

This is a major milestone for the project, as it is an early indication that we're self hosting.

Multiple Architecture Support

At this point in time, we now have build support for two targets: x86_64 and ARMV8a. Our intent is to support haswell and above, or zen and above, for the x86_64 target.

With our ARMv8 target, we're currently looking to support the Pine Book Pro, if we can manage to get hold of some testing hardware. It will likely be some time after full x86_64 support that we'd officially support more hardware, however it is very important that our bootstrap-scripts can trivially target multiple platforms.

ARMv8

An interesting change when cross-compiling for other architectures, is the chicken & egg situation with compiler-rt and other LLVM libraries. When we detect cross-compilation, we'll automatically bootstrap compiler-rt before building musl, and then cross-compile libc++, libc++abi and libunwind to ensure stage1 can produce native binaries for the target with correct linkage.

Next Steps

As we've mentioned, we'll push ahead with 2.5 and 3, which will complete the initial Serpent OS bootstrap, producing a self-hosting, self-reliant rootfs. This is the point at which we can begin to bolt-on package management, boot management, stateless configuration utilities, etc.

Our initial focus is x86_64 hardware with UEFI, and as we gain access to more hardware we can enable support for more targets, such as ARMv8a. Our bootstrap-scripts will always remain open source, as will all processes and tooling within Serpent OS, or anything used to build and deploy Serpent OS.

This will make it much easier in future to create custom spins of Serpent OS for different configurations or targets, without derailing the core project. It should therefore be the simplest thing in the world to fork Serpent OS to one's liking or needs.

If you want to support our work, you can jump onto our IRC channel (#serpentOS on freenode) or support us via the Team page.

Stage1 Complete

Short and sweet, stage1 of the bootstrap is complete. As I indicated on the Lispy Snake blog, I'm still in the process of settling into new accommodation. This is going well, but still awaiting proper broadband connectivity. Work has begun, however, and we're now moving onto stage2 of the bootstrap.

This is handled via our bootstrap-scripts project and can be run by anyone on a relatively modern Linux distribution.

Validating stage1 cross-compiler

Next on the list is completing stage2, which we've already started on. This is simply a cross-compiled chroot environment with all the basic bits in place to build stage3, sanitizing and cleansing the toolchain.

Even though it is early days, we can already, automatically produce a working cross-compiler that targets LLVM's libc++, the musl libc.so, and x86_64-serpent-linux-musl host triplet.

Probably a dull update for most, but an update it is.

Welcome: Project Manager

The Serpent OS team wishes to formally welcome Aydemir Ulaş Şahin to the core team in the primary capacity of Project Manager. Aydemir will help us to coordinate and manage the project goals and infrastructure, being responsible for certain "keys to the kingdom".

Aydemir has extensive experience with Linux, both as a user and developer, having worked in the media industry for many years. Additionally he will be contributing to the core system, code and packaging.

Our initial trio has been announced, and you can always access information on the current team at the new Team page. Now that much of our groundwork has been laid, we can get on with building the project. Check progress at the new Roadmap page.

Welcome Peter

Recently we did reveal on Twitter that long time friend and colleague-of-many-projects, Peter O'Connor, has formally joined the core team for Serpent OS. Many of you know him as sunnyflunk. He will be working on core toolchain optimisations, benchmarking, and eventually Plasma inclusion.

Peter has long wanted to get the most out of his computing experience, starting from Gentoo back in the 2000's to ensure there was no wasted resources on features he didn't use. The desire to over-tinker and use unstable software often led to poor outcomes. While the itch laid dormant for over a decade, it has always been there.

{{

}}

We'd like to officially extend our welcome to Peter, as the ramp up gets in place. It should be noted, he is partly to blame for Serpent OS becoming manifest, as the core team got together after discussions on IRC. He has some fantastic ideas, and is a brilliant distribution engineer with proven success.

As an aside, we're working on improving the website to incorporate a team page, ready for the third team member reveal.

The Great Experiment

So as many have come to realise, we had to rush out a website super quick yesterday as the cat was already out of the bag. One thing that should also be clarified, is our approach to development.

We're A Lab

Primarily (but not exclusively) Serpent OS (future: Serpent Linux) is an endeavour worked on by the Lispy Snake, Ltd crew. So we're all about creating awesome technology and trying to further the playing field.

Serpent OS is not, however, owned by Lispy Snake. It is a contributor-based open source project, with specific roles. Those will be clarified over the coming week, but there is a dedicated Project Manager. Hint: It's not me.

Measuring Success

So, how does an experimental project.. measure success? Quite simply - if others adopt our methodologies or technology, we've succeeded. We're really not looking at "number of installs" or "size of userbase" as a metric, as our chief aim is to build technology.

Options for Longevity

If the experiment is a success, which of course means having tight controls on scope and timescale, then one would assume the primary way to use Serpent OS would be through some downstream repackaging or reconfiguration, i.e. basing upon the project, to meet specific user demand.

It Should Be Fun

This is the heart and soul of all Linux development. Developers should enjoy the involvement and seeing the end result. If we build something more in the end, then that's a journey we can create together.

First Post

In a nutshell, this post wasn't meant to be coming out for a few weeks. As it turns out, we had to get ahead of the cat coming out of the bag, and ensure clarity of goals and ambition.

We've got an awful lot of work ahead of us across the next few months and weeks, and we strongly suggest checking out the About page which should answer most questions.

TL;DR This is not your typical user-facing Linux distribution.

So go ahead, check out the About page.

Posts will follow in the coming weeks, introducing the core team. It should be pointed out, internally, we're in a stage1 bootstrap phase, with stage2 ready. Tooling is next on the agenda.

If you're looking for a modern, lightweight, user-friendly, privacy-focused Linux desktop distribution, then you're in the wrong place.

Otherwise, stay tuned for the updates. We know our website is insanely basic, but it will be improved when it becomes a priority. You're more than welcome to join us on #serpentOS on freenode to become part of the community discussion in the mean time.

Til next time.