The great systemd register, Mk II

JoshuaFlynn · 2017-09-28 01:52:41

Foreword

After waiting out on Devuan to mature for some time, I decided I'd try to make a relatively barebones systemd-free OS on top of Devuan. In trying to at least reasonably certify that the walking DNS remote code execution backdoor that only the NSA could love was gone, I stumbled across this thread invoking a bash script that helped identify processes relying on systemd (or what was left of it), and much to my surprise even my ripped out installation had 72 registries.

'Not to worry', I thought to myself 'I'll just get rid of the software components that directly and indirectly rely on systemd and install ones that don't'. Problem: I found even the kernel image (dependent on udev) relied on systemd. And after doing an initial package list search, I found to my horror that a lot of things Red Hat(e) had tinkered with, including lvm2 (responsible for disc encryption, I believe, now who could possibly want a remote code execution backdoor to have access to that?) were carriers of the systemd disease. 'I'm sorry, but it's xfterminal'

I decide what I needed to do was simply identify systemd using packages. I documented 42. Seemed a bit low. I did a manual scan. I estimated hundreds, if not thousands of packages via indirect dependencies. What to do? I decided I'd create a register documenting packages that rely on systemd so eager beavers like me would be able to shun it if they wanted to. I was lazy, so what was a guy to do?

The steps used

1) I first modified a relatively default devuan installation to include non-free and contrib, and made the insane decision of also sticking in a Debian repository (simply so I could make life harder for myself with even more systemd packages). Apt-get update was, of course, run.

2) I then ran the dpkg query command that pulled every available package name and descriptor from said setup. 3.2mb. 45970 packages or thereabouts Zoiks.

3) I threw it over to my main system to run in python and split on spaces, extracting the first term, and then converting each and every package into an apt-cache depends command which fed out into a text file named after the package. Sure, CPU inefficient, but I was doing lazy coding. I also filtered out the blatantly systemd package names at this stage to feed into a blacklist for later (no point getting the dependencies of a systemd item to see if it's systemd related).

4) After several hours (of running the massive 45970 or so entry of depends commands as a bash script), I got an entire directory of every package imaginable. Properties said it only took up a few mere megabytes, but in truth it was hogging over half a gigabyte due to all the separate files. There was a reason for this.

5) I opened the directory, and spat out it's contents from ls into a text file, meaning I had a list of every file.

6) I fed this list back into another python program who parsed the first word in the file (package name) and captured every dependency (note, it ignores recommends and suggests). Each dependency was put into square brackets after the first package name. Example:

0ad-data-common  [ttf-dejavu-core] [ttf-freefont] [tex-gyre]

This was stored in a line delimited list, so each package name was first, then it's dependencies in brackets. The dependencies list was 3.8mb in size. You can see a copy of it here on this 6 month expiring paste bin (a reupload to a permanent source would be appreciated).

7) Packages that apparently had no dependencies were stripped out, viewable here, which amounted to over 4000 packages. Mostly documentation and useless stuff, though.

8) I constructed a blacklist file from the extracted systemd dependencies, plus the packages I detected with the code found at the other thread link (along with said packages also mentioned at said link). This would be the 'seeder' file for detecting indirect dependencies.

9) I built a basic python program that converted the blacklist items into the square bracket dependency format (note: ignores optional dependencies placed in angular brackets) which populated a unique list item with the terms to detect. The program would then iterate over the list doing scans for packages dependent on systemd, adding it to the detection list and making a note to repeat the scan from the beginning once it completes it's current scan. It then compares every line (containing package name and dependencies) and identifies packages indirectly relating to systemd.

10) Because I know people won't necessarily trust the output from someone because they say so, I made a human readable output list as well, which mentions how a package is related to systemd (note, the program only detects the first systemd association and ignores others). So it'll say [<package name>] via [<dependency>], for example: "[isight-firmware-tools] via [udev]". You can then manually trace it back with Ctrl+F (not efficient, but at least traceable).

The program:

I know the list isn't complete, isn't perfect, and yes it is based on a Debian and Devuan repository hybrid plus user insights, but I figured I'd post it up for other people:

Raw dependencies data:
Dependencies.txt (subject to expiry, 6 months)
NoDependencies.txt

Program stuff itself:

HasDependencies.txt (subject to expiry, 6 months)
Blacklist.txt
DetectSystemD2.py

Excuse the sloppy code, it was only built to give me a rough overview of what I'm dealing with

And finally the (easily non-comprehensive) list:

Because the program is very basic, it cannot identify alternative or optional dependencies or close associations (some I had to add to the initial blacklist myself), so this isn't a definite list, feel free to take the datasets, modify it etc (doesn't have to just detect systemd, you can modify the blacklist to detect whatever). Note, this includes the surrogate libsystemd0.

Has, relies on or references systemd

I personally think the list looks a bit too small, so any suggestions for improvements welcomed (feel free to take or tinker with the code).

Hope it helps someone.

Last edited by JoshuaFlynn (2017-09-28 01:56:13)

golinux · 2017-09-28 03:12:03

You might be interested in comparing your lists with these three. I assume parazyd obtained them from apt searches on the output of amprolla3 (which should be operational soon). They were included in his recent ASCII release proposal:

outdated packages in Ascii: http://sprunge.us/AIeF
These packages might and might not need updates, but we should figure out if there is some necessary releases to be done regarding security.
libsystemd packages in Ascii: http://sprunge.us/cfAW
These packages shouldn't have to be forked at this point, since the library doesn't do anything if systemd is not there.
banned packages in Ascii: http://sprunge.us/ALPE

MiyoLinux · 2017-09-28 03:51:26

golinux wrote:

They were included in his recent ASCII release proposal:

Well that's a might-bit interesting! Thanks golinux!

JoshuaFlynn · 2017-09-28 03:55:01

Thank you goadmin. I should have clarified I was on Jessie, my bad.

I took the blocked list from the link you provided, and set it as my blacklist and re-ran the program.

Here's a list of impacted services, note that it will include lxde-core (a desktop environment that Devuan presently supports, and I do like using LXDE minus it's systemd entanglements).

There's others there, but not many from a primitive scan. I know what I've got here isn't too particularly useful, but it's my small way of helping and showing discontent for systemd in the same move.

golinux · 2017-09-28 05:08:32

Every bit of information adds depth and every new pair of eyes a new perspective. Welcome to the forum.

JoshuaFlynn · 2017-09-29 03:01:31

golinux wrote:

Every bit of information adds depth and every new pair of eyes a new perspective. Welcome to the forum.

Thank you for the warm welcome.

Do you guys have a list of tasks you need help with somewhere? I'd like to help Devuan in some way but my real life is extremely busy, so I'm wondering if there are any time-simple tasks I can help with. I'm semi-moderate knowledge wise (more of an 'all-rounder' in terms of resource creation, images, videos etc). Knowledge of C++ (you can eyeball a code library I spent 5 years working on, but it's likely not to be pretty. Public domain, if any of it interests), 'working knowledge' of Python, JavaScript, and even spending some time on it, I'm still stuck in the noob zone when it comes to bash (it has a crazy syntax).

I'm not really able to get stuck in on anything deep-end programming wise, because I already do that as work, but if you're looking for someone who does convenience scripts (I've got a script that auto-converts a default 64 bit Devuan desktop installation into a stripped down barebones LXDE environment complete with background modifiers on both login, desktop, which I'm planning on converting into a base OS [ideally systemd free] called King-Pigeon. Screenshot).

I'd really like to be able to take a chunk out of systemd though (even if that's as simple as pruning out non-functional references), maybe try to assist it's removal by some means.

(I had planned to donate at one point, but Paypal are strong advocates of surveillance, abuse their powers to undermine people, and I really do not trust bitcoin. So I'll donate by other means IE time.)

At the moment I'm quite busy, but would you be interested in some from-scratch open source GIMP based artwork for use as backgrounds? I'm no good with vectors, so it'll be chunky rasters, and if you don't like em, ditch em.

Last edited by JoshuaFlynn (2017-09-29 03:03:05)

golinux · 2017-09-29 04:59:31

@JoshuaFlynn . . .

Working backwards . . .

You can share artwork on the Friends of Devuan wiki.

Packagers who can help to clean out systemd dependencies are always welcome. Have you seen the Devuan packaging helper? Check with parazyd about which packages are a priority.

Once you have your distro together, please post to the derivatives forum.

golinux

The officially official Devuan Forum!

#1 2017-09-28 01:52:41

The great systemd register, Mk II

#2 2017-09-28 03:12:03

Re: The great systemd register, Mk II

#3 2017-09-28 03:51:26

Re: The great systemd register, Mk II

#4 2017-09-28 03:55:01

Re: The great systemd register, Mk II

#5 2017-09-28 05:08:32

Re: The great systemd register, Mk II

#6 2017-09-29 03:01:31

Re: The great systemd register, Mk II

#7 2017-09-29 04:59:31

Re: The great systemd register, Mk II

Board footer