How to create a self-extracting, self-executing tarball

GNUser · 2017-11-02 16:50:25

I enjoy putting together little apps for myself or my penguin friends, and have long looked for a simple way to bundle scripts, binaries, and icons. I realize this can be accomplished with a .deb or an AppImage, but I've always longed for something much simpler.

Well, I finally figured something out. It just involves two scripts.

This one should be named "pack":

#!/bin/bash

# no need to change anything in this script (nothing in here is project-specific)

project_folder="$(dirname "$(realpath "$0")")"
project_name="$(basename "$project_folder")"

tar cvzf ../payload.tar.gz ./*
cd ..
cat "$project_folder/unpack" payload.tar.gz >"$project_name.run" && chmod a+x "$project_name.run"
mv "$project_name.run" $HOME/Desktop 2>/dev/null
rm payload.tar.gz
echo "$HOME/Desktop/$project_name.run created"
exit 0

And this one should be named "unpack":

#!/bin/bash

# the only line in this script that you need to customize is the POST-UNPACK COMMAND

ARCHIVE_STARTLINE=`awk '/^__ARCHIVE_BELOW__/ {print NR + 1; exit 0; }' "$0"` # gets line number below "__ARCHIVE_BELOW__"

if [[ "$1" = "unpack" || "$1" = "extract" ]]; then
	executable_name="$(basename "$0")"
	project_name="${executable_name%.*}" # strip extension
	executable_folder="$(dirname "$(realpath "$0")")"
	DEST_DIR="$executable_folder/$project_name"; [ -d "$DEST_DIR" ] && rm -r "$DEST_DIR"; mkdir "$DEST_DIR"
	tail -n+$ARCHIVE_STARTLINE "$0" | tar xzv -C "$DEST_DIR" &>/dev/null
	exit 0
else
	export TMPDIR=`mktemp -d /tmp/selfextract.XXXXXX`
	tail -n+$ARCHIVE_STARTLINE "$0" | tar xzv -C "$TMPDIR" &>/dev/null
	cd "$TMPDIR"
	./foo "$@" # <-- POST-UNPACK COMMAND
	# cleanup:
	cd -
	rm -rf "$TMPDIR"
	exit 0
fi

__ARCHIVE_BELOW__

Say you have a directory called some-project, which contains shell scripts, binaries, icons, and maybe other things your app needs. Simply put the pack and unpack scripts in the some-project directory. The only thing that needs to be customized is the POST-UNPACK COMMAND line in the unpack script.

Now executing pack will create "some-project.run" on your desktop, which you can use yourself or share with your friends.

Running some-project.run will unpack the some-project directory to /tmp and launch the post-unpack command. If your friend is computer-savvy and wants to inspect and/or change the contents of some-project.run, running some-project.run with "unpack" as an argument will give them the some-project directory so that they can change stuff and then re-pack:

some-project.run unpack # this will create some-project directory
cd some-project
# change stuff in the some-project directory
./pack

P.S. The central idea for all of this was this amazing old post from Jeff Parent. For an example project, check out my portable (no dependencies!) international-keyboard.run.

Last edited by GNUser (2017-11-13 16:36:04)

FOSSuser · 2017-11-02 18:30:22

....or you could just use tar.

Last edited by FOSSuser (2017-11-02 18:30:45)

GNUser · 2017-11-02 18:55:12

Combining multiple files into a single file is trivial (tar) but is just one of three ingredients. The challenging and fun part was figuring out how to make the tarball self-extracting and self-executing. I thought I'd share how I did it, since it was not trivial and could be useful to someone.

Last edited by GNUser (2017-11-02 20:15:05)

ralph.ronnquist · 2017-11-04 00:22:31

Good script.

One (small) remaining problem would be that it's still a script and not a binary; it relies on a pre-installed interpreter (bash), and is thus not fully self contained. Of course, that's not much of a problem except for those who take offence to installing bash, and those who wants a suid program.

But if you want that covered as well, you could write up that header function in C (probably with some minor variation to the separation logic), and then pack such a binary in front of a tar to make a binary, self-executable tar. The point being that you can concatenate anything to a binary executable without that destroying it.

(Or, rather than C, you could use newlisp, which in fact includes an embedding method similar to this.)

On the other hand, as a binary, it'll instead be sensitive to the architecture, and thus less universally distributable -- swings and carousels.

GNUser · 2017-11-04 01:16:18

Nice thought. That hadn't occurred to me, nor could I imagine that some penguinistas might have a problem with bash. Wow, there are some real hard-core minimalists out there!

All of my projects contain binaries in the tarball anyway (with accompanying source code--hence the benefit of user being able to unpack and re-pack the .run file), so using a shell script header does not expand the audience. Therefore, I would only stand to gain by using a binary header instead.

I'll check out using C or newlisp as an alternative when I have a chance. I have a crush on C, but tend to gravitate towards bash and python because I'm lazy. If you can't have the one you love, love the one you're with.

Last edited by GNUser (2017-11-04 01:24:21)

GNUser · 2017-11-04 01:41:53

P.S. Another option would be to ditch the bashisms in the scripts and use a /bin/sh she-bang. Surely all GNU/Linux users would have some shell installed. This would be an easy upgrade.

ralph.ronnquist · 2017-11-04 02:27:53

Surely all GNU/Linux users would have some shell installed

Yes, according to FSSTND, you'll need to venture off the ranch for that one.

GNUser · 2017-11-04 13:37:06

I purged the bashisms and added some polish. Here are the scripts again, tested and working on multiple shells (bash, dash, busybox ash):

#!/bin/sh

# This script should be called "pack". It works as-is for any project.

project_folder="$(dirname "$(realpath "$0")")"
project_name="$(basename "$project_folder")"

cd "$project_folder"
tar cvzf ../payload.tar.gz ./*
cd ..
cat "$project_folder/unpack" payload.tar.gz >"$project_name.run" && chmod a+x "$project_name.run"
mv "$project_name.run" $HOME/Desktop 2>/dev/null
rm payload.tar.gz
printf "$HOME/Desktop/$project_name.run created\n"
exit 0

#!/bin/sh

# This script should be called "unpack". Only the POST-UNPACK COMMAND needs to be customized.

ARCHIVE_STARTLINE=$(awk '/^__ARCHIVE_BELOW__/ {print NR + 1; exit 0; }' "$0") # gets line number below "__ARCHIVE_BELOW__"

if [ "$1" = "unpack" ]; then
	runfile_dir="$(dirname "$(realpath "$0")")"
	project_name="$(basename "$0" .run)"
	DEST_DIR="$runfile_dir/$project_name"; [ -d "$DEST_DIR" ] && rm -r "$DEST_DIR"; mkdir "$DEST_DIR"
	tail -n +$ARCHIVE_STARTLINE "$0" | tar xzv -C "$DEST_DIR"
else
	export TMPDIR=$(mktemp -d /tmp/selfextract.XXXXXX)
	trap "rm -rf $TMPDIR" EXIT HUP TERM INT
	tail -n +$ARCHIVE_STARTLINE "$0" | tar xzv -C "$TMPDIR" 1>/dev/null && cd "$TMPDIR"
	./foo "$@" # <-- POST-UNPACK COMMAND
fi
exit 0

__ARCHIVE_BELOW__

Last edited by GNUser (2017-11-14 20:49:22)

greenjeans · 2017-11-04 20:42:44

GNUser wrote:

P.S. Another option would be to ditch the bashisms in the scripts and use a /bin/sh she-bang. Surely all GNU/Linux users would have some shell installed. This would be an easy upgrade.

I've been using dash for everything i'm making, after login is done (bash) the system switches to dash anyway on Devuan and Debian, it's said to be 4-5 times faster than bash, just got to keep the bashism's outta the script.

I like your script even better now, and FYI i'm stealing it too.

GNUser · 2017-11-04 23:34:20

I had heard dash was faster, didn't know it could be that much faster. Good to know. I'll definitely try avoid bashisms from now on.

Enjoy the scripts, greenjeans! I'm happy to mingle with you here--learn something new every day!

Last edited by GNUser (2017-11-14 19:22:44)

The officially official Devuan Forum!

#1 2017-11-02 16:50:25

How to create a self-extracting, self-executing tarball

#2 2017-11-02 18:30:22

Re: How to create a self-extracting, self-executing tarball

#3 2017-11-02 18:55:12

Re: How to create a self-extracting, self-executing tarball

#4 2017-11-04 00:22:31

Re: How to create a self-extracting, self-executing tarball

#5 2017-11-04 01:16:18

Re: How to create a self-extracting, self-executing tarball

#6 2017-11-04 01:41:53

Re: How to create a self-extracting, self-executing tarball

#7 2017-11-04 02:27:53

Re: How to create a self-extracting, self-executing tarball

#8 2017-11-04 13:37:06

Re: How to create a self-extracting, self-executing tarball

#9 2017-11-04 20:42:44

Re: How to create a self-extracting, self-executing tarball

#10 2017-11-04 23:34:20

Re: How to create a self-extracting, self-executing tarball

Board footer