The officially official Devuan Forum!

You are not logged in.

#1 2017-04-10 22:16:49

KatolaZ
Member
Registered: 2017-03-11
Posts: 12

Using d1h (a.k.a. "Devuan packaging made easier")

d1h, the Devuan packaging helper

In this guide we will show how d1h can be used to speed-up package
maintenance in Devuan. After having read this document, the reader
should be able to import a Debian package in Devuan, and have their
package included in the Devuan building workflow.

Pre-requisites

Before you start, you need the following packages installed in your
system:

  • debhelper and all the usual stuff needed to build Debian packages
      (lintian, build-essential, cdbs, dh-make, fakeroot, and so on...)

  • git

  • git-buildpackage

  • gpg (for package signing)

On top of that, you should also have:

  • a working account on git.devuan.org

  • the environment variables DEBEMAIL and DEBFULLNAME set,
    respectively, to your email and to your full name, e.g.:

            DEBEMAIL=katolaz@freaknet.org
            DEBFULLNAME="Vincenzo (KatolaZ) Nicosia"
            export DEBEMAIL DEBFULLNAME

  • a working Internet connection

Installing d1h

The package d1h is currently in experimental. In order to install
it, you should add the following line to your /etc/apt/sources.list:

deb http://packages.devuan.org/devuan/ experimental main

then run:

 # apt-get update

and finally:

 # apt-get install -t experimental d1h 
The typical d1h workflow

As an example, we assume in this guide that we want to import the
package "adduser" to Devuan Jessie. Actually, there is no need at all
to import this package, since it does not depend on systemd (so
far). But this is a relatively small and self-contained package to
start with.


1) DEVUAN_REPO

Create a new project on your git.devuan.org account, called
"adduser". This new project will be associated to a git URL in the
form:

git@git.devuan.org:USERNAME/adduser

where USERNAME is your username on git.devuan.org. In the following,
we will denote by DEVUAN_REPO the URL associated to your project on
git.devuan.org.

2) CREATE CACHE

Create the d1h repository cache:

$ d1h cache

This will download the full list of repos hosted at
https://anonscm.debian.org, parse it, and save a cache in
~/.d1h/cache.

It is not necessary to create the cache every time you use d1h, but
it is recommended to run "d1h cache" every now and then.

3) SEARCH REPOSITORY

Search the upstream git repo of "adduser". Here d1h comes
handy:

  $ d1h search adduser
  ##########################
  #### cache last updated on  Fri  7 Apr 21:19:30 BST 2017 
  ##########################
  adduser/adduser: https://anonscm.debian.org/cgit/adduser/adduser.git/
  $ 

Notice that d1h has found only one repository matching our query
"adduser". Note down the corresponding URL

https://anonscm.debian.org/cgit/adduser/adduser.git/

The "search" function accepts multiple strings (separated by
spaces). If more than one string is provided, "search" will look for a
repo matching all of them.

4) IMPORT REPOSITORY

Enter the folder where you would like to keep the package repo, and
import the "adduser" repo:

  $ d1h import https://anonscm.debian.org/cgit/adduser/adduser.git/

You can alternatively provide to "import" a destination folder as
second argument:

  $ d1h import https://anonscm.debian.org/cgit/adduser/adduser.git/ ./pkg_adduser

 
In this case, the repo "adduser" will be imported under the folder
"./pkg_adduser".

When "import" has finished, you will find the repo "adduser" under the
destination folder (or in the current directory, if no destination
folder was provided).

(*) For the git-savvies: "import" corresponds to a "clone".

5) LINK REPO

Now we need to link our new "adduser" repository with the DEVUAN_REPO
we have created on git.devuan.org. Just chdir to the folder where your
repo has been imported, and give the command:

$ d1h link DEVUAN_REPO

where DEVUAN_REPO is the git URL of your project on git.devuan.org,
e.g. something like

git@git.devuan.org:KatolaZ/adduser

This URL will be indicated in the home page of your project on
git.devuan.org.

(*) For the git savvies: "link" resets the origin of your repo, to let
it point to, e.g., DEVUAN_REPO. After "link" has finished, DEVUAN_REPO
will be the origin of your local "adduser" repo.

6) PREPARE SUITE

Now we should prepare the suite for which we would like to build the
package. We assume that we want to build the package for
"jessie". However, new packages never go directly into a release main
repository (with the only exception of the "experimental"). Instead, a
new package for "jessie" is normally added to "jessie-proposed". So we
need to prepare the suite "jessie-proposed":

$ d1h prepare jessie-proposed

This step should be done only once.

(*) For the git-savvies: the "prepare" action creates a new branch, in
this case named "suites/jessie-proposed", and adds the appropriate
"debian/gbp.conf" file to it.


7) SWITCH TO SUITE

Before we start working on modifying the package for jessie-proposed,
we must first "switch" to the corresponding suite:

$ d1h switch jessie-proposed

(*) For the git-savvies: "switch" is simply a checkout to the
corresponding branch.


8) MODIFY THE PACKAGE

Now that we have switched into the suite "jessie-proposed", we must
reset the repo to the correct version of the package. Since we want to
package for jessie, we MUST ensure to work on EXACTLY the same
version of the package available in jessie. In this case, the version
of "adduser" available in jessie is 3.113+nmu3. You can easily check
using:

$ apt-cache policy adduser

Luckily, the repo we have imported from Debian keeps track of this
information through a tag. In order to retrieve the correct version of
the package for jessie, we can run:

$ git reset --hard tags/3.113+nmu3

You can now make all the changes you need to the package. In this
specific case, we don't have to do anything in particular, since the
package does not depend on systemd. We will just edit the
"debian/control" file with our preferred editor, and put our name and
email in the "Maintainer:" field. We will also remove the field
"Uploaders:", and save the file.

When we are done with the changes, we just commit everything:

  $ git add --all
  $ git commit

The "commit" command will open an editor asking to insert a message
for the commit. In this case, we can just insert "Devaun version of
adduser", save the file, and quit the editor.

9) TEST BUILD

It is time to build the package. This can be done through the command:

$ d1h testbuild

This command will start building the package. If it complains about a
missing package, just "apt-get install..." the missing dependencies as
needed, and run it again. If you have all the needed dependencies
installed, the build will proceed and create the following files:

  adduser_3.113+nmu3_all.deb
  adduser_3.113+nmu3_amd64.changes
  adduser_3.113+nmu3.tar.gz
  adduser_3.113+nmu3_amd64.build
  adduser_3.113+nmu3.dsc

in the parent folder of the repo. This time the command "testbuild"
will complain about not being able to sign the package, and it will
terminate with a "testbuild error". You can safely ignore this error:
it is due to the fact that you have not yet edited the file
"debian/changelog". This is what we will do in a moment.

10) CHECK THE PACKAGE

You should now test the .deb package you have just created, e.g. by
installing it:

# dpkg -i adduser_3.113+nmu3_all.deb

11) STAGE

When you are sure that the package works as expected, it is time to
assign a version number to it, using the command:

$ d1h stage 3.113+nmu3+devuan1.1

where 3.113+nmu3+devuan1.1 is the new version number. It is warmly
recommended for Devuan-ised packages to retain the same version number
of the corresponding Debian package, with a "+devuanX.Y" label
appended, where "X" and "Y" indicate the Devuan major/minor version of
the package.

(*) For the git-savvies: the command "stage" will run "gbp dch" to
assign a new version to the package. It will also create a tag for the
new version, push the repo to origin (well, to DEVUAN_REPO), and push
the tags.

12) ADD TO CI

At this point, your package might be ready to be added to the
Continuous Integration (CI) server at ci.devuan.org.  In order to do
that, you should contact a Devuan developer on the #devuan-dev IRC
channel (freenode network), who will guide you through the
process. This will usually involve moving your DEVUAN_REPO under the
group "devuan-packages".

13) BUILD ON CI

After your package has been revised from other developers and added to
the CI server, you will be able to request a build. This is done by
opening a new special "Issue" on the gitlab page of the
repository. This issue should have title equal to "build". It is very
important to specify a label as well, since the label determines which
branch will be built and where the resulting packages will be put. In
our example, the package "adduser" should go in jessie-proposed, so we
will use the label "jessie-proposed".  The issue must be assigned to
"autobuild".

You can check the status and health of your builds on

  http://ci.devuan.org

Moreover, messages about build successes and failures are also
reported on the IRC channel #devuan-dev in real-time.

Last edited by KatolaZ (2017-04-11 09:31:51)

Offline

#2 2017-05-01 22:25:09

greenjeans
Member
Registered: 2017-04-07
Posts: 153
Website

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Aborting attempt, going to go have beer now in my happy place.

Last edited by greenjeans (2017-05-02 19:51:01)


https://sourceforge.net/projects/vuu-do/
Vuu-do GNU/Linux, minimal Devuan-based openbox and mate systems to build on, plus maximal versions now.

I am greenjeans, commander of the mighty VUU legions and legend in my own bathtub.

Online

#3 2017-05-01 22:48:41

Ozi
Member
Registered: 2017-03-15
Posts: 15

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Is there process for doing backports as well?

Offline

#4 2017-05-01 23:00:07

KatolaZ
Member
Registered: 2017-03-11
Posts: 12

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Hi Ozi,

yes, the procedure is the same. The only difference is that if you package for jessie-backports you will need to "prepare" a suite named "jessie-backports" instead of "jessie".

HH

KatolaZ

Offline

#5 2017-06-22 15:43:32

mckaygerhard
Member
Registered: 2017-04-21
Posts: 124
Website

Re: Using d1h (a.k.a. "Devuan packaging made easier")

hi katolaz, its there any hot to for those users that not have network connection or intenet? i mean can i download and then process and made my package? and then upload results?

Offline

#6 2017-06-22 15:55:33

KatolaZ
Member
Registered: 2017-03-11
Posts: 12

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Hi,

the workflow is based on git, so you can pull all the needed stuff and prepare your package on your local git repo, and then push the correct branch to gitlab when you are ready (i.e., after you have been able to successfully produce a working package, which you have tested yourself locally),  This is actually what `d1h stage` does.

HTH

KatolaZ

Offline

#7 2017-07-10 21:10:06

ghaverla
Member
From: Dawson Creek, BC, Canada
Registered: 2017-06-19
Posts: 24

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Seeing as I have downloaded guayadeque (0.5.4 I believe) zipball from some git and compiled it, I will try to de-dbus it and package it.  Sort of as practice.  I have 4 machines to get GPUs working on, in the near future (all AMD GPU on amd64 architecture; but an A10 APU, a HD5450, a HD6450 (both are in R600), a R7-250 and a RX-460.  I don't know if any of this will become something of a package as AMD seems to not care very much about GPU/OpenCL support on Linux, especially for older GPUs.

But, I do have an interest in Perl packages.  I know that Debian came up with a rule for naming things (like Perl modules it packages), but I think that the CPAN name for the package has to  be in a few more places with respect to the product.  Just going from memory, but I think there is a CPAN package called XML::LibXML (or is it LibXML2?).  As a Debian package, it is libxml-libxml-perl.  The name has lost information due to all the case being lowered.

I could see myself spending some time working on Perl things.  How much a person can do, and should do, I suppose is to be determined.  I don't think I would go through all of CPAN, too much stuff there.  And quite a bit of stuff that is at best deprecated.

Offline

#8 2017-07-11 00:33:15

mckaygerhard
Member
Registered: 2017-04-21
Posts: 124
Website

Re: Using d1h (a.k.a. "Devuan packaging made easier")

KatolaZ wrote:

the workflow is based on git, so you can pull all the needed stuff and prepare your package on your local git repo, and then push the correct branch to gitlab when you are ready (i.e., after you have been able to successfully produce a working package, which you have tested yourself locally),  This is actually what `d1h stage` does.

umm so i forced to use git ? its no alternative as debian does with mentor system? wil be so in-flexible Devuan!

Offline

#9 2017-07-11 06:17:23

KatolaZ
Member
Registered: 2017-03-11
Posts: 12

Re: Using d1h (a.k.a. "Devuan packaging made easier")

ghaverla wrote:

But, I do have an interest in Perl packages.  I know that Debian came up with a rule for naming things (like Perl modules it packages), but I think that the CPAN name for the package has to  be in a few more places with respect to the product.  Just going from memory, but I think there is a CPAN package called XML::LibXML (or is it LibXML2?).  As a Debian package, it is libxml-libxml-perl.  The name has lost information due to all the case being lowered.

Hi ghaverla,

deb package names are lowercase. In general, it is not worth repackaging something just to change the name, since this will generate a mess (all the packages which depend on it will need to be rebuilt as well). If you want to help package something that is not in Devuan, or to maintain a systemd-freed package, you are very welcome.

Offline

#10 2017-07-11 06:19:11

KatolaZ
Member
Registered: 2017-03-11
Posts: 12

Re: Using d1h (a.k.a. "Devuan packaging made easier")

mckaygerhard wrote:

[
umm so i forced to use git ? its no alternative as debian does with mentor system? wil be so in-flexible Devuan!

Hi,
I am sorry but there is no alternative to git, since Devuan has chosen to have all its workflow based on git. Also Debian and other distros are doing the same, and moving as much as possible to git, and for very good reasons. Actually, d1h minimises the interaction with git and hides most of the complexity of the underlying workflow.

HND

KatolaZ

Offline

#11 2017-07-11 13:21:12

ghaverla
Member
From: Dawson Creek, BC, Canada
Registered: 2017-06-19
Posts: 24

Re: Using d1h (a.k.a. "Devuan packaging made easier")

KatolaZ wrote:

deb package names are lowercase. In general, it is not worth repackaging something just to change the name, since this will generate a mess (all the packages which depend on it will need to be rebuilt as well). If you want to help package something that is not in Devuan, or to maintain a systemd-freed package, you are very welcome.

I wasn't thinking of changing the name of Debian package.  There is (can be) documentation/information that is part of a package that would be improved if the CPAN name was also mentioned.  Man pages are sometimes missing our poor.  Testing scripts could be expanded.  Examples could be improved or added.

Offline

#12 2017-07-11 13:23:26

golinux
Administrator
Registered: 2016-11-25
Posts: 416

Re: Using d1h (a.k.a. "Devuan packaging made easier")

ghaverla wrote:
KatolaZ wrote:

deb package names are lowercase. In general, it is not worth repackaging something just to change the name, since this will generate a mess (all the packages which depend on it will need to be rebuilt as well). If you want to help package something that is not in Devuan, or to maintain a systemd-freed package, you are very welcome.

I wasn't thinking of changing the name of Debian package.  There is (can be) documentation/information that is part of a package that would be improved if the CPAN name was also mentioned.  Man pages are sometimes missing our poor.  Testing scripts could be expanded.  Examples could be improved or added.

Are you offering to provide that documentation?

Offline

#13 2017-07-14 14:13:59

ghaverla
Member
From: Dawson Creek, BC, Canada
Registered: 2017-06-19
Posts: 24

Re: Using d1h (a.k.a. "Devuan packaging made easier")

golinux wrote:
ghaverla wrote:

I wasn't thinking of changing the name of Debian package.  There is (can be) documentation/information that is part of a package that would be improved if the CPAN name was also mentioned.  Man pages are sometimes missing our poor.  Testing scripts could be expanded.  Examples could be improved or added.

Are you offering to provide that documentation?

Having been around a LUG a long time (since the 1.2.13 kernel), I have a passing familiarity with the concept of the documentation not being as good as it could be.  That some of my friends bring this up when they try to install software helps.

So yes, I would try to add to some packages.

Doing Perl development around the concept of writing tests does work.  It isn't my first choice, but I have used it at times.  So adding tests for string and integer modules I think would help some Perl packages.  If nothing else, that is a place to put extra documentation in terms of in code comments, and POD.

I think Perl needs some way to do floating point tests, but the last time I was thinking seriously about this, there were no good solutions.  I have a textbook which talks about range arithmetic, which might be an approach to start from.  But finding something which can handle the inexactness of floating point and how some calculated solutions can diverge from exact solutions is needed.

Some Debian packages have man pages written by Debian maintainers, as the parent package had no man page.  This I believe happens with some Perl modules in Debian packaging.  It might make sense to replace those kinds of man pages.  But, it is easy enough to have POD in Perl code to produce a man page.  The source of that man page, is the product of the original author(s), and I don't think should be replaced.  To provide extra documentation for those packages, would be something added by Devuan.  So having some guidance as to what to call it, and where to place it in the filesystem would be useful.

Offline

#14 2017-07-18 20:19:32

evilham
Member
Registered: 2017-07-02
Posts: 1

Re: Using d1h (a.k.a. "Devuan packaging made easier")

Since installing d1h requires adding experimental to sources.list, there should be a step about pkg pinning, something like:

In order not to upgrade your whole system to experimental, add a file /etc/apt/preferences.d/stay-on-ascii
with following content:

Package: *
Pin: release a=ascii
Pin-Priority: 900

Package: *
Pin: release a=experimental
Pin-Priority: 500

Offline

#15 2017-07-18 20:45:46

golinux
Administrator
Registered: 2016-11-25
Posts: 416

Re: Using d1h (a.k.a. "Devuan packaging made easier")

evilham wrote:

Since installing d1h requires adding experimental to sources.list, there should be a step about pkg pinning, something like:

In order not to upgrade your whole system to experimental, add a file /etc/apt/preferences.d/stay-on-ascii
with following content:

Package: *
Pin: release a=ascii
Pin-Priority: 900

Package: *
Pin: release a=experimental
Pin-Priority: 500

iirc, experimental ONLY downloads the packages you request.  Will try to find a reference for that.  I think maybe it was on dng mixed in with all the silliness of the last few days.

Offline

#16 2017-07-20 01:26:07

PeteGozz
Member
From: Woodside South Australia
Registered: 2017-06-21
Posts: 59

Re: Using d1h (a.k.a. "Devuan packaging made easier")

ghaverla wrote:

Some Debian packages have man pages written by Debian maintainers, as the parent package had no man page.  This I believe happens with some Perl modules in Debian packaging.  It might make sense to replace those kinds of man pages.  But, it is easy enough to have POD in Perl code to produce a man page.  The source of that man page, is the product of the original author(s), and I don't think should be replaced.  To provide extra documentation for those packages, would be something added by Devuan.  So having some guidance as to what to call it, and where to place it in the filesystem would be useful.

2Cents::

/usr/share/doc/PKGNAME/
   
internally.deb something approximating::
      debian/PACKAGE.docs 
      debian/PACKAGE.example

though::

In some cases it is totally appropriate to add extra man pages
and ref them from the footer of the original.

e.g.
    exim4 has a page that captures its many config files and extensions

In others (depending on how you "call" the program) by extension.

e.g
    git  has a man page for  many (sub) command

However python and perl have foo-doc commands.
(as YOUR probably aware)

So maybe they can be made spit man pages ?
As they already have a doc formatting system ... docutils > rst >
so in python like worlds ::
rst2man - generate unix manpages from reStructured text
(which is my personal favourite)

Given perls awesome legendary text handling ....


Own your own tools.
Break them your way.

Offline

#17 2017-07-20 13:48:11

PeteGozz
Member
From: Woodside South Australia
Registered: 2017-06-21
Posts: 59

Re: Using d1h (a.k.a. "Devuan packaging made easier")

evilham wrote:

Since installing d1h requires adding experimental to sources.list, there should be a step about pkg pinning, something like:

In order not to upgrade your whole system to experimental, add a file /etc/apt/preferences.d/stay-on-ascii
with following content:

Package: *
Pin: release a=ascii
Pin-Priority: 900

Package: *
Pin: release a=experimental
Pin-Priority: 500

Its not a bad idea.
dist-upgrade _will_ go there if it needs to.
(guess it depends what you have from experimental)

Doesn't seem to break anything.

Wouldn't hurt to check your default release (if any) in /etc/apt/apt.conf.d/*
(even when set to jessie I was getting stuff auto included from experimental)


Own your own tools.
Break them your way.

Offline

#18 2017-07-22 01:22:09

sherdlu
Member
Registered: 2017-07-10
Posts: 1

Re: Using d1h (a.k.a. "Devuan packaging made easier")

I found the article very usefull.  For my suitation (starting from an existing devuan project) I used some alternate and/or additional steps.  I am starting partway down the d1h workflow.

IMPORT REPOSITORY

* Make a directory to work in
I am on the 8th attempt.  smarter persons may do it in less.
 

  $ cd
  $ mkdir -p Devuan/reportbug8
  $ cd Devuan/reportbug8

Get a copy of the repository
 

$ gbp clone https://git.devuan.org/devuan-packages/reportbug.git

or, equivalently:
 

$ git clone https://git.devuan.org/devuan-packages/reportbug.git reportbug

Then, dive down into the directory which contains the repository
 

$ cd reportbug
PREPARE

KatolaZ has suggested I put my repo in 'suites/ascii-proposed'.

$ d1h prepare ascii-proposed
SWITCH TO THE NEW BRANCH
$ d1h switch ascii-proposed
RESET --HARD

Here I had a problem.  My system is at version 'jessie'.  Apt-cache policy reportbug shows:

reportbug:
  Installed: 6.6.3+devuan1.3
  Candidate: 6.6.3+devuan1.3
  Version table:
*** 6.6.3+devuan1.3 0
        500 http://ca.mirror.devuan.org/merged/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status

But I want to continue on from the version in ascii.

The solution is/was:   take a look at ascii to find a version number or tag

$ d1h switch ascii

'gitk' shows the head is tagged '7.1.6+devuan2.1', so switch back to the branch I will be using, and do the 'git reset':

$ d1h switch ascii-proposed
$ git reset --hard tags/7.1.6+devuan2.1

There is some clean-up required.  The 'git reset --hard' changes 'debian/gbp.conf'; so

$ (edit debian/gbp.conf; change it back to 'ascii-proposed')
TEST BUILD

In debian/control
  added myself to the 'Uploaders:' line.

In reportbug/__init__.py
  set a new version  (7.1.6+devuan2.2)

Commit all outstanding changes.

Update the changelog file:

$ d1h bump 7.1.6+devuan2.2

commit the modified changelog

Do the test build

$ d1h testbuild
Other notes

gpg key

You will need to have created a gpg key set

$ gpg --gen-key

Upload the public key

$ gpg --send-keys --keyserver hkp://keys.gnupg.net <key id, or fingerprint>

gitk
gitk is great.  fire it up as soon as you have changed to the repository directory.  Use it to start git-gui, git-gui makes 'commits' easy.

Offline

Board footer