Archiving Devuan repositories

bimon · 2020-06-04 22:49:00

I keep /var/cache/apt/archives on a mounted dedicated ZFS dataset like system/apt while having Devuan root on the system/rootfs, also doing regular snapshots and replicated ZFS backups of them to another backup server.

Anyway I would prefer to be able to get all available debs at a specific point of time rather than only debs I ever installed earlier.
So that if I need sometimes in the future a package I did not use earlier I would not need to try to find it somewhere on the Internet.

Did you ever try recently (in 2020) to find an unused earlier deb file say for a Wheezy Debian v7 especially without DVD sets and without Debian http://snapshot.debian.org/
Yes, Wheezy is sometimes still used in Virtual machines and works very well,
btw. its KVM host is powered by modern Devuan.

What if say 5 years later I will need some server package for a ASCII release, how to get it?

Does a single 4GB desktop Devuan DVD contain at least all Devuan specific (non Debian) debs ever released and available by apt-get at the time of ISO generation?

I refer to, e.g. for ASCII:
https://mirror.leaseweb.com/devuan/devu … _dvd-1.iso

Do any Devuan specific debs freed from systemD (not present in original Debian) exist beyond of that 4GB ISO only online by apt-get and not on that ISO?

Can your merging magic for Debian+Devuan repos be deployed or installed locally?
Can I have my local http or ftp server which would merge together all available ISOs dynamically on demand like your own mirrors do?
Say 1x4GB Devuan ISO and 3x4GB Debian ISOs would produce a single point of complete Devuan packages set if mounted in some specific way may be with your http redirection installed locally and with a customized sources.list ?

My primary usage of Devuan is for servers though I use it on my Trinity desktop computer too.
For servers it is not a good idea to miss any server package from ISO, if something goes wrong with Devuan website or Internet connection or anything else we require to have all deb files (even those were not ever installed yet) from the distribution locally, it was an absolute MUST before systemD have not left us a choice. It is especially important for older releases. What happens to your online Jessie or ASCII repository when original Debian v8 and v9 are outdated and removed from online Debian repositories?
How to manage old servers which we do not want to upgrade for some reasons, we do not like an idea to always install the latest release, unavoidable upgrades are something like Windows 10 and systemD itself being forcefully imposed on Debian users, it is a lack of freedom to be forced to always upgrade if it is not needed. Though I like current Beowulf release very much I may still need to maintain some older releases in virtual machines, but may be I will not like future Devuan release say in 3-5 releases later since today, hope it will not happen, but who knows?

Also we cannot wait to allow online Devuan mirrors to go back or our Internet connection to be recovered if something happened there, we may need an access to some earlier not installed yet deb file ASAP.

Another point, would not it be a good idea to provide Devuan server releases without any desktop components earlier than desktop capable? I do not need any modern KDE, Gnome or anything like it, I am happy with Trinity on ASCII, most likely I can survive with my current Trinity and ASCII for future 3-5 years or even more without upgrading my desktop, but on some servers I would prefer to have Devuan not much behind of current Debian stable. Would not it be easier (less development resources needed) to maintain special server edition of Devuan without graphics packages at all which could be released soon after corresponding Debian release?

Last edited by bimon (2020-06-05 06:39:01)

bimon · 2020-10-19 04:51:34

I guess aptly is the most convenient tool for the task?

Just: apt-get install aptly
And then:

aptly -architectures="amd64" mirror create devuan-amd64-ascii https://pkgmaster.devuan.org/merged/  ascii main contrib non-free
aptly mirror update devuan-amd64-ascii

then the same repeated also for the following repos: ascii-backports, ascii-proposed-updates, ascii-security, ascii-updates

Is it enough to have a good full and exhaustive offline archive of the Devuan ASCII 64bit?
I have found Devuan ASCII super rock solid and stable and would not like to upgrade it in a few years.
I would like to have all ASCII deb files available for me even after your next Chimaera release is ready, ASCII will become oldoldstable like Jessie is now.

I can use any needed new software in KVM virtual machines with Devuan Beowulf and Chimaera, Alpine Edge, etc. and would like to keep ASCII on the host and enjoy its stability.

And I wonder, what I get if downloading the whole:
https://pkgmaster.devuan.org/merged/pool/DEVUAN/
?
Does it have all Devuan specific packages which are not present in Debian?

Last edited by bimon (2020-10-19 05:44:33)

fsmithred · 2020-10-19 15:20:42

Try archive.devuan.org for jessie packages.

Use this in sources.list

deb http://archive.devuan.org/merged jessie main

Or go here if you want to download our forked packages. Note that pool is in /devuan but not /merged.
http://archive.devuan.org/devuan/pool/

I'm not sure if debian has archived jessie yet. Back at the end of June when it was supposed to be done, they delayed it.

I think this is a howto for setting up amprolla locally. It's in Spanish. I know for a fact that aitor is running amprolla to merge his own packages in gnuinos.
https://dev1galaxy.org/viewtopic.php?id=2405

HTH

aitor · 2020-10-20 23:40:55

fsmithred wrote:

I think this is a howto for setting up amprolla locally. It's in Spanish. I know for a fact that aitor is running amprolla to merge his own packages in gnuinos.
https://dev1galaxy.org/viewtopic.php?id=2405
HTH

This howto needs an update.

bimon · 2020-10-22 00:21:47

For now I prefer to try aptly tool for repo mirroring, it looks very convenient to me.

Why architecture "all" (common for all CPUs, generally just scripts) mentioned on the page:
https://www.debian.org/mirror/size
is not available?

aptly -architectures="all" mirror create devuan-all-ascii https://pkgmaster.devuan.org/merged/  ascii main contrib non-free

Downloading https://pkgmaster.devuan.org/merged/dists/ascii/InRelease...
gpgv: Signature made Wed 21 Oct 2020 07:59:30 PM EDT using RSA key ID 61FC752C
gpgv: Good signature from "Devuan Repository (Amprolla3 on Nemesis) <repository@devuan.org>"
ERROR: unable to fetch mirror: architecture all not available in repo [devuan-all-ascii]: https://pkgmaster.devuan.org/merged/ ascii, use -force-architectures to override

An example of a package:
https://packages.debian.org/buster/all/firehol/download

Last edited by bimon (2020-10-22 00:29:50)

aitor · 2020-10-22 12:21:10

If you don't specify -force-architectures, then aptly will treat Multi-Arch packages including them in both devuan-i386-ascii and devuan-amd64-ascii at the same time, I guess. The following should work:

aptly -force-architectures -architectures="all" mirror create devuan-all-ascii https://archive.devuan.org/merged/ ascii main
aptly mirror update devuan-all-ascii

larsH · 2020-10-22 19:04:29

Hi

All old debian releases are here http://archive.debian.org/debian/dists/ and will probably be there for a long time to come

Have a nice day
Lars H

PedroReina · 2020-10-23 14:55:59

larsH wrote:

and will probably be there for a long time to come

I read time ago that removing binaries from archive was planned, but I saw just now binaries even for the very first version. Do you know something about the removing of binaries? Thank you.

larsH · 2020-10-23 20:53:09

Hi

Yes, they are keeping a central quite complete archive. I am currently not following Debian as close as idid when I used it :-) , but what I have heard was cleaning up of all the repository and download mirrors, so they did not have these old unsupported releases laying around.

Have a nice day
Lars H

bimon · 2020-10-23 23:45:23

Hello,

Earlier I specified the components explicitelly like:

Components="main contrib non-free";

Now I have made following new config for aptly which includes all components by default without specifying them explicitly :

#aptly mirror show devuan-ascii                                                                                                                                                                    

Name: devuan-ascii                                                                                                                                                                                                                          
Archive Root URL: https://pkgmaster.devuan.org/merged/                                                                                                                                                                                           
Distribution: ascii                                                                                                                                                                                                                              
Components: main, contrib, non-free, raspi, beaglebone, droid4, n900, n950, n9, sunxi, exynos                                                                                                                                                     
Architectures: i386, amd64, armhf                                                                                                                                                                                                                 
Download Sources: yes                                                                                                                                                                                                                             
Download .udebs: no                                                                                                                                                                                                                               
Last update: never                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
Information from release file:                                                                                                                                                                                                                                                 
Architectures: amd64 arm64 armel armhf i386 ppc64el                                                                                                                                                                                                                            
Codename: ascii                                                                                                                                                                                                                                                                
Components: main contrib non-free raspi beaglebone droid4 n900 n950 n9 sunxi exynos                                                                                                                                                                                            
Date: Fri, 23 Oct 2020 03:30:13 UTC                                                                                                                                                                                                                                            
Label: Devuan                                                                                                                                                                                                                                                                  
Origin: Devuan                                                                                                                                                                                                                                                                 
Suite: oldstable                                                                                                                                                                                                                                                               
Valid-Until: Fri, 30 Oct 2020 03:27:17 UTC
Version: 2.1

And after running aptly update I get an error about missing components except general ones: main contrib non-free:

Downloading https://pkgmaster.devuan.org/merged/dists/ascii/InRelease...
gpgv: Signature made Thu 22 Oct 2020 11:30:13 PM EDT using RSA key ID 61FC752C
gpgv: Good signature from "Devuan Repository (Amprolla3 on Nemesis) <repository@devuan.org>"
Downloading & parsing package files...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/source/Sources.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/source/Sources.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/source/Sources.gz...
ERROR: unable to update: no candidates for https://pkgmaster.devuan.org/merged/dists/ascii/raspi/binary-i386/Packages found

How shall this be handled? Why there are more board specific components and they do not work in aptly anyway?

What is the purpose of the components: raspi beaglebone droid4 n900 n950 n9 sunxi exynos ?
What do I miss if not including them into my archive?

Last edited by bimon (2020-10-23 23:56:04)

PedroReina · 2020-10-24 08:27:31

larsH wrote:

Yes, they are keeping a central quite complete archive.

Thank you. These are very good news. Very long term support can be very useful to people running old systems. Sure they have security concerns, but any sysadmin should know the risks. My point is that beeing able to apt install more packages in an old box is very convinient.

aitor · 2020-10-25 11:47:42

Now I have made following new config for aptly which includes all components by default without specifying them explicitly :

#aptly mirror show devuan-ascii                                                                                                                                                                    

Name: devuan-ascii                                                                                                                                                                                                                          
Archive Root URL: https://pkgmaster.devuan.org/merged/                                                                                                                                                                                           
Distribution: ascii                                                                                                                                                                                                                              
Components: main, contrib, non-free, raspi, beaglebone, droid4, n900, n950, n9, sunxi, exynos                                                                                                                                                     
Architectures: i386, amd64, armhf                                                                                                                                                                                                                 
Download Sources: yes                                                                                                                                                                                                                             
Download .udebs: no                                                                                                                                                                                                                               
Last update: never                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
Information from release file:                                                                                                                                                                                                                                                 
Architectures: amd64 arm64 armel armhf i386 ppc64el                                                                                                                                                                                                                            
Codename: ascii                                                                                                                                                                                                                                                                
Components: main contrib non-free raspi beaglebone droid4 n900 n950 n9 sunxi exynos                                                                                                                                                                                            
Date: Fri, 23 Oct 2020 03:30:13 UTC                                                                                                                                                                                                                                            
Label: Devuan                                                                                                                                                                                                                                                                  
Origin: Devuan                                                                                                                                                                                                                                                                 
Suite: oldstable                                                                                                                                                                                                                                                               
Valid-Until: Fri, 30 Oct 2020 03:27:17 UTC
Version: 2.1

Raspi, Beaglebone, Sunxi (Allwinner) and so on... aren't Components, but low-power computers with ARM processors, each of them using its own linux kernel flavor. Am i wrong at this point?

On the other hand, isn't the config file of aptly defined in JSON format?

Caluser2000 · 2021-06-28 04:26:22

fsmithred wrote:

Try archive.devuan.org for jessie packages.
Use this in sources.list
deb http://archive.devuan.org/merged jessie main
Or go here if you want to download our forked packages. Note that pool is in /devuan but not /merged.
http://archive.devuan.org/devuan/pool/
I'm not sure if debian has archived jessie yet. Back at the end of June when it was supposed to be done, they delayed it.
I think this is a howto for setting up amprolla locally. It's in Spanish. I know for a fact that aitor is running amprolla to merge his own packages in gnuinos.
https://dev1galaxy.org/viewtopic.php?id=2405
HTH

Thanks, That is good to know. I'm currently carrying out a wee project with Davuan Jessie with my old P200mmx box(I did my first foray into Linux on it over 17 years ago) It's going to open the eyes of the Linux naysayers on n X86 forum I frequent. Should be fun .At that moment synaptic/apt say the downloads are not autenticated. they install just fine thoiugh. Is there any way to eliminate this warning? I'm currently downloading 15gigs of game related stuf that will run on the old gal fine.

Thanks in advance.

Mark

The officially official Devuan Forum!

#1 2020-06-04 22:49:00

Archiving Devuan repositories

#2 2020-10-19 04:51:34

Re: Archiving Devuan repositories

#3 2020-10-19 15:20:42

Re: Archiving Devuan repositories

#4 2020-10-20 23:40:55

Re: Archiving Devuan repositories

#5 2020-10-22 00:21:47

Re: Archiving Devuan repositories

#6 2020-10-22 12:21:10

Re: Archiving Devuan repositories

#7 2020-10-22 19:04:29

Re: Archiving Devuan repositories

#8 2020-10-23 14:55:59

Re: Archiving Devuan repositories

#9 2020-10-23 20:53:09

Re: Archiving Devuan repositories

#10 2020-10-23 23:45:23

Re: Archiving Devuan repositories

#11 2020-10-24 08:27:31

Re: Archiving Devuan repositories

#12 2020-10-25 11:47:42

Re: Archiving Devuan repositories

#13 2021-06-28 04:26:22

Re: Archiving Devuan repositories

Board footer