The officially official Devuan Forum!

You are not logged in.

#1 2020-06-04 22:49:00

bimon
Member
Registered: 2019-09-09
Posts: 94  

Archiving Devuan repositories

I keep /var/cache/apt/archives on a mounted dedicated ZFS dataset like system/apt while having Devuan root on the system/rootfs, also doing regular snapshots and replicated ZFS backups of them to another backup server.

Anyway I would prefer to be able to get all available debs at a specific point of time rather than only debs I ever installed earlier.
So that if I need sometimes in the future a package I did not use earlier I would not need to try to find it somewhere on the Internet.

Did you ever try recently (in 2020) to find an unused earlier deb file say for a Wheezy Debian v7 especially without DVD sets and without Debian http://snapshot.debian.org/
Yes, Wheezy is sometimes still used in Virtual machines and works very well,
btw. its KVM host is powered by modern Devuan.

What if say 5 years later I will need some server package for a ASCII release, how to get it?

Does a single 4GB desktop Devuan DVD contain at least all Devuan specific (non Debian) debs ever released and available by apt-get at the time of ISO generation?

I refer to, e.g. for ASCII:
https://mirror.leaseweb.com/devuan/devu … _dvd-1.iso

Do any Devuan specific debs freed from systemD (not present in original Debian) exist beyond of that 4GB ISO only online by apt-get and not on that ISO?

Can your merging magic for Debian+Devuan repos  be deployed or installed locally?
Can I have my local http or ftp server which would merge together all available ISOs dynamically on demand like your own mirrors do?
Say 1x4GB Devuan ISO and 3x4GB Debian ISOs would produce a single point of complete Devuan packages set if mounted in some specific way may be with your http redirection installed locally and with a customized sources.list ?

My primary usage of Devuan is for servers though I use it on my Trinity desktop computer too.
For servers it is not a good idea to miss any server package from ISO, if something goes wrong with Devuan website or Internet connection or anything else we require to have all deb files (even those were not ever installed yet) from the distribution locally, it was an absolute MUST before systemD have not left us a choice. It is especially important for older releases. What happens to your online Jessie or ASCII repository when original Debian v8 and v9 are outdated and removed from online Debian repositories?
How to manage old servers which we do not want to upgrade for some reasons, we do not like an idea to always install the latest release, unavoidable upgrades are something like Windows 10 and systemD  itself being forcefully imposed on Debian users, it is a lack of freedom to be forced to always upgrade if it is not needed. Though I like current Beowulf release very much I may still need to maintain some older releases in virtual machines, but may be I will not like future Devuan release say in 3-5 releases later since today, hope it will not happen, but who knows?

Also we cannot wait to allow online Devuan mirrors to go back or our Internet connection to be recovered if something happened there, we may need an access to some earlier not installed yet deb file ASAP.

Another point, would not it be a good idea to provide Devuan server releases without any desktop components earlier than desktop capable? I do not need any modern KDE, Gnome or anything like it, I am happy with Trinity on ASCII, most likely I can survive with my current Trinity and ASCII for future 3-5 years or even more without upgrading my desktop, but on some servers I would prefer to have Devuan not much behind of current Debian stable. Would not it be easier (less development resources needed) to maintain special server edition of Devuan without graphics packages at all which could be released soon after corresponding Debian release?

Last edited by bimon (2020-06-05 06:39:01)

Offline

#2 2020-10-19 04:51:34

bimon
Member
Registered: 2019-09-09
Posts: 94  

Re: Archiving Devuan repositories

I guess aptly is the most convenient tool for the task?

Just: apt-get install aptly
And then:

aptly -architectures="amd64" mirror create devuan-amd64-ascii https://pkgmaster.devuan.org/merged/  ascii main contrib non-free
aptly mirror update devuan-amd64-ascii

then the same repeated also for the following repos: ascii-backports, ascii-proposed-updates, ascii-security, ascii-updates

Is it enough to have a good full and exhaustive offline archive of the Devuan ASCII 64bit?
I have found Devuan ASCII super rock solid and stable and would not like to upgrade it in a few years.
I would like to have all ASCII deb files available for me even after your next Chimaera release is ready, ASCII will become oldoldstable like Jessie is now.

I can use any needed new software in KVM virtual machines with Devuan Beowulf and Chimaera, Alpine Edge, etc. and would like to keep ASCII on the host and enjoy its stability.

And I wonder, what I get if downloading the whole:
https://pkgmaster.devuan.org/merged/pool/DEVUAN/
?
Does it have all Devuan specific packages which are not present in Debian?

Last edited by bimon (2020-10-19 05:44:33)

Offline

#3 2020-10-19 15:20:42

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,645  

Re: Archiving Devuan repositories

Try archive.devuan.org for jessie packages.

Use this in sources.list

deb http://archive.devuan.org/merged jessie main

Or go here if you want to download our forked packages. Note that pool is in /devuan but not /merged.
http://archive.devuan.org/devuan/pool/

I'm not sure if debian has archived jessie yet. Back at the end of June when it was supposed to be done, they delayed it.

I think this is a howto for setting up amprolla locally. It's in Spanish. I know for a fact that aitor is running amprolla to merge his own packages in gnuinos.
https://dev1galaxy.org/viewtopic.php?id=2405

HTH

Offline

#4 2020-10-20 23:40:55

aitor
Member
From: basque country
Registered: 2016-12-03
Posts: 42  
Website

Re: Archiving Devuan repositories

fsmithred wrote:

I think this is a howto for setting up amprolla locally. It's in Spanish. I know for a fact that aitor is running amprolla to merge his own packages in gnuinos.
https://dev1galaxy.org/viewtopic.php?id=2405

HTH

This howto needs an update.


If you work systematically, things will come by itself (Lev D. Landau)

Offline

#5 2020-10-22 00:21:47

bimon
Member
Registered: 2019-09-09
Posts: 94  

Re: Archiving Devuan repositories

For now I prefer to try aptly tool for repo mirroring, it looks very convenient to me.

Why architecture "all" (common for all CPUs, generally just scripts) mentioned on the page:
https://www.debian.org/mirror/size
is not available?

aptly -architectures="all" mirror create devuan-all-ascii https://pkgmaster.devuan.org/merged/  ascii main contrib non-free

Downloading https://pkgmaster.devuan.org/merged/dists/ascii/InRelease...
gpgv: Signature made Wed 21 Oct 2020 07:59:30 PM EDT using RSA key ID 61FC752C
gpgv: Good signature from "Devuan Repository (Amprolla3 on Nemesis) <repository@devuan.org>"
ERROR: unable to fetch mirror: architecture all not available in repo [devuan-all-ascii]: https://pkgmaster.devuan.org/merged/ ascii, use -force-architectures to override

An example of a package:
https://packages.debian.org/buster/all/firehol/download

Last edited by bimon (2020-10-22 00:29:50)

Offline

#6 2020-10-22 12:21:10

aitor
Member
From: basque country
Registered: 2016-12-03
Posts: 42  
Website

Re: Archiving Devuan repositories

If you don't specify -force-architectures, then aptly will treat Multi-Arch packages including them in both devuan-i386-ascii and devuan-amd64-ascii at the same time, I guess. The following should work:

aptly -force-architectures -architectures="all" mirror create devuan-all-ascii https://archive.devuan.org/merged/ ascii main
aptly mirror update devuan-all-ascii

If you work systematically, things will come by itself (Lev D. Landau)

Offline

#7 2020-10-22 19:04:29

larsH
Member
Registered: 2020-05-05
Posts: 143  

Re: Archiving Devuan repositories

Hi

All old debian releases are here http://archive.debian.org/debian/dists/ and will probably be there for a long time to come

Have a nice day
Lars H

Offline

#8 2020-10-23 14:55:59

PedroReina
Member
From: Madrid, Spain
Registered: 2019-01-13
Posts: 157  
Website

Re: Archiving Devuan repositories

larsH wrote:

and will probably be there for a long time to come

I read time ago that removing binaries from archive was planned, but I saw just now binaries even for the very first version. Do you know something about the removing of binaries? Thank you.

Offline

#9 2020-10-23 20:53:09

larsH
Member
Registered: 2020-05-05
Posts: 143  

Re: Archiving Devuan repositories

Hi

Yes, they are keeping a central quite complete archive. I am currently not following Debian as close as idid when I used it :-) , but what I have heard was cleaning up of all the repository and download mirrors, so they did not have these old unsupported releases laying around.

Have a nice day
Lars H

Offline

#10 2020-10-23 23:45:23

bimon
Member
Registered: 2019-09-09
Posts: 94  

Re: Archiving Devuan repositories

Hello,

Earlier I specified the components explicitelly like:

Components="main contrib non-free"; 

Now I have made following new config for aptly which includes all components by default without specifying them explicitly :

#aptly mirror show devuan-ascii                                                                                                                                                                    

Name: devuan-ascii                                                                                                                                                                                                                          
Archive Root URL: https://pkgmaster.devuan.org/merged/                                                                                                                                                                                           
Distribution: ascii                                                                                                                                                                                                                              
Components: main, contrib, non-free, raspi, beaglebone, droid4, n900, n950, n9, sunxi, exynos                                                                                                                                                     
Architectures: i386, amd64, armhf                                                                                                                                                                                                                 
Download Sources: yes                                                                                                                                                                                                                             
Download .udebs: no                                                                                                                                                                                                                               
Last update: never                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
Information from release file:                                                                                                                                                                                                                                                 
Architectures: amd64 arm64 armel armhf i386 ppc64el                                                                                                                                                                                                                            
Codename: ascii                                                                                                                                                                                                                                                                
Components: main contrib non-free raspi beaglebone droid4 n900 n950 n9 sunxi exynos                                                                                                                                                                                            
Date: Fri, 23 Oct 2020 03:30:13 UTC                                                                                                                                                                                                                                            
Label: Devuan                                                                                                                                                                                                                                                                  
Origin: Devuan                                                                                                                                                                                                                                                                 
Suite: oldstable                                                                                                                                                                                                                                                               
Valid-Until: Fri, 30 Oct 2020 03:27:17 UTC
Version: 2.1

And after running aptly update I get an error about missing components except general ones: main contrib non-free:

Downloading https://pkgmaster.devuan.org/merged/dists/ascii/InRelease...
gpgv: Signature made Thu 22 Oct 2020 11:30:13 PM EDT using RSA key ID 61FC752C
gpgv: Good signature from "Devuan Repository (Amprolla3 on Nemesis) <repository@devuan.org>"
Downloading & parsing package files...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/main/source/Sources.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/contrib/source/Sources.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-i386/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-amd64/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/binary-armhf/Packages.gz...
Downloading https://pkgmaster.devuan.org/merged/dists/ascii/non-free/source/Sources.gz...
ERROR: unable to update: no candidates for https://pkgmaster.devuan.org/merged/dists/ascii/raspi/binary-i386/Packages found

How shall this be handled? Why there are more board specific components and they do not work in aptly anyway?

What is the purpose of the components: raspi beaglebone droid4 n900 n950 n9 sunxi exynos ?
What do I miss if not including them into my archive?

Last edited by bimon (2020-10-23 23:56:04)

Offline

#11 2020-10-24 08:27:31

PedroReina
Member
From: Madrid, Spain
Registered: 2019-01-13
Posts: 157  
Website

Re: Archiving Devuan repositories

larsH wrote:

Yes, they are keeping a central quite complete archive.

Thank you. These are very good news. Very long term support can be very useful to people running old systems. Sure they have security concerns, but any sysadmin should know the risks. My point is that beeing able to apt install more packages in an old box is very convinient.

Offline

#12 2020-10-25 11:47:42

aitor
Member
From: basque country
Registered: 2016-12-03
Posts: 42  
Website

Re: Archiving Devuan repositories

Now I have made following new config for aptly which includes all components by default without specifying them explicitly :

#aptly mirror show devuan-ascii                                                                                                                                                                    

Name: devuan-ascii                                                                                                                                                                                                                          
Archive Root URL: https://pkgmaster.devuan.org/merged/                                                                                                                                                                                           
Distribution: ascii                                                                                                                                                                                                                              
Components: main, contrib, non-free, raspi, beaglebone, droid4, n900, n950, n9, sunxi, exynos                                                                                                                                                     
Architectures: i386, amd64, armhf                                                                                                                                                                                                                 
Download Sources: yes                                                                                                                                                                                                                             
Download .udebs: no                                                                                                                                                                                                                               
Last update: never                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
Information from release file:                                                                                                                                                                                                                                                 
Architectures: amd64 arm64 armel armhf i386 ppc64el                                                                                                                                                                                                                            
Codename: ascii                                                                                                                                                                                                                                                                
Components: main contrib non-free raspi beaglebone droid4 n900 n950 n9 sunxi exynos                                                                                                                                                                                            
Date: Fri, 23 Oct 2020 03:30:13 UTC                                                                                                                                                                                                                                            
Label: Devuan                                                                                                                                                                                                                                                                  
Origin: Devuan                                                                                                                                                                                                                                                                 
Suite: oldstable                                                                                                                                                                                                                                                               
Valid-Until: Fri, 30 Oct 2020 03:27:17 UTC
Version: 2.1

Raspi, Beaglebone, Sunxi (Allwinner) and so on... aren't Components, but low-power computers with ARM processors, each of them using its own linux kernel flavor. Am i wrong at this point?

On the other hand, isn't the config file of aptly defined in JSON format?


If you work systematically, things will come by itself (Lev D. Landau)

Offline

Board footer