The officially official Devuan Forum!

You are not logged in.

#1 2018-06-09 12:19:46

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

(Un)Limited TTL for mirror Release files

Not sure this is the right forum for this particular topic, but here it goes.

First, some background info:
I work for a recycling company/thrift shop chain. Earlier this year I was added to the already running PC-refurbishment project. Officially this consists of 3 blokes, including yours truly, but one is a web designer and hasn't visited the premises housing the project since he'd introduced me, the other chap is a volunteer with ADD and an addiction who uses it as some sort of rehab. Meaning that effectively, I'm running the place.

Hardware is donated by private individuals but also (semi-) commercial enterprises as well as governmental dept's (including UN tribunals, which are/were located in or in the vicinity off my home town) and ranges from Lenovo Intel-based Core2 systems (sans hard-drive and RAM) to Dell P4's. I even have a P3 server (2U, SCSI-based) set aside as the drive is too small (70GB is not enough).

When I started out, Linux Mint was used to install Linux on whatever hardware was picked. Imagine the "joy" of systemd on a P4 :-\  We were lucky to install 2 systems a day!  A few weeks ago I made the decision to switch over to Devuan instead. Regrettably, for a novice the default DE is too spartan, so I wanted Mate instead. But that can't be installed from the install DVD (missing policykit packages) as well as quite a few machines can't boot from DVD at all. So I decided to create a local mirror. Fortunately I found enough network stuff to make it happen (switches, machines with sufficient space to host everything one needs for an offline mirror) as our location has no internet connection! Not having done this before it took a fair few days to fiddle everything and get a working system. Next step, obtain the packages from the Devuan mirrors. That went reasonably well, copying the lot over also worked. But after the first week, I got reports that the mirror was outdated and therefore would not be used to install stuff from. This was traced to a just-under-a-week TTL of the various Release files. That in turn meant I need to take the server home every weekend to update it and bring it back in on Monday. Cumbersome. Quite. So, how do I prevent these low TTL files messing things up as it's my intention to inform purchasers of these systems they need to update their PC's after connecting it to the web first thing. Ideally, I'd like to see no TTL at all and I'll update the repo on a monthly basis in order not to get too far out of sync with the main Devuan repo's.

(FYI: I'm running Linux exclusively since 2005, initially Debian after earlier testing of RedHat 8&9 and Suse, but when the last pre-systemd version became too old I switched to Funtoo in 2015. Regrettably Portage is not as good in conflict-resolving as the Debian package managers, so when I learned about Devuan I made the switch when Portage threw me one too many wobbly updating my system :-\ )

Thanks if you've read it this far wink

Offline

#2 2018-06-09 13:15:20

devuser
Member
Registered: 2018-04-30
Posts: 176  

Re: (Un)Limited TTL for mirror Release files

I am not entirely sure if that helps in your situation but adding

-o Acquire::Check-Valid-Until=false

to your apt-get update command would ignore the TTL i think.

Offline

#3 2018-06-10 09:30:27

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

Re: (Un)Limited TTL for mirror Release files

Thanks for that. I was hoping for an easier solution as I'd have to type in every command by hand on any newly installed system and I'm one very lazy bu99er tongue (I'm a big fan of tab-completion and bash history scrolling wink )

I'll try using an external hard-drive (USB3) this week for updating the mirror. Tried it before but didn't work as hoped, perhaps better luck this time now I have a better understanding of the processes and tools involved.

Offline

#4 2018-06-10 09:40:07

devuser
Member
Registered: 2018-04-30
Posts: 176  

Re: (Un)Limited TTL for mirror Release files

Dutch_Master wrote:

Thanks for that. I was hoping for an easier solution as I'd have to type in every command by hand on any newly installed system and I'm one very lazy bu99er tongue (I'm a big fan of tab-completion and bash history scrolling wink )

Well, you could always put it in apt's config or add an alias to your shell. You'd just have to remember to remove it before delivering the system but i agree somehow working around it on the mirror would be nicer but sadly i have no idea about how to do that.

Offline

#5 2018-06-10 22:38:22

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

Re: (Un)Limited TTL for mirror Release files

Hmmm, there's a thought. I could put it in a script (Bash), put that on the mirror, d/l it from there, execute on the local system then remove the script again have the script remove itself. wink

I'll have a brain-storm about that sometime tomorrow.

Thx again!

Offline

#6 2018-06-11 17:41:30

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

Re: (Un)Limited TTL for mirror Release files

Quick note to inform you lot that the mirror transfer via external hard drive works as planned/hoped for, so no more moving about of the actual server smile

I also gave some thought to (partially) automating the install itself, but it would be site-specific so not really suitable for inclusion into the Devuan archive. As it's a different subject it warrants its own thread in due cause.

Offline

#7 2018-11-15 14:14:10

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

Re: (Un)Limited TTL for mirror Release files

Coming back to this, it seems the TTL, which was about 6 days, has been cut considerably, meaning I should now update the mirror virtually essentially every time I need to work with it. That's obviously not happening, so basically this is dead. I understand the underlaying security stuff and all, but for off-line mirrors w/o any chance of a web connection, this is the killer. The message is clear: if you want to have a local mirror, it needs to be online 24/7. Which kindof defeats the purpose of having a local repository/mirror :-\

Offline

#8 2018-11-28 17:08:04

pcalvert
Member
Registered: 2017-05-15
Posts: 30  

Re: (Un)Limited TTL for mirror Release files

Dutch_Master wrote:

A few weeks ago I made the decision to switch over to Devuan instead. Regrettably, for a novice the default DE is too spartan, so I wanted Mate instead. But that can't be installed from the install DVD (missing policykit packages) as well as quite a few machines can't boot from DVD at all.

Why don't you connect a DVD drive via USB? You won't need to boot from it since you'd only be pulling packages from the DVD(s). And, yes, it can be done because that's what I used to do for an old laptop that wouldn't reliably boot from a DVD, even though it had a DVD-ROM drive. With this solution, installing the additional packages from the DVD(s) would need to be done post-installation.

Here's the command you would need to make this work:

# apt-cdrom -d <path-to-the-USB-DVD-drive>

After installation and set-up is complete, you would need to edit sources.list to remove the reference(s) to the DVD(s).

Phil


“Property is the fruit of labor; property is desirable; it is a positive good
in the world. That some should be rich shows that others may become
rich, and hence is just encouragement to industry and enterprise.”
— Abraham Lincoln

Offline

#9 2018-11-28 17:35:32

chris2be8
Member
Registered: 2018-08-11
Posts: 42  

Re: (Un)Limited TTL for mirror Release files

Could you set the clock back on the server? If it's only used as a mirror to install Linux from it doesn't matter if it has the date wrong.

Chris

Offline

#10 2018-11-28 19:17:47

GNUser
Member
Registered: 2017-03-16
Posts: 441  

Re: (Un)Limited TTL for mirror Release files

I also have a local mirror and don't give a hoot if it's "old". To stop apt's whining about the mirror's release file being out of date, create /etc/apt/apt.conf.d/10-unlimited-ttl on the machines that use your mirror. File should contain nothing but this:

Acquire::Check-Valid-Until "0";

Last edited by GNUser (2018-11-28 19:30:37)

Offline

#11 2018-12-01 07:34:22

KatolaZ
Member
Registered: 2017-03-11
Posts: 76  

Re: (Un)Limited TTL for mirror Release files

Hi All,

I think disablng Check-Valid-Until is not a particularly good idea, and I would not recommend it for anything more than a local mirror. Also, it is a good habit to check the signatures on Release and InRelease files (and the Valid-Until field) at least one after every rsync of your local mirror from a remote mirror, by either re-ebabling Acquire::Check-Valid-Until and giving an `apt-get update`,  or by manually verifying the signature with gpg. Well, to be honest apt would complain anyway if something looks dodgy...

My2Cents

KatolaZ

Offline

#12 2019-01-11 13:22:04

Dutch_Master
Member
Registered: 2018-05-31
Posts: 24  

Re: (Un)Limited TTL for mirror Release files

Many thanks guys, I had given up on this. ATM I'm having a holiday so it's not really urgent.

@pcalvert: this is exactly what I'm doing, installing a Devuan base system (no GUI!) from an install CD (I only use CD1 from the set) then edit sources.list to remove the cdrom entry and enable the local mirror. I know I can provide a path during the install phase, but as my co-worker(s) has/have little to no Linux experience, I'd like to keep it simple for him/them in the install guide I wrote. Which also teaches him/them a bit about Linux wink

As for the ttl-workaround: I update from the official Devuan mirror on my desktop, then copy the entire tree (rsync is an asset here!) to a USB harddrive. This harddrive is then used to transfer the tree onto the local offline mirror, again using rsync. So that's not a problem. The issue is with the clients on the offline network that need installing. Now the ttl of the mirror is set so low I essentially have to sync the mirror each time I go there. (it's a part-time section of my job, just 2 days/week and not consecutive)  Our IT guys have promised me I can get online access as soon as the fibre connection works, but that's delayed since at least Sept. last year roll

Offline

Board footer