sources.list confusion.

golinux · 2020-01-25 01:42:10

Hear ye, hear ye intrepid Devuan users!!

Please use deb.devuan.org in your sources list with http NOT https. It is a round-robin of all available Devuan package mirrors.

Country Codes do not work properly so do not use them.

We will let you know if that changes.

Please use the release name - jessie, ascii, beowulf - rather than the suite name - oldstable, stable, testing - for reasons explained in the Release Information.

Finally . . . all Devuan mirrors pull from pkgmaster.devuan.org so please do not add extra load by using it in your sources.list. Use the round-robin - deb.devuan.org - instead.

golinux

mmaglis · 2020-01-25 19:41:09

I get an error when I switch to deb.devuan.org. It seems besides country codes not working properly so do some of the mirrors...
That was the reason I switched back to pkgmaster.devuan.org at some point in the past.

My sources.list entries:

# Devuan repositories
deb https://deb.devuan.org/merged ascii main
deb https://deb.devuan.org/merged ascii-security main
deb https://deb.devuan.org/merged ascii-updates main
#deb https://deb.devuan.org/merged ascii-backports main

# Devuan sources
deb-src https://deb.devuan.org/merged ascii main
deb-src https://deb.devuan.org/merged ascii-security main
deb-src https://deb.devuan.org/merged ascii-updates main
#deb-src https://deb.devuan.org/merged ascii-backports main

Result of apt-get update:

Ign:28 https://deb.devuan.org/merged ascii-security i386 Contents (deb)
Err:29 https://deb.devuan.org/merged ascii-updates/main Sources
  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
Ign:30 https://deb.devuan.org/merged ascii-updates/main i386 Packages
Ign:31 https://deb.devuan.org/merged ascii-updates/main all Packages
Ign:32 https://deb.devuan.org/merged ascii-updates/main amd64 Packages
Ign:33 https://deb.devuan.org/merged ascii-updates/main Translation-en
Ign:34 https://deb.devuan.org/merged ascii-updates/main i386 Contents (deb)
Ign:35 https://deb.devuan.org/merged ascii-updates/main all Contents (deb)
Ign:36 https://deb.devuan.org/merged ascii-updates/main amd64 Contents (deb)
Ign:37 https://deb.devuan.org/merged ascii-updates i386 Contents (deb)
Ign:38 https://deb.devuan.org/merged ascii-updates amd64 Contents (deb)
Ign:39 https://deb.devuan.org/merged ascii-updates all Contents (deb)
Reading package lists... Done
W: The repository 'https://deb.devuan.org/merged ascii Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: The repository 'https://deb.devuan.org/merged ascii-security Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: The repository 'https://deb.devuan.org/merged ascii-updates Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii-security/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii-updates/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Some index files failed to download. They have been ignored, or old ones used instead.

golinux · 2020-01-25 21:17:48

IIRC the round-robin uses http not https.

rolfie · 2020-01-25 21:42:03

I can confirm that https does not work. There is at least one thread somewhere in this forum about this topic.

A simple http://deb.devuan.org/merged without country code works fine (ignore the Devuan web page).

rolfie

golinux · 2020-01-25 22:51:55

rolfie wrote:

(ignore the Devuan web page).

@rolfie . . . please explain what needs to be corrected. I updated the pages before I posted this thread. Maybe I missed something. I should probably add a specific note about the round-robin requiring http here and on the website.

rolfie · 2020-01-26 09:11:21

@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.

There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.

rolfie

Last edited by rolfie (2020-01-26 09:12:50)

golinux · 2020-01-26 09:56:31

rolfie wrote:

@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.
There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.

That section is a partial that is called on both pages. Last I looked, both pages were picking it up. Try refreshing the page if you're not seeing it.

Jens · 2020-01-26 10:08:17

* If you used https:// before, please change to http://.

mmaglis · 2020-01-26 10:27:07

Jens wrote:

* If you used https:// before, please change to http://.

Noted. I changed my sources.list from HTTPS to HTTP and I confirm package updates work correctly now with deb.devuan.org URL.

This, to a degree, is not secure/private.
Is there any plan to ever make HTTPS round-robin work with all mirrors?

Why is pkgmaster.devuan.org allowing connections from non-mirrors (e.g. clients)?
Isn't there or shouldn't be a kind of mirror registration process that would additionally check the quality of mirror configuration (e.g. certificate)?

Jens · 2020-01-26 11:07:04

mmaglis wrote:

Jens wrote:
* If you used https:// before, please change to http://.
Noted. I changed my sources.list from HTTPS to HTTP and I confirm package updates work correctly now with deb.devuan.org URL.
This, to a degree, is not secure/private.
Is there any plan to ever make HTTPS round-robin work with all mirrors?
Why is pkgmaster.devuan.org allowing connections from non-mirrors (e.g. clients)?
Isn't there or shouldn't be a kind of mirror registration process that would additionally check the quality of mirror configuration (e.g. certificate)?

In doubt: there are too few developers to to do the work. Though IMHO you are right, https is better.

fsmithred · 2020-01-26 12:24:15

We don't have control over the mirrors' choice of providing https or not.

If you want to use https, use a mirror in your sources.list that provides https. There's a list of them here -
http://pkgmaster.devuan.org/mirror_list.txt

If you use https, your ISP won't be able to see what you're installing. Package security is provided by gpg signing keys.

Edit: Corrected links: I posted this link first. This is the list of mirrors for downloading isos, not for getting packages.
https://devuan.org/get-devuan

Head_on_a_Stick · 2020-01-26 12:52:59

Note that the apt-transport-https package is needed to take advantage of https sources.

mmaglis · 2020-01-26 15:02:13

fsmithred wrote:
We don't have control over the mirrors' choice of providing https or not.

I think in this day and age HTTPS is a must.
I am suggesting the Devuan project to restrict the round-robin to an "official" list of HTTPS enabled, properly configured and quality checked mirrors and leave all others in an unofficial list.
Is this feasible?

Last edited by mmaglis (2020-01-28 10:20:02)

HevyDevy · 2020-01-26 15:18:43

I think openbsd has nailed this in regards to package security.

https://www.openbsd.org/papers/bsdcan-signify.html

Last edited by HevyDevy (2020-01-26 15:20:09)

rolfie · 2020-01-26 16:01:54

Head_on_a_Stick wrote:

Note that the apt-transport-https package is needed to take advantage of https sources.

Tried this and it didn't work. I think I discussed this in a thread a while ago, but I couldn't find it.

golinux wrote:

rolfie wrote:
@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.
There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.
That section is a partial that is called on both pages. Last I looked, both pages were picking it up. Try refreshing the page if you're not seeing it.

Called that page pointing to /etc/apt/sources.list with my up-to-date FF-ESR and Chromium, still get the hint to the country codes.

rolfie

golinux · 2020-01-26 20:45:55

rolfie wrote:

Called that page pointing to /etc/apt/sources.list with my up-to-date FF-ESR and Chromium, still get the hint to the country codes.

You are correct rolfie. Revisions are now in place at https://devuan.org/os/etc/apt/sources.list

These days, I am seriously doubting my mental competence and am hoping to pass on some of my responsibilities to the next generation sooner rather than later. Will post about that project separately. Hopefully some volunteers will step forward to take up the slack.

tlathm · 2020-01-27 16:57:26

rolfie wrote:

Head_on_a_Stick wrote:
Note that the apt-transport-https package is needed to take advantage of https sources.
Tried this and it didn't work. I think I discussed this in a thread a while ago, but I couldn't find it.

Maybe I'm off base here, but this may mean that apt-transport-https may allow use of https sources that come from the round robin, but NOT https used directly in sources.list(??). That was sort of how I'd interpreted it.

Tom

fsmithred · 2020-01-27 18:49:13

Here's the link for the list of https package mirrors. I gave the link for iso mirrors yesterday by mistake. That post above has been corrected.

http://pkgmaster.devuan.org/mirror_list.txt

If you use https with deb.devuan.org and you are lucky enough to get directed to a mirror that provides https, it should work. But if you get directed to a mirror that only uses http, you will get errors. To use https, your sources.list should have sources that provide https.

Note that you need to add (append) "/merged" to the end of the Base URL given for the mirrors, even if they end in /devuan. For example:
BaseURL: sledjhamr.org/devuan
looks like this in sources.list
deb https://sledjhamr.org/devuan/merged ascii main

mmaglis · 2020-01-28 10:29:09

Jens wrote:
In doubt: there are too few developers to do the work. Though IMHO you are right, https is better.

fsmithred wrote:
We don't have control over the mirrors' choice of providing https or not.

mmaglis wrote:
I think in this day and age HTTPS is a must.
I am suggesting the Devuan project to restrict the round-robin to an "official" list of HTTPS enabled, properly configured and quality checked mirrors and leave all others in an unofficial list.
Is this feasible?

So why not control the mirrors served by round-robin deb.devuan.org and demand HTTPS?
Or alternatively have two round-robins one serving HTTPS only and the other HTTP only.
Is this really a lot of work and not currently feasible?
Are there additional reasons?

Jens · 2020-01-28 11:46:31

mmaglis wrote:

So why not control the mirrors served by round-robin deb.devuan.org and demand HTTPS?
Or alternatively have two round-robins one serving HTTPS only and the other HTTP only.

Instead of deb.devuan.org: debs.devuan.org or deb-ssl. or deb-https.

Then every server needs the same certificate for deb-ssl.devuan.org or you get warnings/error about wrong certs.

Could there be a devuan (web/deb) server with ssl that provides for every package request a "Content moved temporary to https://dev.otherserver.org"?

Every server needs the same content, the same package versions and the same package list. That sounds more like a https proxy. Would it be less work for the devuan server to get just the https request for a package/packagelist, answer with a moved temporarily and the proxy answers with the data or requests the data from the devuan server and answers with the data.

mmaglis · 2020-01-28 12:22:58

Jens wrote:

That sounds more like a https proxy.

This solution implies that:

the proxy to mirror communication may be unencrypted.
the proxy then becomes the bottleneck

To sum up, the problems I want to solve at the same time are:

use a mirror round-robin to avoid load on pkgmaster or specific mirrors
use HTTPS end-to-end for security and privacy

Current Devuan infrastructure does not allow this.
Any work-around mentioned so far solves either one or the other of my above problems.

If there is some low cost idea of how to achieve both, I can contribute to the extent of my time and abilities.
Thank you for your contributions anyway.

golinux · 2020-01-28 16:39:07

Talk is easy and cheap. Perceived problems do not magically solve themselves.

Study amprolla3. When you understand it thoroughly, suggest a solution for consideration.

mmaglis · 2020-01-28 21:41:28

golinux wrote:

Talk is easy and cheap. Perceived problems do not magically solve themselves.
Study amprolla3. When you understand it thoroughly, suggest a solution for consideration.

I found the information in this file (http://pkgmaster.devuan.org/devuan_mirr … hrough.txt) more useful.

6) ENTERING THE deb.devuan.org DNS ROUND-ROBIN
We have put in place a DNS Round-Robin for the domain deb.devuan.org,
which points to all the available package mirrors which can serve
requests for the domanin "deb.devuan.org".
The easiest to have your mirror added to the Round-Robin is to add a
named VirtualHost to your web server conf to serve files for
deb.devuan.org. Sample configuration files for apache and nginx are
available under Section 6.1) and 6.2) below. Please amend them as
necessary, and incorporate them in your webserver configuration. In
particular, be careful in setting the document root and rewrite rules
appropriately.
*** IMPORTANT: THE DNS ROUND-ROBIN WILL NOT WORK FOR HTTPS ***
It is nevertheless recommended to keep your mirror reachable in
*both* ways, i.e., directly through your own URL and via deb.devuan.org,
since we will also advertise a list of existing mirrors with their
corresponding URL (and HTTPS is not supported through deb.devuan.org).
As usual, please shout if you need help with this configuration.
6.1) Sample apache conf for a deb.devuan.org named virtual host
<VirtualHost *:80>

ServerName deb.devuan.org
#### the root must be the folder containing "amprolla.txt"
DocumentRoot /home/mirror/devuan

RewriteEngine on

RewriteRule /merged/pool/DEVUAN/(.*) /devuan/pool/$1
RewriteRule /merged/pool/DEBIAN-SECURITY/(.*) http://deb.debian.org/debian-security/pool/$1
RewriteRule /merged/pool/DEBIAN/(.*) http://deb.debian.org/debian/pool/$1

</VirtualHost>

So it looks HTTPS is recommended for direct mirror connections only, but not supported by the DNS round-robin.
The only reasons, I can see, for this are:

reaching HTTP or FTP ONLY mirrors via https://deb.devuan.org
the mirror does not serve deb.devuan.org through HTTPS due to lack of proper certificate. An Alternative name of deb.devuan.org does not exist in the certificate.

Then you get an error like the one I got:

SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'

How does Debian work? Some info available if you reach https://deb.debian.org/
Compare to https://deb.devuan.org/

I may be wrong, but it looks to me the problem could be solved by either DNS reconfiguration or by mirror policy change (request suitable certificates?).
I do not see another technical reason for not supporting https with dns round-robin.
Perhaps someone from the authors of the above document can enlighten my ignorance :-)

golinux · 2020-01-28 22:08:56

A better place for that discussion would probably be the devuan-dev mail list.

mmaglis · 2020-01-29 20:06:00

golinux wrote:

A better place for that discussion would probably be the devuan-dev mail list.

For completeness of this thread it is worth to mention the below email from devuan-dev list back in 2017:
https://lists.dyne.org/lurker/message/2 … 9d.en.html

Evilham wrote:

We gave the TLS issue a lot of thought and it's not quite doable without
having some sort of control of the servers.

As I suspected, it looks more of an organisational topic rather than technical.
And I also suspect it will not be of higher priority now compared to then...

The officially official Devuan Forum!

#1 2020-01-25 01:42:10

sources.list confusion.

#2 2020-01-25 19:41:09

Re: sources.list confusion.

#3 2020-01-25 21:17:48

Re: sources.list confusion.

#4 2020-01-25 21:42:03

Re: sources.list confusion.

#5 2020-01-25 22:51:55

Re: sources.list confusion.

#6 2020-01-26 09:11:21

Re: sources.list confusion.

#7 2020-01-26 09:56:31

Re: sources.list confusion.

#8 2020-01-26 10:08:17

Re: sources.list confusion.

#9 2020-01-26 10:27:07

Re: sources.list confusion.

#10 2020-01-26 11:07:04

Re: sources.list confusion.

#11 2020-01-26 12:24:15

Re: sources.list confusion.

#12 2020-01-26 12:52:59

Re: sources.list confusion.

#13 2020-01-26 15:02:13

Re: sources.list confusion.

#14 2020-01-26 15:18:43

Re: sources.list confusion.

#15 2020-01-26 16:01:54

Re: sources.list confusion.

#16 2020-01-26 20:45:55

Re: sources.list confusion.

#17 2020-01-27 16:57:26

Re: sources.list confusion.

#18 2020-01-27 18:49:13

Re: sources.list confusion.

#19 2020-01-28 10:29:09

Re: sources.list confusion.

#20 2020-01-28 11:46:31

Re: sources.list confusion.

#21 2020-01-28 12:22:58

Re: sources.list confusion.

#22 2020-01-28 16:39:07

Re: sources.list confusion.

#23 2020-01-28 21:41:28

Re: sources.list confusion.

#24 2020-01-28 22:08:56

Re: sources.list confusion.

#25 2020-01-29 20:06:00

Re: sources.list confusion.

Board footer