The officially official Devuan Forum!

You are not logged in.

#1 2020-01-25 01:42:10

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

sources.list confusion.

Hear ye, hear ye intrepid Devuan users!!

Please use deb.devuan.org in your sources list with http NOT https.  It is a round-robin of all available Devuan package mirrors.

Country Codes do not work properly so do not use them.

We will let you know if that changes.

Please use the release name - jessie, ascii, beowulf - rather than the suite name - oldstable, stable, testing - for reasons explained in the Release Information.

Finally . . . all Devuan mirrors pull from pkgmaster.devuan.org so please do not add extra load by using it in your sources.list.  Use the round-robin - deb.devuan.org - instead.

golinux

Offline

#2 2020-01-25 19:41:09

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

I get an error when I switch to deb.devuan.org. It seems besides country codes not working properly so do some of the mirrors...
That was the reason I switched back to pkgmaster.devuan.org at some point in the past.

My sources.list entries:

# Devuan repositories
deb https://deb.devuan.org/merged ascii main
deb https://deb.devuan.org/merged ascii-security main
deb https://deb.devuan.org/merged ascii-updates main
#deb https://deb.devuan.org/merged ascii-backports main

# Devuan sources
deb-src https://deb.devuan.org/merged ascii main
deb-src https://deb.devuan.org/merged ascii-security main
deb-src https://deb.devuan.org/merged ascii-updates main
#deb-src https://deb.devuan.org/merged ascii-backports main

Result of apt-get update:

Ign:28 https://deb.devuan.org/merged ascii-security i386 Contents (deb)
Err:29 https://deb.devuan.org/merged ascii-updates/main Sources
  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
Ign:30 https://deb.devuan.org/merged ascii-updates/main i386 Packages
Ign:31 https://deb.devuan.org/merged ascii-updates/main all Packages
Ign:32 https://deb.devuan.org/merged ascii-updates/main amd64 Packages
Ign:33 https://deb.devuan.org/merged ascii-updates/main Translation-en
Ign:34 https://deb.devuan.org/merged ascii-updates/main i386 Contents (deb)
Ign:35 https://deb.devuan.org/merged ascii-updates/main all Contents (deb)
Ign:36 https://deb.devuan.org/merged ascii-updates/main amd64 Contents (deb)
Ign:37 https://deb.devuan.org/merged ascii-updates i386 Contents (deb)
Ign:38 https://deb.devuan.org/merged ascii-updates amd64 Contents (deb)
Ign:39 https://deb.devuan.org/merged ascii-updates all Contents (deb)
Reading package lists... Done
W: The repository 'https://deb.devuan.org/merged ascii Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: The repository 'https://deb.devuan.org/merged ascii-security Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: The repository 'https://deb.devuan.org/merged ascii-updates Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii-security/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Failed to fetch https://deb.devuan.org/merged/dists/ascii-updates/main/source/Sources  SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'
E: Some index files failed to download. They have been ignored, or old ones used instead.

Offline

#3 2020-01-25 21:17:48

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

IIRC the round-robin uses http not https.

Offline

#4 2020-01-25 21:42:03

rolfie
Member
Registered: 2017-11-25
Posts: 1,047  

Re: sources.list confusion.

I can confirm that https does not work. There is at least one thread somewhere in this forum about this topic.

A simple http://deb.devuan.org/merged without country code works fine (ignore the Devuan web page).

rolfie

Offline

#5 2020-01-25 22:51:55

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

rolfie wrote:

(ignore the Devuan web page).

@rolfie . . . please explain what needs to be corrected.  I updated the pages before I posted this thread.  Maybe I missed something.  I should probably add a specific note about the round-robin requiring http here and on the website.

Offline

#6 2020-01-26 09:11:21

rolfie
Member
Registered: 2017-11-25
Posts: 1,047  

Re: sources.list confusion.

@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.

There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.

rolfie

Last edited by rolfie (2020-01-26 09:12:50)

Offline

#7 2020-01-26 09:56:31

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

rolfie wrote:

@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.

There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.

That section is a partial that is called on both pages.  Last I looked, both pages were picking it up.  Try refreshing the page if you're not seeing it.

Offline

#8 2020-01-26 10:08:17

Jens
Member
Registered: 2018-06-10
Posts: 4  

Re: sources.list confusion.

* If you used https:// before, please change to http://.

Offline

#9 2020-01-26 10:27:07

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

Jens wrote:

* If you used https:// before, please change to http://.

Noted. I changed my sources.list from HTTPS to HTTP and I confirm package updates work correctly now with deb.devuan.org URL.

This, to a degree, is not secure/private.
Is there any plan to ever make HTTPS round-robin work with all mirrors?

Why is pkgmaster.devuan.org allowing connections from non-mirrors (e.g. clients)?
Isn't there or shouldn't be a kind of mirror registration process that would additionally check the quality of mirror configuration (e.g. certificate)?

Offline

#10 2020-01-26 11:07:04

Jens
Member
Registered: 2018-06-10
Posts: 4  

Re: sources.list confusion.

mmaglis wrote:
Jens wrote:

* If you used https:// before, please change to http://.

Noted. I changed my sources.list from HTTPS to HTTP and I confirm package updates work correctly now with deb.devuan.org URL.

This, to a degree, is not secure/private.
Is there any plan to ever make HTTPS round-robin work with all mirrors?

Why is pkgmaster.devuan.org allowing connections from non-mirrors (e.g. clients)?
Isn't there or shouldn't be a kind of mirror registration process that would additionally check the quality of mirror configuration (e.g. certificate)?

In doubt: there are too few developers to to do the work. Though IMHO you are right, https is better.

Offline

#11 2020-01-26 12:24:15

fsmithred
Administrator
Registered: 2016-11-25
Posts: 2,409  

Re: sources.list confusion.

We don't have control over the mirrors' choice of providing https or not.

If you want to use https, use a mirror in your sources.list that provides https. There's a list of them here -
http://pkgmaster.devuan.org/mirror_list.txt

If you use https, your ISP won't be able to see what you're installing. Package security is provided by gpg signing keys.

Edit: Corrected links: I posted this link first. This is the list of mirrors for downloading isos, not for getting packages.
https://devuan.org/get-devuan

Offline

#12 2020-01-26 12:52:59

Head_on_a_Stick
Member
From: London
Registered: 2019-03-24
Posts: 3,125  
Website

Re: sources.list confusion.

Note that the apt-transport-https package is needed to take advantage of https sources.


Brianna Ghey — Rest In Power

Offline

#13 2020-01-26 15:02:13

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

fsmithred wrote:
We don't have control over the mirrors' choice of providing https or not.

I think in this day and age HTTPS is a must.
I am suggesting the Devuan project to restrict the round-robin to an "official" list of HTTPS enabled, properly configured and quality checked mirrors and leave all others in an unofficial list.
Is this feasible?

Last edited by mmaglis (2020-01-28 10:20:02)

Offline

#14 2020-01-26 15:18:43

HevyDevy
Member
Registered: 2019-09-06
Posts: 358  

Re: sources.list confusion.

I think openbsd has nailed this in regards to package security.

https://www.openbsd.org/papers/bsdcan-signify.html

Last edited by HevyDevy (2020-01-26 15:20:09)

Offline

#15 2020-01-26 16:01:54

rolfie
Member
Registered: 2017-11-25
Posts: 1,047  

Re: sources.list confusion.

Head_on_a_Stick wrote:

Note that the apt-transport-https package is needed to take advantage of https sources.

Tried this and it didn't work. I think I discussed this in a thread a while ago, but I couldn't find it.

golinux wrote:
rolfie wrote:

@golinux: the current page https://devuan.org/os/ is corrected now, as you said. I looked at his page a few days ago, and it still had the hint to the country codes.

    There is a link to https://devuan.org/os/etc/apt/sources.list which still refers to country codes. I would patch that page too.

That section is a partial that is called on both pages.  Last I looked, both pages were picking it up.  Try refreshing the page if you're not seeing it.

Called that page pointing to /etc/apt/sources.list with my up-to-date FF-ESR and Chromium, still get the hint to the country codes.

rolfie

Offline

#16 2020-01-26 20:45:55

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

rolfie wrote:

Called that page pointing to /etc/apt/sources.list with my up-to-date FF-ESR and Chromium, still get the hint to the country codes.

You are correct rolfie. Revisions are now in place at  https://devuan.org/os/etc/apt/sources.list

These days, I am seriously doubting my mental competence and am hoping to pass on some of my responsibilities to the next generation sooner rather than later.  Will post about that project separately.  Hopefully some volunteers will step forward to take up the slack.

Offline

#17 2020-01-27 16:57:26

tlathm
Member
Registered: 2017-11-25
Posts: 78  

Re: sources.list confusion.

rolfie wrote:
Head_on_a_Stick wrote:

Note that the apt-transport-https package is needed to take advantage of https sources.

Tried this and it didn't work. I think I discussed this in a thread a while ago, but I couldn't find it.

Maybe I'm off base here, but this may mean that apt-transport-https may allow use of https sources that come from the round robin, but NOT https used directly in sources.list(??). That was sort of how I'd interpreted it.

Tom

Offline

#18 2020-01-27 18:49:13

fsmithred
Administrator
Registered: 2016-11-25
Posts: 2,409  

Re: sources.list confusion.

Here's the link for the list of https package mirrors. I gave the link for iso mirrors yesterday by mistake. That post above has been corrected.

http://pkgmaster.devuan.org/mirror_list.txt

If you use https with deb.devuan.org and you are lucky enough to get directed to a mirror that provides https, it should work. But if you get directed to a mirror that only uses http, you will get errors. To use https, your sources.list should have sources that provide https.

Note that you need to add (append) "/merged" to the end of the Base URL given for the mirrors, even if they end in /devuan. For example:
BaseURL:  sledjhamr.org/devuan
looks like this in sources.list
deb https://sledjhamr.org/devuan/merged ascii main

Offline

#19 2020-01-28 10:29:09

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

Jens wrote:

In doubt: there are too few developers to do the work. Though IMHO you are right, https is better.

fsmithred wrote:

We don't have control over the mirrors' choice of providing https or not.

mmaglis wrote:
I think in this day and age HTTPS is a must.
I am suggesting the Devuan project to restrict the round-robin to an "official" list of HTTPS enabled, properly configured and quality checked mirrors and leave all others in an unofficial list.
Is this feasible?

So why not control the mirrors served by round-robin deb.devuan.org and demand HTTPS?
Or alternatively have two round-robins one serving HTTPS only and the other HTTP only.
Is this really a lot of work and not currently feasible?
Are there additional reasons?

Offline

#20 2020-01-28 11:46:31

Jens
Member
Registered: 2018-06-10
Posts: 4  

Re: sources.list confusion.

mmaglis wrote:

So why not control the mirrors served by round-robin deb.devuan.org and demand HTTPS?
Or alternatively have two round-robins one serving HTTPS only and the other HTTP only.

Instead of deb.devuan.org: debs.devuan.org or deb-ssl. or deb-https.

Then every server needs the same certificate for deb-ssl.devuan.org or you get warnings/error about wrong certs.

Could there be a devuan (web/deb) server with ssl that provides for every package request a "Content moved temporary to https://dev.otherserver.org"?

Every server needs the same content, the same package versions and the same package list. That sounds more like a https proxy. Would it be less work for the devuan server to get just the https request for a package/packagelist, answer with a moved temporarily and the proxy answers with the data or requests the data from the devuan server and answers with the data.

Offline

#21 2020-01-28 12:22:58

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

Jens wrote:

That sounds more like a https proxy.

This solution implies that:

  • the proxy to mirror communication may be unencrypted.

  • the proxy then becomes the bottleneck

To sum up, the problems I want to solve at the same time are:

  • use a mirror round-robin to avoid load on pkgmaster or specific mirrors

  • use HTTPS end-to-end for security and privacy

Current Devuan infrastructure does not allow this.
Any work-around mentioned so far solves either one or the other of my above problems.

If there is some low cost idea of how to achieve both, I can contribute to the extent of my time and abilities.
Thank you for your contributions anyway.

Offline

#22 2020-01-28 16:39:07

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

Talk is easy and cheap.   Perceived problems do not magically solve themselves.

Study amprolla3.  When you understand it thoroughly, suggest a solution for consideration.

Offline

#23 2020-01-28 21:41:28

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

golinux wrote:

Talk is easy and cheap.   Perceived problems do not magically solve themselves.

Study amprolla3.  When you understand it thoroughly, suggest a solution for consideration.

I found the information in this file (http://pkgmaster.devuan.org/devuan_mirr … hrough.txt) more useful.

6) ENTERING THE deb.devuan.org DNS ROUND-ROBIN

We have put in place a DNS Round-Robin for the domain deb.devuan.org,
which points to all the available package mirrors which can serve
requests for the domanin "deb.devuan.org".

The easiest to have your mirror added to the Round-Robin is to add a
named VirtualHost to your web server conf to serve files for
deb.devuan.org. Sample configuration files for apache and nginx are
available under Section 6.1) and 6.2) below. Please amend them as
necessary, and incorporate them in your webserver configuration. In
particular, be careful in setting the document root and rewrite rules
appropriately.

  *** IMPORTANT: THE DNS ROUND-ROBIN WILL NOT WORK FOR HTTPS ***

It is nevertheless recommended to keep your mirror reachable in
*both* ways, i.e., directly through your own URL and via deb.devuan.org,
since we will also advertise a list of existing mirrors with their                                                                   
corresponding URL (and HTTPS is not supported through deb.devuan.org). 

As usual, please shout if you need help with this configuration.

6.1) Sample apache conf for a deb.devuan.org named virtual host

<VirtualHost *:80>                                                                                                                   
                                                                                                                                     
        ServerName deb.devuan.org                                                                                                     
#### the root must be the folder containing "amprolla.txt"                                                                           
        DocumentRoot /home/mirror/devuan                                                                                             
                                                                                                                                     
        RewriteEngine on                                                                                                             
                                                                                                                                     
        RewriteRule /merged/pool/DEVUAN/(.*) /devuan/pool/$1                                                                         
        RewriteRule /merged/pool/DEBIAN-SECURITY/(.*) http://deb.debian.org/debian-security/pool/$1                                   
        RewriteRule /merged/pool/DEBIAN/(.*) http://deb.debian.org/debian/pool/$1                                                     
                                                                                                                                     
</VirtualHost>

So it looks HTTPS is recommended for direct mirror connections only, but not supported by the DNS round-robin.
The only reasons, I can see, for this are:

  • reaching HTTP or FTP ONLY mirrors via https://deb.devuan.org

  • the mirror does not serve deb.devuan.org through HTTPS due to lack of proper certificate. An Alternative name of deb.devuan.org does not exist in the certificate.

Then you get an error like the one I got:

SSL: certificate subject name (ftp.fau.de) does not match target host name 'deb.devuan.org'

How does Debian work? Some info available if you reach https://deb.debian.org/
Compare to https://deb.devuan.org/

I may be wrong, but it looks to me the problem could be solved by either DNS reconfiguration or by mirror policy change (request suitable certificates?).
I do not see another technical reason for not supporting https with dns round-robin.
Perhaps someone from the authors of the above document can enlighten my ignorance :-)

Offline

#24 2020-01-28 22:08:56

golinux
Administrator
Registered: 2016-11-25
Posts: 3,137  

Re: sources.list confusion.

A better place for that discussion would probably be the devuan-dev mail list.

Offline

#25 2020-01-29 20:06:00

mmaglis
Member
From: Berlin - Germany
Registered: 2018-03-16
Posts: 32  

Re: sources.list confusion.

golinux wrote:

A better place for that discussion would probably be the devuan-dev mail list.

For completeness of this thread it is worth to mention the below email from devuan-dev list back in 2017:
https://lists.dyne.org/lurker/message/2 … 9d.en.html

Evilham wrote:

We gave the TLS issue a lot of thought and it's not quite doable without
having some sort of control of the servers.

As I suspected, it looks more of an organisational topic rather than technical.
And I also suspect it will not be of higher priority now compared to then...

Offline

Board footer