How to mirror ascii?

Count_Cucaracha · 2017-07-17 19:34:43

I need to have a local copy of all packages that comprise ascii. What's the most efficient way (for the mirror server) to do so?
I only need packages for amd64 archictecture.

Should I just use lftp or rsync?

I am new to Devuan (and have not really done much with debian previously). I say this so you have an idea of who you're dealing with. But not new to linux.

Count_Cucaracha · 2017-07-21 00:23:50

Seems that I will need to mirror debian and then use amprolla to get an ascii mirror. Is there a plan by anyone anywhere to do a mirror like this?

golinux · 2017-07-21 01:16:29

From a link on the download page: https://files.devuan.org/MIRRORS.txt

yeti · 2017-07-21 05:48:46

Count_Cucaracha wrote:

I need to have a local copy of all packages that comprise ascii. What's the most efficient way (for the mirror server) to do so?

Maybe `debmirror` is the right tool for that packages job?

#!/bin/sh
sudo -u ruth debmirror \
        -a amd64 \
        -d ascii \
        -s main -s contrib -s non-free \
        --nosource \
        --method http -h auto.mirror.devuan.org -r merged \
        --progress --no-check-gpg \
        /wrk/mirrors/devuan

I was not patient enough to test it more than a few minutes.
"ruth" is my account for administrative work not needing "root" privileges because "ruth" sounds like "root" in german... ;-)

I don't know about a secret ascii images/installers download location.
A simple mirror for jessie's files could look like...

#!/bin/sh
##
## stdout is tty?
##
if [ -t 1 ]
then TTYOPTS=PSv
else TTYOPTS=
fi
##
## ...aaaand action!!!
##
sudo -u ruth rsync --delete -raX${TTYOPTS} files.devuan.org::devuan/. /wrk/mirrors/files.devuan.org/.

Last edited by yeti (2017-07-21 06:00:49)

Count_Cucaracha · 2017-07-21 23:34:33

golinux wrote:

From a link on the download page: https://files.devuan.org/MIRRORS.txt

Should I see ascii directory/files there?

rsync files.devuan.org::devuan/devuan_jessie/

drwxr-xr-x          4,096 2017/05/25 10:27:28 .
-rw-r--r--          1,453 2017/05/24 02:51:30 README.txt
-rw-r--r--         64,857 2017/05/25 10:27:28 devuan-devs.gpg
drwxr-xr-x          4,096 2017/05/21 17:08:39 desktop-live
drwxr-xr-x          4,096 2017/05/31 04:12:19 embedded
drwxr-xr-x          4,096 2017/05/23 23:39:52 installer-iso
drwxr-xr-x          4,096 2017/05/18 11:48:35 minimal-live
drwxr-xr-x          4,096 2017/05/23 04:22:36 virtual

rsync files.devuan.org::devuan/devuan_ascii
rsync: link_stat "/devuan_ascii" (in devuan) failed: No such file or directory (2)

Count_Cucaracha · 2017-07-21 23:37:44

yeti wrote:

Maybe `debmirror` is the right tool for that packages job?

That looks interesting and I also don't see ascii dirs/files anywhere so not sure where they are either.

golinux · 2017-07-21 23:50:31

Count_Cucaracha wrote:

I also don't see ascii dirs/files anywhere so not sure where they are either.

https://devuan.org/os/etc/apt/sources.list

Count_Cucaracha · 2017-07-22 00:36:21

golinux wrote:

https://devuan.org/os/etc/apt/sources.list

From that it looks like rsync will not work. I tried amprolla and it just gets 404s.

I'll keep chipping away at it and eventually figure it out.

mckaygerhard · 2017-07-24 15:46:37

how then i could mirror the repositoryes of devuan? please help to provide collaboration from the users to the project!

i tested and debmirror does not work! any suggestion!?

if users can do thing how uses can use the system? collaborate with project!?

yeti · 2017-07-24 17:43:10

mckaygerhard wrote:

i tested and debmirror does not work! any suggestion!?

Show the error.

mckaygerhard · 2017-07-24 18:38:26

with debmirro using debian : "too many redirects"

with debmirror using Devuan: 404 not found

yeti · 2017-07-24 19:06:59

mckaygerhard wrote:

with debmirro using debian : "too many redirects"

When?
Directly after starting or after running some time?
Here (currently on Debian/Jessie) it is slowly doing it's task but I never had the patience to test it more than half an hour or so and sure that is far away from completing...

Currently I have no Devuan system with enough free harddisk space to test it there...

mckaygerhard · 2017-07-24 19:09:44

that response was a week ago.. i'll test tonight and reportd feedback, i'l test more deeply and maybe need some tune upt setting parameters..

yeti · 2017-07-26 05:53:20

There are problems in the `release` files...

(yeti@kumari:13)/wrk/mirrors$ ./devuan.sh 
Mirroring to /wrk/mirrors/devuan from http://auto.mirror.devuan.org/merged/
Arches: arm64
Dists: ascii,ascii-backports,ascii-security
Sections: main,contrib,non-free
Pdiff mode: use
Not checking Release gpg signatures.
Will clean up after mirroring.
Attempting to get lock ...
Updating remote trace files (using rsync) ...
rsync: failed to connect to auto.mirror.devuan.org (2001:41d0:8:2c55::a2): Connection refused (111)
rsync: failed to connect to auto.mirror.devuan.org (46.105.191.77): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(128) [Receiver=3.1.1]
Warning: failed to use rsync to download extra files.
Getting meta files ...
[  0%] Getting: dists/ascii/Release...   #** GET http://auto.mirror.devuan.org/merged/dists/ascii/Release ==> 200 OK
ok
[  0%] Getting: dists/ascii/Release.gpg...       #** GET http://auto.mirror.devuan.org/merged/dists/ascii/Release.gpg ==> 200 OK
ok
[  0%] Getting: dists/ascii-backports/Release...         #** GET http://auto.mirror.devuan.org/merged/dists/ascii-backports/Release ==> 200 OK
ok
[  0%] Getting: dists/ascii-backports/Release.gpg...     #** GET http://auto.mirror.devuan.org/merged/dists/ascii-backports/Release.gpg ==> 200 OK (1s)
ok
Duplicate dist ascii.
WARNING: releasing 1 pending lock...

`http://ftp.debian.org/debian/dists/stretch-backports/Release`:

Origin: Debian Backports
Label: Debian Backports
Suite: stretch-backports
Codename: stretch-backports
Date: Wed, 26 Jul 2017 02:29:47 UTC
Valid-Until: Wed, 02 Aug 2017 02:29:47 UTC
NotAutomatic: yes
ButAutomaticUpgrades: yes
Acquire-By-Hash: yes
Architectures: amd64 arm64 armel armhf i386 mips mips64el mipsel ppc64el s390x
Components: main contrib non-free
Description: Backports for the Stretch distribution
MD5Sum:
 d41d8cd98f00b204e9800998ecf8427e        0 contrib/Contents-amd64

`http://auto.mirror.devuan.org/merged/dists/ascii-backports/Release`:

Origin: Devuan
Label: ascii-backports
Suite: ascii-backports
Version: 2.0.0
Codename: ascii
Date: Tue, 25 Jul 2017 20:31:36 UTC
Valid-Until: Tue, 01 Aug 2017 20:31:36 UTC
NotAutomatic: yes
ButAutomaticUpgrades: yes
Architectures: alpha amd64 arm64 armel armhf hppa i386 ia64 mips mipsel powerpc ppc64el s390x sparc
Components: main contrib non-freeMD5Sum:
  8a4b59cb71d78f27c2cd19431cccd2f8      808 non-free/binary-amd64/Packages.xz

There's a linefeed missing between `non-free` and `MD5Sum:` and the field `Codename:` probably should be `ascii-backports` (same for `ascii-proposed-updates` and `ascii-updates`).

Apart from these showstoppers for mirroring `ascii-backports`, `ascii-proposed-updates` and `ascii-updates` the remaining results look ok so far:

(yeti@kumari:13)/wrk/mirrors$ du -sh devuan/*/*
16M     devuan/dists/ascii
180K    devuan/dists/ascii-security
0       devuan/dists/testing
0       devuan/dists/testing-security
56G     devuan/pool/DEBIAN
20M     devuan/pool/DEVUAN
8,0K    devuan/project/trace

Currently I have no ASCII/arm64⁽¹⁾ system, so I cannot test installing from this mirror.

(1): Last weekend I reinstalled all my Devuan-PIs with Jessie because...
...eeeehhh... ***cough³*** <whisper> ASCII </whisper>

——————————

Edit@20170726-1042-UTC: Bonus question.

What's the difference between `auto.mirror.devuan.org` and `packages.devuan.org`?

——————————

Edit@20170726-1054-UTC: Jessie has the same `Codename:` problems.

E.g.: .../jessie-proposed-updates/Release

Last edited by yeti (2017-07-26 10:55:58)

mckaygerhard · 2017-07-26 12:07:22

well i was tired now i can confirm its not just me only!

auto mirror its a main entry to proxy' the packages for users in a automatic way.. umm maybe amprola must be the reason ?

Last edited by mckaygerhard (2017-07-26 12:07:36)

golinux · 2017-07-26 14:26:48

Since the devs don't have time to babysit this forum, I took the initiative to ask for you. Here's the answer:

<golinux> You might want to have a quick look at this: https://dev1galaxy.org/viewtopic.php?pid=3675#p3675
<Centurion_Dan> we don't provide rsync access to the packages at this stage.

You might want to read more about amprolla. It's just a search away if you would care to try:

https://git.devuan.org/devuan-infrastructure/amprolla

https://talk.devuan.org/t/amprolla-pack … -proxy/298

https://github.com/parazyd/amprolla

yeti · 2017-07-26 14:46:49

golinux wrote:

Since the devs don't have time to babysit this forum,

Babysit?
Someday I will come up with an answer to that and it will be with a noteworthy interest rate!
:-P

The "Codename:" problem still is unanswered.
Trying to debmirror a dist with wrong codename results in mirroring the dist matching that codename...
That can't be correct!

golinux · 2017-07-26 16:30:37

yeti wrote:

golinux wrote:
Since the devs don't have time to babysit this forum,
Babysit?
Someday I will come up with an answer to that and it will be with a noteworthy interest rate!
:-P

LOL! Looking forward to something 'noteworthy'!!

yeti wrote:

The "Codename:" problem still is unanswered.
Trying to debmirror a dist with wrong codename results in mirroring the dist matching that codename...
That can't be correct!

Have you filed a bug report? Or tried to contact a dev on #devuan or #devuan-dev? It is unlikely anyone who could fix this would see it here.

yeti · 2017-07-26 16:58:01

golinux wrote:

yeti wrote:
The "Codename:" problem still is unanswered.
Trying to debmirror a dist with wrong codename results in mirroring the dist matching that codename...
That can't be correct!
Have you filed a bug report? Or tried to contact a dev on #devuan or #devuan-dev?

Did the "or" part...

Count_Cucaracha · 2017-07-26 17:14:42

From what I can tell the entire project pivots on amprolla3. So mirroring and apt-file and probably lots of other stuff depends on amprolla3. And also inrfastructure setup/config.

In summary: infrastructure followed by amprolla3.

golinux · 2017-07-26 17:18:00

yeti wrote:

golinux wrote:
yeti wrote:
The "Codename:" problem still is unanswered.
Trying to debmirror a dist with wrong codename results in mirroring the dist matching that codename...
That can't be correct!
Have you filed a bug report? Or tried to contact a dev on #devuan or #devuan-dev?
Did the "or" part...

Yeah saw it in the backlog and just now connected it to you. I can't remember if it was actually solved. (Too much to remember . . .) A bug report is a more permanent way of recording it. irc logs quickly fade into oblivion.

mckaygerhard · 2017-07-26 21:08:58

yeti wrote:

golinux wrote:
Since the devs don't have time to babysit this forum,
Babysit?
Someday I will come up with an answer to that and it will be with a noteworthy interest rate!
:-P
The "Codename:" problem still is unanswered.
Trying to debmirror a dist with wrong codename results in mirroring the dist matching that codename...
That can't be correct!

again its debuan for the users or for "dev"'s only?

if documentation were put in right place, "babysit"'s dont ask so many to the "dev"'s ok, more respect to the users AGAIN!

golinux · 2017-07-27 00:36:15

mckaygerhard wrote:

again its debuan for the users or for "dev"'s only?

If you don't know the answer by now, you're not paying attention so I'm not going to repeat myself.

if documentation were put in right place . . .

This fragment is somewhat intelligible. Documentation is important. We all know that. It also needs to be written. It doesn't write itself. As to where is the right place? Well, that depends . . .

"babysit"'s dont ask so many to the "dev"'s ok, more respect to the users AGAIN!

I have no idea what you're trying to say here but I do know that you end by again wrongly judging and and insulting those of us working so hard on a project that you bitch about in nearly every post. Some gratitude for the tremendous effort it's taken to get this far would be appropriate. But then finding anything positive or remotely helpful in one of your posts is the last thing I'd ever expect. If you don't like it here and have such a low estimation of our efforts, please move on elsewhere . . .

mckaygerhard · 2017-07-27 01:01:58

golinux wrote:

This fragment is somewhat intelligible. Documentation is important. We all know that. It also needs to be written. It doesn't write itself. As to where is the right place? Well, that depends . . .

well i can made, but where? spell check can made by others.. and dont said "hard to track your write", documentation its missing and are need!

I have no idea what you're trying to say here but I do know that you end by again wrongly judging and and insulting those of us working so hard on a project that you bitch about in nearly every post. Some gratitude for the tremendous effort it's taken to get this far would be appropriate. But then finding anything positive or remotely helpful in one of your posts is the last thing I'd ever expect. If you don't like it here and have such a low estimation of our efforts, please move on elsewhere . . .

does users can go here and ask for problems? and then when ask call babyes and other .. so then i still waithing for all the reported bugs, i have problems to track repositories (well at least that was rsync lack of)

i noted that Devuan said in the main page that its a dristribution not copy of debian, but i also read in some where here and in git.debuan that 90% are DIRECTLY copy of debian...

in that case why not copy the debian documentation in a system wiki and then users can made changes over the copy.. if devuan its just a copy without the sytemd, only we must change the systemd related and done!

its a obvious choice and the more inteligent way! please! event pretent to do something that have many lacks

Last edited by mckaygerhard (2017-07-27 01:03:01)

golinux · 2017-07-27 01:14:39

I surrender . . . Someone else with psychic powers and more patience that I is going to have to continue this adventure down a rabbit hole. Better yet I'm just going to lock this thread and put it out of its misery.

The officially official Devuan Forum!

#1 2017-07-17 19:34:43

How to mirror ascii?

#2 2017-07-21 00:23:50

Re: How to mirror ascii?

#3 2017-07-21 01:16:29

Re: How to mirror ascii?

#4 2017-07-21 05:48:46

Re: How to mirror ascii?

#5 2017-07-21 23:34:33

Re: How to mirror ascii?

#6 2017-07-21 23:37:44

Re: How to mirror ascii?

#7 2017-07-21 23:50:31

Re: How to mirror ascii?

#8 2017-07-22 00:36:21

Re: How to mirror ascii?

#9 2017-07-24 15:46:37

Re: How to mirror ascii?

#10 2017-07-24 17:43:10

Re: How to mirror ascii?

#11 2017-07-24 18:38:26

Re: How to mirror ascii?

#12 2017-07-24 19:06:59

Re: How to mirror ascii?

#13 2017-07-24 19:09:44

Re: How to mirror ascii?

#14 2017-07-26 05:53:20

Re: How to mirror ascii?

#15 2017-07-26 12:07:22

Re: How to mirror ascii?

#16 2017-07-26 14:26:48

Re: How to mirror ascii?

#17 2017-07-26 14:46:49

Re: How to mirror ascii?

#18 2017-07-26 16:30:37

Re: How to mirror ascii?

#19 2017-07-26 16:58:01

Re: How to mirror ascii?

#20 2017-07-26 17:14:42

Re: How to mirror ascii?

#21 2017-07-26 17:18:00

Re: How to mirror ascii?

#22 2017-07-26 21:08:58

Re: How to mirror ascii?

#23 2017-07-27 00:36:15

Re: How to mirror ascii?

#24 2017-07-27 01:01:58

Re: How to mirror ascii?

#25 2017-07-27 01:14:39

Re: How to mirror ascii?

Board footer