Local Mirror Package Caching using approx and xinetd

PeteGozz · 2018-07-30 00:55:38

This Walk through documents setting up a HTTP based proxy using
*approx_ and xinetd_*.

Features of this approach include::

- Reliable and _low_ maintenance
- Small and light
- efficient enough for enterprise level deployment.
Should scale up to 100's of LAN clients including virtuals
(depends on hardware and bandwidth)

- Tunable access by network space and optionally time of day. (xinetd)
No need to worry your iptables or other filtering.
- Tunable system loads. (xinetd)

- A simply structured *pool based mirror* under /var/cache/approx (default).
- Readily and cleanly supports other Debian type systems
and 3rd party repositories

- Low standby resource demand (xinetd)
- Fast start up and shutdown. (xinetd)

- Logs flexibly. To file or syslog. ( approx and / or xinetd )

- flexible client side configurations

Downsides::

- For simple use its easier to modify the clients apt sources.list (s)
- NOT a single apt install (and walk away)
- _You_ have to create a new config file for xinetd

The Process ::

With *minimal examples* (for some version of clarity.)

_All network addressing / names etc will need to be adapted to your LAN_.
(so don't just cut and paste)

# apt-get install xinetd approx

1. edit the servers /etc/approx/approx.conf
(this is a minimal example)
::

devuan http://deb.devuan.org/merged

2. edit a clients /etc/apt/sources.list.d/01-stable.list
(? disable the rest while you test ?)
*The client can be the same machine as the server.*

::

deb http://10.1.1.10:9999/devuan stable  main contrib non-free

3. create xinetd approx service description file at ::

/etc/xinetd.d/approx

A somewhat more than minimal example.

::

   # /etc/xinetd.d/approx
    service approx
    {
    id           = approx
    disable      = no
    bind         = 10.1.1.10    
    port         = 9999
    only_from    = 127.0.0.1/8 10.1.1.1/8 
    protocol     = tcp
    socket_type  = stream
    wait         = no
    user         = approx
    server       = /usr/sbin/approx
    }

4. Restart xinetd

::

root@yours~#  /etc/init.d/xinetd  restart

(or)

root@yours~# service xinetd restart

check xinetd has enabled your new service ::

you@yours~$  netstat -tan | grep 9999

A simple test if you have netcat onboard.

you@yours~$  nc 10.1.1.10 9999

"get foo"

HTTP/1.1 400 Bad Request
Server: approx/5.10 Ocamlnet/4.1.2

(yes that *nc get* is deliberately broken)

 you@yours~$  CTL c

to close the "conversation".

5. check default logging (and any errors like the one above)

::

  $ tail -f /var/log/daemonlog

something like: *Nethttpd: Bad request line*

6. simple test from the configured client

::

# apt-get update

Operations::

see: /var/cache/approx/ (your new pool(s) of packages)
see: /var/log/daemon.log (default logging)
see: service xinetd status

References::

see: /usr/share/doc/approx/examples
see: /usr/share/doc/xinetd/examples
$ apropos approx
$ apropos xinetd

Longer Verbose version at::

https://wombatsandemus.blogspot.com/201 … -with.html
(yeah sorry)

ASIDE::

Basic testing and checking of the approx-import tool seems to be OK.
As do the devuan "passthroughs to debian" at the repository.
Though this requires another doc or two.

Last edited by PeteGozz (2018-07-30 03:35:27)

PeteGozz · 2018-07-30 01:01:19

There are other good options as well.
squid and apt-cacher-ng

This approach works reasonably well for my home LAN
AND builds sane mirrors that are amenable to dpkg / apt / file:://

(system tools)

Last edited by PeteGozz (2018-07-30 01:02:27)

golinux · 2018-07-30 01:03:29

WOW! Nice to see you PeteGozz. It's been way too long . . .

PeteGozz · 2018-07-30 01:10:39

Hi
Um yeah lost my password

Thought I would/should contribute some tested configs I have here.
The spin offs just from this one could probably fill a small book.

golinux · 2018-07-30 01:14:53

PeteGozz wrote:

Hi
Um yeah lost my password

For future reference . . . all you have to do is ping one of the admins to get a new one.

PeteGozz wrote:

Thought I would/should contribute some tested configs I have here.
The spin offs just from this one could probably fill a small book.

Your contributions are always welcome. Don't be a stranger . . .

PeteGozz · 2018-07-30 03:25:43

golinux wrote:

PeteGozz wrote:
Hi
Um yeah lost my password
For future reference . . . all you have to do is ping one of the admins to get a new one.
PeteGozz wrote:
Thought I would/should contribute some tested configs I have here.
The spin offs just from this one could probably fill a small book.
Your contributions are always welcome. Don't be a stranger . . .

I will try not to be.

PeteGozz · 2018-08-02 02:48:45

Update:
The weekly cron script is simple and tweakable .
A 1000 days _may_ be too long or maybe not.

It is simply a couple of transparent find commands.

It would also make a great template for other runs or even uses by other services.

The version below has modified comments

see: /etc/cron.weekly/approx

#!/bin/sh
# /etc/cron.weekly/approx
# Remove old files from the approx(8) cache

# Location of the cache
cache=/var/cache/approx

# Maximum age of files to keep, in days
max_age=1000

# Delete  _files_  modified   +more than "max_age" days ago 
# and _print_ the activity noise to cron so it can tell somebody if needs be.

find $cache -type f -mtime +$max_age -delete -print

# Remove empty files and directories

find $cache -empty -delete

The officially official Devuan Forum!

#1 2018-07-30 00:55:38

Local Mirror Package Caching using approx and xinetd

The Process ::

#2 2018-07-30 01:01:19

Re: Local Mirror Package Caching using approx and xinetd

#3 2018-07-30 01:03:29

Re: Local Mirror Package Caching using approx and xinetd

#4 2018-07-30 01:10:39

Re: Local Mirror Package Caching using approx and xinetd

#5 2018-07-30 01:14:53

Re: Local Mirror Package Caching using approx and xinetd

#6 2018-07-30 03:25:43

Re: Local Mirror Package Caching using approx and xinetd

#7 2018-08-02 02:48:45

Re: Local Mirror Package Caching using approx and xinetd

Board footer