You are not logged in.
I have a box behind a router. Said box currently works as a vpn server (openvpn) and wifi AP (hostapd). openvpn and hostapd run on their own subnets. There are, therefore, three subnets: one for ethX (router), tunX (vpn), and wlanX (ap). Right now, everything behind the box can access the internet just fine.
I'd like to configure said box to redirect all network traffic to-from an external vpn (tunY). I can connect to this external vpn from said box, as an openvpn client (eg, sudo openvpn ./external.ovpn). However, when that happens, all of my devices behind the box -- on the ap or vpn server -- lose internet until I close tunY.
Google et al were pretty unhelpful.
I assume this will require some iptables or ip commands that I'm unaware of. I'm hoping someone can help me figure out what's missing. Please let me know if I need to provide any other information that would be helpful.
Last edited by siva (2018-06-11 20:43:36)
Offline
What you need are bridging tools to 'transfer' traffic between the various subnets and the VPN. Never tried it myself though, but google should be able to give you some info on bridging ethernet/LAN traffic.
HTH!
Online
Could it be that the outbound traffic through tunY needs NAT (i.e. a MASQUERADE rule)?
Offline
@dutch: I've never set up a tunnel, either. My understanding is that they can link one adapter to another, but I would need something that can funnel several adapters to tunY, and visa-versa.
@rrq: I tried iptables -t nat -A POSTROUTING -s 10.8.x.x/24 -o tunY -j MASQUERADE, and that worked okay for devices connected to the AP, but not on the VPN from an outside network. Outside devices hang at UDP link remote [AF_INET] public.ip.address:port.
Offline
I think, since those are locally generated but with remote sources, you'll also need a MASQUERADE rule on the OUTPUT chain of the nat table. I suppose the set up of the first openvpn (the server) should have dealt with it, but it perhaps worked out to use something else (eth0) as its outbound interface. In fact, maybe a restart of the server, after the outbound openvpn (client) is set up makes this happen?
Offline
you'll also need a MASQUERADE rule on the OUTPUT chain of the nat table.
In this context, I'm not completely sure what this would look like.
it perhaps worked out to use something else (eth0) as its outbound interface
Correct, that's been the deal since I first set up the "original" vpn (connect to box from wherever). My iptables rules for that are as follows:
iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -s 10.8.y.0/24 -j ACCEPT
iptables -A FORWARD -j REJECT
iptables -t nat -A POSTROUTING -s 10.8.y.0/24 -o ethX -j MASQUERADE
maybe a restart of the server, after the outbound openvpn (client) is set up makes this happen?
No idea. I've restarted both client and server to no avail.
Last edited by siva (2018-06-12 02:15:49)
Offline
I have a feeling the VPN server you are connecting to might be pushing you a default route and thereby killing your internet connection by taking packets that should really go to eth0. I'd inspect the output of route -n before and after connecting to the VPN. If the default route gets taken over you might have your culprit and you can adjust your VPN client config to ignore the pushed route.
Edit: Iirc OpenVPN does not actually replace the default route but adds 2 /1 routes and thereby takes precedence (should show up as 0.0.0.0 and 128.0.0.0) so it might not be obvious what is happening.
Last edited by devuser (2018-06-12 06:45:44)
Offline
Yes, forget my nonsense about masquerading on the output chain, and check routing regarding the ethX network. In particular, the server-vpn return traffic must not be routed through the client-vpn.
Offline
inspect the output of route -n before and after connecting to the VPN.
Here's the output of route -n before and after:
# BEFORE VPN
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.x.1 0.0.0.0 UG 0 0 0 ethX
10.8.x.0 10.8.x.2 255.255.255.0 UG 0 0 0 tunX
10.8.x.2 0.0.0.0 255.255.255.255 UH 0 0 0 tunX
169.xxx.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 ethX
192.168.x.0 0.0.0.0 255.255.255.0 U 0 0 0 ethX
192.168.y.0 0.0.0.0 255.255.255.0 U 0 0 0 wlanX
#AFTER VPN
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
0.0.0.0 192.168.x.1 0.0.0.0 UG 0 0 0 ethX
10.8.x.0 10.8.x.2 255.255.255.0 UG 0 0 0 tunX
10.8.x.2 0.0.0.0 255.255.255.255 UH 0 0 0 tunX
10.8.y.0 0.0.0.0 255.255.255.0 U 0 0 0 tunY
xx.xx.xxx.xx 192.168.x.1 255.255.255.255 UGH 0 0 0 ethX
128.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
169.xxx.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 ethX
192.168.x.0 0.0.0.0 255.255.255.0 U 0 0 0 ethX
192.168.y.0 0.0.0.0 255.255.255.0 U 0 0 0 wlanX
check routing regarding the ethX network. In particular, the server-vpn return traffic must not be routed through the client-vpn.
In the routing table, it doesn't look like it is. I'm not sure the best way to make that route; I'm still pretty new at this and doing a lot of reading on the fly.
One thing that wasn't immediately obvious is the public IP. When I connect to the external VPN server, my network's public IP changes (in the routing table, it's the xx.xx.xxx.xx IP); and even though everything connected to the box on the internal network plays nicely with tunY, the clients that connect to the same box from outside the internal network hang at the connection process. The error is the same as if the box were offline, and the second I kill the box's client connection (tunY), my IP changes back, and those external clients connect exactly as expected.
I'm wondering if this is causing this grief for my external clients. I assumed that I should still be able to access my router from the "actual" (my ISP's) public IP. I wonder if that assumption was in err.
(Also, just for fun, I tried to redirect the external clients to xx.xx.xxx.xx instead of my ISP public ip, but to no avail, as I suspected.)
Last edited by siva (2018-06-12 18:40:05)
Offline
Tbh i don't think i fully understood your post but those entries are OpenVPNs way of setting a default route:
0.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
128.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
Since they are more specific they override this route:
0.0.0.0 192.168.x.1 0.0.0.0 UG 0 0 0 ethX
As for this route:
xx.xx.xxx.xx 192.168.x.1 255.255.255.255 UGH 0 0 0 ethX
I guess xx.xx.xxx.xx is actually the IP of the VPN server you are connecting to, right? It's added by OpenVPN to keep routing packets to the VPN server over the external interface. Otherwise the new default route would break the connection to the server.
Bottom line: Unless you want to route packets to the internet over tunY (in which case you should give your VPN server some NAT ability) you should adjust your OpenVPN client config to ignore the default route pushed by the server. https://community.openvpn.net/openvpn/w … ectGateway has some information on how to ignore it. Of course if you control the server you could edit it's config to not push a route (or at least not such a broad one) also.
Last edited by devuser (2018-06-12 21:36:57)
Offline
devuser might be right, though one thing is that you do need to allow forwarding between tunX and tunY.
server-vpn might keep its clients up-to-date with its own gateway, which changes from 192.168.x.1 to be 10.8.y.1. It thus configures its clients' routing to pass 10.8.y.1 traffic through itself, and that traffic would stop at the forwarding block.
Offline
Yeah, the routing is just one part of the puzzle. Forwarding is the other. I guess the routing is fine beyond the superfluous default route though. The clients-lose-internet symptom is a dead giveaway and lots of OpenVPN server howtos assume the user wants run NAT and have the default route push in their configs which iirc is simply called something like push def1 so it's easy to miss that one doesn't really need this when following those.
Offline
Thanks for the responses so far. I've been researching routing with openvpn, to little avail.
One issue that I'm noticing is that openvpn seems to ignore routing information. For example, based on this post, I added this line to my external client's (which creates tunY) .ovpn file:
route 192.168.0.0 255.255.0.0 net_gateway
redirect-gateway def1
According to the posts, this should exclude any devices on any 192.168.(...) networks from being routed through the vpn. In fact, it does not.
My personal favorite so far is this one:
Here are a couple of simple lines for your CCD files that will redirect all traffic except for a specific IP range, in this case 10.0.0.0/16:
push "route 10.0.0.0 255.255.0.0 net_gateway" push "redirect-gateway def1"
Seems simple enough, so I also modified the route to 192.168.0.0 255.255.0.0, in hopes to exclude that IP range.
Despite the enthusiastic accolades in the topics comments, the same results occur: all network traffic is routed over the VPN.
I'm having the exact opposite problem when using the route-nopull option, coupled with custom route or push "route options.
Last edited by siva (2018-06-15 17:41:04)
Offline
I don't think you have to set anything routing related in OpenVPNs config. The routes on tunX and tunY were fine as far as i can tell (besides the default route pointing torwards tunY obviously). If i understand what you want to do correctly "redirect-gateway def1" (that is what route-nopull would ignore if you could not control the server config) is not what you want at all. Could you please remove all routing related config (including redirect-gateway def1) from client and server and post the output of route -n?
Last edited by devuser (2018-06-15 18:41:56)
Offline
"redirect-gateway def1" (that is what route-nopull would ignore if you could not control the server config) is not what you want at all.
What I'd really like to do is use the external VPN service as something like a proxy for devices connected on all the interfaces.
Devices on tunX connect to the box from outside the local network. Ideally, the flow of information would be something like this:
1) Laptop at a coffee shop connects to this box at home (tunX). It performs this by connecting to my ISP's public IP address.
2) The box at home does some filtering and monitoring, then redirects the packets through the VPN service (tunY).
3) The box receives the reply from tunY, filters/monitors, and uses tunX to send the packets to my laptop at the coffee shop.
The problem is that, when tun1 is up, step 1 doesn't happen on the laptop at the coffee shop -- it hangs while trying to connect. So, if this is impossible for devices on tunX, then as a fallback, I'd like to use it for at least wlanX and ethX, and ignore tunX altogether.
Could you please remove all routing related config (including redirect-gateway def1) from client and server and post the output of route -n?
With push "redirect-gateway def1" removed from server.conf, the output is the same as what I posted earlier:
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
0.0.0.0 192.168.x.1 0.0.0.0 UG 0 0 0 ethX
10.8.x.0 10.8.x.2 255.255.255.0 UG 0 0 0 tunX
10.8.x.2 0.0.0.0 255.255.255.255 UH 0 0 0 tunX
10.8.y.0 0.0.0.0 255.255.255.0 U 0 0 0 tunY
xx.xx.xxx.xx 192.168.x.1 255.255.255.255 UGH 0 0 0 ethX
128.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
169.xxx.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 ethX
192.168.x.0 0.0.0.0 255.255.255.0 U 0 0 0 ethX
192.168.y.0 0.0.0.0 255.255.255.0 U 0 0 0 wlanX
Last edited by siva (2018-06-15 20:34:04)
Offline
devuser wrote:"redirect-gateway def1" (that is what route-nopull would ignore if you could not control the server config) is not what you want at all.
What I'd really like to do is use the external VPN service as something like a proxy for devices connected on all the interfaces.
Devices on tunX connect to the box from outside the local network. Ideally, the flow of information would be something like this:
1) Laptop at a coffee shop connects to this box at home (tunX). It performs this by connecting to my ISP's public IP address.
2) The box at home does some filtering and monitoring, then redirects the packets through the VPN service (tunY).
3) The box receives the reply from tunY, filters/monitors, and uses tunX to send the packets to my laptop at the coffee shop.The problem is that, when tun1 is up, step 1 doesn't happen on the laptop at the coffee shop -- it hangs while trying to connect. So, if this is impossible for devices on tunX, then as a fallback, I'd like to use it for at least wlanX and ethX, and ignore tunX altogether.
So basically you want to have all all outgoing traffic directed towards the internet go over tunY?
Could you please remove all routing related config (including redirect-gateway def1) from client and server and post the output of route -n?
With push "redirect-gateway def1" removed from server.conf, the output is the same as what I posted earlier:
Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY 0.0.0.0 192.168.x.1 0.0.0.0 UG 0 0 0 ethX 10.8.x.0 10.8.x.2 255.255.255.0 UG 0 0 0 tunX 10.8.x.2 0.0.0.0 255.255.255.255 UH 0 0 0 tunX 10.8.y.0 0.0.0.0 255.255.255.0 U 0 0 0 tunY xx.xx.xxx.xx 192.168.x.1 255.255.255.255 UGH 0 0 0 ethX 128.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY 169.xxx.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 ethX 192.168.x.0 0.0.0.0 255.255.255.0 U 0 0 0 ethX 192.168.y.0 0.0.0.0 255.255.255.0 U 0 0 0 wlanX
That's kinda strange since tunY still is the default route but i'll wait with drawing any conclusions until i know for sure if you actually want to route traffic to the internet over it or not.
Really, if you want to connect to the internet over tunY is the main question at this point since as is it's configured to do so by these routes (which act as a default route):
0.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
128.0.0.0 10.8.y.1 128.0.0.0 UG 0 0 0 tunY
If you just want packets toward 10.8.y.0/24 to go over tunY this route is enough
10.8.y.0 0.0.0.0 255.255.255.0 U 0 0 0 tunY
and we have to figure out why it even adds the default route or at least stop it from doing so.
Last edited by devuser (2018-06-15 21:43:56)
Offline
So basically you want to have all all outgoing traffic directed towards the internet go over tunY?
That's the idea. It works okay from the box, and from devices connected to its hotspot. Also, when I'm on the box's hotspot, I can connect through openvpn, and everything works as normal.
It's only when I'm outside the lan that I have any issues. My conjecture is it has something to do with the outside VPN service changing the box's public IP -- it overrides the ISP public IP. Yet, my client.ovpn is set to reach out to the ISP's public ip, and as I mentioned, it works okay on the LAN, just not outside of it. That's weird to me. (Edit: this behaviour changes when route-nopull is used; see below.)
That's kinda strange since tunY still is the default route
Why wouldn't it be? All routing information was taken off. I'm assuming that the external VPN service I'm using pulls everything by default.
Really, if you want to connect to the internet over tunY is the main question at this point
Yeah, that's the idea. I want the box to be a middle-man for filtering and monitoring network traffic. Otherwise, I would connect directly to the external VPN service.
The problem is that, anytime it's running, I lose VPN access to tunX from outside the network. (Inside the local network, though, it's okay.)
If you just want packets toward 10.8.y.0/24 to go over tunY this route is enough
10.8.y.0 0.0.0.0 255.255.255.0 U 0 0 0 tunY
This is the behavior when route-nopull is used. (If you want, I can post route -n so you can see the full table.) The issue is that when this happens, it seems like nothing at all is routed through tunY: browserleaks.com/ip reveals my ISP public IP, not the VPN service's (xx.xx.xxx.xx from my previous post).
Interestingly, when route-nopull is used, my devices outside of the local network can connect to the box. It's just that, as I said, when it's used, it seems like nothing at all is going over tunY.
and we have to figure out why it even adds the default route or at least stop it from doing so.
My conjecture is that my external VPN service pulls all routes by default. I want to use tunY to access websites. The problem is that when it's running, I can't access the box from outside the LAN.
Last edited by siva (2018-06-15 22:25:11)
Offline
OK, i get it now. Yes, in that case the default route makes sense and your internal VPN server should also have "redirect-gateway def1" (sorry i thought you had access to the external one when asking to remove that). Well, that's going to be a bit ugly then but something along the lines of the following should work:
echo 200 isp >> /etc/iproute2/rt_tables
iptables -A PREROUTING -i eth0 -t mangle -p udp --sport [YOUR-INTERNAL-VPN-SERVER-PORT] -j MARK --set-mark 1
ip rule add fwmark 1 table isp
ip route add default via 192.168.x.1 dev eth0 table isp
Obviously can't really test it but it should basically mark packets coming from your internal VPN server and add a rule to make them use an entirely different routing table which does nothing but send the packets through your ISP instead of the external VPN. Rationale being: I think the problem isn't really not being able to reach the box from out side (the packets arrive just fine) the packets that the internal VPN server sends back to it's outer client get routed through the external VPN instead of being routed through your ISP as the client expects.
Last edited by devuser (2018-06-15 22:41:35)
Offline
echo 200 isp >> /etc/iproute2/rt_tables iptables -A PREROUTING -i eth0 -t mangle -p udp --sport [YOUR-INTERNAL-VPN-SERVER-PORT] -j MARK --set-mark 1 ip rule add fwmark 1 table isp ip route add default via 192.168.x.1 dev eth0 table isp
Obviously can't really test it but it should basically mark packets coming from your internal VPN server and add a rule to make them use an entirely different routing table which does nothing but send the packets through your ISP instead of the external VPN. Rationale being: I think the problem isn't really not being able to reach the box from out side (the packets arrive just fine) the packets that the internal VPN server sends back to it's outer client get routed through the external VPN instead of being routed through your ISP as the client expects.
Interesting idea. Unfortunately, the behaviour is the same. :\ (I tried it with and without route-nopull; those behaviours haven't changed at all.) Should this be used in conjunction with some option added to my server/client configs?
Offline
Well, as long as you have to default route on tunY VPN config should be fine. What i'd do at this point is fire up tcpdump to debug:
tcpdump -ni eth0
Do you see the packets from your client connecting to your internal VPN server? If yes do also see the replies going from the server to the client? If you see the packets from the client but don't see any replies can you see replies when running
tcpdump -ni tunY
Edit: You can add
port [YOUR-INTERNAL-VPN-SERVER-PORT]
to the tcpdump commandline to filter traffic. Might be a bit hard to spot anything otherwise.
Edit2: It not working with nopull (as that should have eliminated the default route toward tunY) seems a bit fishy to me. If it worked and the default route points towards eth0 i don't see how it could stop you from connecting from outside. Are you sure the packets aren't getting blocked by iptables?
Last edited by devuser (2018-06-15 23:36:09)
Offline
Well, as long as you have to default route on tunY VPN config should be fine. What i'd do at this point is fire up tcpdump to debug:
tcpdump -ni eth0
Do you see the packets from your client connecting to your internal VPN server? If yes do also see the replies going from the server to the client? If you see the packets from the client but don't see any replies can you see replies when running
tcpdump -ni tunY
Edit: You can add
port [YOUR-INTERNAL-VPN-SERVER-PORT]
to the tcpdump commandline to filter traffic. Might be a bit hard to spot anything otherwise.
Okay, so I have no idea why I never thought to run tcpdump, but you nailed part of the diagnosis, which puts me at ease. With route-nopull, I can see packets to-and-from my mobile device, but it doesn't use tunY.
On the other hand, with route-nopull disabled, I saw the packets trying to reach ethX, but as you said, no replies. And I've gotta be totally honest with you, but I have no idea what that means.
Are you sure the packets aren't getting blocked by iptables?
Again, in all honesty, I'm not sure. What would be the best way to check?
Last edited by siva (2018-06-18 21:10:25)
Offline
What would be the best way to check?
Remove the rule
iptables -A FORWARD -j REJECT
to see if it works without it.
Then add it back, preceded by rules that allow forwarding between tunX and tunY.
Offline
What would be the best way to check?
Remove the rule
iptables -A FORWARD -j REJECT
to see if it works without it.
Same behavior. Here's my current config:
# iptables -L --line-numbers
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT udp -- anywhere anywhere udp dpt:domain
2 ACCEPT udp -- anywhere anywhere udp dpt:domain
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
2 ACCEPT all -- anywhere anywhere
3 ACCEPT all -- anywhere anywhere
4 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
5 ACCEPT all -- anywhere anywhere
6 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT udp -- anywhere anywhere udp spt:domain
2 ACCEPT udp -- anywhere anywhere udp spt:domain
Last edited by siva (2018-06-18 21:36:01)
Offline
On the other hand, with route-nopull disabled, I saw the packets trying to reach ethX, but as you said, no replies. And I've gotta be totally honest with you, but I have no idea what that means.
Well, it's not 100% sure but it pretty much implies that the routing "works". Sadly not "works" as in does what you but rather does what it says though. I am quite certain the missing replies aren't really missing but just exit on the wrong interface as the default route that's in place when you remove route-nopull tells them. I guess you could see them when point tcpdump to tunY rather than eth0. That's basically what i was aiming at with the custom routing table but it seems that didn't work for some reason. I am quite sure that would be the way to go though.
could you post outputs of the following commands
ip route show table isp
ip rule show
iptables -vnL
iptables -t mangle -vnL
before and after running
iptables -A PREROUTING -i eth0 -t mangle -p udp --sport [YOUR-INTERNAL-VPN-SERVER-PORT] -j MARK --set-mark 1
ip rule add fwmark 1 table isp
ip route add default via 192.168.x.1 dev eth0 table isp
I know this is going to be a lot of output but i am trying to wrap my head around where it could go wrong and it's nothing i can easily test locally. Also your internal OpenVPN server uses UDP and when looking at tcpdump it's reply packets used [YOUR-INTERNAL-VPN-SERVER-PORT] as source port, right? Otherwise the iptables rule that should mark the packets would be wrong.
Last edited by devuser (2018-06-18 21:52:57)
Offline
That's a good set of rules for doing no filtering at all, yes
If you don't mind, rather dump the output if iptables-save, just to confirm all the tables. Basically, you shouldn't need any iptables rules, except the masquerading of the outbound traffic on tunY. That one is necessary to allow packets with original source from IP 10.8.x.0/24 (i.e., your server-vpn clients) or 192.168.y.0/24 (your wlanX neighbors), as well as allowing packets from 192.168.x.0/24 (your ethX neighbors) to be forwarded and masqueraded through tunY.
Offline