cwx/op-packages
1
0 Fork 0
mirror of https://github.com/kiddin9/op-packages.git synced 2026-11-06 20:14:10 +08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

⛅ Sync 2026-08-24 23:49:27

Browse Source
This commit is contained in:
github-actions[bot] committed 2026-08-24 23:49:27 +08:00
1 parent e187a41f8d
commit 1bd7b2c053
48 files changed
+4980 -4592
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

shunt/README.md
+777
View File
@@ -0,0 +1,777 @@
<!-- markdownlint-disable -->
# shunt - policy based routing by mac, source, destination and domain
## Table of Contents
* [Description](#description)
* [Quick Start](#quick-start)
* [Main Features](#main-features)
* [Prerequisites](#prerequisites)
* [Which tunnels work](#which-tunnels-work)
* [Installation and Usage](#installation-and-usage)
* [shunt CLI interface](#shunt-cli-interface)
* [shunt config options](#shunt-config-options)
* [How addresses are learned](#how-addresses-are-learned)
* [What polling costs](#what-polling-costs)
* [Examples](#examples)
* [What it shells out to](#what-it-shells-out-to)
* [What shunt creates on the system](#what-shunt-creates-on-the-system)
* [Coexistence with pbr and mwan3](#coexistence-with-pbr-and-mwan3)
* [Troubleshooting & debug options](#troubleshooting-and-debug-options)
* [Known limitations](#known-limitations)
* [Support](#support)
* [Removal](#removal)
* [Donations](#donations)
<a id="description"></a>
## Description
shunt routes selected traffic into a policy interface - a VPN tunnel, a second
uplink, a mobile connection - chosen by client address, client MAC,
destination or domain. It keeps its own nftables table and one routing table
per policy, so it coexists with fw4 and with other routing tools instead of
competing with them.
The one thing that defines the project: **shunt is not bound to any DNS
backend.** It works unchanged with dnsmasq, unbound, smartdns, AdGuard Home or
anything else, because it never asks the resolver for anything and never sits
in the DNS path. Domain to address mapping comes from two sources shunt owns
itself, described under [How addresses are learned](#how-addresses-are-learned).
<a id="quick-start"></a>
## Quick Start
For a typical setup these few steps are enough - see the sections below for
details:
1. Install the LuCI companion package: `apk update && apk add luci-app-shunt`
(this pulls in the `shunt` backend as a dependency).
2. Make reverse path filtering loose and give the policy interface a
masquerading firewall zone - both once, both shown under
[Prerequisites](#prerequisites). Without the first, marked traffic is
dropped; without the second it is marked and routed and then goes nowhere,
which looks exactly like shunt not working.
4. Open LuCI under `Services -> shunt`, add a policy on the `Policies` tab: pick
the `Interface`, name the clients under `Source addresses` or
`Source MAC addresses`, and list the `Domains` you want routed.
5. Start and verify:
```sh
/etc/init.d/shunt enable
/etc/init.d/shunt start
shunt check
```
**Please note:** a domain policy only takes effect for a client's *next*
connection to an address that has just been learned - see
[Known limitations](#known-limitations).
<a id="main-features"></a>
## Main Features
* Routes by client address, client MAC, destination CIDR and domain, in any
combination
* Resolver independent: works with any DNS backend, and with an encrypted
upstream, because it reads the plaintext leg between client and resolver
* Wildcard domains (`*.example.com`), learned passively as clients use them
* Per-policy killswitch: hold the traffic when the interface drops, instead of
leaking it out of the normal uplink
* Own nftables table and routing tables, disjoint mark range - runs beside
`pbr` and `mwan3`
* IPv4 and IPv6 throughout, with a MAC selecting a host in both at once
* Per-element counters on every set, a ubus status object and a LuCI frontend
* No dependency on a specific DNS backend, no resolver configuration, no
include files, no hooks into fw4's ruleset
<a id="prerequisites"></a>
## Prerequisites
* OpenWrt with fw4/nftables
* `ucode` plus `ucode-mod-fs`, `ucode-mod-socket`, `ucode-mod-uci`,
`ucode-mod-uloop`, `ucode-mod-resolv`, `ucode-mod-ubus`, `ucode-mod-rtnl`,
`ucode-mod-log` and `rpcd-mod-ucode` - all pulled in by the package
`ucode-mod-resolv` and `ucode-mod-ubus` are soft at runtime: without resolv,
poll is skipped and the observer carries the service alone; without ubus,
gateway discovery and interface events are skipped and the config's own values
are used. Both cost one warning in the log, not a failed start.
<a id="which-tunnels-work"></a>
### Which tunnels work
Any of them, and there is no supported-protocols list to check against, because
shunt never asks what protocol an interface speaks. It consumes two things: the
device to route into, and a gateway if one is needed. Both come from netifd,
and a device netifd does not manage is taken as given.
* **Point to point tunnels** - wireguard, OpenVPN `tun*`, L2TP, PPTP,
Tailscale, NetBird and the like - need no gateway at all. The route is
`default dev <device> table <n>`.
* **Ethernet style interfaces** - OpenVPN `tap*`, a second wired uplink, a
mobile connection - use the gateway discovered from netifd, or `gw4`/`gw6` if
you set them.
* **Interfaces netifd does not manage** work by name too. On OpenWrt this is
the exception rather than the rule - wireguard, OpenVPN, L2TP and the rest
all have netifd protocols and are managed like any other interface. It
applies to a tunnel brought up outside netifd, by `wg-quick` or a script of
your own. If such an interface does need a gateway, discovery cannot find one
and you have to set `gw4`/`gw6` yourself.
The one thing that does not work is anything that is not a routable interface.
Tor is the usual example: it normally offers a SOCKS port, and sending traffic
there is a redirect, not a route. shunt marks a packet and looks up a routing
table; without a device to put a default route on, there is nothing for it to
do. Transparent proxying is out of scope by design, not for want of a special
case.
Two kernel-side prerequisites that shunt does **not** configure for you:
**`rp_filter` must be loose on the policy interface.** Marked traffic takes an
asymmetric path, so strict reverse path filtering drops it. The kernel decides
per incoming packet using `max(net.ipv4.conf.all.rp_filter,
net.ipv4.conf.<dev>.rp_filter)`, where `2` is loose - so setting the policy
interface alone to `2` suffices even while `all` stays strict. Prefer this: it
leaves reverse path filtering intact on every other interface.
```sh
cat > /etc/sysctl.d/99-shunt.conf <<'EOF'
net.ipv4.conf.phy0-sta0.rp_filter=2
EOF
sysctl -p /etc/sysctl.d/99-shunt.conf
```
Replace `phy0-sta0` with your policy interface's device - the `Interface`
column on the overview shows it - one line per policy device.
There is a boot-order catch. A device that does not exist yet - a tunnel, or a
wifi client interface brought up late - has no `conf/<dev>` entry at boot, so
`sysctl -p` cannot set it and skips the line. When the device finally appears
it inherits `net.ipv4.conf.default.rp_filter`, and if that is strict the device
comes up strict and stays that way until the next `sysctl -p` - which for most
setups means until the next reboot, i.e. never in practice. Two ways around it:
set `net.ipv4.conf.default.rp_filter=2` as well, which makes every
later-appearing interface inherit loose (a little broader, but far short of
`all`), or let shunt handle it with the option below.
**`rp_filter_manage` (optional, off by default).** With it set, shunt itself
sets `rp_filter=2` on its own policy devices - at start and again whenever one
comes up, which is exactly the boot-order moment a static file misses. It only
ever touches the devices shunt routes into, never `all` or `default`, and only
while the service runs. It is off by default because changing a security
setting should be a deliberate choice:
```sh
uci set shunt.@global[0].rp_filter_manage='1'
uci commit shunt
/etc/init.d/shunt restart
```
Whichever way you choose, the daemon checks the live per-device value and warns
- naming the device - only for a policy device that is still strict, so with
`rp_filter_manage` on the warning simply does not appear. The package
deliberately ships no box-wide sysctl file: `rp_filter` on `all`/`default` is a
distribution default OpenWrt sets strict in `/etc/sysctl.d/10-default.conf`, and
loosening it there weakens anti-spoofing on every interface, well beyond
shunt's own traffic.
**Masquerading stays fw4's job.** shunt marks and routes; it does not touch
the firewall's NAT. The policy interface needs a zone with `masq` enabled and
forwarding from `lan`, exactly as any other uplink. If the interface is a
netifd one - say a wireguard interface named `vpn`:
```sh
uci add firewall zone
uci set firewall.@zone[-1].name='vpn'
uci set firewall.@zone[-1].input='REJECT'
uci set firewall.@zone[-1].output='ACCEPT'
uci set firewall.@zone[-1].forward='REJECT'
uci set firewall.@zone[-1].masq='1'
uci set firewall.@zone[-1].mtu_fix='1'
uci add_list firewall.@zone[-1].network='vpn'
uci add firewall forwarding
uci set firewall.@forwarding[-1].src='lan'
uci set firewall.@forwarding[-1].dest='vpn'
uci commit firewall
/etc/init.d/firewall reload
```
`network` names a **logical interface**, not a device. For a device netifd does
not manage - a tunnel brought up outside netifd - use
`uci add_list firewall.@zone[-1].device='wg0'` instead. And if the policy
interface already has a zone, because it is an ordinary second uplink, there is
nothing to do here.
Symptoms of getting this wrong are worth knowing, because they do not look like
a firewall problem: the prerouting counters rise, `nft list set` shows the
learned address being hit, and the client's connection simply times out.
<a id="installation-and-usage"></a>
## Installation and Usage
* Update your router's apk repository (`apk update`)
* Install the LuCI companion package `luci-app-shunt`, which also installs the
main `shunt` package as a dependency
* Make `rp_filter` loose and give the policy interface a masquerading firewall
zone - both are one-time steps with copy-paste commands under
[Prerequisites](#prerequisites)
* Configure at least one policy, either in LuCI under `Services -> shunt` or by
editing `/etc/config/shunt`
* Enable and start the service, then run `shunt check` - it prints the mark,
routing table and rule priority of every accepted policy, and every rejected
value with its reason
* Check the `Set Reporting` tab to see which addresses were learned, and the
`Processing Log` tab for the service's own messages
<a id="shunt-cli-interface"></a>
## shunt CLI interface
All functions are available from the command line, and the config file can be
edited directly if you prefer that to LuCI.
```sh
shunt check # render everything, print marks and issues, change nothing
shunt run # foreground, the procd service entry point
shunt flush # tear down table, rules, routes and the mapping file
shunt -v <cmd> # echo every message to the terminal as well
```
`shunt check` is safe at any time, including while the service runs, because it
only renders - it never touches the kernel. Run it after every config change.
Note that it says nothing about whether the service is *running*; that is what
`/etc/init.d/shunt status` and the LuCI overview are for.
Exit codes: 0 ok, 1 runtime failure, 2 usage or unusable config.
Logging goes to syslog under the tag `shunt`, so `logread -e shunt` shows
everything - the daemon's own lines carry its pid, `shunt[1234]:`. Debug lines
stay off unless `-v` is given or `option debug '1'` is set - under procd there
is no command line, so a bug report needs the config switch. Expect volume: on
a router running adblock roughly half of all observed answers are error
replies, and debug gives each one a line.
`shunt flush` is the escape hatch if the daemon ever dies without tearing
down. It is idempotent and safe on a box that never ran shunt.
<a id="shunt-config-options"></a>
## shunt config options
### Global section
| Option | Default | Description |
| :--- | :--- | :--- |
| enabled | `1` | master switch; `0` means the service starts and exits |
| debug | `0` | log every observed answer and every set write |
| rp_filter_manage | `0` | set rp_filter=2 on shunt's own policy devices, at start and on ifup |
| poll_interval | `300` | seconds between poll cycles, at least 30 |
| entry_ttl | `1200` | nftables timeout on learned elements, at least 60 |
| snoop | `1` | enable the passive DNS observer |
| snoop_device | `br-lan` | LAN devices to observe, a list, one entry per segment |
Values below the minimum are clamped, not rejected, and the clamp is logged.
`entry_ttl` should stay well above `poll_interval` - an element is rewritten
once its remaining timeout drops below half of `entry_ttl`, so the default pair
refreshes comfortably within two poll cycles.
### Policy sections
Each `config policy` section is one routing policy. **The section must be
named, and the name must match `[A-Za-z0-9_]{1,24}`** - it becomes an nftables
identifier, so a section without a name, or one with a hyphen or a dot in it,
is rejected as an issue and never rendered. LuCI enforces the same pattern
when a policy is added.
| Option | Description |
| :--- | :--- |
| enabled | `0` skips the section entirely |
| interface | netifd logical name (`wan`, `trm_wwan`) or raw netdev (`wg0`, `phy0-sta0`) |
| fallback | `main` (default) or `block`, see below |
| gw4 / gw6 | gateway override; normally unnecessary |
| src | client addresses or CIDRs whose traffic this policy owns |
| src_mac | client MAC addresses, ORed with `src` |
| proto | `tcp`, `udp`, or both; a port without one covers both |
| dport | destination ports, single or a range like `8000-8080` |
| dst | destination addresses or CIDRs |
| domain | domain patterns, see below |
`src`, `src_mac`, `dst` and `domain` are lists and may repeat.
Interfaces are resolved through netifd: a logical name resolves to its
`l3_device`, a raw netdev is adopted if netifd knows it, and a device netifd
knows nothing about passes through as given - which on OpenWrt means a tunnel
started outside netifd, since wireguard and the other tunnel types have netifd
protocols of their own. Gateways are discovered from the same dump, merged
across sibling entries, because netifd splits families. `gw4`/`gw6` override
discovery and always win; on a point to point interface no gateway is needed
at all.
There is deliberately no list of supported tunnel protocols. shunt asks netifd
for the device and the gateway and renders a default route into the policy
table - `default via <gw> dev <device>` when a gateway is known,
`default dev <device>` when none is needed. Wireguard, OpenVPN in both `tun`
and `tap` mode, L2TP, PPTP, Tailscale, NetBird, a second physical uplink or a
mobile connection all reduce to those two shapes, so none of them needs a case
of its own. See [Which tunnels work](#which-tunnels-work) for the one thing
that genuinely does not fit.
### Selectors are ANDed, client selectors OR each other
* `src` or `src_mac` alone marks everything from those clients
* `dst`, `domain`, `dport` or `proto` alone marks that traffic from everyone
* clients plus destinations marks only those clients' traffic to those
destinations
`dport` and `proto` AND with everything else, so a policy with a client, a
domain and `dport 443` covers that client's HTTPS traffic to that domain and
nothing more. A port without a protocol matches **both** tcp and udp - "port
443" almost always means QUIC too, and requiring the protocol would let it
slip through unnoticed. If ports or protocols were configured and none of them
is usable, the policy is skipped rather than rendered without the narrowing.
This is the single most common source of "it did not work" reports: with a
client and a domain both set, a generic `curl ifconfig.me` from that client
correctly takes the normal uplink, because `ifconfig.me` is not in the domain
list. That is the policy working, not failing.
A client MAC and a client address OR each other, so a host may be named either
way. If client selectors were configured and **none** of them is usable - a
typo in the only address, say - the policy is skipped with an issue rather
than falling back to "every client", which is what an absent client selector
otherwise means.
### Selecting clients by MAC
`src_mac` exists mainly for IPv6. Clients prefer rotating privacy addresses
for outgoing traffic, so a single IPv6 address is not a usable selector and the
LAN prefix covers every host in the segment. A MAC picks exactly one host, in
both address families, and keeps doing so when the addresses change. A policy
with only `src_mac` therefore needs no v6 address to route v6.
Three limits, none of them guessable:
* **Same layer 2 segment only.** Anything behind another router arrives with
that router's MAC.
* **Never the router itself.** The `output` chain sees traffic the router
generated, which has no ethernet sender, so MAC rules are not installed
there. An address based policy does cover the router; a MAC-only one does not.
* **Phones randomise their MAC**, though usually stable per network. Use the
address the client shows in your DHCP leases, not the one on the label.
### Domain patterns
```
example.com matches the apex only
*.example.com matches subdomains only, at any depth, NOT the apex
```
List both to cover both. This is more typing than dnsmasq's implicit subdomain
inclusion, and it is deliberate: dnsmasq's behaviour surprises people
regularly, this one does not.
Precedence, in order:
1. an exact match always beats any wildcard
2. among wildcards the longest suffix wins, so `*.cdn.example.com` beats
`*.example.com` regardless of which policy declared them
3. the same pattern in two policies belongs to **both**
Rule 3 is what makes one domain usable by two client groups over two different
uplinks: the address is written into each policy's set, and each policy's rule
matches only its own clients, so they stay apart. Rules 1 and 2 still decide
specificity - a shared `*.example.com` never overrides somebody's exact
`www.example.com`.
Matching is label aligned, never string suffix: `evilexample.com` does not
match `*.example.com`. A bad pattern is collected as an issue, never fatal.
### Policy precedence
Section order in `/etc/config/shunt`, top to bottom. There is no `priority`
option - one less value to set wrong. A packet matching two policies takes the
earlier one; rule evaluation ends at the first match.
Note that domain precedence is resolved *before* this, at the matcher: the most
specific pattern wins even if it sits in a later section.
### Fallback: main or block
`fallback 'main'` (default) renders no default route into the policy table, so
when the policy interface is down the table is empty and marked traffic falls
through to `main` - the normal uplink. Traffic keeps flowing, unpolicied.
`fallback 'block'` adds a blackhole default at metric 9999 to the policy table.
While the interface is up its own default has the lower metric and wins; when
the interface drops, the kernel withdraws that route and the blackhole catches
everything. That is the killswitch: traffic belonging to the policy stops
rather than leaking out of the wrong interface.
<a id="how-addresses-are-learned"></a>
## How addresses are learned
Two sources feed the same nftables sets, union with an element timeout. They
are complementary, not alternative modes.
* **poll** resolves the configured names through whatever system resolver
exists, on a fixed interval. It warms the sets before the first client
packet, so first contact does not race. Wildcards are not names and cannot be
polled.
* **snoop** passively observes DNS responses on the LAN side via AF_PACKET with
a BPF filter matching **UDP source port 53** - answers, not questions -
including one level of VLAN tagging. It covers CDN variance and wildcards,
which poll cannot. It reads; it never writes anything back onto the wire and
never sits between a client and its resolver. If it dies, DNS keeps working
and only the policy stops applying.
<a id="what-polling-costs"></a>
### What polling costs
"Polling" invites the assumption of waste, so here is the arithmetic. One cycle
is a single call asking for A and AAAA of every listed name: two lookups per
name per interval, against the **local** resolver. Ten names at the default 300
seconds is 240 lookups an hour - about what a dozen web page loads cost, on a
network whose own DNS traffic runs to hundreds of answers in a few minutes.
There is no polling of anything else: no interface scanning, no ruleset
re-rendering, no periodic writes. An element is only rewritten when its
remaining lifetime has dropped below half.
Two costs worth knowing:
* The query is synchronous. A name that does not resolve blocks the cycle until
it times out (2s, one retry), which delays the service start noticeably if
several are wrong. This is why an unresolvable name is reported by name.
* For names with a TTL shorter than the interval the local cache has expired,
so poll does refetch upstream rather than answering from cache.
What that means in practice:
* **Pick the device the answers cross on their way to the clients**, normally
`br-lan`. It must be an **Ethernet type** device - a bridge, a VLAN device, a
physical port, a wireless interface. A tunnel or PPP interface has no
ethernet header, so neither the packet filter nor the decoder can read it,
and the failure is silent: nothing matches, nothing is logged.
* **Only the client-to-resolver leg matters, and only whether *it* is
encrypted.** What the resolver does upstream is irrelevant: the usual OpenWrt
setup - unbound or dnsmasq on the router, forwarding upstream over DoT or DoH
- is fully covered, because the client asked in plain text over the LAN and
the answer comes back the same way.
* **A client that speaks DoH or DoT itself is invisible**, because it bypasses
the local resolver. That is the one encryption case that costs coverage.
* **One entry per layer 2 segment.** A guest or IoT VLAN on its own device never
carries the answers of the main LAN, so it needs its own `snoop_device` entry.
A device that cannot be opened costs one warning; the others keep running.
* **The router's own lookups are not seen.** poll's queries leave through the
uplink, not the LAN device.
* Not seen either: DNS over TCP, DNS on a port other than 53, and a second
stacked VLAN tag.
A name in a `domain` list that never resolves is reported once, by name and
policy:
```
poll: www.example.com (policy vpn) has no address - the policy entry has no effect until it resolves
```
Once on the way in and once on recovery, never in between. The first cycle runs
immediately at start, so a typo shows up within seconds. Wildcards cannot
produce this message - nothing can tell whether `*.example.com` was ever meant
to match anything.
<a id="examples"></a>
## Examples
**One client, one domain family, over a wireguard tunnel**
```
config policy 'vpn'
option enabled '1'
option interface 'wg0'
option fallback 'main'
list src '192.168.1.50'
list domain 'example.com'
list domain '*.example.com'
```
**A whole IoT VLAN over a mobile uplink, killswitch on**
The client is named by MAC, so it is covered in both address families without
listing a rotating IPv6 address:
```
config global
list snoop_device 'br-lan'
list snoop_device 'br-iot'
config policy 'iot'
option enabled '1'
option interface 'trm_wwan'
option fallback 'block'
list src_mac 'aa:bb:cc:dd:ee:ff'
list domain '*.vendor-cloud.com'
```
**The same domain for two client groups over two uplinks**
Both policies claim `www.example.com`; each routes only its own clients:
```
config policy 'wwan'
option interface 'trm_wwan'
list src '10.168.30.70'
list domain 'www.example.com'
config policy 'vpn'
option interface 'wg0'
list src '10.168.1.20'
list domain 'www.example.com'
```
**A destination range without any domain**
```
config policy 'office'
option interface 'wg0'
list src '192.168.1.0/24'
list dst '10.0.0.0/8'
```
<a id="what-it-shells-out-to"></a>
### What it shells out to
Almost nothing. The daemon and the rpcd backend work through ucode's native
bindings - `fs`, `socket` for the AF_PACKET observer, `uci`, `uloop`, `ubus`,
`resolv`, `rtnl` and `log` - and rpcd carries the LuCI side, so there is no
shell glue, no `awk`, no temporary state files. Logging goes to syslog through
the binding, not through a `logger` process per line.
Two external commands remain:
| Command | Why |
| :--- | :--- |
| `nft` | the ruleset is applied and read as one atomic batch; ucode has no nftables binding |
| `ip` | routes and rules are written this way, although `rtnl` already reads them - replaceable |
Nothing is ever handed to a shell for parsing: `system()` and `popen()` take an
argument array, and where stderr has to be captured the wrapper is
`sh -c 'exec "$0" "$@"'`, which passes arguments through untouched.
<a id="what-shunt-creates-on-the-system"></a>
## What shunt creates on the system
```
table inet shunt own table, survives fw4 reloads
chain prerouting filter hook prerouting, priority mangle
chain output route hook output, priority mangle
set d4_<policy> / d6_<policy> learned, flags timeout, per-element counter
set s4_<policy> / s6_<policy> static dst, flags interval, counter
set c4_<policy> / c6_<policy> client src selectors, interval, counter
set m_<policy> client MACs, no family digit, counter
fwmark <index> << 24, mask 0xff000000
ip rule pref 31000 + <index>
routing table 8000 + <index>
/etc/iproute2/rt_tables.d/shunt.conf the table name mapping
```
The mark mask is fixed at `0xff000000`, which allows 255 policies. The `output`
chain is `type route` so the router's own marked traffic is re-routed after the
mark is set.
Every set carries per-element counters, so "is this element ever hit" is one
look at `nft list set inet shunt <set>` rather than a tcpdump session. The two
kinds count different things: nftables tests a rule left to right, so a
**client** set counts every packet that matched the selector, whether or not
the destination matched afterwards; a **learned** set is the last lookup in the
rule, so a hit there means the packet really was marked. A busy client beside
learned addresses at zero is a client that has not visited any of the routed
domains, not a fault.
**Writes are batched, and the interval adapts.** `nft -f` reads the entire
ruleset from the kernel before it resolves a single name, so on a box that
also runs a tool with very large sets - banIP with 238k elements, measured -
one `add element` costs seconds of CPU, and `nft --check` alone costs the
same. That is a known bug in nftables (netfilter bugzilla #1735, open
since 2024), not something shunt can fix, so observed addresses are collected
and applied together by a timer.
The interval follows what the last write actually cost, between 2 and 60
seconds: on an ordinary box a write takes milliseconds and the interval stays
at its floor, where the batching is invisible. Where it is expensive the
interval grows until nftables takes a bounded share of the machine instead of
all of it, at the price of a learned address reaching its set later. Both
numbers show up under `debug`.
**A reload wipes learned state.** Applying the configuration destroys and
re-creates the table atomically, so the learned sets start empty. poll rewarms
them within one interval and snoop refills from live traffic; expect a short
window after a restart where domain policies do not apply yet.
<a id="coexistence-with-pbr-and-mwan3"></a>
## Coexistence with pbr and mwan3
shunt is an independent implementation, not a fork of `pbr` and not a drop-in
for it - there is no config migration and no attempt at feature parity. Within
its scope it is a full alternative.
Running both at once during a migration is safe by construction:
| | pbr | mwan3 | shunt |
| :--- | :--- | :--- | :--- |
| fwmark mask | `0x00ff0000` | `0x00003f00` | `0xff000000` |
| ip rule pref | 30000 counting down | ~1001-3250 | 31000 counting up |
| routing tables | dynamic from ~256 | 1-250 | 8000+n |
| nft | chains in fw4's table | | own `inet shunt` table |
The mark bits are disjoint and all three mask their writes. Where pbr and shunt
both match, pbr's lower rule priority wins, deterministically. So move policies
over one at a time and retire pbr once its config is empty.
Anything shunt cannot see is worth knowing about: marks set via `SO_MARK` on a
daemon socket (OpenVPN's `--mark`) or by an eBPF program are invisible to any
inspection. If a box uses those, check the mark ranges by hand.
<a id="troubleshooting-and-debug-options"></a>
## Troubleshooting & debug options
### Did the policy actually match?
The authoritative check needs no route lookups at all:
```sh
nft reset counters table inet shunt
# generate traffic from the client
nft list chain inet shunt prerouting # rule counters moved?
tcpdump -ni <policy-interface> host <addr> # the flow leaves where it should
```
The wire capture is ground truth - but capture a **learned address**, not
everything: on a router whose policy interface is also a normal uplink, an
unfiltered capture shows traffic that has nothing to do with shunt. An IP echo
service is a convenient confirmation, but it only discriminates uplinks that
actually have different exits.
The route lookup variant asks the kernel directly:
```sh
ip route get <addr> mark 0x1000000
ip route get <addr>
```
The first answer must name the policy table, the second the normal uplink.
**The first line needs iproute2's `ip`** (`ip-full`), because BusyBox's
`route get` does not understand `mark` - one build rejects it outright, the
OpenWrt one sends an incomplete netlink request that the kernel answers with
`EINVAL`. The second line, without a mark, works with either. Check which one
you have with `readlink -f $(command -v ip)`.
Adding `from <client> iif br-lan` makes the lookup more precise, with one
further catch worth a confused test session: **`iif` is not optional** there.
Without it, `from` a non-local address makes the kernel validate a locally
originated lookup and answer `ENETUNREACH` regardless of any table's content,
which reads like broken routing and is not.
### What the observer discards, and why
Most DNS answers on a network are of no use to a routing policy, so snoop
counts what it discarded and why. `ubus call shunt status` reports those
counters, and the LuCI overview shows them with readable labels.
| Verdict | Meaning |
| :--- | :--- |
| qtype | the question was not for an address at all |
| noaddr | an address was asked for, the answer carried none |
| nomatch | the name belongs to no policy |
| dns:E_* | the message did not parse, e.g. `E_RCODE` for NXDOMAIN |
| frame:E_* | the packet did not decode, e.g. `E_FRAG` for a fragment |
`qtype` is usually the largest category and that is expected: current browsers
and operating systems ask for **HTTPS records (type 65)** alongside every A and
AAAA. `noaddr` is the other half of that distinction: the question *was* A or
AAAA, the reply is well formed, and the answer section still holds no address -
NODATA.
The checks run in order and the **first** one wins, so these are "first reason
to discard" rather than independent counters: a PTR query for a name you route
counts as `qtype`, never as `nomatch`.
A high discard count is therefore not a fault. The one number that says whether
the observer is doing its job is the matched count next to them.
### When a policy stops applying after a reconnect
The kernel removes routes from a policy table when the interface goes down,
after which the fwmark rule falls through to `main` while every counter keeps
counting. shunt handles this on two levels: a ubus listener on
`network.interface` rebuilds the route half on ifup/ifdown, and every poll tick
replays the route commands as a keeper. Learned sets survive both. If ubus is
unavailable, only the keeper remains, so recovery takes up to one
`poll_interval`.
### Debug logging
```sh
uci set shunt.@global[0].debug='1'
uci commit shunt
/etc/init.d/shunt restart
logread -e shunt
```
Set it back to `0` afterwards. Every observed answer and every set write gets a
line, which on a busy network is a lot.
<a id="known-limitations"></a>
## Known limitations
These are consequences of the design, stated rather than worked around:
* **Clients that speak DoH or DoT themselves are invisible to snoop.** poll
still covers the names you list explicitly; wildcards do not work for those
clients. A *resolver* forwarding upstream over DoT or DoH changes nothing.
* **Wildcards require snoop.** poll can only resolve names it was given, and
`*.example.com` is not a name.
* **One CDN address serves many domains.** If a policy routes `example.com` and
the address behind it also serves a thousand other sites, those sites follow
the same policy. This is unsolvable at layer 3 by anything that routes on
addresses.
* **The first connection to a newly seen address takes the old path.** snoop
learns from the response the client is reading at that moment, so the client's
SYN is usually out before the element reaches the set. Measured on a live
router: the entire first connection stayed on the normal uplink, and the next
connection to the same host started on the policy interface. The switch
happens at a connection boundary; shunt does not touch conntrack, so no
established flow is ever yanked to a different exit mid-stream. Listing the
entry point explicitly closes the gap, because poll warms it before any client
asks.
* **DNS over TCP is not observed.** Port 53 over TCP needs reassembly, which is
out of scope; answers large enough to force TCP are rare in the traffic shunt
cares about.
* **Route and rule application is best effort.** At boot a tunnel interface may
not exist yet. A rule over an empty table falls through to `main`, so the
failure mode is "policy not applied yet", never "traffic broken". Each
distinct reason is one warning line.
* **No interface hotplug.** A device that appears later is picked up on the next
`ifup` event or within one poll interval, not immediately.
**Out of scope permanently:** resolver-integrated set population (dnsmasq
`nftset`, AdGuard Home etc.). Being independent of the DNS backend is the entire
point of the project, so adopting a backend-specific mechanism would give up
the one property that distinguishes it. Also out: DSCP tagging and user
include files.
<a id="support"></a>
## Support
Please report issues with as much detail as possible - the output of
`shunt check`, the relevant part of `logread -e shunt` with `debug` enabled,
your `/etc/config/shunt`, and the OpenWrt version of the device.
<a id="removal"></a>
## Removal
Stop the service with `/etc/init.d/shunt stop`, which also tears down the
nftables table, the routing tables and the ip rules, then remove the `shunt`
and `luci-app-shunt` packages if necessary. `shunt flush` does the teardown
alone, should anything be left behind.
<a id="donations"></a>
## Donations
You like this project - is there a way to donate? Generally speaking "No" - I have a well-paying full-time job and my OpenWrt projects are just a hobby of mine in my spare time.
If you still insist to donate some bucks ...
* I would be happy if you put your money in kind into other, social projects in your area, e.g. a children's hospice
* Let's meet and invite me for a coffee if you are in my area, the “Markgräfler Land” in southern Germany or in Switzerland (Basel)
* Send your money to my [PayPal account](https://www.paypal.me/DirkBrenken) and I will collect your donations over the year to support various social projects in my area
No matter what you decide - thank you very much for your support!
Have fun!
Dirk