Transparent proxying with squid and pf

squid is a caching web proxy, it's set up between web browsers and servers, fetching documents from servers on behalf of browsers. It can accelerate web access by caching frequently requested pages and serving them from its cache. It can also be used to filter pop-up ads and malware or to enforce access control (which clients may request what pages based on different authentication methods).

Traditionally, the proxy is an optional component, and browsers are configured to actively use the proxy. Transparent proxying means forcing all web traffic through the proxy without the cooperation (or knowledge) of the clients. Once all browser connections pass through the proxy, outgoing connections to external hosts can be restricted to the proxy, and direct connections from local clients can be blocked.

The OpenBSD packet filter (pf) can be used to redirect connections based on various criteria, including source and destination addresses and ports. For instance, one can redirect all TCP connections with destination port 80 (HTTP) that arrive through an interface connected to local workstations to a squid proxy running on a different address and port.

Since the destination address is translated for such connections, the squid proxy needs some way to find the originally intended destination address of the web server to fetch the document from. If the client sends a HTTP 1.1 compliant Host: header in its HTTP request, squid uses the specified host. Older clients don't provide a Host: header, in which case squid can query the packet filter about the original destination address of the redirected connection. The latter approach requires the proxy to run on the firewall itself, otherwise the proxy can run on a separate host.

The following instructions describe how to install and configure squid on the firewall itself, querying the packet filter when no Host: header is provided.

Installing squid

Install squid from the OpenBSD ports tree. If you're unfamiliar with the ports system, read this first.

# cd /usr/ports/www/squid
# env FLAVOR=transparent make install

Configuring squid

The configuration for squid is found in /etc/squid/squid.conf, you'll need to make at least the following changes from the default configuration:

http_port 127.0.0.1:3128
http_access deny to_localhost
acl our_networks src 10.0.0.0/8
http_access allow our_networks
visible_hostname insomnia.benzedrine.ch
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

Start squid with option -z for the first time, to create the swap directories:

# squid -z
2003/04/11 16:26:13| Creating Swap Directories

After that, you can start squid with just:

# squid

When you make changes, you can tell squid to reload its configuration with:

# squid -k reconfigure

Configuring pf

The pf configuration is /etc/pf.conf. The file is documented in pf.conf(5). This is a minimal example of the required rdr rule. Make sure you also allow the redirected connections to pass, they'll have destination address 127.0.0.1 when the filter rules are evaluated. Redirection does not automatically imply passing. Also, the proxy must be able to establish outgoing connections to external web servers.

int_if="gem0"
ext_if="kue0"

rdr on $int_if inet proto tcp from any to any port www -> 127.0.0.1 port 3128

pass in on $int_if inet proto tcp from any to 127.0.0.1 port 3128 keep state
pass out on $ext_if inet proto tcp from any to any port www keep state

Note that squid needs to open /dev/pf in order to query the packet filter. The default permissions for this file allow access only to root. squid is running as user _squid, group _squid, so one way to allow access to squid is by changing the group ID of the file to _squid and make it group-accessable:

# chgrp _squid /dev/pf
# chmod g+rw /dev/pf

Troubleshooting

Make sure processes running on the firewall can connect to local clients and external servers. Try ping and telnet to internal and external hosts. Note that when running a bridge(4), the internal and external interface both need an IP address, otherwise the firewall's userland (where squid is running) is isolated from both networks. You can block unwanted connections to the firewall itself using pf.

Verify that clients can browse external servers with the redirection rule disabled, and that those connections actually pass through the firewall. The firewall's internal address is usually used as default gateway for all clients.

The packet filter might be blocking packets unintentionally due to a mistake in a more complex ruleset. Add 'log' to all 'block' rules and watch pflog for blocked packets and responsible rule number. Try the minimal ruleset above, and once that works, change it to the more complex ruleset in smaller steps.