ComputersConsumer ElectronicsCell PhonesHome Theater & AudioInternetGraphic Design & Video EditingIndustrial Technology

How to Set Up a Transparent Squid Proxy Server Using pfSense

Updated on September 19, 2016
skear profile image

Sam works as a Network Analyst for an algorithmic trading firm. He obtained his Bachelors Degree in Information Technology from UMKC.

Source

Proxy servers act as an intermediary for clients on a network requesting resources from another server. The most common type of proxy is a web proxy.

Proxy servers can be very useful for improving the speed of an internet connection by caching, logging internet usage, or filtering the traffic. The proxy server will store local copies of HTML pages, images, and other files in its cache.

Caching proxy servers can greatly improve the internet performance of corporate networks or internet cafe's where many users may be requesting similar pages.

When a client requests a web page the proxy checks to see if has any of the files stored in cache, if it does it serves them to the client without having to download them from the web server.

This reduces latency and saves internet bandwidth. Transparent proxys route the clients traffic through the proxy server automatically, unlike traditional proxys which require configuration changes on the client systems.

If you are unfamiliar with pfSense check out an Introduction to pfSense.

Package installation
Package installation
Accessing the proxy server menu
Accessing the proxy server menu
Squid proxy configuration
Squid proxy configuration

Getting Started

The first thing you'll need to do is install the squid package in pfSense. This can be done from the package manager found under the system menu.

Locate the Squid package and click the + symbol next to it to begin the installation. The installation process normally takes a few minutes to complete.

Configuration

After the installation is completed you will have a new menu option under 'Services' called 'Proxy Server'. Click on the new menu option to bring of the configuration page.

Here you will need to set the proxy interface which is typically LAN. Next check to box 'Allow users on the interface'. Then check the box 'Enable transparent proxy'. Now scroll down to the bottom and hit save. This will start the squid service using the settings you have defined.

At this point you have a fully functional transparent proxy server running on pfSense. You do not need to make any changes to the computers on your network for them to use the proxy. Any clients requesting web pages on port 80 will be automatically redirected through the proxy. The users on your network won't even know their traffic is going through a proxy!

Traffic Management Settings
Traffic Management Settings

Traffic Management

The traffic management tab has some settings that are useful if you want to place bandwidth usage restrictions on the proxy.

Using these settings you can configure a maximum download or upload size which will restrict transfers over a certain size limit.

You can also set the proxy to throttle binary files, cd images, or any other file type that you specify. Per host throttling sets the maximum amount of bandwidth an individual host can use.

Performance Tweaks

There are various options on the cache tab of the squid configuration page that you can modify to improve performance in your environment. Below are some of the settings I recommend modifying. If the computer running the proxy has a limited amount of disk and ram you should be cautious not to use overly aggressive settings. On the other hand if you have lots of resources to spare you can increase the settings to improve performance.

  • Hard disk cache size - This sets the total amount of hard disk space squid will use to cache objects. If you have a large hard drive you can increase this setting to cache more objects. Just remember that objects cached in memory will be retrieved faster than objects on hard disk.
  • Memory cache size - If your pfSense system has plenty of ram I recommend increasing the size of the memory cache. Objects that squid can't store in memory end up getting swapped to disk which is much slower than RAM.
  • Maximum object size -The default of 4K is pretty small, I recommend increasing this to 50. You could set it larger but most cache hits tend to take place on small files anyway.
  • Edit /boot/loader.conf.local - This change needs to be done via SSH. Using a text editor such as vi add kern.ipc.nmbclusters="32768" to the file then save the file and reboot the pfSense router. This increases the total amount of memory used for socket buffers to 32M.

Visit the pfSense documentation site for more Squid performance tuning tweaks.

Cache Management Settings
Cache Management Settings

Manually Clearing The Squid Cache

Squid has it's own system for purging old objects from the cache but occasionally you may want to clear the entire Squid cache.

I recommend logging into pfSense using SSH to run these commands. It is possible to run them using the command prompt function in the diagnostics menu but I don't advise doing so.

First you'll need to stop the proxy service.

squid -k shutdown

The command below will delete all of the files in the cache directory. This command can take a long time to run especially if you have defined a large cache size so be patient.

rm -fr /var/squid/cache/*

Before you restart Squid you should recreate the swap directly structure.

If you receive a message that Squid is already running go ahead and shutdown the service and again and rerun the command. PfSense seems to restart Squid on it's own when it notices it's not running.

squid -z

Finally you can restart Squid to begin using the proxy again.

/usr/local/sbin/squid -D

To make sure it started you can check the status on the services menu which is found under the status menu of the web interface.

Lightsquid Report
Lightsquid Report

Proxy Reports

If you are interested in tracking the usage of your proxy you can install an additional package called Lightsquid.

Lightsquid will generate interactive reports that track all of the websites visited by users, as well as a list of top sites.

You can even determine which IP visited a certain site, and what time it was visited.

Lightsquid can be installed through the pfSense package manager the same way you installed squid.

After installation a new menu item will be created under Status called Proxy Report.

On the settings page you can set the report refresh schedule to an interval between 10 minutes and 24 hours. This determines how often Lightsquid generates a new report. You can manually refresh the report using the refresh now button. To view the report click on the Lightsquid Report tab.

Conclusion

I hope this hub has demonstrated how easy it is to setup a transparent proxy using the power of pfSense. Transparent proxys can add value to small home networks or large corporate networks with hundreds of users.

Feel free to comment if you have any questions and please let me know what else you would like to learn about pfSense.

If you found this hub useful please take a moment to rate it or leave a comment below.

Comments

    0 of 8192 characters used
    Post Comment

    • profile image

      Timothy Lathen 3 months ago

      I have a pfsense firewall that connects and gets out to the net however I have devices that need to get to a particular proxy server and all traffic is being blocked to that proxy from within the firewall.

    • profile image

      SURYA 5 months ago

      Dear Mr. Sam Kear

      Hello.. i'm INDONESIAN want to learn about PFsense, i would like to use PFsense at my workplace who's have two ISP, the first i use for LAN connection and the second for WiFi.. which topology will you suggest for me??

      before i would to say Thank's for shared your knowledge...

    • profile image

      Shayan 6 months ago

      This awesome post. Thanks for sharing

    • profile image

      Opensolve 16 months ago

      Hi,

      Is there a way to clear the cache without downtime?

    • profile image

      himanshu 17 months ago

      hello,

      can some some tell me how can i block torrent, facebook and youtunbe through squidguard in pfsense. i try to do in target category domain: Facebook.com youtube.com and so on and denied in common access list but it s not working can some one tell me where am i wrong and whats more should i do ????

    • skear profile image
      Author

      Sam Kear 18 months ago from Kansas City

      @sikhan

      You could either block TCP port 443 to block all HTTPS traffic, or decrypt and filter the requests.

      Check out my article on HTTPS decryption.

      https://turbofuture.com/internet/Intercepting-HTTP...

    • profile image

      sikhan 18 months ago

      how i can stop https request on pfsense.

      like facebook and othere.

    • profile image

      javed 2 years ago

      Hi,

      How can i restart squid3 gracefully in pfsense. Like as in centos i can restart with the following command :

      service squid reload

      How can i achieve this in SQUID3 with pfsense.

    • profile image

      Pherez 2 years ago

      hi guys

      im trying to set up an open VPN on my pfsense server but seems like the tunneling bit of it is confusing me ,

      my ip 10.17.37.23

      sub.255.255.255.0

      gate.10.17.37.250

      dns are the 8.8.8.8 and 8.8.4.4

      how should my tunneling ip addresses be

      the tunel and local one.

    • skear profile image
      Author

      Sam Kear 2 years ago from Kansas City

      @TTGReviews

      In order for the proxy to be transparent (meaning no client settings need changed) then it would replace your router and become your default gateway.

      You can set it up as a stand alone proxy on your network but you will need to manually adjust the browser settings on your clients to target the proxy.

    • TTGReviews profile image

      TTGReviews 2 years ago

      Would I set this up alongside with my router, or would this become my router. I know pfSense is a router OS, but could I keep my current router and just add this as well?

    • Hezekiah profile image

      Hezekiah 3 years ago from Japan

      Thanks for this guide, I have been looking to set up something like this for some time.

    • profile image

      Shenin 3 years ago

      I have gone through your pfsense articles and was great. Do you have any idea how to configure BGP in pfsense

    • profile image

      Adams 4 years ago

      i have a psense proxy on my network, and the users enter their usernames and passwords up to 7 times before it finally accepts and log them on while using internet explorer, how do i reduce this to just one authentication. we are using IE8.

    • profile image

      John 4 years ago

      Thank for this nice tutorial...is it possible to edit /boot/loader.conf.local in pfsense 2.0.3

    • profile image

      Morrison 5 years ago

      Hi Skear,

      Nice work!

      I have one question:

      What do you mean by this:"Before you restart Squid you should recreate the swap directly structure."?

      Which command can do that?

      Thanks forward.

    • profile image

      AlexArt 5 years ago

      BIG THANKS to you man! ). It was very usefull to me.

    • profile image

      Mark 5 years ago

      Nice work

    • profile image

      yohans 5 years ago

      Guys on the forum page of pfsense.org, there is a step by step on how to enabled multiwan and squid in one box.

    • profile image

      jva1601 6 years ago

      Thanks for the info but I am running into an issue on my new pfsense install 2.0. I am unable to download packages. An error comes up something about check internet connection. I am to do an nslookup, ping, traceroute from the pfsense box to www.pfsense.com. So I don't think that is my problem. Any ideas what else to check?

    • profile image

      rb 6 years ago

      great article - worked like a charm!

    • skear profile image
      Author

      Sam Kear 6 years ago from Kansas City

      @Yohans, Currently it doesn't appear to be possible, squid will only use one of the wan links.

      The workaround is to install pfSense on a second machine running the proxy. The proxy machine would point to the dual wan router as it's gateway.

      The clients would point to the transparent proxy as their default gateway.

      Clients -> Squid Proxy -> Dual Wan Router

    • profile image

      Yohans C 6 years ago

      and is not possible to have multiple WAN and squid proxy in the same box?

    • profile image

      Pandoro 6 years ago

      I love you O_O

    • skear profile image
      Author

      Sam Kear 6 years ago from Kansas City

      @mon

      In order to run the proxy in transparent mode it must be on the same box as the router. So potentially you could set up one pfsense router/transparent proxy whose gateway pointed toward a second pfsense multi wan router.

      I haven't tried this before but when I find some time I will test this theory and update this hub.

      @weseppers

      I'm glad you've found the hub useful, thanks!

      @hmeister

      Thanks for your kind words! I have found pfSense to be a very valuable resource. I hope my articles will allow others to discover this to.

    • profile image

      hmeister 6 years ago

      skear --

      Thanks for this great site. It is one of a few that actually show what can be done with PFSense in a practical way. Your other articles are valuable also as a primer to use PFSense "Newbies"...

      Regards;

      H.

    • weseppers profile image

      weseppers 6 years ago

      Wow, What the great information. Thank you.

    • profile image

      mon 6 years ago

      hi Skear,

      is it possible to setup up a separate pfsense box for squid proxy server connected to a pfsense box for multi wan with loadbalancer and failover on the same subnet?HOW?

      Thanks

    • skear profile image
      Author

      Sam Kear 6 years ago from Kansas City

      Hi Mon,

      To edit the file you'll need to first log in using SSH. For instructions on setting up SSH check out http://samkear.com/how-to-guides/how-to-configure-... , the SSH instructions are at the very end of that post.

      Once you're logged in type vi /boot/loader.conf.local then hit insert and add the line kern.ipc.nmbclusters="32768". To save the file hit ESC, then type :wq! enter.

    • profile image

      mon 6 years ago

      how to to edit boot/loader.conf.local?