How to Set Up a Transparent Squid Proxy Server Using pfSense

Updated on January 15, 2018
skear profile image

Sam works as a Network Analyst for an algorithmic trading firm. He obtained his Bachelors Degree in Information Technology from UMKC.

Proxy servers act as an intermediary for clients on a network requesting resources from another server. The most common type of proxy is a web proxy.

Proxy servers can be very useful for improving the speed of an internet connection by caching, logging internet usage, or filtering the traffic. The proxy server will store local copies of HTML pages, images, and other files in its cache.

Caching proxy servers can greatly improve the internet performance of corporate networks or internet cafe's where many users may be requesting similar pages.

When a client requests a web page the proxy checks to see if has any of the files stored in cache, if it does it serves them to the client without having to download them from the web server.

This reduces latency and saves internet bandwidth. Transparent proxys route the clients traffic through the proxy server automatically, unlike traditional proxys which require configuration changes on the client systems.

If you are unfamiliar with pfSense check out an Introduction to pfSense.

Source

Getting Started

The first thing you'll need to do is install the squid package in pfSense. This can be done from the package manager found under the system menu.

Locate the Squid package and click the + symbol next to it to begin the installation. The installation process normally takes a few minutes to complete.

Package installation
Package installation

Configuration

After the installation is completed you will have a new menu option under 'Services' called 'Proxy Server'. Click on the new menu option to bring of the configuration page.

Here you will need to set the proxy interface which is typically LAN. Next check to box 'Allow users on the interface'. Then check the box 'Enable transparent proxy'. Now scroll down to the bottom and hit save. This will start the squid service using the settings you have defined.

At this point you have a fully functional transparent proxy server running on pfSense. You do not need to make any changes to the computers on your network for them to use the proxy. Any clients requesting web pages on port 80 will be automatically redirected through the proxy. The users on your network won't even know their traffic is going through a proxy!

Squid proxy configuration
Squid proxy configuration

Traffic Management

The traffic management tab has some settings that are useful if you want to place bandwidth usage restrictions on the proxy.

Using these settings you can configure a maximum download or upload size which will restrict transfers over a certain size limit.

You can also set the proxy to throttle binary files, cd images, or any other file type that you specify. Per host throttling sets the maximum amount of bandwidth an individual host can use.

Traffic Management Settings
Traffic Management Settings

Performance Tweaks

There are various options on the cache tab of the squid configuration page that you can modify to improve performance in your environment. Below are some of the settings I recommend modifying. If the computer running the proxy has a limited amount of disk and ram you should be cautious not to use overly aggressive settings. On the other hand if you have lots of resources to spare you can increase the settings to improve performance.

  • Hard disk cache size - This sets the total amount of hard disk space squid will use to cache objects. If you have a large hard drive you can increase this setting to cache more objects. Just remember that objects cached in memory will be retrieved faster than objects on hard disk.
  • Memory cache size - If your pfSense system has plenty of ram I recommend increasing the size of the memory cache. Objects that squid can't store in memory end up getting swapped to disk which is much slower than RAM.
  • Maximum object size -The default of 4K is pretty small, I recommend increasing this to 50. You could set it larger but most cache hits tend to take place on small files anyway.
  • Edit /boot/loader.conf.local - This change needs to be done via SSH. Using a text editor such as vi add kern.ipc.nmbclusters="32768" to the file then save the file and reboot the pfSense router. This increases the total amount of memory used for socket buffers to 32M.

Visit the pfSense documentation site for more Squid performance tuning tweaks.

Cache Management Settings
Cache Management Settings

Manually Clearing The Squid Cache

Squid has it's own system for purging old objects from the cache but occasionally you may want to clear the entire Squid cache.

I recommend logging into pfSense using SSH to run these commands. It is possible to run them using the command prompt function in the diagnostics menu but I don't advise doing so.

First you'll need to stop the proxy service.

squid -k shutdown

The command below will delete all of the files in the cache directory. This command can take a long time to run especially if you have defined a large cache size so be patient.

rm -fr /var/squid/cache/*

Before you restart Squid you should recreate the swap directly structure.

If you receive a message that Squid is already running go ahead and shutdown the service and again and rerun the command. PfSense seems to restart Squid on it's own when it notices it's not running.

squid -z

Finally you can restart Squid to begin using the proxy again.

/usr/local/sbin/squid -D

To make sure it started you can check the status on the services menu which is found under the status menu of the web interface.

Proxy Reports

If you are interested in tracking the usage of your proxy you can install an additional package called Lightsquid.

Lightsquid will generate interactive reports that track all of the websites visited by users, as well as a list of top sites.

You can even determine which IP visited a certain site, and what time it was visited.

Lightsquid can be installed through the pfSense package manager the same way you installed squid.

After installation a new menu item will be created under Status called Proxy Report.

On the settings page you can set the report refresh schedule to an interval between 10 minutes and 24 hours. This determines how often Lightsquid generates a new report. You can manually refresh the report using the refresh now button. To view the report click on the Lightsquid Report tab.

Lightsquid Report
Lightsquid Report

Conclusion

I hope this hub has demonstrated how easy it is to setup a transparent proxy using the power of pfSense. Transparent proxys can add value to small home networks or large corporate networks with hundreds of users.

Feel free to comment if you have any questions and please let me know what else you would like to learn about pfSense.

If you found this hub useful please take a moment to rate it or leave a comment below.

Questions & Answers

    Comments

      0 of 8192 characters used
      Post Comment

      • profile image

        Fahad Hameed 

        4 months ago

        can I block URLs for certain Wifi users and allow for other Wifi User.

        Like If I have more than one Wifi AP and wanted to restrict URLs on all except mine Wifi AP ??

        Is it possible in Transparent Squid Proxy Server Using pfSense.

        thanks

      • profile image

        ACSM 

        8 months ago

        How can i block proxy sites accessing & searching from google (Ex- kpproxy , hidemyass , proxyninja)

      • profile image

        lekan 

        8 months ago

        Instead of IP showing on lightsquid report. I want to use their usernames. how can I do that .

      • profile image

        Timothy Lathen 

        12 months ago

        I have a pfsense firewall that connects and gets out to the net however I have devices that need to get to a particular proxy server and all traffic is being blocked to that proxy from within the firewall.

      • profile image

        SURYA 

        14 months ago

        Dear Mr. Sam Kear

        Hello.. i'm INDONESIAN want to learn about PFsense, i would like to use PFsense at my workplace who's have two ISP, the first i use for LAN connection and the second for WiFi.. which topology will you suggest for me??

        before i would to say Thank's for shared your knowledge...

      • profile image

        Shayan 

        15 months ago

        This awesome post. Thanks for sharing

      • profile image

        Opensolve 

        2 years ago

        Hi,

        Is there a way to clear the cache without downtime?

      • profile image

        himanshu 

        2 years ago

        hello,

        can some some tell me how can i block torrent, facebook and youtunbe through squidguard in pfsense. i try to do in target category domain: Facebook.com youtube.com and so on and denied in common access list but it s not working can some one tell me where am i wrong and whats more should i do ????

      • skear profile imageAUTHOR

        Sam Kear 

        2 years ago from Kansas City

        @sikhan

        You could either block TCP port 443 to block all HTTPS traffic, or decrypt and filter the requests.

        Check out my article on HTTPS decryption.

        https://turbofuture.com/internet/Intercepting-HTTP...

      • profile image

        sikhan 

        2 years ago

        how i can stop https request on pfsense.

        like facebook and othere.

      • profile image

        javed 

        2 years ago

        Hi,

        How can i restart squid3 gracefully in pfsense. Like as in centos i can restart with the following command :

        service squid reload

        How can i achieve this in SQUID3 with pfsense.

      • profile image

        Pherez 

        3 years ago

        hi guys

        im trying to set up an open VPN on my pfsense server but seems like the tunneling bit of it is confusing me ,

        my ip 10.17.37.23

        sub.255.255.255.0

        gate.10.17.37.250

        dns are the 8.8.8.8 and 8.8.4.4

        how should my tunneling ip addresses be

        the tunel and local one.

      • skear profile imageAUTHOR

        Sam Kear 

        3 years ago from Kansas City

        @TTGReviews

        In order for the proxy to be transparent (meaning no client settings need changed) then it would replace your router and become your default gateway.

        You can set it up as a stand alone proxy on your network but you will need to manually adjust the browser settings on your clients to target the proxy.

      • TTGReviews profile image

        TTGReviews 

        3 years ago

        Would I set this up alongside with my router, or would this become my router. I know pfSense is a router OS, but could I keep my current router and just add this as well?

      • Hezekiah profile image

        Hezekiah 

        3 years ago from Japan

        Thanks for this guide, I have been looking to set up something like this for some time.

      • profile image

        Shenin 

        3 years ago

        I have gone through your pfsense articles and was great. Do you have any idea how to configure BGP in pfsense

      • profile image

        Adams 

        4 years ago

        i have a psense proxy on my network, and the users enter their usernames and passwords up to 7 times before it finally accepts and log them on while using internet explorer, how do i reduce this to just one authentication. we are using IE8.

      • profile image

        John 

        4 years ago

        Thank for this nice tutorial...is it possible to edit /boot/loader.conf.local in pfsense 2.0.3

      • profile image

        Morrison 

        6 years ago

        Hi Skear,

        Nice work!

        I have one question:

        What do you mean by this:"Before you restart Squid you should recreate the swap directly structure."?

        Which command can do that?

        Thanks forward.

      • profile image

        AlexArt 

        6 years ago

        BIG THANKS to you man! ). It was very usefull to me.

      • profile image

        Mark 

        6 years ago

        Nice work

      • profile image

        yohans 

        6 years ago

        Guys on the forum page of pfsense.org, there is a step by step on how to enabled multiwan and squid in one box.

      • profile image

        jva1601 

        6 years ago

        Thanks for the info but I am running into an issue on my new pfsense install 2.0. I am unable to download packages. An error comes up something about check internet connection. I am to do an nslookup, ping, traceroute from the pfsense box to www.pfsense.com. So I don't think that is my problem. Any ideas what else to check?

      • profile image

        rb 

        6 years ago

        great article - worked like a charm!

      • skear profile imageAUTHOR

        Sam Kear 

        6 years ago from Kansas City

        @Yohans, Currently it doesn't appear to be possible, squid will only use one of the wan links.

        The workaround is to install pfSense on a second machine running the proxy. The proxy machine would point to the dual wan router as it's gateway.

        The clients would point to the transparent proxy as their default gateway.

        Clients -> Squid Proxy -> Dual Wan Router

      • profile image

        Yohans C 

        6 years ago

        and is not possible to have multiple WAN and squid proxy in the same box?

      • profile image

        Pandoro 

        7 years ago

        I love you O_O

      • skear profile imageAUTHOR

        Sam Kear 

        7 years ago from Kansas City

        @mon

        In order to run the proxy in transparent mode it must be on the same box as the router. So potentially you could set up one pfsense router/transparent proxy whose gateway pointed toward a second pfsense multi wan router.

        I haven't tried this before but when I find some time I will test this theory and update this hub.

        @weseppers

        I'm glad you've found the hub useful, thanks!

        @hmeister

        Thanks for your kind words! I have found pfSense to be a very valuable resource. I hope my articles will allow others to discover this to.

      • profile image

        hmeister 

        7 years ago

        skear --

        Thanks for this great site. It is one of a few that actually show what can be done with PFSense in a practical way. Your other articles are valuable also as a primer to use PFSense "Newbies"...

        Regards;

        H.

      • weseppers profile image

        weseppers 

        7 years ago

        Wow, What the great information. Thank you.

      • profile image

        mon 

        7 years ago

        hi Skear,

        is it possible to setup up a separate pfsense box for squid proxy server connected to a pfsense box for multi wan with loadbalancer and failover on the same subnet?HOW?

        Thanks

      • skear profile imageAUTHOR

        Sam Kear 

        7 years ago from Kansas City

        Hi Mon,

        To edit the file you'll need to first log in using SSH. For instructions on setting up SSH check out http://samkear.com/how-to-guides/how-to-configure-... , the SSH instructions are at the very end of that post.

        Once you're logged in type vi /boot/loader.conf.local then hit insert and add the line kern.ipc.nmbclusters="32768". To save the file hit ESC, then type :wq! enter.

      • profile image

        mon 

        7 years ago

        how to to edit boot/loader.conf.local?

      working

      This website uses cookies

      As a user in the EEA, your approval is needed on a few things. To provide a better website experience, turbofuture.com uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

      For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at: https://turbofuture.com/privacy-policy#gdpr

      Show Details
      Necessary
      HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
      LoginThis is necessary to sign in to the HubPages Service.
      Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
      AkismetThis is used to detect comment spam. (Privacy Policy)
      HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
      HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
      Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
      CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
      Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the googleapis.com or gstatic.com domains, for performance and efficiency reasons. (Privacy Policy)
      Features
      Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
      Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
      Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
      Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
      Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
      VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
      PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
      Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
      MavenThis supports the Maven widget and search functionality. (Privacy Policy)
      Marketing
      Google AdSenseThis is an ad network. (Privacy Policy)
      Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
      Index ExchangeThis is an ad network. (Privacy Policy)
      SovrnThis is an ad network. (Privacy Policy)
      Facebook AdsThis is an ad network. (Privacy Policy)
      Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
      AppNexusThis is an ad network. (Privacy Policy)
      OpenxThis is an ad network. (Privacy Policy)
      Rubicon ProjectThis is an ad network. (Privacy Policy)
      TripleLiftThis is an ad network. (Privacy Policy)
      Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
      Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
      Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
      Statistics
      Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
      ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
      Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)