Sam works as a network analyst for an algorithmic trading firm. He obtained his bachelor's degree in information technology from UMKC.
Proxy servers act as an intermediary for clients on a network requesting resources from another server. The most common type of proxy is a web proxy.
Proxy servers can be very useful for improving the speed of an internet connection by caching, logging internet usage, or filtering the traffic. The proxy server will store local copies of HTML pages, images, and other files in its cache.
Caching proxy servers can greatly improve the internet performance of corporate networks or internet cafe's where many users may be requesting similar pages.
When a client requests a web page, the proxy checks to see if it has any of the files stored in cache, if it does, it serves them to the client without having to download them from the web server.
This reduces latency and saves internet bandwidth. Transparent proxys route the client's traffic through the proxy server automatically, unlike traditional proxys, which require configuration changes on the client systems.
If you are unfamiliar with pfSense, check out an Introduction to pfSense.
The first thing you'll need to do is install the squid package in pfSense. This can be done from the package manager found under the system menu.
Locate the Squid package and click the + symbol next to it to begin the installation. The installation process normally takes a few minutes to complete.
After the installation is completed, you will have a new menu option under 'Services' called 'Proxy Server'. Click on the new menu option to bring to the configuration page.
Here you will need to set the proxy interface, which is typically LAN. Next, check to box 'Allow users on the interface'. Then check the box 'Enable transparent proxy'. Now scroll down to the bottom and hit save. This will start the squid service using the settings you have defined.
At this point, you have a fully functional transparent proxy server running on pfSense. You do not need to make any changes to the computers on your network for them to use the proxy. Any clients requesting web pages on port 80 will be automatically redirected through the proxy. The users on your network won't even know their traffic is going through a proxy!
The traffic management tab has some settings that are useful if you want to place bandwidth usage restrictions on the proxy.
Using these settings, you can configure a maximum download or upload size, which will restrict transfers over a certain size limit.
You can also set the proxy to throttle binary files, cd images, or any other file type that you specify. Per host throttling sets the maximum amount of bandwidth an individual host can use.
There are various options on the cache tab of the squid configuration page that you can modify to improve performance in your environment. Below are some of the settings I recommend modifying. If the computer running the proxy has a limited amount of disk and ram, you should be cautious not to use overly aggressive settings. On the other hand, if you have lots of resources to spare, you can increase the settings to improve performance.
- Hard disk cache size: This sets the total amount of hard disk space squid will use to cache objects. If you have a large hard drive, you can increase this setting to cache more objects. Just remember that objects cached in memory will be retrieved faster than objects on the hard disk.
- Memory cache size: If your pfSense system has plenty of ram, I recommend increasing the size of the memory cache. Objects that squid can't store in memory end up getting swapped to disk which is much slower than RAM.
- Maximum object size: The default of 4K is pretty small; I recommend increasing this to 50. You could set it larger, but most cache hits tend to take place on small files anyway.
- Edit /boot/loader.conf.local: This change needs to be done via SSH. Using a text editor such as vi add kern.ipc.nmbclusters="32768" to the file, then save the file and reboot the pfSense router. This increases the total amount of memory used for socket buffers to 32M.
Visit the pfSense documentation site for more Squid performance tuning tweaks.
Manually Clearing the Squid Cache
Squid has its own system for purging old objects from the cache, but occasionally, you may want to clear the entire Squid cache.
I recommend logging into pfSense using SSH to run these commands. It is possible to run them using the command prompt function in the diagnostics menu, but I don't advise doing so.
First, you'll need to stop the proxy service.
squid -k shutdown
The command below will delete all of the files in the cache directory. This command can take a long time to run especially if you have defined a large cache size, so be patient.
rm -fr /var/squid/cache/*
Before you restart Squid, you should recreate the swap directly structure.
If you receive a message that Squid is already running, go ahead and shut down the service and again and rerun the command. PfSense seems to restart Squid on its own when it notices it's not running.
Finally, you can restart Squid to begin using the proxy again.
To make sure it started, you can check the status on the services menu, which is found under the status menu of the web interface.
If you are interested in tracking the usage of your proxy, you can install an additional package called Lightsquid.
Lightsquid will generate interactive reports that track all of the websites visited by users, as well as a list of top sites.
You can even determine which IP visited a certain site and what time it was visited.
Lightsquid can be installed through the pfSense package manager the same way you installed squid.
After installation, a new menu item will be created under Status called Proxy Report.
On the settings page, you can set the report refresh schedule to an interval between 10 minutes and 24 hours. This determines how often Lightsquid generates a new report. You can manually refresh the report using the refresh now button. To view the report, click on the Lightsquid Report tab.
Additional pfSense Guides
- How to configure a DNS blacklist using pfSense
If your looking for an easy way to block domains on your network based on many common categories DNS blacklist can do the job easily. DNS blacklist is a package for the popular pfSense platform.
- Dual Wan Router - How to Load Balance Using pfSense
Purchasing a dual wan router can easily set you back several hundred bucks. Besides the high prices many of the models on the market lack many features. So instead of shelling out cash for a router with...
- Setting up a pfSense router
If your looking to replace your home router with something that offers more control, features, and performance pfSense is an excellent choice. pfSense can act as both a router and firewall offering lots of features for free that are often only found
This article is accurate and true to the best of the author’s knowledge. Content is for informational or entertainment purposes only and does not substitute for personal counsel or professional advice in business, financial, legal, or technical matters.
Questions & Answers
Question: We have an ASA firewall on our LAN and we want to use Pfsense and the ASA at the same time, is this possible?
Answer: Yes, you can use pfSense in combination with the ASA if you wish. To use pfSense as a transparent proxy you would set the default gateway on your clients to point to the pfSense system. You would then set the gateway of pfSense to point to the ASA.
If you do not want to modify the default gateway of the clients you can modify the client proxy settings to point to pfSense. Again pfSense would need to point to the ASA as its default gateway.
beryani on December 17, 2018:
how to setup squid then i use web page when disconnect internet?
Fahad Hameed on March 29, 2018:
can I block URLs for certain Wifi users and allow for other Wifi User.
Like If I have more than one Wifi AP and wanted to restrict URLs on all except mine Wifi AP ??
Is it possible in Transparent Squid Proxy Server Using pfSense.
ACSM on December 06, 2017:
How can i block proxy sites accessing & searching from google (Ex- kpproxy , hidemyass , proxyninja)
lekan on November 22, 2017:
Instead of IP showing on lightsquid report. I want to use their usernames. how can I do that .
Timothy Lathen on August 11, 2017:
I have a pfsense firewall that connects and gets out to the net however I have devices that need to get to a particular proxy server and all traffic is being blocked to that proxy from within the firewall.
SURYA on June 16, 2017:
Dear Mr. Sam Kear
Hello.. i'm INDONESIAN want to learn about PFsense, i would like to use PFsense at my workplace who's have two ISP, the first i use for LAN connection and the second for WiFi.. which topology will you suggest for me??
before i would to say Thank's for shared your knowledge...
Shayan on April 25, 2017:
This awesome post. Thanks for sharing
Opensolve on July 20, 2016:
Is there a way to clear the cache without downtime?
himanshu on June 15, 2016:
can some some tell me how can i block torrent, facebook and youtunbe through squidguard in pfsense. i try to do in target category domain: Facebook.com youtube.com and so on and denied in common access list but it s not working can some one tell me where am i wrong and whats more should i do ????
Sam Kear (author) from Kansas City on May 23, 2016:
You could either block TCP port 443 to block all HTTPS traffic, or decrypt and filter the requests.
Check out my article on HTTPS decryption.
sikhan on May 23, 2016:
how i can stop https request on pfsense.
like facebook and othere.
javed on September 08, 2015:
How can i restart squid3 gracefully in pfsense. Like as in centos i can restart with the following command :
service squid reload
How can i achieve this in SQUID3 with pfsense.
Pherez on July 07, 2015:
im trying to set up an open VPN on my pfsense server but seems like the tunneling bit of it is confusing me ,
my ip 10.17.37.23
dns are the 18.104.22.168 and 22.214.171.124
how should my tunneling ip addresses be
the tunel and local one.
Sam Kear (author) from Kansas City on January 28, 2015:
In order for the proxy to be transparent (meaning no client settings need changed) then it would replace your router and become your default gateway.
You can set it up as a stand alone proxy on your network but you will need to manually adjust the browser settings on your clients to target the proxy.
TTGReviews on January 26, 2015:
Would I set this up alongside with my router, or would this become my router. I know pfSense is a router OS, but could I keep my current router and just add this as well?
Hezekiah from Japan on September 11, 2014:
Thanks for this guide, I have been looking to set up something like this for some time.
Shenin on August 29, 2014:
I have gone through your pfsense articles and was great. Do you have any idea how to configure BGP in pfsense
Adams on November 16, 2013:
i have a psense proxy on my network, and the users enter their usernames and passwords up to 7 times before it finally accepts and log them on while using internet explorer, how do i reduce this to just one authentication. we are using IE8.
John on November 13, 2013:
Thank for this nice tutorial...is it possible to edit /boot/loader.conf.local in pfsense 2.0.3
Morrison on April 16, 2012:
I have one question:
What do you mean by this:"Before you restart Squid you should recreate the swap directly structure."?
Which command can do that?
AlexArt on February 14, 2012:
BIG THANKS to you man! ). It was very usefull to me.
Mark on January 02, 2012:
yohans on November 25, 2011:
Guys on the forum page of pfsense.org, there is a step by step on how to enabled multiwan and squid in one box.
jva1601 on November 02, 2011:
Thanks for the info but I am running into an issue on my new pfsense install 2.0. I am unable to download packages. An error comes up something about check internet connection. I am to do an nslookup, ping, traceroute from the pfsense box to www.pfsense.com. So I don't think that is my problem. Any ideas what else to check?
rb on October 13, 2011:
great article - worked like a charm!
Sam Kear (author) from Kansas City on October 09, 2011:
@Yohans, Currently it doesn't appear to be possible, squid will only use one of the wan links.
The workaround is to install pfSense on a second machine running the proxy. The proxy machine would point to the dual wan router as it's gateway.
The clients would point to the transparent proxy as their default gateway.
Clients -> Squid Proxy -> Dual Wan Router
Yohans C on October 07, 2011:
and is not possible to have multiple WAN and squid proxy in the same box?
Pandoro on May 12, 2011:
I love you O_O
Sam Kear (author) from Kansas City on May 08, 2011:
In order to run the proxy in transparent mode it must be on the same box as the router. So potentially you could set up one pfsense router/transparent proxy whose gateway pointed toward a second pfsense multi wan router.
I haven't tried this before but when I find some time I will test this theory and update this hub.
I'm glad you've found the hub useful, thanks!
Thanks for your kind words! I have found pfSense to be a very valuable resource. I hope my articles will allow others to discover this to.
hmeister on May 08, 2011:
Thanks for this great site. It is one of a few that actually show what can be done with PFSense in a practical way. Your other articles are valuable also as a primer to use PFSense "Newbies"...
weseppers on May 05, 2011:
Wow, What the great information. Thank you.
mon on May 03, 2011:
is it possible to setup up a separate pfsense box for squid proxy server connected to a pfsense box for multi wan with loadbalancer and failover on the same subnet?HOW?
Sam Kear (author) from Kansas City on March 05, 2011:
To edit the file you'll need to first log in using SSH. For instructions on setting up SSH check out http://samkear.com/how-to-guides/how-to-configure-... , the SSH instructions are at the very end of that post.
Once you're logged in type vi /boot/loader.conf.local then hit insert and add the line kern.ipc.nmbclusters="32768". To save the file hit ESC, then type :wq! enter.
mon on March 05, 2011:
how to to edit boot/loader.conf.local?