Cell PhonesComputersConsumer ElectronicsGraphic Design & Video EditingHome Theater & AudioIndustrial TechnologyInternet

URL Filtering - How To Configure SquidGuard in pfSense

Updated on April 28, 2016
skear profile image

Sam works as a Network Analyst for an algorithmic trading firm. He obtained his Bachelors Degree in Information Technology from UMKC.

URL filtering is a method of blocking access to certain websites based on the web address. There are several commercial products available for URL or content filtering but you can actually set up a very robust system on your own using SquidGuard and pfSense. SquidGuard is a very useful plugin for the popular Squid proxy server that can be used for blocking,or redirecting web requests on your network.

SquidGuard has a long list of features that can be customized to meet your needs. It is also extremely fast and won't slow down internet access for your users. If you need to block access to a list of undesirable websites or only allow access to certain sites then SquidGuard can assist you with this.

SquidGuard is very flexible making it easy to adapt to different applications. Whether you need to do basic URL filtering on your home network or you need to create complex rules for a large public network SquidGuard can do the job.

Before you can set up SquidGuard you will need a functioning pfSense proxy server. If you're new to pfSense I would recommend reading through the introduction to pfSense guide first.

Install the SquidGuard Package
Install the SquidGuard Package

Installing SquidGuard

SquidGuard can be installed using the pfSense package manager. To access the package manager click on packages in the system menu. Select the available packages tab and scroll down until you find SquidGuard, then click the plus symbol next to the description to begin the installation.

Once the installation is complete you will have a new menu item under services called proxy filter.

Adding a Blacklist
Adding a Blacklist

Enabling a Blacklist

To configure the blacklist feature open the general settings page (Services \ Proxy Filter). Click on the checkbox to enable the blacklist feature.

You can use one of your own blacklists or one of the publicly available lists on the web. You can find a list of several blacklists at http://www.squidguard.org/blacklists.html. Once you determine which blacklist you are going to use enter the URL of the blacklist into the blacklist URL box.

Click save, then click on the blacklist tab and click on download. It can take several minutes for the list to download and process. Once it is finished it will display "Blacklist update complete" in the status box below.

Uploading a blacklist to SquidGuard
Uploading a blacklist to SquidGuard
Editing the access control list
Editing the access control list

Access Control Lists

After uploading your blacklist you will need to configure which categories should be allowed, blocked, or white listed. The simplest method of configuration is to use the common ACL tab. The common access list settings will apply to all users of the proxy.

If you want to apply different rules to other source networks you should use the groups ACL tab. For example if you want to apply heavy filtering on a group of computers in a lab while granting unrestricted access to computers in another network you would use the group configuration.

In the drop down box next to each of the target rules you can select one of the following actions.

  • Allow - Grant acces to the target category unless blocked by another rule or exception
  • Deny - Block all access to sites in the target category
  • Whitelist - Always allow access to the target category.

I would recommend enabling the setting to block IP addresses in the URL. This will prevent your users from bypassing the filter by simply using the IP of the web site instead of the URL.

By default when a user attempts to visit a blocked page they will see an internal error page indicating that the page was blocked and which target category it falls under. You can change the redirect page to be a blank page, or any other internal or external URL. For example you could redirect the users to google if you want to.

Whenever you modify a target rule you will need to click 'apply' on the general settings tab in order for the changes to take affect.

Excluding URLs from the Blacklist

There may be certain sites that you need to allow your users to access. To prevent these sites from being blocked you can create a new target category and add a list of domains or URLS that should not be blocked.

To do this click on the target categories tab then click the plus symbol to add a new category. You must assign a unique name to the new category, the name you choose cannot contain spaces.

The target category can filter by domain name, URL, or an expression. Listing a site by domain will grant access to the main site and all of its sub pages. Entering a URL will allow access only to that exact web page. Expressions allow you to grant access based on certain keywords.

When you are finished click on save, then go back to the common or group ACL tab (where ever you created your rule) and select and action of whitelist for your new category.

You can also use this same method to add additional sites to your blacklist.

Creating new target categories
Creating new target categories

Filtering by Expression

In additional to domain and URL filtering SquidGuard can create filters using regular expressions. These types of filters are great when you want to search for certain strings of text in a URL to make a filtering decision. If you are unfamiliar with regular expressions they can be somewhat confusing at first but there are many online resources on the subject so I won't go in to much detail about them in this article.

To create a filter that uses an expression click on the target categories tab and either create a new category or edit an existing one. Enter the expression you want to filter on in the expression box and then click save. Then go back to the common or group ACL tab and select the action (deny, allow, etc ) for your target category.

Below are a few examples of filtering expressions. These could be edited depending on what you intend to filter out. For more useful information about filtering on regular expressions check out http://www.squidguard.org/Doc/Examples.

Block downloads based on file extension

(.*\/.*\.(zip|rar|exe|msi|mpeg|avi))

Block certain top level domains

(.gov|.xxx|.mil|.net)

Block searches for "proxy bypass" on Google and Yahoo

(.*(google|yahoo).*(search_query|keywords|search|query|q|p)=.*(\+|\%20)*(proxy|bypass).*(\-|\+|\%20).*(proxy|bypass).*)

Scheduling Rules
Scheduling Rules

Time Based Rules

SquidGuard also allows you to apply URL filtering based on schedules. Schedules are useful for applying rules at different times during the day, or only on certain days of the week.

For example you could apply strict URL filtering rules during business hours and automatically disable the rules after 5PM. If you are filtering your home network you may not want the kids visiting certain sites during the school week, this is another example where you would use a time based rule.

To create a time based rule click on the times tab and then click the plus sign to create a new schedule. You can create as many different schedules as you need.

Schedules can be applied using the groups ACL tab. Create a new group ACL or edit an existing one, then in the 'time' drop down box select the schedule you created.

Don't forget to click apply on the general tab for the settings to take affect.

Conclusion

Commercial web filtering appliances can be very expensive and difficult to manage. SquidGuard and pfSense are completely free and very powerful. SquidGuard offers many other features not covered in this hub. For more in-depth information visit SquidGuard.org and check out the documentation section. Also be sure to check out some of my other hubs to learn about more ways to use pfSense on your network.

Comments

    0 of 8192 characters used
    Post Comment

    • ahmadml profile image

      ahmadml 5 years ago

      Hi,

      I am using pfsense 2. I have 2 ISP connections. I have created 2 Aliases and create 2 LAN rules for these aliases to use different ISP.

      Group1 and Group2.

      Group1 is using WAN1

      Group2 is using WAN 2

      After Installing squid in transparent mode I am having the following Problem.

      Group1 traffic is going through WAN1

      Group2 traffic is also going through WAN1

      Is it possible to setup squid to use different gateways for LAN_subnet or aliases at the same time? I want

      Group1 traffic through WAN1

      Group2 traffic through WAN2

      using Squid?

      Thanks. I will Appreciate your help in this regard,

      Ahmad

    • skear profile image
      Author

      Sam Kear 5 years ago from Kansas City

      @ahmadml

      By default Squid will only use the WAN interface. Although it is possible to configure Squid to take advantage of a dual wan connection.

      Basically you will need to setup a floating rule that load balances traffic from the pfSense system itself and not just the LAN clients.

      I've linked to a forum post below that describes the process in more detail.

      http://forum.pfsense.org/index.php?topic=41252.0#m...

    • profile image

      rama 5 years ago

      Hi skear,

      nice to know you.

      About squidguard, can squidguard doing a content filtering, not just url?

      Thanks,

      rama

    • skear profile image
      Author

      Sam Kear 5 years ago from Kansas City

      @rama

      Unfortunately Squidgaurd cannot do any filtering of content within the page, it only looks at the URL's.

      You might want to look at DansGuardian, it is a true open source content filter.

    • profile image

      josh4trunks 5 years ago from San Luis Obispo, California

      Getting this under LightSquid Report, not sure what I did wrong...

      report folder '/var/lightsquid/report' not contain any valid data! Please run lightparser.pl (and check 'report' folder content)

    • skear profile image
      Author

      Sam Kear 5 years ago from Kansas City

      @josh4trunks

      Make sure Squid logging has been turned on. Then manually try to update lightsquid by clicking 'refresh now' and 'refresh full'.

      If you have a lot of Squid logs it can take a while to generate the reports the first time.

      If all else fails re-install both Squid and Lightsquid.

    • profile image

      butterflyweb 5 years ago

      Hi,

      With pfSense 2.0, can I install a platform with squidGuard 1.1? ...

      I have already installed squid and Lightsquid that work very well!

      During a previous installation, squidGuard 1.1 did not appear in menus (but I was a squid

    • profile image

      faisal 5 years ago

      i used squidguard too for my office... its powerfull

    • profile image

      Dawg Von T 4 years ago

      Hopefully there is still someone here.

      I installed SquidGuard and such, but when I set all the rules, it won't block the "Denied" websites. And yes i've saved it.

      How can i fix this ?

      -Thanks!

    • skear profile image
      Author

      Sam Kear 4 years ago from Kansas City

      @ Dawg Von T

      In order for the blocks to work both the Squid, and SquidGuard service must be running. You can check the status of both services in the status \ service menu.

      If you don't have Squid installed yet you can do so using the guide below.

      https://turbofuture.com/internet/How-to-setup-a-tr...

      If you have trouble getting both services to run try re-installing the packages. The latest version of SquidGuard has a bug that sometimes breaks the Squid package after installation.

      If both services are running, make sure the blacklist as successfully loaded in the blacklist tab of SquidGuard.

    • profile image

      Sekrit Skworl 3 years ago

      I am following your awesome guides, step by step. But, as I'm new to a lot of things outside of things "consumer grade MS", I am at a loss.

      I am running my PFSENSE box, and saving like a bastard on every change (unlike the last try).

      This is my concern:

      I got to the portion on installing "Squid Guard". However, PFSENSE only shows betas for 1.4.x & 1.4 Dev. The Log shows that 1.4 was stable but an incremental release was beta.

      So, my question is: Where is the latest stable Pfsense package?

      I appreciate your guidance.

    • profile image

      John Kap 2 years ago from Sydney, Australia - I come from a land down under.

      Hello,

      thanks for your documentation, helped me set up my system.

      I have configured a target category using a regular expression and a time based rule to stop the kids from accessing youtube on a school night. It works a treat when they try to access youtube via http, however if they type in httpS://youtube.com the filter does not work.

      I understand why it doesn't work. It's is set up to work with http, i.e. port 80, and a secure connection httpS uses port 443, hence it bypasses the system above.

      I've tried to get it operational with port 443, and the closest I can get is a proxy timeout error message when they hit httpS://youtube.com. I've obviously missed something in the setup and can't figure it out, any help pointing me in the right direction is greatly appreciated.

      John

    • profile image

      Luca 2 years ago

      Hello skear, thank you for the wonderful tutorial, i've got a problem on my squid guard proxy filter: I've created an ACL, when I set di IP address of a generic user in my LAN ti be filtered, it works great, when I set an username (of a user created in the section User Manager) it doesn't filter... the user is logged whit captive portal, i set the name like: 'name', but the result remane the same.... do I have to add some privilege for this user? thanks :)

    • profile image

      WORC 2 years ago

      Good day , can you pls. help me in blocking youtube.com(https) or blocking streaming sites. Thank you so much and God Bless

    • TTGReviews profile image

      TTGReviews 2 years ago

      How easy would it be to set this up in combination with the Captive Portal that you have also written about?

    • profile image

      Ricardo Mejias 2 years ago

      Hi Sam, how we can use the Times in squidguard so we can restrict the access in a especific time...!

    • profile image

      realComp 19 months ago

      I understand that you cant block facebook with squid since they use https://facebook.com

      How can I block that?

      Any advice using OpenDNS

      Thanks,

      Omar

    • profile image

      Mohammad qader 14 months ago

      after installing and configuring squid and squidgaurd the internet ok, but suddenly the internet disconnect. when uninstalling squid and squidgaurd the internet comes back. plz help me what should i do?

    • profile image

      Luis 11 months ago

      Hi.

      How I can configure the pfsense to see all the url and not only blocked?

    • profile image

      Chris 9 months ago

      Hi. I've been using squid successfully but by enable the blacklist, it seems to priorities it. What I mean is this. I've enabled a blacklist and it all works perfectly. I've denied the ads blacklist called [blk_BL_adv] under Common ACL however I want to allow a particular ad website through, called mailchimp.com. I than go to Target Categories and I create an Allowed_sites list with that domain. I go back to Common ACL but puts it at the top of the list so it wouldn't matter if I allow/whitelist it. How to I put the target category at the bottom?

      Cheers for the tutorial

    • profile image

      happybenng 6 months ago

      I often use allavsoft to download other YouTube videos or download video from Facebook, SBS, Blip TV, etc.

    • profile image

      TL84 2 months ago

      HI Sam,

      when using the group acl, it only seems to work when entering an IP address in the Client (Source) field. is there a way for me to be able to use a pfsense alias or pc hostname in this field and the proxyfilter does a DNS resolve? because at the moment I have to create DHCP Reservations for some of the client PCs I need to use for certain ACLs.

      Thanks

    Click to Rate This Article