Monitoring Internet Usage With LightSquid and pfSense
LightSquid provides an easy and free method of monitoring internet usage on your network. LightSquid is a Squid log analyzer that runs on pfSense. By parsing through the proxy access logs the package is able to produce web based reports that detail the URLs accessed by each user on the network.
This package works well for both small and large networks. The reports have some useful features that allow you to see bandwidth usage, URL access by date and time, and top site reports.
Since LightSquid runs directly on your pfSense router it is both centralized and stealth. Users on the network have no way of knowing their traffic is being logged and analyzed using this method.
Requirements for LightSquid
Lightsquid works by analyzing Squids access logs so you must already have a Squid proxy set up in order to use Lightsquid. I always set up my proxies in transparent mode, this way all of the users traffic automatically passes through the proxy creating logs for Lightsquid to look at.
Lightsquid expects the Squid logs to be stored in the default location (/var/squid/log), so if you have Squid configured to store them somewhere else you will need to rervert to the original log location.
Lightsquid can easily be installed through the pfSense package manager. To access the package manager click on packages in the system menu. Click the plus symbol on the right side of the package to start the installation.
When the installation is complete there will be a new entry in status menu called proxy report.
LightSquid is very easy to configure, the default installations options are perfectly sufficient. At the very least I would recommend setting the refresh cycle to something reasonable for your needs.
To change the settings for LightSquid click on proxy report which is found under the status menu.
Below is an explanation of each of the settings that are available.
Language - The language setting can be used to change what language the LightSquid reports are displayed in.
Bar color - This setting lets you change the color of the bars in the reports.
Report scheme - Think of this as the theme for the appearance of the reports. The base theme is clean and simple but I like the NovoSea scheme the best.
IP resolve method - LightSquid attempts to resolve the IP address into domain names. You can change the method it uses to resolve the IPs with this setting. In my experience DNS seems to work the best.
Refresh sheduler - This setting affects how often the Squid logs are analyzed. Decreasing the value will make the reports stay more up to date but will consume more system resources. Be careful not to set the refresh cycle to occur too frequently, if the system can't finish one update before another one is requested you will eventually crash the system.
Skip url - If there are any URLs that you don't want to show up in the reports you can list them here.
Viewing the Reports
To view the LightSquid reports click on proxy report under the status menu, then click on the LightSquid report tab. The reports are very intuitive to navigate through. After you select a day you will see a list of clients that accessed the proxy on that day.
Once you select a host from the list you will see all of the URLs accessed by that client. Clicking the clock icon at the top of the page will show you the time of day that each URL was accessed.
Error attempting to access the reports
If you are getting an error when you attempt to view the reports you may need to manually update them, this is very common if you attempt to view the reports soon after LightSquid is first installed. To start a manual update click "refresh now" , then "refresh full".
Sometimes it takes a while for the initial reports to be generated, if you have a large amount of accumulated Squid logs it can take even longer so be patient.
The reports don't contain any data
If your reports don't contain any data first make sure that squid is enabled and running in transparent mode. Also make sure that logging is enabled in Squid and the log store directory is set to /var/squid/log.
You can SSH into pfSense and check the squid log directory to verify that log files are actually being created. If the Squid log files exist in the correct directory and reports are not working then something is wrong with LIghtSquid.
If all else fails try re-installing LightSquid.
© 2011 Sam Kear
More by this Author
Wireshark is a very useful tool for capturing, and analyzing network traffic. This hub explains how to run Wireshark remotely over an SSH session to analyze traffic in real time. This method saves time because it...
Heavy bandwidth users can slow down your entire network. This hub will show you how to use pfSense to set up traffic shaping to prioritize internet traffic.
By default the Squid Proxy has no visibility of encrypted HTTPS traffic. Enabling HTTPS interception will allow you to monitor and log encrypted web traffic passing through the Squid proxy server.