5. Monitoring

Introduction

In production environments, monitoring and alerting of the FreeNAC server is recommended . This section discusses several such tools included in FreeNAC. You may choose to use these scripts, do nothing, or do similar monitoring with your own tools.

The monitoring scripts that need to be individually tested and enabled in root cron. In Previous chapters other cron scripts were mentioned, for example for monitoring MySQL. This section, which is addressed at the Linux adept, covers other housekeeping scripts that inducate to the system administrator if the FreeNAC system is behaving properly, or if specific switches or ports are having issues.

All of these are focussed on the main server, some such as process monitoring may also be used on secondary servers.

Monitoring syslog

Monitoring syslog for unusual events, is done by the logcheck which basically does a grep on the logs. See also the syslog configuration chapter.

0 8,12  * * 1-5        /opt/nac/logcheck/logcheck.sh 

The following two check that a minimum number of devices are being regularly allowed onto the network (i.e. FreeNAC is actually seeing and VMPS authenticating end-devices), and that a port is not flapping between several vlans.

30   6-22 * * 1-5 /opt/nac/bin/monitor_allows.sh
*/4 *     * * *   /opt/nac/bin/flap_detect.php

Are there any 'MAC-NOT-RECONFIRMED' from switches or vmps requests with MAC 000000 that might indicate communication problems between switches and the NAC server?

*/10 7-18 * * 1-5 /opt/nac/bin/monitor2.sh

FreeNAC Updates

Notify if there are updates to NAC (query FreeNAC.net and report if there is a new version)

0 0 * * 0        /opt/nac/bin/updates.php 

Process monitoring

Monitor_processes just does a grep on the process list and send an email alert if a process dies. This tool is run regularly from cron.

*/20 7-18 * * 1-5 /opt/nac/bin/monitor_processes.pl proctst vmpsd_external postconnect

In 802.1x mode, check samba & free radius too.

*/20 7-18 * * 1-5 /opt/nac/bin/monitor_processes.pl winbindd smbd nmbd radiusd

proctst: There is an aletrnative process monitoing with the proctst daemon. proctst (as opposed to monitor_processes) is a daemon: it does not need cron, and not just alerts when a process dies but also restarts it.
With proctst there are also unexpected side effects: if you should shutdown a daemon manually, because you want to do some debugging or so, proctst immediately restarts it.
e.g. You shutdown mysql, want to do backups or maintenance, in fact mysql was immediately restarted by proctst and you may not realise it is running.
So consider using proctst when there is an actual problem with a daemon dying, or for production servers where everything has to be as automated as possible.

Configuring proctst:

• Copy the example configuration contrib/etc/proctst.conf to /etc, review and adapt.
• Copy the startup file from contrib/startup_init.d/proctst to /etc/init.d.
• Start the daemon:
  /etc/init.d/proctst start
• Test that it works as expected; stop daemons, watch syslog etc.
• Then enable proc to automatically start when the system is rebooted:
  chkconfig proctst on [Suse/Redhat]
  update-rc.d proctst defaults [Debian/Redhat]

Other tools

To do: check_disk watches system load and disk space usage.

0 8-18 * * 1-5 /opt/nac/bin/check_disk 90 800