Configuring Network Switches

Introduction

This document aims to explain how to configure Switches for use with FreeNAC, and how to troubleshoot. Focus is currently on Cisco.

This document is divided into several subpages, if you wish to see it all in one page, please click the "Printer-friendly version" link below.

VMPS parameters

Vlans

The Vlan names and number must be configured on switches exactly the same as in the Vlan table in FreeNAC. NAC does not configure this on the switches for you.

So for example, if NAC is going to attribute the Vlans 'Printer' and 'Workstation', these two must be defined exactly with the same name and number on the Switches, and in FreeNAC.

FreeNAC also allows 'location based vlans' i.e. the vlans names do not have to be the same on all switches, refer to the FreeNAC Users Guide >> Windows GUI >> Configuration: Vlans.

syslog

Its recommended to configure the switches to send a copy of their logs to the NAC server, helping in troubleshooting.

vmps server

CATOS:

  set vmps server   192.168.245.40

  clear vmps server 192.168.245.18

  reconfirm vmps

  sho vmps

IOS:

conf t   
vmps server     192.168.245.40  
no vmps server  192.168.245.18 
end
vmps reconfirm
sho vmps

VMPS “retry” switch parameter

The following is an extract from "Troubleshooting Connectivity Between the VMPS Client and the VMPS server", http://www.cisco.com/warp/public/473/157.html#topic1-3

VMPS reconfirmation occurs when the VMPS client asks the VMPS if the dynamic port assignments are correct and if the correct MAC addresses have been assigned to the right ports. By default, this happens about every 60 minutes. Issue a show vmps command on the VMPS client to determine the VMPS reconfirmation time.
If the connectivity between the VMPS client and VMPS is intermittent (some data gets lost along the way) then you can try to increase the VMPS retry interval on the VMPS client, as a workaround. Issue the set vmps server retry command. By default, the VMPS client will try three times. In an environment with intermittent connectivity, when you increase the VMPS retry interval, you give the client more chances to connect to the VMPS before it gives up and VLAN membership fails.

Since Version 2.0, FreeNAC queries an SQL database in real time when authenticating end devices. There is also an optional "hub detection" feature which means it tries to detect and ping all devices already on a hub. Thus authentication can take seconds.

This can lead to the switch getting impatient, sending several requests and logging MACNOTRECONFIRMED messages to syslog, especially when reconfirming all ports each hour. One solution is to increase the vmps retry count from the standard 3, to say, 10.

When there is a loss of connectivity between a VMPS client and a VMPS, the VMPS reconfirmation might fail and produce the DVLAN-2-MACNOTRECONFIRMED error message. The port will lose its DVLAN assignment, as in this example:

     %DVLAN-2-MACNOTRECONFIRMED:Mac [00-00-f4-11-11-0f] is not reconfirmed
     %DVLAN-1-DENYHOST:Host 00-00-11-11-11-0f denied on port 3/10

Cam” timeouts on “silent” servers

After the end-device transmits and the switch receives a valid response from the VMPS server, the switch enables the interface in the correct VLAN. If the client sits idle for a while causing the bridge aging timer to expire for the entry, the Catalyst returns the port to an unassigned state.

Therefore

'silent' servers, (or printers for example) would be disconnected from the network if they did not transmit packets at least every 5 minutes.
If the VMPS daemon died during the night, users would not be able to login in the morning. (Of course this can be mitigated by with redundnacy mechanisms).

The aging timer (or CAM: content addressable memory) can be viewed on CatOS switches with:

sh cam agingtime (The default value is 300 seconds).

This timeout can be increased to several hours. This increases the risk of arp flooding (we think), but this is a low risk on internal network hopefully. It is recommended to set a value like 12 hours for dynamic/VMPS ports. This is important for switches that have servers/printers that may not send out any packets for several minutes or hours.

Its also recommended to use logcheck or a similar tool, to watch for unusual Switch syslog entries, especially floods.

CatOS:
The value can be set in seconds and per vlan. It needs to be set for each VLAN, for example on VLAN 4:

 show cam agingtime VLAN_NR

set cam agingtime VLAN_NR XXX  (secs, e.g. 24h=86400, 12h=43200)

IOS:

arp mac-address-table aging-time XXX  (secs)

Other notes

A graphical example

IOS example1

Example of migrating Switches to use a new VMPS server

Assuming we had two previous VMPS servers 192.168.245.18 and 192.168.245.19, and we now wish to change the switches to use a new server 192.168.245.40. Then logon on to the switches and do the following.

Monitoring: watch the syslog entries on the vmps server, the updating of the “last seen” times and “Server log” in the Windows GUI.

CATOS:
set vmps server 192.168.245.40
clear vmps server 192.168.245.19
clear vmps server 192.168.245.18
reconfirm vmps
sho vmps

IOS:
conf t
vmps server 192.168.245.40
no vmps server 192.168.245.18
no vmps server 192.168.245.19
end
vmps reconfirm
sho vmps

Cisco CatOS configuration examples

CatOS

Initially, Switches must be configured to send a copy of syslog messages, and given the name of the vmps servers, where it can send requests for dynamic port assignment. See also http://www.cisco.com/en/US/products/hw/switches/ps607/products_configura....

# Setting up syslog servers

set logging server 192.168.245.40

# Set VMPS servers

set vmps  server 192.168.245.40

set vmps  server 192.168.245.19 primary

set vmps  server 192.168.245.18

# Remove a VMPS server & show status

clear vmps server 192.168.245.19

show vmps

# Lets make a port dynamic & ask the switch to re-authenticate all dyn ports, i.e. use VMPS

  set port membership 2/36 dynamic

  reconfirm vmps

# To switch a port back to static Vlan (if you had problems)

  set port membership 2/36 static

# To verify port

  show port status 2/36

# to disable/enable port (simulate cable being removed)

  set port disable 2/36

  set port enable 2/36

# The switch tries to contact a server 3 times by default, before stopping. This value can be programmed on the switch (to a maximum of 10):

  set vmps server retry 5

# The switch reconfirms by default every 60 minutes, set it to 120:

set vmps server reconfirminterval 120

# Other useful commands:

  show mac-address-table address 00:04:dd:b6:5c:c2

  show cdp neighbors

  show cdp neighbors Gi4/5

  show cdp neighbors Gi4/5 detail

# Tag a name to a port (to document usage)

  set port name 2/32 webcam

# Look at the MAC table:

  show arp

ARP Aging time = 1200 sec
+ - Permanent Arp Entries
* - Static Arp Entries
192.168.1.19 at 00-03-ba-17-fa-bf port 2/49 on vlan 2
192.168.1.18 at 00-03-ba-18-06-4b port 2/49 on vlan 2

show port status 2/32

Port Name Status Vlan Level Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
2/32 inactive dyn- normal auto auto 10/100BaseTX

show cam dynamic 2/43

* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry.
X = Port Security Entry $ = Dot1x Security Entry
VLAN Dest MAC/Route Des [CoS] Destination Ports or VCs / [Protocol Type]
---- ------------------ ----- -------------------------------------------
3 00-04-76-15-48-30 2/43 [ALL]
Total Matching CAM Entries Displayed =1

show cam 00-04-76-15-48-30

Problems with “clear vmps server” on old CatOS

The command for removing vmps server “clear VMPS server” seems to be missing from older CatOS versions, there is no known workaround except either upgrading CatOS, or avoiding deleting the server IP address!

The offending Switches had the following version.
> (enable) show version
WS-C2948 Software, Version NmpSW: 6.3(1)
Copyright (c) 1995-2001 by Cisco Systems, Inc.
NMP S/W compiled on Jul 24 2001, 12:55:29
GSP S/W compiled on Jul 24 2001, 10:36:29
System Bootstrap Version: 4.4(1)
Hardware Version: 2.1 Model: WS-C2948

Cisco IOS SNMP v3 setup

Please note that this guide doesn't apply anymore to FreeNAC 3,In FreeNAC 3, the programming of the switches is done using PHP's SNMP libraries, instead of using the Linux utilities. This guide applies to prior versions of FreeNAC used along with SuSE Linux.

SNMP v3 setup

(contribution from 'immi')
To use authentication and encryption with SNMP and also restriction by access-list who can access my device.
For SNMP write I enabled only limited part of SNMP tree (.1.3.6.1.2), read is open.

1. Cisco Switch part in config mode:

snmp-server group secure v3 priv
snmp-server group secure v3 priv read secure-ro write secure-wr access 1
snmp-server view secure-ro internet included
snmp-server view secure-wr mgmt included
snmp-server user snmpusr secure v3 auth md5 cisco123 priv des56 cisco123

access-list 1 permit host x.x.x.x
access-list 1 deny any log

# then you can check
VMPSclient#sho run | incl snmp
snmp-server group secure v3 priv read secure-ro write secure-wr access 1
snmp-server view secure-ro internet included
snmp-server view secure-wr mgmt included
snmp-server location /CZ/PRG/ROOM249
snmp-server contact CallMe ext.: xxxx
VMPSclient

VMPSclient#sho snmp group
groupname: secure security model:v3 priv
readview : secure-ro writeview: secure-wr
notifyview: <no notifyview specified>
row status: active access-list: 1

VMPSclient#sho snmp user
User name: snmpusr
Engine ID: 8000000903000014A86637C0
storage-type: nonvolatile active
Authentication Protocol: MD5
Privacy Protocol: DES
Group-name: secure

# two examples for check if it is working:
snmpwalk -v 3 -u snmpusr -l authPriv -a MD5 -A cisco123 -x DES -X cisco123 172.16.1.1 system
snmpwalk -v 3 -u snmpusr -l authPriv -a MD5 -A cisco123 -x DES -X cisco123 172.16.1.1 sysUpTime

2. Then modify the default SNMP values on Freenac server, it is in /usr/share/snmp/snmp.conf:
vmpssrv:~ # cat /usr/share/snmp/snmp.conf
(comments are erased)
defversion 3
defsecurityname snmpusr
defsecuritylevel authPriv
defauthtype MD5
defauthpassphrase cisco123
defprivtype DES
defprivpassphrase cisco123

To test, snmpwalk 172.16.1.1 system

3. Modify /opt/nac/etc/config.inc
(just part for port reset)
## restart_port
# $snmpwalk="/usr/bin/snmpwalk -v 1 -c public"; # SNMP Read community
# $snmpset ="/usr/bin/snmpset -v 1 -c private"; # SNMP Write community
$snmpset ="/usr/bin/snmpset"; # SNMP Write community
$snmpwalk="/usr/bin/snmpwalk"; # SNMP Read community

Cisco IOS vmps configuration

Network Switch Configuration & Tips for Cisco IOS

Initially, Switches must be configured to send a copy of syslog messages, and given the name of the vmps servers, where it can send requests for dynamic port assignment. Relevantr Cisco docs:

Configuring VMPS

conf t 
no vmps server 192.168.245.41
vmps server 192.168.245.40
vmps reconfirm 120
end
show vmps

Re-authenticate all current connections

vmps reconfirm

Re-authenticate all current connections, by emptying the MAC table. Note that the previous “vmps reconfirm” will not re-allow systems that were previously denied. For that we need to clear the MAC table.

clear mac-address-table dynamic

Enable VMPS on port fa0/2:

conf t
int fa0/2
switchport access vlan dynamic

(Re-)enable static Vlan 8 on port fa0/2:

conf t
int fa0/2
switchport access vlan 8

The switch tries to contact a server 3 times by default, before stopping. This value can be programmed on the switch (to a maximum of 10):

vmps retry 5

The switch reconfirms by default every 60 minutes, make it 2hrs :

vmps reconfirm 120

Other commands

show vmps stat
clear vmps statistics

show vlan
sh mac-address-table
sh mac-address-table | inc DYNAMIC
sh mac-address-table | inc BLOCKED

Debug the switches logic: when and how does it send queries and how does it interpret answers?

ter mon
debug vqpc all

Why to add a "clear_mac" feature?

A problem in newer IOS Cisco switches has been detected.

When an unknown
computer connects, a DENY from FreeNAC is received and the switch port
blocks access. If later the properties of the connecting device are
modified in order to allow it access the vlan, the port will remain in
the blocked stated for that device, preventing any further VMPS
requests from reaching the FreeNAC server. The amount of time the port
remains in the blocked state is variable. A port restart doesn't change
the port status, neither does disconnection of the network cable from
the switch port.

After some analysis, it has been discovered that
removing the MAC address from the switch's CAM table will remove the
blocked state and the port will work as expected. Therefore such a 'clear mac'
function has been added to FreeNAC in V3.0.3 as a complement to
port_restart

See the thread in the forum where this problem was initially discussed.

Cisco 802.1x tests

Introduction

This sections contains results from some test with 802.1x on Cisco switches and FreeRadius.

Setup on an access point on port 2/22

Lets say there is an access point on port 22, first set it to static and assign a trunk with the appropriate vlans:

set port membership 2/22 static
Port 2/22 vlan assignment set to static.
Spantree port fast start option set to default for ports 2/22.

set trunk 2/22 on
clear trunk 2/22
Port(s) 2/22 trunk mode set to auto.
Port(s) 2/22 trunk type set to dot1q.

sw0503> (enable) set trunk 2/22 11-12,15
Vlan(s) 11-12,15 already allowed on the trunk
Please use the 'clear trunk' command to remove vlans from allowed list.

Setting up 802.1x on port 0/2

logging 192.168.245.40
aaa authentication dot1x default group radius
dot1x system-auth-control
radius-server host 192.168.245.40 auth-port 1812 acct-port 1813 key 7 141E1C040D14
radius-server retransmit 3

# a port with static Vlans:
interface FastEthernet0/2
switchport access vlan 15
switchport mode access
dot1x port-control auto
no cdp enable
spanning-tree portfast

# dynamic vlans: vlan is returned by the radius server
interface FastEthernet0/2
switchport access
switchport mode access
dot1x port-control auto
no cdp enable
spanning-tree portfast

## Option: reauthenticate every two hours
dot1x timeout reauth-period 7200
dot1x reauthentication

## Other options
#dot1x default
#dot1x guest-vlan 524
#dot1x auth-fail vlan 522

##Enabling MAC-auth-bypass in switches that allow this option
#dot1x mac-auth-bypass

##Timing options specially for MAC-auth-bypass
#dot1x max-reauth-req 3 #Number of EAP requests sent to the client before trying MAC-auth-bypass
#dot1x timeout quiet-period 5 #Number of seconds to retry auth after a failed auth
#dot1x tx-period 5 #Number of seconds to wait for an answer after an EAP request has been sent to the client

##aaa authorization network default group NAC

testing

#sh dot1x

Sysauthcontrol = Enabled
Supplicant Allowed In Guest Vlan = Disabled
Dot1x Protocol Version = 1

#sh dot1x interface fastEthernet 0/2

Supplicant MAC <Not Applicable>
AuthSM State = CONNECTING
BendSM State = IDLE
Posture = N/A
ReAuthPeriod = 3600 Seconds (Locally Configured)
ReAuthAction = Reauthenticate
TimeToNextReauth = N/A
PortStatus = UNAUTHORIZED
MaxReq = 2
MaxAuthReq = 2
HostMode = Single
Port Control = Auto
ControlDirection = Both
QuietPeriod = 60 Seconds
Re-authentication = Enabled
ReAuthPeriod = 3600 Seconds
ServerTimeout = 30 Seconds
SuppTimeout = 30 Seconds
TxPeriod = 30 Seconds
Guest-Vlan = 0
AuthFail-Vlan = 0
AuthFail-Max-Attempts = 3

debug dot1x ?
all All Dot1x debugging messages turned on
errors Error codes
events Events
packets Packets
registry Registries
state-machine State machine
undebug all

#debug dot1x errors
Dot1x Errors debugging is on

References

http://www.cisco.com/en/US/products/hw/switches/ps5213/products_configur...
http://wiki.freeradius.org/Rlm_perl
http://www.cisco.com/univercd/cc/td/doc/product/lan/cat3560/12225see/scg...

Mac bypass authentication: (note not all IOS switches have this..)
http://www.cisco.com/en/US/products/hw/switches/ps708/products_configura...

Notes

Note: For FreeRadius assigning VLANs dynamically, do a users file with:
> DEFAULT Auth-Type == MS-CHAP or
> NAS-IP-Address==x.y.z.w, NAS-Port = 50001
> Tunnel-Type = VLAN,
> Tunnel-Medium-Type = 6,
> Tunnel-Private-Group-ID = VLAN_number
>
> DEFAULT Auth-Type := Reject
>
> You need to keep this file for every vlan you want to return and the
> request attributes you want to check.
> In fact, the script I have does exactly this. It outputs just those
> values at the end of the authentification process (post_auth), and
> then the switch assigns the client the vlan that VMPS has returned.
> I think it is easier than maintaining the users file by ourselves

0008.7446.2aa5

------------------------------
/opt/nac/bin/rad2vmps

$request{server_ip}='freenac'
in the post_auth function

Then modify radiusd.conf accordingly
// Radiusd.conf in the modules section add
verify_mac {
module = "/opt/nac/bin/rad2vmps"
}
//Authorize section
authorize {
verify_mac
eap
}
// Add a post-auth section
post-auth {
verify_mac
}

Setting up the nas-port attribute
-----------------------------------------
conf t
radius-server attribute nas-port format X
where X can be

a Format is type, channel or port
b Either interface(16) or isdn(16), async(16)
c Data format(bits): shelf(2), slot(4), port(5), channel(5)
d Data format(bits): slot(4), module(1), port(3), vpi(8), vci(16)

Recommended for FreeNAC: a (default)

Reference: http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_...

Sending vendor specific attributes
------------------------------------------
conf t
radius-server vsa send authentication
end

Using 'ciscocmd'

Introduction

'ciscocmd' is a useful tool for remotely executing commands or querying cisco swithes. Is is briefly described here as it is useful when operating FreeNAC in a large environment.

Cisco-centric Open Source Initiative
http://sourceforge.net/projects/cosi-nms
http://cosi-nms.sourceforge.net/

This is a great tool for 'remote control' of Cisco switches. Some examples are below.

Download and extract, no compilation is needed.
These tests were done with v1.4, I installed in /opt/nac/ciscocmd-1.4.

Single switch example

Example switch is SWITCH1)
./ciscocmd -u USER -p MYPASS -t SWITCH1 -c "show vmps"
./ciscocmd -u USER -p MYPASS -t SWITCH1 -c "reconfirm vmps" -e -s MYPASS
./ciscocmd -u USER -p MYPASS -t SWITCH1 -c "ping 192.168.1.40" -e -s MYPASS

(change USER, MYPASS; and the enable password as needed..)

Several switches

# Get all CatOS switches from the FreeNAC DB (hw type 2948, store in catos.txt),
echo "select name from switch where hw like '%2948%' order by name;" | mysql opennac |egrep -v name > catos.txt

# and check their vmps status:
./ciscocmd -u USER -p MYPASS -T catos.txt -c "show vmps" | egrep "VMPS Action|VMPS Last Accessed|Last Reconfirmation|show vmps"

Configuring Network Switches

Introduction

Contents

VMPS parameters

Vlans

syslog

vmps server

VMPS “retry” switch parameter

Cam” timeouts on “silent” servers

Other notes

A graphical example

Example of migrating Switches to use a new VMPS server

Cisco CatOS configuration examples

CatOS

Problems with “clear vmps server” on old CatOS

Cisco IOS SNMP v3 setup

SNMP v3 setup

Cisco IOS vmps configuration

Why to add a "clear_mac" feature?

Cisco 802.1x tests

Introduction

Setup on an access point on port 2/22

Setting up 802.1x on port 0/2

testing

References

Notes

Using 'ciscocmd'

Single switch example

Several switches