How to tune Cisco AIP-SSM IPS module

Note: this article has been written for AIP-SSM 7.0.3(E4) OS image with Cisco IPS Manager Express 7.0.3


The company for which I work bought me:) Cisco AIP-SSM module since I asked for something to use for IPS/IDS purposes. AIP-SSM was logical choice since I replaced our "old" 2811 IOS firewall + IPS router with ASA 5520 since the router's CPU was melting with our 50 megs Internet pipe and about 60 connections / sec + IOS FW+IOS IPS+traffic shaping with dozens of CBWFQ policies nested + NAT of course;) ASA 5520 does the same job (except QoS) with 3-5 % of average CPU usage.

In this blog post I want to share my experience with tuning this AIP-SSM baby.

Ok first I had to upgrade the thing (that is install new OS image) because I wanted to use new fancy Cisco IPS Manager Express software which I think is great and easy to use. After that I configured policy-map on my ASA firewall to inspect only traffic destined for my Internet facing web and ftp server which serves for secondary purposes so if the IPS breaks something very few people will notice:) And then the first issue. Suddenly I couldn’t connect to the web server from Internet any more. WTF? IPS event log is empty, but somehow I had a feeling that my connection was denied by the IPS. Ok. What was my first mistake? Well, since my web server is published to the Internet with static NAT configured on my ASA I placed IPS policy-map onto its outside interface and in class-map I specified web server's NAT inside global address. This (I guess) confused the ASA which routed traffic matched by IPS class-map to some nasty black hole. Very strange. I didn't expected such behavior. I expected that in the case of wrong traffic definition or misplaced IPS policy-map ASA does not catch anything to send to the AIP-SSM module, but not to block that traffic. I didn't sniffed Cisco Bug Toolkit for it, but this smells me like some ASA OS bug. I run 7.0 OS of the ASA for now, so if you run some later versions of ASA OS it may behave differently. Whatever, since ASA order of operations for inbound traffic says that inbound packets are first NATed back to the inside local address and then processed by modular policy framework I moved my IPS policy-map little closer to me - onto inside interface and class-maped internal IP address of the web server. And it worked like a charm. IPS detected some nmap port scanning I tried against my web server, root ftp password logins, etc.

Next after that little trial period I decided to put my entire Internet communication under the AIP-SSM microscope since the module has enough capacity to inspect my whole Internet tube.

Now folks, let's go serious so here is the rule number one: never, I mean really NEVER place your brand new out-of-the box IPS device in IPS mode for the first time. No matter how good and accurate your IPS might be there are good chances that you’re going to get some false positives. In the case of Cisco AIP-SSM if those false positives are with risk rating score of 90 or more then Event Action Override (if enabled and left with default actions) will ban potential attacker from your network for one hour period. This "attacker" might be your CEO's workstation or even worse: your Exchange Server:)) That is the reason why for the first time (for me it was about 1 month) you should configure IPS policy-map to work in IDS mode. In this mode ASA firewall will send only copy of your traffic to the AIP-SSM module, so it will be able to only detect, but not really block potential attackers and kill legitimate traffic. Also in IPS module configuration under Event Action Override uncheck the 'Reset TCP Connection' action checkbox for the duration of this testing (learning) phase because default Event Action Override rule states that any fired event with risk rating of 90 or more will cause AIP-SSM to (among other restrictive actions) send TCP resets to the attacker in the case that attack was carried on TCP protocol which is mostly the case. TCP resets are sent even if the device works in IDS module causing potential false positives to disrupt legitimate traffic.

Now when you have your module in completely Monday-morning-emergency-NoInternetConnection-calls free :) mode you should start tuning your IPS.

Before you start analyzing events a slight configuration of AIP-SSM module is strongly advised to make your life a little bit easer:

1. Define OS Identifications for your valuable and most risky assets (Internet facing servers for example) on this way IPS will know which attacks are relevant to your infrastructure.
2. Define Event variables - this helps later in the tuning phase 'cause you will create rules (Event Action Filters) to catch false positive traffic and prevent blocking it. In the Event Action Filters it's much easier to say for example $internal for all of your internal subnets then to count all your internal address space in each filter. This will also give you great flexibility if you renumber your IP network in the future in which case you only need to follow this change in Event Action Variables while leaving filters intact. On the same way network/host/protocol object grouping in ASA firewall configuration works.
3. Target Value Rating - specify hosts on your network that needs the greatest level of protection. Doing this AIP-SSM will greatly increase risk rating score for the attacks tried against these critical hosts.
4. Clear "Test Global Correlation" check box to make IPS stops blacklisted attackers immediately even before they try something evil.
5. Turn off "Network Participation" - this may sounds selfish but I don't want my internal hosts firing some false positive alarms to be listed on some black lists.

After implementation of the steps above it's time to examine IPS logs.

Wait for a week and then look for events with the risk rating of 90 or more. After you isolate these events dig even further searching for actions taken such as: deniedAttacker not performed, etc. These "not performed" actions are not performed because the module operates in IDS mode. In IPS mode these actions would be performed, of course. Now each of isolated events (risk rating >= 90 with some restrictive actions taken) should be analyzed by whoising victim and attacker IP address, examining traffic flow, packet capture traffic in some hard-to-distinguish cases, etc. For this you should really know your network and what kind of traffic and protocols are normal and good and which are pure evil:). Simply said you must be able to distinguish good from evil or else you have great chances to miss something important. After you analyzed events you should create Event Action Rule for each of them. In Event Action Rules subtract only blocking actions while leaving logging actions active. Some authors advise to subtract even logging events but I like to have everything logged for just a case if I need some historical data for forensic work, etc. Of course with lot of false positives in it your log will become a reading nightmare, so in IPS Manager Express I can create nice event reports filtering only important events without false positives. For example I usually like to see all events generated for my critical servers (even false positives if there are only few, otherwise only medium or high severities). For other things like outbound traffic I usually display only medium and high risk events to check if some worms are trying to spread or for some client side vulnerabilities exploitation attempts and so on.

As I previously said you should monitor your IPS and analyze logs on the way I described above for some period of time let say one month or even more. It depends on how quickly you need IPS protection in place, traffic volume (more traffic more like hood for false positives). Don't give up of IDS mode too quickly. I made a mistake and left it in IDS mode for a couple of days only and wrongly concluded that all false positives generated by my network are covered with Event Action Rules, but I was wrong. Here is the story: since we are small ISP for daughter firms inside our holding company one of our customers informed me by panic call:) that they don't have Internet connection anymore. Upon the call I was assumed that my new IPS has something with it:) and I was right. Signature ID 5930 (Generic SQL Injection) was fired for the traffic from public IP address of this customer. Risk Rating of this event was more then 90 and this customer was blocked for Internet access automatically. Normally some paranoid rookie admin would simply assume that IPS was right and someone from this network tried to SQL injection some web application on the Internet. But not me:) first I whoised the victims IP address and discovered that it is owned by Google:) Then I asked my DNS for RR record in hope that this IP has RR record. And bingo:) it was Google Analytics. So it was perfectly legitimate traffic since server from my customer updated some web statistics to Google Analytics and my IPS found SQL 'union select' statements in HTTP requests. I immediately created Event Action Rule subtracting blocking actions for such signature if "malicious" traffic flows from inside to outside. Now my IPS shall not disturb Google Analytics but it will prevent some SQL injection attacks against my Internet facing web applications.

Also there are a couple of noisy sweep signatures which normally should fire in the event of port or IP range scanning attempts but the ugly truth is that they fires for many multi connection applications. As you know some applications will try to connect to different hosts one by one (like network printer manager apps) which look like port scanning. It is completely safe to disable those signatures and leave port scanning and worm propagation detection to the Anomaly Detection feature of Cisco AIP-SSM.

And of course it's always good practice to test your IPS implementation against couple of known application exploits. For this I recommend Nessus scanner and Metasploit Framework combination. It's just to see if the IPS really works. Don't fool yourself that you will be able to test how effective your IPS is against tools like Metasploit framework because no way you can try all exploits and all the ways of tunneling/sneaking them to the target. Leave that to professional labs and read their reports (don't trust them blindly;)). When choosing IPS don't try to pen test it (you can create a false sense of security!) but instead watch how many false positives they generate for your network and usability of its user interface and compare all of that to competitive products. Yet another simply way to try IPS quickly without playing with Metasploit (anyway I advise you to learn this tool - pure fun and adrenalin:)) is to send a packet with the same source and destination IP address. Simply from your router or firewall ping it's own outside interface. The IPS should generate 'impossible IP packet' signature. I tried this "technique" with IOS IPS and AIP-SSM. I'm not sure if it works with other products. Of course this test isn't applicable to the standalone IPS sensors, but only to the integrated ones such as AIP-SSM module or IOS IPS. Anyway I don't practice this test by myself because it cannot check other important signature micro engines such as HTTP. I rather use Metasploit and try some most famous working exploits like ms08_067_netapi against unpatched Windows XP SP2 or earlier or for the client side ms10_002_aurora - a legendary IE Aurora bug against unpatched MS Internet Explorer 6 and 7.

If these tests fire signatures then you now that your IPS is working, but it's not "set and forget" kind of device - it requires at least daily event monitoring, hunting potential false and true positives... And don't forget to educate at least one member of your networking crew to handle the IPS because you don't need a spoiled vacation while trying to unblock good traffic that false-positively fired some killer signature;)