Security Stuffs

Creating a Threat Feed with Splunk

I operate (loosely) a couple of small websites.  None of them make any money but all of them seem to get attacked on fairly regular basis.  Of course, I keep track of the attacks in Splunk so I can make pretty graphs and such but I’ve not been using the data to it’s fullest potential.  This is something I’ve been meaning to build for a while, but have just recently gotten around to doing..  With that, I introduce the Spork Threat Feed.  *applause*

The world really needs another threat feed?  Not really, but mine isn’t for public consumption.  It’s really an efficiency issue.  Right now, my WAF scans all these requests and makes a GOOD/BAD determination about each one.  In relative terms, this layer 7 scanning is an expensive operation.  Much more expensive than say, throwing away traffic that matches an IP address in a threat feed.  This threat feed just keeps track of the GOOD/BAD determination made by the WAF so maybe the WAF can do less work.


Extracting the Data from Splunk

In my WAF, I created a custom log type that contained the rating of the offense (1-5) that was assigned by ASM, the source IP, the URI and a few other things.  These logs arrive in Splunk and because I’ve defined the format, it’s pretty easy to identify the entries that I care about.

Using the Splunk reporting and the stats command, I count up the number of attacks from each source and group them by rating.  I simply multiply the Rating by the Quantity to arrive at something termed “Annoyance”.  If Annoyance exceeds whatever threshold of pain I feel is appropriate, that IP is added to the threat feed.  This lets a particular user (generally myself) have a policy violation or two without getting added to the list.  A guy could have quite a few low rated violations (using curl without permission) or a very small number of high rated requests (SQL Injections) before being added to the list.  Yes this is a pretty simplistic model but for now it seems to work ok for now.


You Have a Threat Feed – Now What?

My layer3/4 firewall has the ability to consume a number of different threat feeds.  I just point it to a URL and every so often it will request an updated list and begin blocking.  Splunk has an option to export a report as CSV via REST, but it has a column header on it and wants to put quotes around the IP addresses..  It’s also a two-step process.  Bleh.  This means I had to write a small bit of code to grab the report and format it a bit, but it was fairly straight forward.


Couple of Notes

This post is purposefully vague as I’m still honing the reports and scripts a bit.  I’d like to be able include more data in the threat feed so I need to do some optimization of the reporting.

The graph below shows the number of requests that are being rejected based on the different categories of threat feed data. As of right now, the custom feed isn’t pick up much traffic.  I’ve not done the analysis to understand if it’s because the attackers are spurious (hit a couple hundred requests and never come back) or if I’m just not including a long enough history for my feed creation.


Only the “Sporked” items are hits against the custom feed list.  Others are hits against other threat feeds..

Leave a Reply

Your email address will not be published. Required fields are marked *