« Getting started with honeyd

Kippo-Graph 1.3 released! »

Aug 09 2014

Adding ElasticSearch support to Kippo SSH honeypot

I am very fond of ElasticSearch as a storage infrastructure and I do believe it is very useful for storing attack data, especially from honeypots. If you follow my blog, you would have seen my first attempts at transferring Kippo’s data to ElasticSearch, or creating Kibana dashboards to visualize SSH attacks. These eventually led to the Kippo2ElasticSearch script, a simple way to transfer your logged Kippo data from MySQL to an ES instance.

But, having just a script (which keeps no state by the way) is not the best way to go about it. So I decided to add ElasticSearch support to Kippo itself. For that purpose I have created a fork of Kippo which is now available for testing. The git repo is hosted on GitHub: https://github.com/ikoniaris/kippo

The way it works is by filling out a new section in Kippo’s config file, where you put all the details regarding your ES instance. An example is shown below:

[database_elasticsearch]
host = 127.0.0.1
port = 9200
index = kippo
type = auth

Before you use it you will have to install two additional requirements:

  1. pyes: https://pypi.python.org/pypi/pyes
  2. GeoIP: https://pypi.python.org/pypi/GeoIP

You then have to make sure the ES service is running and you’re ready to start Kippo. Using this fork, every connection attempt against your honeypot will be logged in your ElasticSearch instance automatically. You can then use the exported dashboard (.json file) from Kippo2ElasticSearch to visualize your data with Kibana. And just a extra note, the logging components of Kippo can be used together, so you can have MySQL and ES logging enabled at the same time.

I have also submitted my changes as a pull request to be included in the official Kippo codebase, hopefully it will be accepted. Until then you can help a lot if you give this fork a try and report back some feedback!

  • Pingback: Security-Vision » Adding ElasticSearch support to Kippo SSH honeypot()

  • psy

    why not use logstash / logstash-forwarder for that?

    its not a goog solution to have an unprotected elasticsearch running and of course not, on the same server as the honeypot runs!

    • http://bruteforce.gr/ Ion

      Hi psy, thanks for your message.

      I get what you’re saying, but of course you can have an ES instance somewhere internally, not on the same server as the honeypot. In any case, having ElasticSearch output seems very beneficial to me.

      Regarding Logstash, that was my first thought but then I decided to implement the output directly. If you’re capable of creating a Logstash parsing configuration file please let me know and I can help spread the word.

      Regards,
      Ion

      • psy

        Hi Ion,

        I’m shipping the glastopf.log with logstash-forwarder, using the following logstash configuration to parse it: http://pastebin.com/6NTLMSTL

        Greetings,
        psy

      • psy

        Err, i somehow misstook that.

        For kippo i also use lgostash-forward, but this logstash filter: http://pastebin.com/J5wp4cvZ

        Sorry for the mess!

      • http://bruteforce.gr/ Ion

        Hey psy, this is great nonetheless :)

        I will definitely do a blog post at some point about combining Logstash (so the whole ELK stack, not just ES) with honeypots.

        Thanks,
        Ion

  • Craig Valli

    okay works well except if you have multiple kippo hosts feeding the elasticsearch
    sorting out a solution nowish…

    • Craig Valli

      and now it works just individual index for each kippo instance…and we are good

  • Zachary Hardie

    Great addition. However, after importing existing data with the kippo2elasticsearch.py script and then updating my kippo instance to this ES enabled one, I am noticing that only the originally imported data is displaying on the Attack Maps and below. Any idea why this would be the case?

    • http://bruteforce.gr/ Ion

      Hi Zachary, unfortunately I’m not sure what might cause this discrepancy you’re seeing. Have you tried manual testing/trial’n’error?

More in Honeypots, Programming, Visualization
Getting started with honeyd
HoneyDrive 3 VMware guide
HoneyDrive 3 Royal Jelly edition
Vagrant configuration for Thug honeyclient
Kippo-Graph 1.2 released!
Close