Video

2015 Honeynet Project Annual Workshop – Stavanger, Norway – Save the date!

Dec 08 2014

New tool: kippo-log2db.pl

I’m copying an interesting email from SANS’ mailing list, by Jim Clausing. Jim has developed a new tool as a replacement for Kippo2MySQL. The new tools is called kippo-log2db.pl and you can download it here (local copy).

I’ve been running kippo for several years now on a couple of honeypots that I have around and when I started I was just logging to the text logs that kippo can create.  Since then, kippo now supports logging directly to a MySQL database and some other folks (especially Ioannis “Ion” Koniaris at bruteforce.gr) have created some nice tools to generate reports from kippo data.  These tools expect the data to be in the kippo MySQL database schema.  Having logged several years worth of stuff to the text log files, I didn’t want to lose all that data, but I did want to be able to take advantage of some of the neat tools that Ion has developed, so I needed a way to get that data from the text logs to the supported db schema.  Now Ion had created a script that he called Kippo2MySQL, but that converted things to his own schema and lost some data in the process.  Using that as inspiration, however, I have created a script that will read the kippo text logs and populate a kippo database (using the same schema that kippo can now log to directly).  The only hitch that I discovered is that when kippo is logging to text logs and restarts, it doesn’t maintain unique session ids, it starts over again from 1.  This caused me have to make a small change to the sessions table.  I had to change the primary key from ID to (ID,STARTTIME).  Fortunately, I haven’t had an collisions where multiple sessions with the same id actually had ttylogs which is where things might get a bit sketchy.  This was accomplished with

mysql> alter table sessions drop primary key, add primary key(id,starttime);

yielding

mysql> show create table sessions\G
*************************** 1. row ***************************
       Table: sessions
Create Table: CREATE TABLE `sessions` (
  `id` char(32) NOT NULL,
  `starttime` datetime NOT NULL,
  `endtime` datetime DEFAULT NULL,
  `sensor` int(4) NOT NULL,
  `ip` varchar(15) NOT NULL DEFAULT '',
  `termsize` varchar(7) DEFAULT NULL,
  `client` int(4) DEFAULT NULL,
  PRIMARY KEY (`id`,`starttime`),
  KEY `starttime` (`starttime`,`sensor`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.01 sec)

I’ve imported about 800K login attempts and can now play with kippo-graph or (soon, I haven’t had the chance yet) kippo2elasticsearch.  The script can be found here though I have one small issue that I’ll try to fix shortly, I think it is printing out too many #’s, I set it to print out 1 every 10,000 lines it reads from the log files and it seems like I’m getting way more than that, but that is a minor annoyance, maybe I’ll just add a switch to turn that off later.  In the meantime, enjoy and if you find any problems or have ideas for improvement, let me know either in the comments or by e-mail at my address below.

References:

http://handlers.sans.org/jclausing/kippo-log2db.pl

—————
Jim Clausing, GIAC GSE #26
jclausing –at– isc [dot] sans (dot) edu

Nov 19 2014

Run HoneyDrive 3 on Hyper-V server

Todd from Computer and Network Security Services, LLC has published a great blog post about running HoneyDrive 3 on a Microsoft Hyper-V server. I’m reposting it below:

Having a Honeypot in your network can help to alert you to malicious traffic. However, installing and maintaining one can be a bit troublesome, particularly if you haven’t done it before. The complexity only increases if you aren’t familiar with Linux operating systems. I have written a previous blog on the results I received from a Honeypot I set up on my home network which was accessible to the Internet. What I didn’t write about was how long it took me to get it going. I used Dionaea and had it setup in its own subnet. I also had a firewall between it and my network, “just in case.” I turned the Honeypot off after a couple months and got busy with other things. I wanted to get back to it but I didn’t want to go through all the hassle again.

 Enter HoneyDrive3 from Ioannis Koniaris at http://bruteforce.gr. He has built a Linux Distro with honeypots already built and ready to run. I first learned of the tool in the October issue of the ISSA journal written by Russ McRee.

Rather than covering the tool in detail I would like to document the steps I took to get it up and running on a Hyper-V server. Ioannis has it configured to download in an .ova file format which can be imported to VirtualBox. The hard disk itself is in the .vmdk (VMWare) format. Hyper-V uses the .vhd format. Converting the file is straight forward but there are a couple hurdles. (It would be nice if everybody supported a standard format, but I digress).

There are two blogs that got me headed in the right direction. The first is, http://blog.opensecurityresearch.com/2013/05/setting-up-your-hacking-playground.html. Here, they go into a lot of detail about the problems of converting a guest Linux OS from a .vmdk to a .vhd. I won’t say much about it since they give a very detailed description but the tool they used successfully, Starwind V2V, didn’t work on HoneyDrive3. The second site I won’t point to because there is a current XSS attack on the page, according to my browser.

A little background on the conversion process of a .vmdk to a .vhd. Hyper-V has built-in tools that will do the conversion very nicely as long as the guest operating system is a Windows machine. I have used it to convert other servers successfully. The Starwind V2V tool is free and I have used it successfully in the past but when I tried it on HoneyDrive3 I got the following error:

Invalid file format (10) [0]

D:\HoneyDrive_3_Royal_Jelly\HoneyDrive_3_Royal_Jelly-disk1.vmdk – Invalid format. EOS marker not found

I found the necessary steps on the second website. We will use the VirtualBox command line tools to do the conversion. It is only one command but there are some pre-requisites.

 You can download HoneyDrive here, http://sourceforge.net/projects/honeydrive/

 Here are the specs of my systems.

Hyper-V Server 2008 R2 running on an HP ProLiant server as the host.

For VirtualBox I have an I7 laptop with 8Gb of memory. I also have a second box running HoneyDrive on an Intel core2duo box. The required specs are really low.

You will need to have VirtualBox installed on a separate computer.

  1. Download the .ova file and import it into VirtualBox. Then start up the machine. As an aside, you can extract files from .ova by changing the file extension to .tar and using 7-zip to extract them.
  2. The VirtualBox Guest Additions are installed already. It needs to be uninstalled. The following steps are performed inside HoneyDrive
    1. Insert the Guest Additions cd by clicking Device and selecting Insert Guest additions CD.
    2. Open Terminator in HoneyDrive
    3. Type ls /media to see the version of Guest Additions. My version is 4.3.8_92456.
    4. sudo sh /media/VBOXADDITIONS_4.3.8_92456/VBoxLinuxAdditions.run uninstall
  3. Shut down the machine normally and close VirtualBox manager. I did not have any snapshots on mine.
  4. You need to have the VirtualBox Manager opened as an administrator. When I tried it the first time I right-clicked it and selected run as administrator. It didn’t work and it threw and error. I then opened its properties and selected the box to run as admininstrator. This worked. Go figure.
  5. Open VirtualBox Manager as Administrator. It must be open when you run the command below.
  6. Open a command prompt, also as administrator, and navigate to the VirtualBox installation location.
  7. In my set up, I copied the HoneyDrive.vmdk to the VirtualBox installation directory so I didn’t have to path to it.
  8. Use this command for the conversion:  VBoxManage clonehd –format vhd honeydrive filename.vmdk> < new name.vhd>
  9. I had errors with this command before I was running as admin
  10. The conversion took less than 10 minutes and came out to about 9Gb
  11. Copy the file to your Hyper-V  server.
  12. Depending on your network, you might want to order a pizza
  13. After you are done eating the pizza your file is probably copied
  14. On the Hyper-V server create a new VM but when you get the part where it asks you to create a new hard disk select the newly converted hard drive. I am assuming the reader has a basic knowledge of Hyper-V. If you have questions let me know.
  15. Finish the wizard and start up the VM.

Contrary to previous IT experiences this worked the first time I tried it.

Now go to http://bruteforce.gr for some good reading. If you have a subscription to the ISSA Journal you can use the Toolsmith article in the October issue for a great getting started guide.

Happy hunting, or perhaps, trapping.

Video

s06 Bringing PWNED To You Interesting Honeypot Trends Elliott Brink

Video

Black Hat USA 2014 – Incident Response: Secure Because Math A Deep Dive on Machine Learning

Sep 07 2014

How to install Perl DBD on Mac OS X Mavericks with MAMP Stack

Today I decided to work on Honeyd-Viz a bit which I feel I have abandoned the last year. In order to do so, I needed to have a sample database to play with. As you know, you can create a MySQL database with entries from Honeyd’s honeyd.log file using the Honeyd2MySQL script. Honeyd2MySQL uses Perl’s DBI::DBD module for MySQL operations. I have also been using MAMP Stack from BitNami for development. The problem I had was the installation of DBI::DBD on my Mac OS X which I needed in order to use the DBD::mysql driver. If you’re having troubles you can follow the guide below (written with some help from StackOverflow):

  1. Install XCode from the App Store. Then, open XCode, go to the Preferences –> Downloads menu and install the Command Line Tools.

  2. Install MAMP Stack from BitNami. Write down or take a mental note of the password value for MySQL’s root user. Choose to start the services (Apache & MySQL).

  3. Add MAMP’s MySQL binaries to your PATH (this is particularly needed for mysql_config). Example:

    locate mysql_config
    /Applications/mampstack-5.4.32-0/mysql/bin/mysql_config

    Get that directory and add it to your .bash_profile file, appending it to the PATH variable:

    nano ~/.bash_profile
    PATH={ ... }:/Applications/mampstack-5.4.32-0/mysql/bin

    Logout and open a new shell session.

  4. Create symlinks for MySQL’s lib files in your local lib path:

    cd /usr/local
    mkdir lib #it might already exist, e.g. if you're using Homebrew
    cd lib
    sudo ln -s /Applications/mampstack-5.4.32-0/mysql/lib/plugin/ plugin
    sudo ln -s /Applications/mampstack-5.4.32-0/mysql/lib/*.dylib .

  5. Initialize CPAN and install cpanm:

    cpan #accept defaults
    sudo cpan App::cpanminus

  6. Install DBI and download DBD::mysql:

    sudo cpanm DBI
    sudo perl -MCPAN -e 'shell' #opens a CPAN shell session
    cpan> get DBD::mysql
    cpan> exit

  7. Manually install DBD::mysql:

    cd ~/.cpan/build/DBD*
    sudo perl Makefile.PL --testuser='root' --testpassword='<mysql_root_password>' #use the password you entered during MAMP's installation
    sudo make install

  8. (optional) Symlink your MAMP’s MySQL sock file if needed (e.g. if you get an error while trying to connect to MySQL server running on ‘localhost’):

    ln -s /Applications/mampstack-5.4.32-0/mysql/tmp/mysql.sock /tmp/mysql.sock
    chmod 777 /tmp/mysql.sock

That’s it! Hopefully everything would be good to go.

Aug 25 2014

DionaeaFR: adding parameterized date range

UPDATE: this change has been merged into the official DionaeaFR repo.

As you might know, DionaeaFR is a very good frontend for Dionaea malware honeypot. It is developed by @rubenespadas, is written in Python and uses the Django web framework. I have covered DionaeaFR in the past in my post Visualizing Dionaea’s results with DionaeaFR and of course I have included it in HoneyDrive.

But, DionaeaFR had an issue that was bugging me a lot; it only displayed data for the last 7 days (starting from the current day and going backwards). This is a problem when dealing with old databases or when you want to get a more comprehensive overall impression of the honeypot’s activity or when you simply decided to stop your capturing activities for some days and then want to visualize what was going on.

So, I decided to fix it (along with some other small issues). You can find a fork of DionaeaFR on my GitHub account here: https://github.com/ikoniaris/DionaeaFR where there is a RESULTS_DAYS variable in the settings.py file that you can set to the number of days you want DionaeaFR to show data for (starting from the current day and going backwards). I have also submitted that as a pull request but I haven’t got a response yet, thus I decided to post this.

Enjoy, and please let me know of any feedback.

Page 1 of 2812345...1020...Last »