Signature Updates: Month In Review

Since I will be busy this coming week with other priorities, I am posting an early month in review blog on signature updates.

In the last 3 weeks we have not seen a whole lot of action on in-the-wild malware, most of what is propagating at the moment are variants of already detected content. That is however not to say there has not been new signatures extracted, allot of this months signatures have come from account level compromises on vulnerable e107, wordpress and joomla installations along with user submissions. There is not a whole lot of ground breaking malware threats, it is more of the usual such as mass mailers, perl/php command shells, irc bots and php socket flooding tools.

In total, the 3 weeks ending Sat July 24th, there has been 128 new signatures in 54 classifications with 65 signatures being added in the last 7 days. This brings us to a total of 2,588 (1002 MD5 / 1586 HEX) signatures, an increase of 117 signatures over the last blog post on signature updates. For those paying attention, there is a discrepancy of -11 signatures between the 128 new signatures and the +117 change since the last update, this is because there has also been 11 signatures removed for poor performance/false positives.

As always new signatures are automatically updated daily or can be manually updated with the -u|–update command line options. The 128 new signatures fall into the following classification groups:

base64.inject.unclassed    exp.linux.unclassed
perl.cmdshell.n0va         perl.ircbot.Arabhack
perl.ircbot.BaMbY          perl.ircbot.devil
perl.ircbot.fx29           perl.ircbot.genol
perl.ircbot.karawan        perl.ircbot.oldwolf
perl.ircbot.plasa          perl.ircbot.putr4XtReme
perl.ircbot.rafflesia      perl.ircbot.UberCracker
perl.md5browser.avi        perl.shell.cgitelnet
php.cmdshell.antichat      php.cmdshell.avi
php.cmdshell.aZRaiL        php.cmdshell.c100
php.cmdshell.DxShell       php.cmdshell.h4ntu
php.cmdshell.hackru        php.cmdshell.KAdot
php.cmdshell.lama          php.cmdshell.Macker
php.cmdshell.mic22         php.cmdshell.myshell
php.cmdshell.NCC           php.cmdshell.r3v3ng4ns
php.cmdshell.r57           php.cmdshell.s72
php.cmdshell.Safe0ver      php.cmdshell.SimShell
php.cmdshell.SRCrew        php.cmdshell.Storm7
php.cmdshell.unclassed     php.cmdshell.winx
php.cmdshell.wls           php.cmdshell.xakep
php.cmdshell.ZaCo          php.cpcrack.Aria
php.exe.globals            php.include.remote
php.ircbot.NewLive         php.mailer.DALLAS
php.mailer.unclassed       php.mailer.YoUngEST
php.nested.base64          php.pktflood.unclassed
php.rshell.0wned           web.malware.unclassed

Signatures For The Masses

Today I found the time and energy, despite how tedious it was, to go over the last two weeks worth of malware submissions and missed edge IPS data from when I was away. This resulted in a total of 126 new signatures (67 MD5 / 59 HEX) which brings LMD to a total of 2,471 signatures (894 MD5 / 1577 HEX). This now also gives the project a unique distinction among anti-virus and malware detection offerings, as the single largest project, commercial or open source, detecting Linux malware.

To further illustrate the lapse in coverage by other vendors, we can turn to CYMRU analysis of the MD5 hashes in LMD, as discussed on the LMD home page, CRYMRU provides malware data to vendors such as trendmicro, symantec, kaspersky, microsoft, google and more.

KNOWN MALWARE:       301
 % AV DETECT (AVG):  57
 % AV DETECT (LOW):  58
 % AV DETECT (HIGH): 71
 UNKNOWN MALWARE:    593

This in short shows that of all the vendors that CYMRU provides data for, only 301 of LMD’s 894 MD5 signatures are detected by competing solutions and of those threats detected, on average, only 57% of vendors detect each threat. This information really has no other significance than to reinforce the validity of this project and the time I am investing into it, chalk one up for stroking own ego!

New signatures in this update are classified into the following groups, you will notice ALLOT of command shells in this update, including an interesting addition, a JSP command shell!

base64.inject.unclassed     exp.linux.unclassed
jsp.cmdshell.zerocnbct      perl.cmdshell.n0va
perl.ircbot.Arabhack        perl.ircbot.BaMbY
perl.ircbot.devil           perl.ircbot.genol
perl.ircbot.karawan         perl.ircbot.rafflesia
perl.ircbot.UberCracker     perl.md5browser.avi
php.cmdshell.antichat       php.cmdshell.avi
php.cmdshell.aZRaiL         php.cmdshell.DxShell
php.cmdshell.h4ntu          php.cmdshell.hackru
php.cmdshell.KAdot          php.cmdshell.lama
php.cmdshell.Macker         php.cmdshell.myshell
php.cmdshell.NCC            php.cmdshell.r3v3ng4ns
php.cmdshell.s72            php.cmdshell.Safe0ver
php.cmdshell.SimShell       php.cmdshell.SRCrew
php.cmdshell.unclassed      php.cmdshell.winx
php.cmdshell.wls            php.cmdshell.xakep
php.cmdshell.ZaCo           php.include.remote
php.mailer.DALLAS           php.rshell.0wned

I am Back: Signature Updates

I am back, fresh off a trip home to Montreal, which I must say was an absolutely amazing time. It has left me reflecting on a lot of things, most importantly that there really is no place like home — I miss Montreal more than I can even describe. That said though, time to get back into the mix of things — there is a mountain of malware submissions to review, 91 to be exact. Today I really could not find the energy or time to go through them all but I did process the edge IPS data to extract some in the wild signature data which generated 8 new signatures that are now live. In the coming days, I will work through the malware submissions and get those signatures out as soon as possible.

Linux Malware Detect v1.3.6: Loose Ends

In LMD 1.3.3 there was allot of changes, 29 to be exact, that made LMD much more robust and especially the monitoring component, much more usable. If that release was about making good things better, then this release is about bringing loose ends together. I spent a couple of days running LMD through its paces along with having many people help me test it and during that process, we brought allot of little things to the surface that needed fixing or revising.

In total, there has been 31 changes, fixes or new additions to LMD since that 1.3.3 release on the 15th, most of these changes were completed days ago but I wanted to take the time to make sure they were working as intended and that no more bugs/issues came to the surface. At the moment, since releasing LMD on the 11th, there has been a total of 1349 downloads, so to say that there is plenty of opportunity for bug reports would be understated. I am comfortable in saying that the changes from 1.3.3 to 1.3.6 are stable, reliable and working as intended.

The version changes aside for the moment, there has also been a mountain of user submitted files with the –checkout feature, I processed many of those yesterday and earlier last week for a total of 71 new signatures for the week. Those signatures will have automatically been updated to your install through the cron.daily run of –update, or you can run it yourself if you do not use the default cronjob.

So, what of significance has changed since 1.3.3? The biggest changes are that there is now a -d|–update-ver feature that performs a version update check and if a new version of LMD is available, it will install it. This feature does both a version number check and hashes the main LMD files checking for differences with the server side files, when one of the two checks fails, an update is forced. The version update is not automatically run for a number of reasons that I am to lazy to explain, just think about it a bit. All session and quarantine data is migrated on update.

Most of the other changes are fixes and improvements on existing features, especially the monitoring component which of the 31 changes since 1.3.3, 17 of them are all within the monitoring component. There has also been a few changes to the README file to reflect some minor usage changes, to clarify better some usage of the monitoring service and to explain some new ignore options.

That is all from me, changelog is below, enjoy.

Project Page: http://www.rfxn.com/projects/linux-malware-detect/

Change Log v1.3.3 => v1.3.6:
[Fix] session data gets recreated if it disappears during scan
[Fix] tlog now handles data that logged between 0bytes and first wake cycle
[Fix] monitor_check now properly handles CREATE,ISDIR events
[Change] –alert-daily|weekly alerts have been changed similar to manual alerts
[Fix] cleaner was not properly running on monitor_check calls to scan files
[Fix] quar_suspend was not properly running on monitor_check calls to quar()
[Change] monitor tracker files now pass through trim_log to avoid oversizing
[Fix] monitor_check now properly handles path names with spaces
[Fix] monitor_check was throwing nx file/directory error for monitor.pid
[Fix] older bash versions were having trouble with the [[ =~ ]] regexp search
[Change] set all script files from shebang/bin/sh to shebang/bin/bash
[Change] –alert-daily|weekly will now only send alerts if hits were found
[New] -d|–update-ver now compares file hashes to determine update status
[Fix] suspend events were not properly being added to monitor alerts
[Change] all alerts have had spacing changes to make them more readable
[Fix] signature names now properly list for daily|weekly alerts hit list
[Fix] monitor_check will now recursive monitor newly created directories
[New] monitor daily|weekly alerts now save as a pseudo scan report with SCANID
[Fix] monitor reports now generate properly when quar_hits=0
[Fix] cleaner function was not properly executing under certain conditions
[Change] additional error checking/output added to the cleaner function
[Change] default status output of scans changed for better performance
[New] added ignore_intofiy for ignoring paths from the monitor service
[Change] updated ignore section of README
[Fix] backreference errors kicking from scan_stage1 function
[New] -d|–update-ver option added to update installed version from rfxn.com
[Change] updated short and long usage output for update-ver usage
[Fix] -k|–kill-monitor now properly kills only the inotifywait/monitor pid’s
[Fix] monitor_cycle function now correctly stores its pid in the pidfile
[Fix] files with multiple events in the same waking cycle are only scanned once
[Change] install.sh now symlinks maldet executable to /usr/local/sbin/lmd

Linux Malware Detect v1.3.3: Making good things better

This morning I have put out LMD v1.3.3, this is on the back of two other successive releases in recent days that improved LMD in many areas, along with correcting some bugs that were graciously reported by those helping to break-in the project. I have also listened to feedback and revised a number of features along with completely redoing how the inotify monitoring operates, to provide a much more robust model for real-time file monitoring.

I am also happy to say that people are embracing the use of the -c|–checkout option to send me malware that is not currently detected, which is being processed daily with my regular signature maintenance tasks. I have today added 24 new signatures, all of them created from user submissions.

There are a few big changes in this release…

First and foremost is that the configuration file conf.maldet has been completely revised with more granular options provided for quarantine, scan and monitoring, along with better commenting. Adding to the configuration convenience is that the install.sh script will now import config settings from previous install along with migrating session data.

Next up and something I am excited about, is a rule driven – signature based – cleaner function that can remove string based malware injections from files. The cleaner has two default rules created by me that will clean files of base64 and gzinflate injected strings very accurately. Through the next couple of days/weeks, I will be adding more cleaner rules that will allow for a much broader base of signatures that we can clean files for.

Finally, inotify monitoring got some loving with a top-down review of things and I came up with a less-invasive way of spawning the inotifywait processes that no longer requires a process for each path/user monitored. There is now a single master process that will monitor all configured paths, with better dynamic scaling of the sysctl hooks for inotify based on system resources. In addition, I added an option to pass the monitor service a comma spaced paths list or file containing line spaced paths, from the command line. This is in addition to preserving the users monitoring feature which has also been improved but is no longer the default, you must now call -m|–monitor with one of the USERS|FILE|PATHS options, see –help or the README file for more details.

Please be mindful that although LMD is considered stable it is still a relatively new project and as such your mileage may be a little bumpy, if you run into any issues please post comments on the project page, in this post or send me an email to ryan rfxn.com.

Home: http://www.rfxn.com/projects/linux-malware-detect/
Current Release:
http://www.rfxn.com/downloads/maldetect-current.tar.gz
http://www.rfxn.com/appdocs/README.maldetect
http://www.rfxn.com/appdocs/CHANGELOG.maldetect

v1.3.3 | May 15th 2010:
[Fix] quarantined files were not properly dropping owner
[New] signature based, rule driven, cleaner component added
[New] base64.inject cleaner rule
[New] gzbase64.inject cleaner rule
[New] -n|--clean SCANID option added to batch clean scan all files from a scan
[Fix] made default install file/path permissions more strict (750/640)
[New] install.sh now preserves conf.maldet settings
[New] install.sh now links backups of old installation to INSTALL_PATH.last
[Fix] install.sh now properly imports session data from previous install
[New] -s|--restore can now take a SCANID to batch restore all files from a scan
[Change] improved the layout of conf.maldet; more scan options and commenting
[New] added quar_susp_minuid option for suspend user minimum user id
[Fix] inotify monitor now properly acts on MODIFY,MOVE_TO,MOVE_FROM states
[Change] inotify monitor now can take a list of paths or file for path input
[Change] inotify monitor now has no default use, must specifiy USER|FILE|PATHS
[Change] revised short and long usage output for new options/usage changes
[Change] inotify monitor now spawns only one process for all monitored paths
[Change] inotify monitor sets max_user_instances to processors*2
[Change] inotify monitor sets max_user_watches to inotify_base_watches*users
[Change] migrated all inotify options from internals.conf to conf.maldet
[New] added inotify_base_watches to conf.maldet for max file wathces multiplier
[New] added inotify_nice to conf.maldet for run-time prio of inotifywait
[New] added inotify_webdir to conf.maldet for html/web root only monitoring
[Change] extensive format change to README
[Change] rewrote inotify section of README to reflect the many changes
[Change] -q|--quarantine now calls cleaner if quar_clean=1
[Change] -n|--clean can now do in place cleaning without quarantine