Tag: linux

Posted on

LMD 1.4: Little Something For Everyone!

The much awaited for 1.4 release of Linux Malware Detect is here! In this release there is quite literally something for everyone, from massive performance gains to FreeBSD support and everything in between :). For those who wish to dive straight into it, you can run the -d or –update-ver option to update your install to the latest build and check out the change log for full details.

I will try cover some of the highlights of this release for those with the appetite for it, here goes…

One of the more exciting changes is that Clam Anti-Virus is now supported as an optional scanner engine. When LMD detects that ClamAV is installed on the local system, through detection of the clamscan binary, it will default to using clamscan as the default scanner engine. The use of clamscan as the scanner engine leverages LMD in a couple of ways. First, it allows for ClamAV’s threat database to be used in detecting threats, over 900k strong, in addition to the LMD signatures which are ClamAV compatible. Secondly and more importantly, it improves scan performance greatly, over five times faster. Finally, it also improves the accuracy of threat detection as ClamAV is more efficient at doing hex payload analysis of files using LMD’s hex pattern match signatures. To enable this all you need to do is have ClamAV installed and LMD will detect it all on its own, if you wish to override the detection/usage of clamscan then you can set clamav_scan=0 in conf.maldet.

Another change that I am excited to announce, is that LMD 1.4 is now compatible with FreeBSD, less the inotify real-time monitoring as it is a Linux specific feature that requires me to design a new monitoring subsystem around FreeBSD’s inotify equivalent, kqueue. That said, allot of testing went into ensuring FreeBSD compatibility but it did not end there, I also went to great pains to improve Linux compatibility both with RH variants and non-RH variants alike, the officially supported set of distributions is as follows:
– FreeBSD 9.0-CURRENT
– RHEL/CentOS 5.6
– RHEL 6
– Fedora Core 14
– OpenSuse 11.4
– Suse Linux Enterprise Server 11 SP1
– Ubuntu Desktop/Server 10.10
– Debian 6.0.1a

This supported list is not meant as an exclusive list, it is simply a “test” set of distributions that I work with that give LMD the best expectation of working on an even wider set of Linux distributions. This improved compatibility will open up LMD to a larger community of users and there-in allow the project to grow and prosper in new and exciting ways.

The way LMD updates itself has now been improved, traditionally the daily signature updates only updated the core hex and md5 signature files but that proved to create some gaps in ensuring that all dynamic components for detecting threats are current. As such, now the update feature also pulls down the most current set of cleaner rules and LMD signatures in ClamAV format. In addition, the update process has seen an improvement in error checking; the signature files are now validated for length and missing files, if either validation checks fail then all signatures are forcibly updated.

The hex scanner (internally known as stage2 scanner) has been improved in that it now makes use of a named pipe (FIFO – first in first out) for processing file hex payload data, this allows for greater depth penetration into files and at a much lower cost in overhead. This means more accurate threat detection, fewer false positives and improved scan speeds; although it still pales in comparison to when clamscan is used as the scanner engine but nevertheless it is an improvement and an important one at that.

Further adding to the threat detection capabilities of LMD, is a new statistical analysis component that will see allot of expansion in later releases. The first feature in the statistical analysis component is called the string length test. The string length test is used to identify threats based on the length of the longest uninterrupted string within a file. This is useful as obfuscated code is often stored using encoding methods that produce very long strings without spaces (e.g: base64, gzip etc.. encoded files). This feature is presented in conf.maldet through the string_length variables, it is disabled by default as it can in some situations have a relatively high false-positive rate, especially on .js files. Future releases will see extension and file type based filtering specifically wrapped around the statistical analysis components to reduce false positives, however it is still a very powerful feature in detecting obfuscated/encoded malware.

There is a number of usage changes that have been made, the most notable and important being in ignore files, specifically the ignore_inotify and ignore_file_ext files.

The first, ignore_inotify is a specific file designed for ignoring paths from inotify real time monitoring, previous to LMD 1.4 this file only accepted absolute directory/file paths which was very limiting and created headaches for many people. The ignore_inotify file now fully supports posix extended regular expressions, meaning you can ignore absolute paths still or create regular expressions to cover specific file types or dynamic path/directory structures. An example of this is that temporary sql files may write out to /var/tmp in the format of /var/tmp/#sql_12384_4949.MYD, previously you would have to ignore /var/tmp completely which exposed the system more than it helped. Now, you can add an entry to ignore_inotify such as ^/var/tmp/#sql_.*\.MYD$ and it will properly ignore the temporary SQL files while retaining full monitoring of /var/tmp.

The second, ignore_file_ext was a feature added in the 1.3.x branch that was pulled back due to technical issues. The file speaks for itself, it allows you to ignore files from scan results based on file extensions, this has now been fixed and is working properly. The usage of the file is straight forward, simply add one extension per line to ignore_file_ext and it will be excluded from scan results (e.g: .tar.gz , .rpm , .html , .js etc…), there is no need to use an asterisk (*) in entries in the ignore file.

Further usage changes include that the -c|–checkout flags now supports directories instead of just absolute files, so you can upload threats to rfxn.com from an entire directory (please make sure all threats within the directory are actual malware, I would prefer not to sort through hundreds of html/web files). The -r|–scan-recent and -a|–scan-all flags now support single file scans, previously only directory paths were accepted. A background option has been added in the form of -b|–background that allows scans to be run in the background, the -b|–background option must come before the scan options, such as (see –help for more details):

maldet --background --scan-recent /home/?/public_html 7
maldet --background --scan-all /home/?/public_html
maldet -b -r /home/?/public_html 7
maldet -b -a /home/?/public_html

There have also been a couple of changes to the -e|–report flags allowing for the listing of available reports and emailing of previous scan reports. The usage of these changes is straight forward and is as follows:

maldet --report list
maldet --report SCANID [email protected]

That about covers things, there have been a number of smaller changes and fixes in LMD 1.4 which are detailed in the change log. To ensure you are running the latest build please run the -d or –update-ver option to have LMD auto-update or visit the project home page and download the latest build.

Posted on

ATA Over Ethernet: As an Alternative

New technologies, new toys — Oh how I love getting my hands dirty with them. Today I am going to have a look at ATA Over Ethernet (AoE) as an alternative solution to NFS in the role of a NAS/SAN implementation. We will look at both the server side vblade setup and the client side AoE kernel module along with a practical deployment setup which includes a convenience script I developed to make vbladed slightly less of a nuisance to maintain.

First things first though, what exactly is ATA Over Ethernet? Straight off the wikipedia page, here are the important parts that describe AoE best:

"ATA over Ethernet (AoE) is a network protocol developed by the Brantley Coile Company, designed for simple, high-performance access of SATA storage devices over Ethernet networks. It is used to build storage area networks (SANs) with low-cost, standard technologies.
...
AoE runs on layer 2 Ethernet, it does not use internet protocol (IP), so it cannot be accessed over the Internet or other IP networks. In this regard it is more comparable to Fibre Channel over Ethernet.
...
SATA (and older PATA) hard drives use the Advanced Technology Attachment (ATA) protocol to issue commands, such as read, write, and status. AoE encapsulates those commands inside Ethernet frames and lets them travel over an Ethernet network instead of a SATA or 40-pin ribbon cable. By using an AoE driver, the host operating system is able to access a remote disk as if it were directly attached."

OK, of note here is that AoE is an ATA implementation over Ethernet, being layer 2 it is a dumb protocol with no knowledge of the TCP/IP stack, as such it can only communicate in the simplest of ways inside a switched network (its packets cant be routed between multiple networks). As such, AoE is ideal when used on a private network or better yet a network dedicated to SAN (Storage Area Network), it can however be used on a public facing network as so long as the hosts in the AoE network are all within the same switched segment of the network (More info here on routable AoE).

That all said, what makes AoE a viable alternative to NFS? Well in the role of storage access in its simplest capacity, NFS is just bloated and adds a significant amount of overhead and complexity to something that deserves to be simple. Further, NFS is woefully inadequate at maintaining the level of reliability required when you are, for example, exporting an entire file system to another device for the purpose of high-availability usage such as a /home extension or MySQL file system. Personally, I am slightly biased as I hate NFS; I use it, but only for a lack of often anything better to meet the role in exporting file systems and directory trees across networks. Although it does what its supposed to just fine, more often than not you can get woken up at 4AM with the most mysterious and sudden of NFS issues that are notorious for being mind numbing to resolve. It is for this simple reason — NFS’s lack of reliability, that sent me searching for a simple, scalable and reliable alternative. AoE has managed to meet two of these three points — simple and reliable, while coming up short on the scalable side, more on that in a bit.

There are two components of an AoE setup, the server side storage device that will run vblade and the client side that will access the exported storage using the AoE kernel module under Linux. I should note that although the vblade server package is for Linux, the client side drivers are available for Windows, OS X, FreeBSD and more; in Linux the AoE kernel module is part of the mainline kernel.

The server you choose to run vblade can be any device that you want to export files or devices on, there is little in the way of requirements as vblade is a pretty slim package and doesn’t consume much in the way of resources other than CPU. For a modest environment where you plan to export to no more than 10-15 clients, a dual core system with 2GB RAM is more than sufficient for the vblade server. For my deployment, I run vblade on a quad core Xeon 3.0Ghz, 6GB RAM and 9TB Raid5 array that exports to 54 client servers. More on my setup later when we review scalability but for now lets jump right into the vblade setup and usage.

Lets go ahead and grab the vblade package, compile and install it:


# wget http://iweb.dl.sourceforge.net/project/aoetools/vblade/20/vblade-20.tgz
# tar xvfz vblade-20.tgz
# cd vblade-20
# make && make install
install vblade /usr/sbin/
install vbladed /usr/sbin/
install vblade.8 /usr/share/man/man8/

There is no compile time configure script or any other real configuration required, vblade installs straight into /usr/sbin and is an overall painless process. The simplicity of the vblade package comes at a cost, in that there is no support for a configuration file to control multiple vblade instances, making things slightly tedious. This should not detract from the use of vblade, it is a mature and reliable package but one with a very simple approach that does little in the way other than what it is supposed to do.

To make life easier for myself, I created a wrapper of the sorts to add support for a configuration file along with limited error checking and some command line conveniences — we’ll grab the wrapper and default config template as follows:


# wget http://rfxn.com/downloads/vbladed.conf
# wget http://rfxn.com/downloads/vbladectl
# mv vbladed.conf /etc/
# mv vbladectl /usr/sbin
# chmod 640 /etc/vbladed.conf
# chmod 750 /usr/sbin/vbladectl
# ln -s /usr/sbin/vbladectl /etc/init.d/vbladed
# chkconfig --level 2345 on

You will note, that we enabled vblade to start on boot through init, although the wrapper is not technically an init script, it does support being called from init and managed through chkconfig for convenience. Lets look at the configuration file /etc/vbladed.conf then we’ll review the vbladectl usage after that:

##
# vbladed export configuration file
##

# unique shelf identifier for this vblade server
SHELF="0"     # must be numeric 0-254, default 0

##
# AOESLOT FILE MAC IFACE ALIAS
# 0 /data/server.img FF:FF:FF:FF:FF:FF eth1 server

The configuration file is pretty straight forward, the SHELF variable only matters if you intend to run multiple vblade servers on the same network, if that is the case then this value must be unique to each vblade server or you will run into client side conflicts of being unable to distinguish between vblade servers. The export definitions follow in the format of “AOESLOT FILE MAC IFACE ALIAS” which the below breaks down further:
AOESLOT is a per-client identifier for EACH exported file or device to the SAME client; in other words if you configure multiple exports to the same client server then this value needs to be unique for each.
FILE is the full path to a device or file you want to export, this can be an unformatted raw device such as /dev/sdb, a preformatted partition such as /dev/sdb5 or a loopback image such as /data/server.img.
MAC is the MAC address of the client-side interface that is attached to the network you intend AoE traffic to move over; more appropriately, it is the interface connected to your private network on the client server
IFACE is the server-side interface that can reach the client-side interface you defined the MAC address for; more appropriately, it is the interface connected to your private network on the vblade server
ALIAS is a reference alias for each configuration entry, this must be unique to each vbladed.conf definition

For the purpose of this article, we will go ahead and create a loopback image, format it and export it for a client server called apollo, then we will review how to import the file system onto the apollo server using the AoE kernel module. First, lets create our image:


# dd if=/dev/zero of=/home/apollo.img bs=1 count=0 seek=10G
# yes | mkfs.ext3 /home/apollo.img

This will create a sparse, zero filled file, meaning it will be 0bytes on disk and allocate space, up to 10G, as data is stored to it. There is a slight performance hit to this as the image file must grow itself as data is written, this however is made up for in improved efficiency of space usage. To create an image that preallocates space on the disk you would run ‘# dd if=/dev/zero of=/home/apollo.img bs=1M count=10000‘, be patient as this will take some time to complete, then format it as described above.

Now that we have the image/device we want to export, we need to add the definition for it into the vbladed.conf file, to do so we need to note the MAC address of the interface on apollo that will communicate with the vblade server, in our case this is a private interface eth1 but in your setup it can be a public facing interface if needed — just make sure its within the same subnet as the vblade server.

[root@apollo ~]# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:16:E6:D3:ED:E5
          inet addr:10.10.6.6  Bcast:10.10.7.255  Mask:255.255.252.0
    ... truncated ...

We now have the client side MAC address (00:16:E6:D3:ED:E5) and we have the device/file we want to export (/home/apollo.img), and we also know the private network interface on our vblade server is eth1 as well, so we can create the vbladed.conf definition:

0 /home/apollo.img 00:16:E6:D3:ED:E51 eth1 apollo

That should be appended into the bottom of /etc/vbladed.conf, then we are ready to start the vblade instance for the configuration we’ve added. The vbladectl wrapper includes start, stop and restart flags which also accept an optional alias for performing actions against only a specific vblade instance, run vbladectl with no options for usage help. Time to start the vblade instance for apollo as follows:

# /usr/sbin/vbladectl start apollo
started vbladed for apollo (pid:16320 file:/home/apollo.img iface:eth1 mac:00:16:E6:D3:ED:E51)
( you could also just pass the start option without an alias to start instances for all entries in vbladectl.conf )

The default behavior for vblade also sends log data to the kernel log, typically /var/log/messages on most systems, so tailing the log will produce the following logs if all is normal:

# tail /var/log/messages
Apr  3 16:49:25 backup5 vbladed: started vbladed for apollo (pid:16320 file:/home/apollo.img iface:eth1 mac:00:16:E6:D3:ED:E51)
Apr  3 16:49:24 backup5 vbladed: pid 16320: e0.0, 419430400 sectors O_RDWR

The important part there is the ‘vbladed: pid 16320: e0.0, 419430400 sectors O_RDWR’ entry in the log as this comes from vblade itself, the other log entry comes from the wrapper. This log entry tells us that vbladed forked off successfully and that it has exported our data for the defined server as e0.0 (etherdrive shelf 0 slot 0), you’ll see the significance of this shortly.

We are now ready to move over to our client server, apollo, and import our new AoE file system. This is an easy task and if you are running a current Fedora / RHEL (CentOS) based distribution, you’ll find the AoE kernel module already included. The module is also part of the mainline kernel so if you are using a custom kernel, please be sure to enable the corresponding config option (CONFIG_ATA_OVER_ETH).

There is really no right way to load a kernel module, you can either use modprobe which I recommend or you can use insmod on the modules full path, which is a matter of preference. Let’s first verify the module exists, which modprobe does for us but for the sake of this article and familiarity, we will check (remember you’re running this on the client server, i.e apollo):

# find /lib/modules/$(uname -r)/ -name "aoe.ko"
/lib/modules/2.6.18-194.32.1.el5PAE/kernel/drivers/block/aoe/aoe.ko

There we have it, the module returned fine, listing the full path to it. If you did not get anything back this may be that you are running a custom kernel by your own choosing, and need to configure the CONFIG_ATA_OVER_ETH option. It may also be that your data center provider or a software vendor installed a custom kernel without this feature and you should contact them requesting it. As an alternative, you could download the etherdrive sources for the AoE kernel module on the coraid website and compile it against your kernel, this requires your kernel build sources or on RHEL based systems the kernel-headers package.

That said, we will now load the module using modprobe, the preferred method:

# /sbin/modprobe aoe
( or you can run /sbin/insmod MODULE-PATH )

If everything went OK, then modprobe will generate no output and you can verify the module is loaded as follows:

# lsmod | grep aoe
aoe                    60385  1

When the AoE module is loaded it will start listening for broadcast traffic from AoE on all available interfaces, a very passive process. If you have done everything correct then the module will quickly detect the exported device/file from the vblade server and inform you in the kernel log along with creating the appropriate /dev/etherd/ device file. Let’s verify this by checking the log and then checking the /dev/etherd path:

# tail /var/log/messages
Apr  4 17:13:02 apollo kernel: aoe: aoe_init: AoE v22i initialised.
Apr  4 17:13:02 apollo kernel: aoe: 003048761643 e0.0 v4014 has 419430400 sectors
Apr  4 17:13:02 apollo kernel:  etherd/e0.0: unknown partition table
# ls /dev/etherd/
e0.0

If for some reason you do not see the log entries described above along with no e0.0 device file under /dev/ethered, this may be a misconfiguration on the vblade server, perhaps you got the interface or mac address in vbladed.conf wrong? Double check all values. If you opted to try run things over a public facing interface, the issue may be that your network VLAN’s each server (which is fairly common), in that case you may need to request that all your hardware be part of the same VLAN or the provisioning of a private switch and private links for your hardware.

Assuming that things went good, that you see the appropriate log entries and the e0.0 device file under /dev/etherd/, we are ready to mount the file system, we will mount it as /mnt/aoe for the purpose of this article:

# mkdir /mnt/aoe
# mount /dev/etherd/e0.0 /mnt/aoe
# df -h /mnt/aoe
Filesystem            Size  Used Avail Use% Mounted on
/dev/etherd/e0.0      5G   36M  4.9G  0% /mnt/aoe

You may run into an issue of unrecognized file system on the device, though the file system we created on it, on the vblade server, should show through. If it does not, simply run an ‘mkfs.ext3 /dev/etherd/e0.0’ on it and you will be all set. There is no hard set rule on creating the file system on the vblade server, you could just export raw images and devices then partition/format file systems on them on a per-client basis as you require it.

The only thing that is left is to set our new file system on apollo to load at boot time, the simplest way to do this is to append a couple of lines to /etc/rc.local as follows:

/sbin/modprobe aoe
sleep 5 ; mount /dev/etherd/e0.0 /mnt/aoe -onoatime

The rc.local script is run at boot time after all other services have started, so if you are loading a file system used for mysql, user home data or similar you will probably want to also add a line after the mount to restart said services. You’ll also notice two things about the entries we added to rc.local; The first is the sleep delay before the mount, this allows the aoe kernel module to complete its discovery process for AoE file systems before we try to mount it. Then, we are using the noatime option on the mount command, which disables the updating of the last access time on files during read/write operations. This is important because traditionally whenever a file is read from disk, it causes a write operation back to disk to update the atime attribute on the file, so disabling atime usage can greatly reduce i/o calls (effectively in half for reads), which is especially significant for networked file systems.

Conclusions
I have had an overall good experience with AoE so far, it is incredibly simple and very reliable as an implementation. The only issue I have seen is the scalability of it and I attribute this more to the vblade server package than AoE as a protocol. There appears to be a degradation in I/O throughput performance for exported file systems that is in-line with the number of (instances) file systems you export on the same physical server. The best usage example of this is that in my environment I run vblade on one server with exports to 54 servers, the throughput when there is 1-10 instances running averages about 51MB/s (408Mbit), as that increases though to 54 instances, the throughput per client server drops drastically to an average of 14MB/s (112Mbit). This is a very sharp decrease in performance, one that makes the viability of vblade in much larger of a setup questionable.

I do need to caution that this issue may be environment specific as speaking to other vblade users has produced mixed feedback, some do not experience this kind of performance loss while others do. I will also note that I run vblade on a second storage device, on the same private network as the 54 instance vblade server, and this second storage device has only 4 instances running with an average throughput of 71MB/s (568Mbit). So the conclusion you draw from this is up to you, at the end of the day I am more than happy with the implementation as a whole and can accept the loss of performance for the larger implementation in the name of reliability and simplicity.

Posted on

Raid Management: Know Whats Really Going On

In today’s hosting environment it is common place for servers to have hardware based raid cards but what is not common place is having a reliable method for checking the status of the raid arrays. Few would question the value to data integrity by making use of raid technology but very few organizations and businesses implement the tools required to proactively maintain raid arrays, they simply hope for a DC tech to hear a raid alarm and assume the technician will handle the failure. The reality is very different, data centers are loud and increasingly server-dense so hearing a raid alarm let alone pin-pointing the server with the alarm going off, is a daunting task. I remember more than a few times where I found myself with a paper towel tube to my ear listening server to server to try find that troubled box with the annoying alarm going off. This is not how servers should be managed.

As server administrators or web host operators, it is your responsibility, your duty, to have tools in place that can proactively monitor the status of raid arrays and alert you when an array becomes degraded. That way you can have a paper trail of sorts when something has went wrong, submit a ticket to your data center technicians and have the situation corrected before a degraded array from a single disk failure turns into a multi-disk failure and failed array with data loss.

I have created and been using for sometime a script that can query the status of raid controllers from Areca, 3Ware & MegaRaid. The MegaRaid support is mostly intended for Dell PowerEdge PERC cards, however it should work for most MegaRaid based controllers, ymmv though.

The principle is very simple, the package contains the proprietary command line tools from Areca, 3Ware & MegaRaid that can query the status of respective controllers and then an accompanied ‘check’ script handles determining what raid controller is on the system and then runs the appropriate tool in order to get the raid status and if it is degraded or in any state other than a consistent one, it will dispatch an alert to a configured e-mail address.

Download and extract the package:

# wget http://www.rfxn.com/downloads/raid_check_pub.tar.gz
# tar xvfz raid_check_pub.tar.gz

The package will extract to raid_check/, you should place this under /root/ as the check script expects to be run from /root/raid_check/. If you wish to change the path then please modify ‘raid_check/check’.

With the package now setup under /root/raid_check/ you need to modify the ‘raid_check/check’ script to set an email address that alerts are going to get sent too. Once this is done you should symlink the check script to cron.daily so that raid failures will be picked up on daily cron runs, you may change this to cron.hourly if so desired.

# ln -s /root/raid_check/check /etc/cron.daily/raid_check

That’s it, you can give the check script a run to see if things are working. If there is a failure or inconsistency detected then it will be shown on console in addition to the email alert being sent. If everything is OK and there is no issues detected, then no output will be presented.

# sh /root/raid_check/check

Tip: You can check if your server has a raid card by running the following command:

# cat /proc/scsi/scsi  | grep Vendor

If you see Vendors listed as ATA followed by hard drive model names (i.e: WDC, HD etc…) then your servers disks are directly connected and there is no raid controller present. If on the other hand you see vendor names such as Areca, AMC, 3ware, or MegaRaid then you have a hardware raid controller.

Posted on

LMD 1.3.7: Milestones, Fixes & Signature Updates

Today marks the release of LMD 1.3.7, which is a minor release update that fixes a few bugs and is also the final 1.x release before version 2.0 as described in the LMD: one year later blog post. The bug list for LMD has remained very small over the last 6 months and this release reflects that by fixing the current outstanding bugs.

Changes 1.3.6 => 1.3.7:
[Fix] package ownership at some point got set to uid 501 instead of root
[Fix] daily cronjob now checks ps output for inotifywait proc instead of pidof
[Fix] monitor mode users would exit prematurely if a user home path did not exist
[Fix] a file hijacking race condition existed with quarantine mode restore function
[Fix] inotify max_user_instances value was being set to a value that would cause inotifywait
to fail

A thanks goes out to Mark McKinstry of Nexcess.net for assistance tracking down and fixing the issue with inotifywait reporting on some systems that inotify support did not exist in the kernel, when it actually did, this was an issue with the value maldet was setting for inotify max_user_instances. A thanks also goes out to Jeff Patersen from webhostsecurity.com for identifying and bringing the file hijacking race condition to my attention. This issue had the potential, under certain circumstances, to allow a user to gain access to root-owned files in user-readable paths. These fixes on their own are reason enough for all users to update, the -d|–update-ver command switches will take care of all update business for users so there is no reason to not update (i.e: # maldet -d).

Today I have also put up a small set of signature updates on top of the regular daily queue processing, this includes 25 HEX signatures for various items in the review queue as well as associated file hashes. This brings the project to over 5,000 signatures, a milestone that has been a long time coming and one that sets this project apart from all other malware projects in the Linux world. Even the top tier AV vendors and open source project ClamAV lack the depth of malware signatures that LMD brings to the Linux community. At the moment, the project is growing by an average of 14 signatures per day with a review queue that I still need to finish processing of over 1300 user submissions.

We also can celebrate another milestone this month, with passing 3,000 confirmed installations of LMD (3,241 as of this writing). We can determine this by checking the number of unique IP addresses (servers) that check-in daily to the rfxn.com server for signature updates. The total downloads of LMD sit at 12,952 to date, which is roughly where we expect it to be having had 3 major releases (minor releases dont get much attention) that most users would have installed or updated to.

As a holiday gift to all LMD users, I am making it my goal to have all pending items in the review queue processed and signatures created by the end of December, so keep your eyes open and i’ll make a post when that has been completed.

Posted on

LMD: One Year Later

With my move back to Canada behind me and adjusting to some new routines with life, its about time to get back into the mix with the projects. Though things have been slow the last couple of months, it has not stopped me from making sure regular and prompt malware updates are released.

Today, we reflect on the first year of Linux Malware Detect, which was released in a very infantile beta release about a year ago. The project has evolved in allot of ways from its original goals, it has certainly changed in every way for the better. What was originally to be a closed project, relegated to mostly internal work related needs, ended up like most of my projects morphing into a public release. The first release saw the world with less than 200 signatures, no reliable signature update method, manual upgrade options and very flawed scanning and detection methods (v 0.7<). Now, we sit at version 1.3.6, with 4,813 signatures, a scanning method that though still needs some work, is far superior than what was originally in place, a detection routine based on solid md5 hashes and hex signatures. We have cleaner rules that can clean some nasty injected malware, we got a fully functional quarantine and restore system, reporting system, real-time file based monitoring, integrated signature updater and version updater and a vibrant community of users that regularly submit malware for review. Yes, LMD has grown up!

The most grown-up part of LMD has to be how signatures are handled and how the processing of them is almost an entirely automated process now, this was detailed a little more in Signature Updates & Threat Database posted in September. The key part here though, is “almost entirely automated”, everyday that the processing scripts run to bring in new malware, there is always a number of files that cant be processed automatically and these are moved to a manual review queue. With how busy life has been the last couple of months, the review queue has slowly risen to 1,097 files pending review. This queue is at the top of my list for tackling over the next couple of days and weeks, its allot of work to review that much malware but it will get done. Many of the files to review are actual user-submissions so if you did submit something and find its still not detected by LMD, this would be why :).

There is still allot on the to-do list for LMD going forward, with the upcoming release of version 2.0 we will see some changes in how LMD does business. The first and to me the biggest will be optional usage statistics, which will allow users to have LMD report anonymized statistics back to rfxn.com. These statistics will show us which malware hits are found on your servers, which in turn contributes towards better focus on what type of malware threats are prioritized in the daily processing queue for hashing & review. The statistics will also help create informative profiles on the soon-to-be-released dailythreats.org web site about how maldet is used and what are the most prevalent threats in the wild.

Other additions to LMD 2.0 will be a refined scanner that will provide greater speed with large file sets (50k – 1M+ files), an ability to fork scans to the background, better and more predictable logging format for 3rd party processing of LMD log data, redesigned reporting system, full BSD support, ability to create custom signatures from the LMD command line, expanded cleaner rules, wildcard support for exclude paths, a number of security and bug refinements and as always, more signatures.

If you have any feature requests for LMD 2.0, go ahead and post them as a comment and I will make sure they get added to the list. Thank you to everyone who continues to support rfxn.com projects through donations, feedback and by just using & spreading the word about the projects. I look forward to another year of LMD and seeing it become the premier malware detection tool for Linux and all Unix variant OS’s.