Overview

MTAutoBan is a plugin for Movable Type. It maintains an Apache webserver configuration file (also known as a “.htaccess” file) that bans IP addresses in the Apache webserver. The set of banned addresses is generated from the junk objects stored in the Movable Type database. Whenever the set of junked objects changes, the Apache configuration file is updated.

The goal of this plugin is to slow down the flood of junk objects by banning the IP source addresses that generate them. This makes it easier to check the junk objects for bad filtering and makes denial of service effects from excessive amount of junk much less likely. By doing the banning at the Apache webserver level instead of in Movable Type itself, the runtime cost of the banning is minimized. With a good set of filters, even large1 groups of systems are rapidly banned with little effect on the weblog.

As a result of further experience, I added the ability to redirect requests instead of simply banning them, because it seems that many junkers notice 403 error eventually and rotate attacks over to new machines. Redirection is intended to thwart this.

Because of the large numbers of compromised computers used to generate junk, the maintenance cost of banning the associated IP addresses must be near zero. This plugin achieves that goal. No maintenance is required once installed, and the defaults are good enough that initial configuration is not even necessary.

To make the banning as automatic as possible, the configuration file is updated whenever

  • An object is marked as “junk”, either explicitly or via junk filtering.
  • A set of objects is marked as “not junk” explicitly.
  • Whenever a junked object is deleted.

Current Version: 1.2.3

Beta Version for MT 4.0 Compatibility: 1.2.4

Requirements

MTAutoBan requires

All other Perl modules required by MTAutoBan are also required by Movable Type.

Installation

  1. Get the distribution.
  2. Create a directory named AutoBan under the plugins directory in your MT installation directory.
  3. Copy all files in the distribution to the AutoBan directory.

Note: If you installed a version prior to 1.1.0, you should remove the no longer in use file

extlib/Transfinitum/Apache/IPAccess.pm

Usage

MTAutoBan is designed to be a “zero configuration” plugin — unless you like to fiddle with things, you should have no need of configuring MTAutoBan. If you do read through this section and there are settings that you don’t understand, simply ignore them and leave them unchanged. These settings are for control-freaks like me who have psychological problems that do not permit them to tolerate anything they do not have absolute control over. Normal, well-adjusted people can get on with their lives instead.

To configure MTAutoBan, go to the global plugins configuration page. This can be reached from the “Plugins” link on the MT main page in the “System Shortcuts” area. MTAutoBan should be listed as one of the plugins available. The plugin has several configuration options.

Ban address if
This sets the number of times an address must appear in the junk objects to be banned. Addresses that appear this many times (or more) are put in the set of addresses to be banned.
Junk Comments
If this is checked, then junk comments are considered to be junk objects.
Junk Trackbacks
If this is checked, then junk trackbacks are considered to be junk objects.
Only junk objects less than #### minutes old.
If this is checked, the set of junk objects is restricted to those less than the specified number of minutes old. One day is 1440 minutes, a week is 10,080 minutes.
Path
Path to the access file. If this path is relative, then it is relative to the MT installation directory. If the final character is a forward slash, then the file name ‘.htaccess’ is automatically added. MTAutoBan will mark its own ban list in the file so that an existing file can be used even if there are other Apache configuration commands in it.
Redirection
MTAutoBan by default denies access to banned addresses. If this setting is “None”, that is what happens. MTAutoBan can also redirect requests from those addresses. This can be done in one of two ways, determined by this setting. In both cases, only requests for files in directories controlled by the access file are affected.
  • ErrorDocument — MTAutoBan inserts an ErrorDocument directive in its section of the access file to override the page that would normally be used.
  • mod_rewrite — Use mod_rewrite to redirect requests instead of denying them. This option has unknown performance implications and so should be used with caution (see here).
Ban page URL
If the previous setting, “Redirection”, is set to be active, then the target of the redirection is this URL. Relative paths are relative to the directory where the access file is (by default the MT install directory). Absolute paths are relative to the base URL for the weblog.
Read #### objects at a time from the database
This is a performance tuning parameter for geeks. When MTAutoBan is generating the access file contents, it reads junk objects from the database. To avoid system limitations, it does this in chunks. This parameter specifies the number of objects per chunk.
Write #### addresses per line in the access file
This controls how many addresses are written per line of access file output.

In addition to these configuration options, there is also a check box labeled “Force access file update”. If checked, when the configuration is saved the access file is updated.

Tags

MTAutoBan has MT tags to display information about its operations.

MTAutoBanBannedCountThe number of banned IP addresses.
MTAutoBanUniqueCountThe number of unique addresses in the junk objects.
MTAutoBanObjectCountThe number of junk objects checked.
MTAutoBanThresholdThe number of junk objects needed to ban an address.
MTAutoBanLastUpdatedThe time and date of the last access file update. This takes a “format” attribute to format the time. It behaves identically to the “format” attribute for standard MT time / date tags.

Known Issues

  • If a junk object is changed to non-junk on the object edit page, the access file is not updated. If the object is changed to non-junk from the listing page, then the access file is updated. This is by design for performance reasons. Fixing this means updating the access file on every non-junk comment and trackback. Currently comments and trackbacks that are not marked junk avoid the cost of access file update. This can be worked around by either using the listing page to mark objects non-junk, or using the configuration panel to force an access file update, or not worrying about it and letting the next automatic update to the access file fix the problem.
  • Using the time limit has some possible performance implications. Because the timestamps on comment and trackbacks are stored in weblog local time instead of UTC, it is not possible to determine how old a potential junk object is without loading the weblog to which it belongs. Therefore enabling this option will cause a load of all weblogs whenever the access file is updated. In addition, rather than doing two database queries (one for comments, one for trackbacks), the update will potentially require two database queries for every weblog. The internal logic will optimize this where possible, combining queries for weblogs that have the same timezone. If all weblogs have the same timezone, then only two queries for all comments and trackbacks are done. Whether this is noticeable in a real deployment is unknown. It shouldn’t be an issue unless you have dozens of weblogs in many different timezones.

Notes

  • Don’t forget to set auto-delete for feedback if you want MTAutoBan to clean up after itself.
  • MTAutoBan is careful to update the access file at most once for each invocation of the MT configuration script.
  • MTAutoBan has been tested on configurations with over 32,000 junk entries and over 10,000 banned addresses.
  • This plugin is not quite as useful as I had hoped, due to the large number of spam sources (over 13,000 on one weblog). It does seem to do well at holding back floods. In addition, many junkers use the same zombie net so that even new attacks that get past the filters don’t generate hundreds of junk objects. I haven’t had a new attack dump more than a dozen or so junk objects since I installed MTAutoBan, whereas it was quite common before that.
  • If the ban threshold is set to a non-numeric value it will be reset to the default.
  • If the path is left blank, it will be reset to the default.

Performance

The overall performance characteristics of this plugin are unknown due to lack of an installed base. It has been running on all of my MT based weblogs for a couple of years without any apparent negative performance impact, but that is not conclusive.

The theory is that while updating the access file is expensive, it is significantly less expensive than loading up the MT application itself, which is done even when junk is caught by the filters. My experience has been that while there are a number of single hit junkers, there are many who flood a single weblog with a large number of requests in a short time period. It is those that MTAutoBan is designed to frustrate.

MTAutoBan does file locking on the access file and if it fails to get the lock, it cancels its access file update on the presumption that another instance of itself is already in the middle of an update. Because all addresses are updated every time2, if an address is not added for this reason it will be on the next access file update so at most there is one more junk object than optimal. In exchange, the total server load per weblog is limited to a single update at any moment in time.

Support for mod_rewrite adds another level of complexity to performance issues. Because MTAutoBan is targeted at access files, it cannot use the RewriteMap directive, which is only available to the main Apache configuration file. Therefore, it uses stacked RewriteCond directives followed by a RewriteRule that rewrites all URLs to the redirection target. I have very little data on the performance impact of processing 100 - 200 RewriteCond directives each with 50 alternate clauses in their regular expression is. It’s a lot of data, but the regular expresions are all literals.

In practice, I used it briefly with ~7,000 addresses without an obvious performance impact, which is rather weak evidence. This weblog has ~5,000 addresses banned with mod_write redirection (you can see the exactly number on the upper right of this page). If there is an impact, it should show up while using the Movable Type interface. If it has, I have not noticed. It may be, however, that I happen to use or run on a lightly loaded webserver. I recomend caution when using this feature.

White listing

MTAutoBan has its own section of the .htaccess file and preserves content outside of its area when updating the file. This is done so that other, non MTAutoBan directives can be kept in the file. The most common expected use of this feature is white listing, i.e. allowing specific hosts / IP addresses access regardless of what AutoBan does. For example, if your ISP supplied IP addresses was 192.168.56.56, you might want to white list it to decrease the risk of losing access. To do that, you would put the following in the .htaccess file.

order deny,allow
allow from 192.168.56.56

This instructs the Apache webserver to favor allow from over deny from and then to allow the IP address 192.168.56.56, achieving the desired effect. The same technique can be used to permit specific proxy addresses or domains (see here for additional information). This content will be preserved by MTAutoBan.

Bugs

Versions prior to 1.0.3 had a bug where multiple instances of could interfere and scrozzle the .htaccess file. The symptoms are that any access to MT scripts causes a 500 (internal server) error. If this occurs, simply delete the .htaccess file via FTP or your webhost’s standard file management then install a version 1.0.3 or later.

History

  • Version 0.1.0 - Initial version
  • Version 0.2.0
    • Added check boxes for each address.
    • Added “Check all” and “Uncheck all” buttons
    • Suppressed buttons when no log data is present or it is all filtered.
  • Version 0.3.0
    • Fixed a problem with the login template.
    • Fixed a bug where a missing .htaccess file would cause a spurious warning.
  • Version 0.4.0
    • The plugin automatically detects which version of MTB is in use and grubs through the appropriate log data. This means that the same install works on Movable Type 2.6X and 3.1X.
  • Version 1.0.0
    • Updated extensively for MT 3.2.
    • Use junk objects instead of activity log.
    • Use plugin configuration to set parameters.
    • Completely automated, no separate operation page.
  • Version 1.0.1
    • Minor formatting cleanup.
    • Added “Force access file update” option.
  • Version 1.0.2
    • Fixed documentation link problem.
  • Version 1.0.3
    • Fixed a potential race condition where the .htaccess file could get scrozzled under high loads.
  • Version 1.1
    • Added the option to time limit the potential junk objects.
    • Cleaned up the distribution so that all files are in the same plugin directory.
    • Moved code around to minimize plugin footprint when not invoked.
    • Cleaned up some language and display in the settings to be more consistent.
    • Added MT tags to display statistics.
  • Version 1.1.1
    • Added MTAutoBanThreshold tag.
    • Cleaned up some code, added additional comments.
    • Cleaned up settings display for more consistent terminology, widened some input boxes.
  • Version 1.1.2
    • Changed IP address sorting algorithm to be faster.
  • Version 1.1.3
    • Fixed MT::PluginData library requirement for the template tags.
  • Version 1.2.2
    • Added ErrorDocument support.
    • Added mod_write redirection support.
    • Added performance tuning parameters to settings.
    • Improved look and feel of settings display.
    • Verified MT 3.31 compatibility.
  • Version 1.2.3
    • Fixed bug where the target URL for mod_rewrite would always be “sand-trap.php” in the access file regardless of the settings value.
  • Version 1.2.4
    • Fixed a minor issue with accessing the junk score constant.
    • Updated settings HTML to work in both MT 3 and MT 4.
    • Did minor testing with MT 4.0 to verify compatibility.

1 As of this writing, the largest observed attack was roughly 1800 distinct source addresses. These were all detected and banned with no noticeable effect on the weblog. Note that this is for posting attacks only — reading attacks are not protected against.

2 You might think that it would be more efficient to do incremental updating, but it’s not. And since it’s not, it is much more robust to do everything every time.