Blacklists keep on rolling :p

After the release of WPBlacklist 2.6, I got a couple of reports about a problem with the path in the installer utility. I had put the path in for my server where the WP installation is in the root server and had not tested things out thoroughly enough to catch the fact that this code would fail if somebody had WP installed in a sub-folder. I’d created a fix and was about to release a bug-fix release when a few other problems caught my eye. So I fixed all of them but then got caught up in some other stuff and didn’t have the time to test any of the changes till today. I guess comment spam is on the rise again because I’ve been getting a lot of reports from people who’ve tried the WPBlacklist plugin and found the error. So, to stem the tide of complaints, I decided to quickly test my changes and release WPBlacklist 2.6.1 :p Enjoy!

Update: Since everybody seems to link to this entry for WPBlacklist, I thought I’d add an update to point to the WPBlacklist page which actually has the latest version, which at the time of this update had a release of WPBlacklist which was 3 sub-versions ahead :p

Onwards and upwards

I can’t seem to stop working on the WPBlacklist plugin :p Now that it seems to work so well in combating the spam on my site, I wanted to add a few more features to make it even better. So, I added the option to allow the plugin to harvest spammer details from comments it automatically deletes. Then, I thought, "hey, wouldn’t it be good to allow the plugin to delete other messages which have already been held for moderation by the WP core?" So I added an option for that too – though I’ve actually turned that option off on my own site because I kinda like seeing what item on the blacklist triggered the deletion of the comment and I can’t get an exact item when the comment has been held by the core :p Anyway, it all seems to work fairly well.

I was about to release the plugin when I was notified by a user of another problem – the fact that if they added a URL manually to the blacklist, that it sometimes results in problems. This turned out to be due to another one of those regular expression related errors and so I fixed that as well and in the process, created another file to the growing list of WPBlacklist files – this time, to hold common functions. Since now I had a lot of files all over the place and the install instructions were getting complicated by different locations for the files to go to, I decided to make things simple by including the directory structure in the ZIP file so that all a user had to do was extract the ZIP file contents and then upload things just as they were 🙂 So that’s all there is to the new release – which I’ve called WPBlacklist 2.6. Enjoy 🙂

Spam free at last!

WPBlacklist 2.5 works like a charm, even if I do say so myself :p For the last couple of days, it has been catching and deleting all spam comments posted to my site without me having to lift a finger 🙂 I get e-mail notifications of comments deleted and what entry in the blacklist triggered the deletion and this lets me refine my blacklist even further by adding any missing stuff (like the author IP or e-mail) to my blacklist so that if the spammer tries again with a different set of URLs or something, he’d still get tagged by WPBlacklist. Yes, as you can see, I’m really pleased with my own work :p Now, all I can think of to add to WPBlacklist is a feature which would automatically harvest details such as author IP, e-mail, URL etc. from automatically deleted entries and add them to the blacklist so that I don’t have to even worry about doing that bit manually based on the e-mails that I receive. That’s not a top priority at the moment though since so far WPBlacklist 2.5 has been 100% effective in combating all comment spam posted to my site …

Another day, another release

My work on WPBlacklist 2.5 (yes, I upped the version to 2.5 since there are a couple of major features in there as well as a complete revamp of the installation system :p) continued much faster than I thought and I now have a fully working release which has been tested on this very blog 🙂 So what’s new you ask? Well, the major feature is the ability to automatically delete comments which are marked as spam by the WPBlacklist plugin. Now while some people would like anything that is blocked to be simply deleted without them having to bother about it, I tend to be a bit cautious about stuff which is automatically deleted. So, I added multiple options to the WPBlacklist plugin so that the administrator can specify what exactly is deleted – whether only matching IPs or IPs marked as spam sources by a real-time blacklist (RBL) or an author e-mail which is marked as spam and so on. You can turn on only the options that you are comfortable with and have the plugin automatically delete comments which match.

But what if something accidentally gets deleted even under those conditions that you really wanted, you ask. Fear not because I thought of this too :p I simply added another option which tells the WPBlacklist plugin to e-mail you with the full comment contents when a comment is automatically deleted. If you don’t want to be bothered by e-mail, all you have to do is turn the option off (it is off by default anyway :p) but if you want to be careful, then you can still get e-mails of all deleted comments but not have to bother with comment moderation on your WP install – best of both worlds? I think so 🙂

I also had to update the WPBlacklist installer since this new release made it necessary for me to do some table structure changes. When I looked at the installer, I realized how unfamiliar with WP I had been when I first started coding the WPBlacklist plugin – because it was clunky and really bad code :p So, I went back and cleaned up the code and made the installer much more simpler and also better suited to upgrades as well. Now you can run the installer to update your database structures without having the installer delete your existing data. Yes, yes, I know that by now you’re probably wondering where you can download the new release from (if you’re reading this at all, that is :p) and so, without much ado, here is the link – enjoy 🙂

Those persnickety bugs …

I’ve been using WPBlacklist 2.0 for a few days now and today all of a sudden, I discover not one but two bugs in it! The first one was something that has bothered me a lot with the original WPBlacklist 1.2 – the fact that you can use regular expressions in the blacklist but not all URLs that you add to the blacklist are regular expression safe :p So I’d added a bit of code to make the URLs regular expression safe but forgot to use that same code in some parts of WPBlacklist 2.0 🙁 I corrected that mistake but at that point realized that the MT blacklist import code wasn’t working properly since it wasn’t taking in new entries even though there were new entries in Jay Allen’s centralized blacklist. I investigated the problem and discovered that a new bug had crept into the code due to some variable name changes. Fixed that too and then I had WPBlacklist 2.01 :p

In the meantime, I’ve been hit by comment spam again. It looks as if this particular spammer has a robot which is smarter than the usual ones – it goes through the index.php file, finds the comment script, parses the comments script and finds the particular variables that are used in the comment script and then submits spam using the information it just discovered. Of course, all this spam is being held for moderation by either the WP moderation system or WPBlacklist but I am getting tired of having to delete all that stuff off the queue later. So, I’m thinking of adding a configurable option to WPBlacklist where the user can specify whether to hold comments identified via IP, e-mail, URL etc. or to delete them immediately. I’d like to make the option individually configurable so that you can specify a different action based on the identification type but am not sure exactly how I’ll end up setting the whole thing up. This probably will be WPBlacklist 2.1 – no idea on an ETA yet though.

Oooh, that dirty, rotten spam!

I’ve been working on the next release of the WPBlacklist plugin whenever opportunity permitted the last few days. I’ve not been able to do as much as I would have liked due to the fact that I have guest at home at the moment and they take precedence. Fortunately, I’d done most of the work over the weekend on Saturday and Sunday and all that remained was to add the finishing touches and do some testing. I haven’t really tested the plugin as much as I’d like but it seems to work fairly well and so I think I’ll release it now since I probably won’t get the chance to do much more for a week or so :p

So what is new? I don’t have the time to do a full post here and so, I’ll basically copy over the feature list from the post I made to the WordPress Forum :p The new features are as follows:

1. Better blacklist management (the ability to delete blacklist entries for instance)
2. The ability to filter out comments based on not just a full IP but a partial IP as well
3. The ability to check a real-time blacklist for spam IPs
4. The ability to search existing comments using the blacklist or a given IP or search expression and then delete the found comments, if necesary
5. The ability to mass-delete comments in the moderation queue while harvesting details from those comments (such as poster IP, URL, e-mail address) and to add them to the blacklist

If you want to get your hands on the plugin after reading all that, you can download it from here :p

Soul Brothers and other stuff

Do you think people can have soul brothers or sisters? Or at least spiritual brothers/sisters? Actually, that is a throwaway question prompted by the fact that I was going to write about somebody who I consider to be a kindred spirit – or at least of his blog :p I’ve known Tyran for quite a long time now and whether he’s writing about wireless or wagons, Firefox or Frodo, blogs or Barsoom, I know what he’s talking about … Or I can empathize with what he said. I’ve always made it a point to drop by Tyran’s blog first thing in the morning because whatever he writes about, it’s bound to be interesting, if not thought provoking. Which is why it surprised me today when I discovered that I had not been by Tyran’s blog in almost a year! Guess, that’s what marriage, shifting to a new job, going back to your old job again and all the other stuff in between does to you :p

Anyway, I decided to take a day off from coding and scripting and debugging and all the other good stuff that keeps me busy during the day, and catch up on my reading over at the Whinery. Of course, as usual, it was interesting and luckily for me, Tyran seems not to have been too prolific in his own blog output since I was able to get through a full year’s worth of entries in one day. Of course, if you’re interested, you can go over there and read his blog yourself :p

Then there is Nigel with his Red Ferret Journal. I’ve known Nige for almost as long as I’ve known Tyran and Nige also probably has the distinction of the Blog user with the most entries in his database :p I’ve been using Blog longer than anybody (I wrote it after all :p) but I’ve only got around 750 entries when you combine four separate blogs (some of which are no longer active) whereas Nigel has a whopping 4000+ entries in his single blog! Nige (like Tyran) has been a very good friend through the years though I’ve never met either of them in person. And this, probably is the greatest gift that I’ve recieved in the years I’ve been developing freeware applications – the great people I get to know and to make friends with. There are many others that I count as friends but I’ve lost touch with them through the years due to some reason or other but this is in remembering all those people who’ve enriched my life over the years 🙂

And that meandering entry is my excuse for not doing too much in the way of coding today :p Speaking of coding, I seem to have caught my first comment spam with the new and improved WPBlacklist 1.22 plugin 🙂 Unlike the old version, I even got an e-mail notification of the fact that a comment was held for moderation. Now, if I could just add the option to check external spam lists and a mechanism to search existing comments for specific keywords and IP addresses, I should be ready to release WPBlacklist 2.0 🙂 I’m kind of eager to get started on it but I think I should get Blog done first. But if you’re looking for information on Blog, this is not the place to be … DC is :p

Blacklists and other bits

I talked about this particular bug in the WPBlacklist plugin that I wrote a few days ago as well. I mean the one where a comment which had been held for moderation by the internal WordPress moderation system could be approved by the WPBlacklist plugin because it doesn’t check the status of the comment before running the comment through the blacklisted item list. I got bitten by this bug just yesterday when a spam comment which was held for moderation because it had multiple URL’s in it, was approved by the blacklist plugin because those URLs were not in the blacklist. I immediately added the necessary changes to the code, tested it out and uploaded the new script to my server. Since this is a bug which will affect others as well, I decided to release WPBlacklist 1.22.

However, I didn’t stop at changing just the plugin code. I also attacked another problem which had been bugging me – the fact that I don’t get a "held for moderation" e-mail when the WPBlacklist plugin holds an e-mail for moderation. This was due to the way the internal WP code worked and so I decided to change the code a little bit so that it worked the way I wanted. I’ve included the changes in the readme.txt file in the WPBlacklist distribution but in case somebody is interested, here is a excerpt direct from the readme.txt file … Umm, I tried posting that bit but it really messes up the paragraphs for entries here on SM :p So I guess you’ll just have to read the readme.txt file to find out what you need to do …

Blacklists and blind-spots

After yesterday’s post about the apparent problems with my WPBlacklist plugin, I decided to do some testing on my own to see if things were indeed not working. And I’ve got both good and bad news :p The good news is that the WPBlacklist plugin does work and there are no hidden loopholes – the bad news is that it doesn’t exactly work the way you think it would :p I realized during the course of my investigation that I had either not paid as much attention to what happens in the WordPress core as I should have when I wrote the WPBlacklist plugin or that I did know one critical piece of information which I had then proceeded to forget all about :p

The piece of information? Well, actually it’s two bits of information :p One was the fact that I had not added any code to notify the user when a comment was held for moderation by the WPBlacklist plugin – it simply holds the comment silently – the strong, silent type :p The second bit of information was that the WP core code actually sends out confirmation of a comment posting (or of it being held for moderation) based on its internal spam list before it initiates the hook which allows external plugins like WPBlacklist to take a shot at comment spam. So basically, if a comment is not caught as spam by the internal spam list in WP, then the user gets a notification saying a comment was posted on his/her site but WPBlacklist in the meantime might have actually caught the problem and put the comment on hold silently. This actually explains several reports I got from users who said that WPBlacklist wasn’t working even when certain words were in their blacklist – they were probably depending on the e-mail confirmation instead of checking the actual comments appearing under an entry – or, at least, that’s what I think now :p

Of course, finding the cause is just half the solution. Now I need to find a way around it. Unfortunately, to provide a solution that works for all, it looks as if I will have to modify some of the core WP code and submit it to the WP devs and hope that they accept the change – or something. I’ve already checked the CVS code from the end of August and it still has the same problem. So, unless they’ve fixed it within the last two weeks, I think I’ll need to submit my own fix to the WP devs. Of course, this also brought up the possibility that I might have my own bug hiding in the WPBlacklist plugin code – I don’t remember whether I checked to see if a comment was already held for moderation and if so, then simply exit the check routine. Because if I didn’t do that, then it is possible that something which got tagged as spam by the internal routines (perhaps because it had too many links in the comment ..) might get untagged by WPBlacklist – now that just wouldn’t do :p So now I’ve got lots of stuff to keep me occupied for the next few days 🙂

The blogger’s bane …

Comment spam is increasingly becoming a major problem 🙁 There used to be a time when I’d get maybe one or two spam entries a month but of late, there are days when I literally get flooded by spam entries. I had written/modified the WPBlacklist plugin soon after I shifted to WordPress just because of this problem. I had been using a blacklist plugin while using Movable Type and I knew that I’d probably get hit by comment spam if I wasn’t protected on WP too. The blacklist plugin seemed to work and I was content for a while but a couple of weeks back, the latest problems started.

I’d been offline for a couple of days and came back to discover that I’d been hit by around a hundred spam comments on various posts scattered over my blog. Fortunately, WP makes it fairly easy to remove comments en masse and an SQL command was even simpler in this particular instance because the spammer used the same URL for all of his comment spam. Once I removed the spam however, I went back and took a look at the spam itself and discovered a few things:

1. The spam was from many different URLs
2. The particular site s/he was spamming for was not in my blacklist
3. I had set the WP option which marks a comment as spam depending on the number of links in the comment to look for five links.

So, I went ahead and added the URL for the site to my blacklist and lowered the link threshold for spam to two. I thought I was safe but that wasn’t the case as I discovered a couple of days later.

I again got hit by spam but this time, I was online when the attack started. I discovered that the new spam comments had only two links and they were getting through WP’s spam protection. I went back and checked the wording on the threshold setting and it seemed to indicate that if I wanted two links to trigger spam blocking, that I should actually set the count to 1. Or so it seemed at that point though I’m not really sure about that since I’ve messed about with that setting a bit since then and it seems to work either way now. Or something. Anyway, I was able to stem the flood of comments to about 75 on this round and nothing got really published except for a couple in between me changing settings since all the comments got held for moderation that time.

I was kind of wondering about my blacklist plugin by this time though since I did have the URL in the blacklist and the blacklist didn’t seem to catch the spam – it was just the built-in moderation in WP based on the number of links in the comments which was putting the stuff in to a queue to be moderated. I decided to add a few more variants on the URL in case the one I had originally added didn’t work and wait for further developments.

A few days ago, I got attacked again. This time, all the comments went into moderation since I had the link count set correctly but when I went in in the middle of the attack and removed the link count check, I started getting comments which were published. So, my blacklist plugin for some strange reason wasn’t working properly! Or maybe the spammer was using a method which bypassed the blacklist checking? I don’t know … I’ll have to take a look at the blacklist plugin again and see what transpires – guess I’ll have to do that soon though since based on the regularity of the attacks, I probably should have one occurring in a day or two …