Plugging Away at Plugins

Interesting how things work sometimes depending on the test data. I use the WP-ShortStats plugin to keep track of the traffic statistics to my blog. And of course, one of the most interesting stats to watch is the search keywords because sometimes you get such weird stuff 🙂

I like the WP-ShortStat plugin because it is compact, unobtrusive and gives a lot of stats which warm the cockles of a geek’s heart 🙂 However, a couple of days ago, I noticed that the plugin was giving me only a couple of search keywords instead of the longer list I had been used to getting before that and none of the terms which had appeared for a couple of weeks before, were now appearing. I thought maybe there was some automatic purging code in place which removed older entries and thought no more about it. At least for the time being.

Today, I felt like doing a little bit of digging since the ShortStat display was getting cluttered by lots of spam referrers and so on. So I took a look at the plugin code. It turns out that the plugin simply pulls the last 100 records matching certain criteria to give a list of search terms. So I ran the query on my data to see what I got. I noticed that of the 100 records returned, very few actually were search queries 🙂

So I took a look at the SQL and the ShortStat plugin code to see if there was a way to improve upon the code. It turned out that I could add an extra condition to the SQL query to return results which were more likely to be search queries. I did that and I suddenly had a full list of search terms in my WP control panel again!

I’m sure that the WP-ShortStat developer did a lot of testing of the plugin before you released it. However, as I mentioned initially, sometimes it’s the data which determines what sort of output you get. And no, this is not really a bug in the plugin but it did explain some unexpected behaviour and in case somebody else is in the same position I was in, hopefully this clarifies what is going on 🙂

A Gallery, My Kingdom for a Gallery!

I meant to write another post in the All Things Art and Good series today since I have received permission from a lot of wonderful artists whose amazing creations I’m eager to share with the blogosphere 🙂 However, I had this little WordPress plugin/coding entry that I had to make since it has some relevance to the art posts.

When I wanted to start writing about artists and their work, I wanted to liven up the posts with some images. Instead of providing links which led in all directions and which people might not be interested in following, I wanted to provide some visual cue as to all the wonder and beauty they might find in my own post. A thumbnail gallery of images seemed the best way to do this.

Unfortunately, I couldn’t find a single gallery plugin for WordPress which did what I wanted 🙁 There were WordPress gallery plugins aplenty which either provided a gallery provided the images were on your own server or on a remote Gallery2 installation or on a remote image sharing service like Flickr. But what if I didn’t want to download somebodyelse’s copyrighted material and put it up on my own server? (That somehow felt a little icky to me …) What if all I wanted was a simple gallery of images mentioned in each post that provided a thumbnail image at the end of the post and linked back to the original image? There just didn’t seem to be such a beast …

So, I decided to code my own. In fact, I spent most of the day before I wrote the first All Things Art and Good entry in coding up the plugin 🙂 At the moment, the plugin simply takes a list of image links in a post as long as they follow a specific format, remove the image links from the post and instead creates a gallery of these images at the end of the post. The plugin will automatically create a thumbnail version of the original image on my server the very first time. After that, the cached thumbnail is used for the gallery each and every time.

I originally created the gallery to follow a rigid grid pattern based on a certain number of columns. This worked fine except that the nitpicky side of me couldn’t abide the fact that when you resized the browser window, sometimes you might have a lot of empty space because you went with a low number of columns 🙂 So I went for a more fluid gallery which would adjust its layout as the browser window was resized. Unfortunately, this dynamic layout results in some rather strange layout issues. However, I’m going with it for the moment.

Of course, there’s another issue. When I wrote the plugin, I was testing things with the entry I did on Ruben de Vela. Now all of Ruben’s images were direct links to a specific image file. So my method of extracting info from an IMG tag and linking back to the original image worked fine. But when I got to Phil McDarby, I realized that I had a problem – Phil had pages for each image and these pages were full of other images and links which would provide a lot more information to a fan of his work. But my thumbnails were linking directly to the main image only, not to the HTML page!

Of course, I mention the pages in my post and link to them directly as well so as to get around this but I’d rather have the thumbnails themselves link to the proper page. I’ll probably have to modify the way the plugin works to do that though. So I’m thinking about how I’m going to do that at the moment. If you’ve got some solutions to the issues I’m facing, or have been looking for a similar plugin and would like to get the plugin from me, let me know. But tomorrow, we get back to our regularly scheduled program and look at the beautiful and funny artwork of Denis Zilber 🙂

First There Was Weird Science …

… And now there’s Lulu-science 🙂 Laurie invented the term yesterday when talking about my curiosity to know how many places you are bumped up the ranks with one book sale on Lulu.

Thanks to my friend Ginosion, who bought a copy of Honest, the Martian Ate Your Dog, we are well on the way to learning what the impact of one book sale is on Lulu rankings 🙂 Of course, the thing is, we probably have to wait a month for the next tabulation of rankings and so it’ll be a while before the next update on Lulu-science, but I will be sure to keep everybody posted as to how things go. (Of course, don’t let that stop you from buying a copy of my book, if you’ve been considering it 😀 Or simply download the free e-book version, see what you think of it and if you like it, then go buy the printed version …)

In other news, I am back to posting more regularly. I don’t know if this has anything to do with the switch back to WordPress or if I simply had been too tired of blogging, but I do find that I want to blog again. I do wonder if the WordPress interface has something to do with it though because in using it, I seem to find the WordPress control panel easier and more responsive than the MovableType one. Of course, this is not empirical data, just subject opinion …

Back To Pressing Word

Yes, it’s happened again :p Once upon an era ago, I used to use MovableType (MT) but I switched to WordPress (WP) when the MovableType license became too restrictive and commercialized. I switched back to MovableType when they opened up the license and at a time when WordPress development was stagnating. And now, to complicate matters even further, I have switched back to WordPress because I hate the "improvements" made to MovableType 4.0 and don’t really want to upgrade to that particular interface/release 🙂

Of course, switching back and forth between blogging applications is never an easy or fun task 🙂 The MT to WP importer included with WP is good enough if all you want is your entries (and perhaps comments, I forget now, I really haven’t used the built-in importer in a long time) transferred over. However, I usually want an exact copy of my current blog transferred over including authors, comments, categories, tags and even blogrolls, if possible 🙂 So, I had to do the work myself.

I had previously written a script to transfer most of the above from MT to WP but when I tried to use the script, I discovered that database changes had left the script behind :p So, I had to sit down and figure out the changes made to both MT and WP table structures all over again. After a couple of hours of work, I had a new script which allowed me to transfer over everything from my existing MT 3.x installation to WP 2.5 🙂 (Yes, I will put the script up for download soon …)

But that’s not all there is to it, right? I had to get my old theme converted over. But I was lucky there because I had originally converted my WP theme over to MT when I switched to MT the last time and I still had the old WP theme. So a few tweaks later, I was ready to go there. But wait … there was more to be done …

I post to my blog using my own offline blogging utility, Blog. So I had to test Blog to make sure that it worked fine with the latest version of WP. And wouldn’t you know it, there was an issue – tagging, which had worked fine under MT, was not working with WP. I had to find the issue and fix it. Then, I had to get all the plugins I wanted, configure them, check that they worked fine and then deploy them.

Finally, all the work is done. And I’m ready to get back to WordPress. So here’s Solipsistic Meanderings once again in WP 🙂

Vacillations of the vague kind

I am thinking of switching back to MovableType once again :p The reason? I am tired of dynamic pages and the headaches they bring with them 🙂 At one point in time, when I first switched to WordPress from MovableType, I was really happy with dynamic pages and all the nifty things they allow you to do. But as time has passed and I’ve come to consider load on the server, spidering by search engine robots etc., I’m beginning to think that perhaps static pages are a much better option.

For instance, I notice that the Google spiders are hitting my WP-Cache pages. Those pages are ephemeral – they will be there for an hour today but gone tomorrow. So why are the search spiders looking for old pages again and again. How much load does that put on the system?

Being in website hosting, I know how much load is put on the server by dynamic sites which depend on database queries. I don’t really want to put that kind of load on my host’s server. I know that I don’t actually put that kind of load on their server because I don’t have a really popular site. But what if the traffic to my site were to increase? I am not sure and I don’t know if I want to find out.

Of course, there are other reasons. I don’t like the pace at which WP development is going nor the direction they seem to be taking. There isn’t that much being done about combating spam for instance. The efforts are all in the form of plugins and none of them seem to be really effective.

So, I’ve been looking at MovableType. They do have the features I want from a blogging platform. The free personal version even allows multiple blogs – something that I’ve been complaining about in WordPress for a while now :p The only snag at the moment is that I hate MT’s import feature because it relies on a text file. All of the WP to MT migration scripts rely on MT’s clunky text-file based import. I think I might have to write a new script which simply transfers the content from one database to another and I should be set to start testing the new MT to see if I can work with it 🙂

The template tango

The design for this site (Solipsistic Meanderings that is, not the original Developer’s Corner) was originally made when the blog was implemented using Movable Type. I knew exactly the kind of look I wanted and I spent quite a bit of time in getting the template for the site to look the way I wanted it to be. I was really happy with how the site design turned out when it was finally complete.

Then came my switch to WordPress and I was faced with the prospect of either going with a new template or switching the Movable Type template over to WordPress. I decided to go with the latter option since I really liked the template :p In those days, WordPress templates were nothing but an index.php file which contained the styling for the blog’s default page. So, it was fairly easy to convert things over once I discovered the WordPress equivalents to the MT template tags that I was using. That I believe was using WordPress 1.2

Then came WordPress 1.5 and then 2.0 and while the template system in WordPress did change radically, they kept it backward compatible and so I was able to continue using the old template that I’d made for WordPress 1.2 back in the day with a few minor tweaks and changes. Of course, this meant that I wasn’t using the latest and greatest advances in WordPess templates but I was happy enough not to have to mess with the template again and so left things alone.

However, recently I had to do some digging into the WordPress template system when I wanted to design a custom template for Shout Out! In the process, I realized that there were a lot of nifty things in the new WordPress template system that I was missing out on because I continued to hang on to the legacy single file template system from the old days. So, once I got Shout Out! working, I decided to switch the SM template over to the new system – mostly because it’s easier to edit sections in the WP web interface than to search for say the sidebar code in the complete file.

Once I got the template split into header, sidebar, body and footer and everything as working fine, I came to know about WordPress Widgets – where you can have your full sidebar be a collection of widgets. This meant that I had to do some further tweaking to my template and sidebar code. So I went ahead and tried it but ultimately gave up on widgets since I like the ability to customize that the standard sidebar configuration provides. But still, the widgets are a neat idea. Just goes to show though that a blogger’s work is never done – there is always something to tweak or to customize :p

That fiendish spam :p

First there were the blogs. Then came the comment systems. And then came blog spam. In the old days when Blogger was new and "blog" wasn’t a buzzword, we didn’t have to deal with stuff like comment spam. We could go online, post our thoughts and that would be that. Now, I spend most of my time weeding out comment spam on my blog than I do on actually writing an entry :p

Of course, comment spam mutated into trackback spam and then pingback spam. For the longest time, I would get up in the morning and would have hundreds of trackback spam entries to wade through and to delete. I then developed the WPBlacklist plugin for WordPress and that made things a bit easier – it would automatically delete all comments, trackbacks and pingbacks marked as spam. But as these things have a tendency to do, spammers got smarter. Simple blacklisting wasn’t enough after a while. They would consistently get around blacklisting, again and again. So I switched WordPress plugins to combat the problem.

I installed Bad Behaviour for WordPress and Spam Karma. For a few months, I had no trackback spam attacks and whatever comment spam that came my way was caught by Spam Karma most of the time. That was till a week or so ago. Instead of a couple of spam messages a night, I suddenly started getting ten or so a night. And then a couple of days ago, I started getting trackback spam again. Now I get about fifty spam messages a night. This is still a far cry from the hundreds that I used to get nightly but it looks as if that I might be back to that soon if this trend continues.

So I have to conclude one of two things – I have either become very popular amongst spammers or Bad Behaviour is not as effective as it used to be. Spam Karma still does tag all of this as spam and does not display it on my blog but the problem is that I have to wade through all this spam in the morning and delete it since Spam Karma simply holds the spam – it does not automatically delete it. The authors of both Spam Karma and Bad Behaviour have said that spammers are becoming smarter in how they write they spambots. Perhaps this is just an indication that this is true. If so, then it’s time for the next generation of weapons in the anti-spam arsenal. I just hope they get here soon :p

Tags, categories and changes

After getting the latest builds of Blog to work, I have decided to remove Ultimate Tag Warrior from the mix here on SM. This is not due to any issues with UTW itself – it’s a great plugin and does the job well – but due to me realizing that UTW was overkill and that for what I was doing, it introduced overhead that wasn’t needed 🙂

Basically, I post all my entries from Blog. I never use the WP admin interface much except to remove spam. I do all my editing, posting and other entry management stuff via Blog itself. The only reason I was looking for a tag cloud mechanism was so that I could show a visual representation of the various topics covered on this site and let somebody interested drill down easily to only the posts that interested them. UTW allows you to do all that but in my case, I was simply converting categories to tags to do this. Since the categories are a built-in feature of WordPress, it occurred to me that I didn’t need the overhead of UTW (which would slow down XMLRPC to an extent) if I had a plugin which displayed a tag cloud based on my categories.

Enter the WordPress Heat Map plugin which does exactly that 🙂 After a brief search of the Net, I actually found a couple of different alternatives. The other one was the Weighted Categories plugin, which apparently inspired the author of the Heat Map Plugin. However, the Heat Map plugin appeared to be better maintained and supported and so, I went with that :p Incidentally, the author of the Weighted Categories plugin is apparently working on a way to make his plugin work with keywords instead of categories. I would be interested in that if it didn’t create too much overhead and worked without any issues with XMLRPC-based posting. But I doubt it … There is another plugin which supports keywords – the Keywords plugin, and the author is apparently working on tag cloud kind of support. Unfortunately, it works basically the same as UTW (just not as well :p) and so, I don’t see the point to pursuing that one either :p

Basically, what I’d like from a keyword tagging plugin for WP, is for the plugin to examine each new post, identify the keywords automatically and then add them as tags. Of course, at the moment, that appears to be too much to ask for :p The biggest problem appears to be identifying the keywords. Yahoo Content Analysis doesn’t do too great a job of it as I’ve mentioned before and while Tagyu does do a much better job, it is abominably slow and tends to end up in my XMLRCP session timing out before the analysis is done :p The best solution would be something that the plugin can do itself instead of contacting an external server but the implementation is the key ….

Midnight fires in the coding forges …

It’s more Blog news again I’m afraid since I haven’t been doing much else :p (Incidentally, I really should get the Blog page re-designed so that it is more consistent with the look of the new site … you know how it goes, so many things to do ..) Anyway, getting back to Blog coding, after doing some extensive coding and light testing, the category feature in Blog works fully and it even publishes via XMLRPC to WordPress :p I was really happy to get that bit working finally since I’ve been working non-stop at that functionality since I got back to working on Blog.

Now that that is done, I have a few more things to fix up in Blog before I can do a release. The internal release versions have been jumping around quite a bit since the last beta release due to the fact that I’ve been sending private builds to a few people who asked 🙂 I need to keep the newer builds compatible with the older builds and so I’m now up to 8.0 beta 5 at the moment. I’m hoping that I can do a public release of beta 5 but I do need to get a few more things fixed up first.

One thing that I have fixed since the category stuff was done was the find feature. It has been broken since at least beta 3 and I finally got around to fixing it :p At the same time, I want to add the option to be able to search the current entry for some given text as well as searching the current blog and all blogs under Blog. I even managed to fix up the search results dialog so that it shows the new WYSIWYG view as well as the old HTML view. However, I just discovered that the HTMLEdit component that I use might not be able to do search and replace of text. This is going to be a bit of an issue with the find feature and the new replace feature that I intend to implement … bummer 🙁 Hopefully, I can find a workaround … There are also the pages of bug reports on the 8.0 betas at the forums that I need to attend to … Work, work, work :p

Coding again …

Well, some of the internal Blog changes are done 🙂 I finally added a data module to the Blog code, transferred over all the data tables scattered over different forms on to the data module and cleaned up the code to work with the data module. Then of course, there was all the work involved in getting the code to compile again. It is finally done and in the process I fixed another issue – the internal Blog categories were not showing up properly and this is something which has been annoying Laurie :p Turns out that the categories were getting assigned fine but were not displaying due to an issue with the sequence of events that were being carried out.

Now that I’ve got the code working again, I plan to redo the entire Journal Management dialog. Instead of the current single dialog with multiple tabs, I’m going to go for four different dialogs which will allow more flexibility in terms of how each area (servers, blogs, templates and uploads) works. This is mostly to accommodate all the new category management stuff which is going to have to come in for the new release. I’m going to have to add category lists for each remote blog so that each installation of WP (or whatever supported blog) is kept distinct from the internal Blog categories. I will also have to add code to merge in the new remote blog categories with existing Blog categories and keep track of what gets assigned where. Ah, joy :p

On other coding fronts, I still have that add-on plugin I started work on. I still want to finalize the feature where it will automatically create tags for all posts without tags even though I will not use the tag for the main purpose it was created 🙂 So, I don’t know which one I will want to work on today – Blog or the plugin but hopefully, both will get worked on …