Can I tag along?

Yesterday’s serendipitous discovery of TagCloud has led to a few other developments since then :p I realized after watching my tag cloud for a while yesterday that it wasn’t changing at all – it was simply displaying the tags for wordsmith.org and nothing at all from my own blog entries. So I decided to do some digging to see what was going on. I still am not sure what exactly is wrong with TagCloud because I had assumed that the Yahoo Content Analysis service basically went through your site and figured out what the relevant tags for your entries were instead of you having to set up tags for each entry. However, nothing seemed to be happening – maybe their indexing service was slow or maybe it wasn’t functional. All I know is that even after 24 hours, my site was not displaying any tags.

So, I decided to look for alternatives tailored specifically towars WordPress 🙂 Or more accurately, I wanted to find something for WP which would generate tags in case I was wrong about TagCloud and they actually needed the tags somehow explicitly specified in the RSS feed. That was when I came across this blog post – now here was a solution (and a total solution at that :p) to what I’d been looking for. So I went over to the Ultimate Tag Warrior page, downloaded the plugin and installed it. I removed my existing TagCloud entry on my blog and added my own personalized tag cloud for my site. But nothing still showed up!

Why not? Because none of my entries had existing tags :p So now I had to find a way to add the tags to get a tag cloud. I spent a little time doing the entries by hand and trying to figure out what tags to assign before giving it up as too time consuming. The Ultimate Tag Warrior plugin does have a button which is supposed to retrieve the suggested tags for each entry from Tagyu but for some reason, the button wouldn’t work – at least at my end. I decided to go through the source and see what was happening and if necessary, write a plugin of my own which would connect to Tagyu and pull suggested tags for each entry at the time of publishing. Perhaps I could adapt the system to also go through all my existing entries and add tags …. So I have a new project to work on … Not that I really wanted one but oh well, need something to keep life interesting, right? :p

Inconvenient breakdowns

Technology always breaks down when you least expect it and when it can cause the most inconvenience :p Of course, Murphy said it better with his law but still this fact was brought home to me once again this week.

I had been using Spam Karma 2 for about a month now. It worked well and caught everything thrown its way during that time period. So, with both Spam Karma and Bad Behaviour in place I went off for a couple of days secure in my mind that my blog would be safe. (BTW, I should mention that Bad Behaviour has been doing an excellent job with all that annoying trackback spam. I see a lot of trackback entries – sometimes all I see in my visitor logs is trackback entries – but none of them seem to make it through. Of course, it is possible that Bad Behaviour is blocking some legitimate ones but then again, I’m not absolutely certain this is the case.)

I come back after two days and what do I find? Not one, not two but three spam comments had gotten through everything and were sitting pretty in my blog. I immediately deleted them. I believe I know why they made it through (no, Spam Karma wasn’t malfunctioning :p) but the interesting thing is that this happened while I was away and so couldn’t deal with them immediately. So I’m left wondering as to whether the three comments were just because the first comment got through and the guy kept going or was it just another incident of things going wrong at the most inconvenient time where it could cause the most damage? :p

Aimless Again

Now that I’ve got the site sort of working the way I want, I am aimless again :p There are plenty of things I really need to do – I need to finish work on WriteTrack and put that up for download, I *really* need to put out a new release of Blog, I should get working on my next novel and so on. But I just don’t feel as if I have the energy to do any of it. I’m taking a couple of days off due to the second of our (meaning Muslim :p) festivals tomorrow. I’m hoping that the time away from computers and work and all the other stuff that go into normal routine will help me recharge my batteries and get back into the thick of things.

WriteTrack is actually almost done. All the major work is over and I have one more section to add – the one for tracking income and expenses. Laurie has been trying to get some beta testers for it on a couple of writing lists that she’s on but we haven’t got that much of a response. There were a few people who downloaded it and one who actually provided some feedback. But then again, I guess one person with feedback is better than none :p

As for Blog, it’s actually ready for another beta release and has been for a while. I’ve been using it to post to WordPress for like the past year or more – I’ve just been kind of reluctant to release it since I know that there are a lot of areas that I personally don’t use which are going to prove to be troublesome. I don’t want somebody to jump in to the beta and then find that they have reduced functionality. So I should actually get off my butt and fix this stuff, right? That’s where the lethargy comes in :p But there are a few niggling things that I really want to fix and hopefully, I’ll get around to fixing those and doing another release after this little break …

And the winner is …

I was looking around for a mailing list/notification list script yesterday for a bit before I decided to customize one for my own use after all :p It seemed to be the easiest course of action since I couldn’t find even one that had all the features that I wanted. So I went to work, combined the code from two scripts that I liked and finally ended up with something which works (sort of) and is both secure enough and extensible enough for my tastes.

Of course, now that I had the mailing list software, I had to go ahead and use it :p That meant a further site update. The reason I’d needed a mailing list in the first place was because I wanted to provide the option to anybody who might be interested in my book(s) to be notified of any changes or events of note – such as the actual publication of the book :p (Not that I really know for certain that anybody is interested mind you … but there is always the off-chance that somebody is :p) So, I hastened to add a new page to the Books section of the site, add links, set up the mailing list subscription and so on and finally it all seems to be working. If you’re interested, you can go here and check it out yourself :p Don’t expect too many mails to come from that list but I’ll try to send out a mailing every time something of note happens … so whadda ya waiting for? Go on, sign up, you know you want to :p

The Work of Blogging

In the old days, running a blog used to be fun. You just set the blog up and writing about whatever took your fancy. But then came the comment spam. And then the referral spam (or was it the other way around? :p). Then the trackback spam, the pingback spam, the tagboard spam … you name it, there was sapm for it. When writing your blog became secondary and combating all this spam was the primary job, that was when blogging became a not-so-fun endeavour – at least for me.

I had not worked on my sites or my blog for such a long time that I had totally given up monitoring what was going on with my sites. Just a week or so ago, I noticed that the then SM sub-domain had about 1GB of traffic in about a week and I had not posted an entry there in months! So I began doing some digging and discovered that I’d done something really stupid back in the day and had not realized it :p I had set up a top 10 referrers list for my blog a long time ago and had not realized that referrer spam had been hitting it in ever increasing numbers till the referrer spam was doing around 300MB of transfers a day! That was when I finally noticed it.

But taking the top 10 referrer list down did nothing of course. These guys probably mark sites with referrer lists, set their bots up and then forget all about it themselves. So nobody was going to notice that I’d actually taken the top 10 referrer list down. I had to do something myself. A quick Google around was all that was needed. There were a lot of people who had to deal with the same issue and they were doing so by using mod_rewrite and .htaccess to do the job. So I sat down and went through the list of spam referrers to my site, came up with a list of rules and then put them up. The next day, the transfers from the SM subdomain went down to about 10MB from a whopping 300MB! I’ve had to keep an eye out on the traffic figures since then and tweak the referrer blocking list (there’s automated scripts out there which will parse your Apache log files and add new referrer filtering rules ….) but it’s been worth it.

Now I need to find a solution for the damn tagboard spam :p (The tagboard’s been disabled for a while now since all I get on it is spam …)

Scamming Scum

I’ve been working on the redesign of the site whenever I have a spare moment – I think I’ll divide the main site up between the blog, my writing and my coding. I want to have a consistent interface which still visually breaks the site into these different sections. The design for SM will remain pretty much the same but I’m adding two new layouts for the other sections – at least, that’s the plan.

In between working on the layout, I’ve been reading up on more scams by fake agents and I must say, it just makes me wonder about humanity all the more. Anyway, before I get into that, I guess I should talk about my own brush with a fake agent. It all started with a standard query. Usually, I would e-mail an agent a query but the Robins Agency had an online form for people to submit queries. So I submitted via that. I must admit, I made one mistake before querying them – I neither checked them out via Google or checked one of the online lists such as Preditors & Editors which provides a list of scam artists who pose as agents. (But I’m wiser now :p). The very next day, I received this response:

Dear Fahim,

Yes, we’d like to take on your work; however, we limit the number of clients we accept to only those with marketable works and who are in a position to pay our annual retainer fee of $3,250 (US). Although many agents charge $4,500 to $6,500 per year for expenses on top of their commission rate of 15% – 20%, we’ve found that we can do a superb job for only $3,250 with a commission rate of 10%-15%.

As to what you can expect. The process is rather simple (as compared to rocket science 🙂 ); we bring your work in, take it through editorial to make sure it is in good shape; take it through marketing and research to find not only direct sales opportunities, but indirect as well (i.e. film, tv, foreign markets); put together a marketing package, then start the submission process. When a buyer is found, then the negotiations begin.

You are informed each step of the way as to what we are doing with your work; and are a key player in the sale negotiations as well. We normally answer e-mails within 24 hours and telephone calls the same day … that is, if you don’t reach us when you call.

What you won’t get is the typical attitude of most agents (you work for them, not the other way around); you won’t get someone cutting your work up because they can, not because it may be needed (yours doesn’t need it); you won’t get an agent selling you or your work short just for their commission check; and you won’t have to wonder what is being done with your work as you’ll know.

Many agents do not provide the scope of services that we do; and they expense back to you (monthly or quarterly) about $4,500 to $6,500 annually.

We think that we provide a solid service for the monies charged; more importantly though — our clients feel they are getting more than their money’s worth.

If this is the way you’d like your agent to work — for you, not the publisher — and if this is within your budget, then let us know and we’ll get the paperwork in order and start straight away. I look forward to seeing how far our agency can take your work.

With regards,

Syd Michaels
********************************
Robins Agency
*******************************

Now the above is, in a manner, fairly straightforward – they do tell you that they charge an annual fee of $3,250 and as an aspiring author, I half-thought, "hey, if they can deliver on what they promise, that might not be so bad" even though I knew that most reputable agents did not charge an up front fee. However, I did decide to do some checking up on them online at this point and what I found was various posts on different forums where people said that they’d paid the Robins agency and then had not heard back from them or that Cris Robins herself would call a writer who’d signed up with the agency and cry (yes, cry) on the phone asking for money.

Now, I have no idea as to the veracity of any of these claims. The Internet after all is the one place people seem to feel secure about making all sorts of bogus claims and indulging in mindless slander. However, there just seemed to be too many instances of the Robins Agency being mentioned as a scammer for it to be just slander. Besides, the Preditors & Editors list (which *is* reputable) mentioned them as well. So, while I still wanted to cling to the hope that they would actually sell my book for me, I decided that these really weren’t the people I should be going with. So I sent them off the following e-mail:

Dear Syd,

Thank you for getting back to me. However, a few things about your response did not ring true with me – such as the fact that you claim that my work would not need any cutting based on just a few pages of my manuscript. I am in search of an agent who believes in my work and wants to sell it. You asking for $3,250 yearly up front to represent me seems to indicate that you do not really believe in my work but are simply doing it by rote. So thank you but I think I will pass. Regards,

Fahim

Two days later, I got the following from Syd:

Fahim,

I did not say your work would not need any cutting based on anything; fact is, I said we’d take it through editorial to make sure it is in good shape.

There is no such thing as a free lunch, Fahim. You get what you pay for — this is a business, not a wish factory. We pick up works we BELIEVE we can sell; but belief doesn’t pay the light bills.

We know we offer the most fair, complete, and honest service on the market. We would have like to work with you to realize your dreams.

Best to you,

Syd

And that was the extent of my dealings with them. Now this might just be conjecture, but the last response to me felt as if they were trying to make me feel bad about rejecting them and perhaps go back to them begging for another chance. Well, not a chance :p

Of course, this encounter was pretty mild compared to some of the others I’ve been reading about online. The agents appear to come after the people who posted negative comments about them, threaten law-suits, stalk the people, mail bomb them, call their neighbours up to get their unlisted numbers – the whole works. Then there are those who actually defend these people – saying that those who complain about them don’t have the talent to make it and so they are simply trying to bring down good people providing a solid service.

I can’t speak for any of these people but what I can say is this – if an agent wants money from me up front to sell my book and also wants a chunk of the money that I’d get for my book, then there really is something wrong. I agree, there is no free lunch – but that works both ways. I don’t see why I should provide a free lunch for an agent to potentially do nothing after taking my money. Their incentive is the commission that they get, my incentive is the (potential) fame and fortune in selling my book. An equitable trade, wouldn’t you say? :p

Change is in the air …

It’s been a very long time since I’ve done anything with my sites (except to try to combat all the spam that is :p). I’ve been taking a break from the programming and have been working more on the writing end of things. And then work became a bit hectic and took over everything and so didn’t have time to do anything at all.

On the writing front, I finally found an agent who wanted to actually read my manuscript instead of simply saying "Not interested" after I queried them. However, the manuscript has been with them for about two months now and haven’t heard much from them 🙁 So, I’m a bit disheartened at the moment and wondering whether the whole thing is worth it.

On the other hand, I’ve also been reading about scammers on the Net who rip new writers off without any compunction. Not that the agent who has got my manuscript is one – I did check them out first :p Interestingly, I did get approached by a scammer at one point and had an interesting exchange of mails with them but don’t want to get into that in this post since that’s a whole new post by itself :p But what I did notice from my reading online was that these scammers can get pretty vindictive when they are called scammers – imagine that :p There were some pretty interesting (and horrifying) stories online but one of the most detailed (and engrossing) ones was this one. After reading that, I do have to wonder if it is even worth it going through all that hassle to get published …

You might have noticed that SM has now moved to be the main focus of my domain. This is in line with other changes that I’m contemplating. If the current agent that I’m corresponding with does not work out, I might try to simply publish my current novel (and the other six in the same cycle) online. My friend Nige has been prodding me to consider some sort of Net-based publishing mechanism like Cory Doctorow does and I might actually do that and be done with all this agent business …

So what does that mean for the software part of this site? Well, the software will still be there and I will continue to provide support as and when possible. I’m even working on a few apps which will help a writer keep track of submissions and story ideas and they will go up when I think the app is ready. However, coding will probably not be the main focus of this site any longer. The old Developer’s Corner content is still there and I will eventually add links from the main page to all that stuff but for the moment, it becomes just another historical layer to this site :p

The Long Road to Publication

I have been totally incommunicado while I finished work on my first novel – I didn’t do much writing here or on DC and I have not been doing much work on my software – except to write a few new tools for my writing but more on that later. I finished the first draft about two months ago and since then, spent a month polishing up the first draft. Laurie helped me immensely with it since I tend to have this bad habit of using adverbs liberally and Laurie is of the opinion that there is only one solution when it comes to adverbs – total genocide :p Between the two of us we hit a happy medium where I do use adverbs but not as profusely as I did at first. I do see her point about adverbs being the lazy way of saying things at times and how it weakens a sentence but sometimes I feel that an adverb can actually make a sentence more succinct rather than having to use many more words to say the same thing without adverb usage 🙂

Anyway, now that I’ve polished things up a bit, I’ve moved up to the next step. There is difference of opinion at this point – some believe that you should go straight to a publisher while others believe that you must find an agent first. The first class of people say that if you find a publisher, any agent will take you on while the latter say that an agent will give you more credibility with a publisher 🙂 I decided to try an agent first and wrote to one and waited a couple of weeks, they took a look at the first few chapters and said that wasn’t for them. (Of course, I had done something really stupid with the first agent – I sent them my unedited first draft because I was too impatient and in the words of Laurie, "Of course, they rejected it" :p) That’s a lesson for anybody who might want to learn from my mistakes – do not send out your unedited first draft no matter how impatient you might be :p

I then queried a second agent and waited two weeks with no response and then wrote to them again to find out that they’d never received the first query. They came back with a "I personally dislike humorous science fiction". Then I queried another well-known agent and was told "that this doesn’t sound like a project for us" without them even seeing my manuscript.

So here I am – struck out three times. This probably is the time when I should take a good look at things and decide where I am going … or maybe not :p I did manage to get a hold of Terry Pratchett’s agent Colin Smythe and I must say that Colin was a really nice bloke and a great human being in that he didn’t know me from Adam and yet, he helped me out as much as he could and pointed me in the right direction for submitting to more agents and publishers. I will be forever grateful to Colin for his kindness.

I have decided to continue to try agents till I find somebody who likes my style of writing because I’m beginning to get the feeling that humorous science fiction might be a dirty word amongst agent or something. However, I ran across this site by a fellow programmer who has actually managed to get his humorous science fiction novel published and that has given me hope. I will continue on my road to publication and will try to keep this page updated in case somebody is still interested in how it goes with me – or not 🙂

Backtracking on those trackbacks …

No sooner had I converted my site over to use WP 1.5 (after making the earlier entry about planning to do so), I got hit by one of those bits of new trackback spam – ugh! My first reaction was that perhaps the new trackback and pingback blacklisting features that I’d added to WPBlacklist weren’t working anymore due to some change in WP 1.5 (I had actually done the work before WP 1.5 was released …). I had almost gotten my development stuff ready to debug the code when I thought to go back and check on the trackback spam I’d received. Turned out that they’d come up with yet another variation on "texas holdem" – in their case, it was texas hold’em -. My initial reaction was to simply add an entry to the blacklist for the new variation and go on but then I decided to check how many entries I had in the blacklist for just the word texas and I was amazed to discover that I had over 300 entries! So I decided to come up with a regular expression which would catch all variations of texas holdem and after a little mucking around, the final masterpiece was "texas.*?hold.*?em" :p

Of course, the above may identify some benign stuff as spam as well but for the moment, I don’t care – I will deal with it when it happens :p (Oh if anybody is going to comment on this entry, you’d better not use the words texas holdem :p) I then set about deleting all the individual blacklist items which would be replaced by the regular expression and discovered that I could remove 316 items! Of course, I don’t know how many (probably all) would come back when I next update from Jay Allen’s central list but for the moment, I am happy the way things are 🙂 And the icing on the cake was just as I got done deleting all those 316 items, a new e-mail comes in saying that a comment was automatically deleted by WPBlacklist. The offender? The self-same guy with that new variation on the dreaded texas poker theme – gotcha sucker!! :p If anybody is eagerly awaiting the new WPBlacklist, I am happy enough with the way it is working at the moment (and what better way to test than in a production environment? :p) and so just have to make the necessary documentation changes, pack it all up, update the WPBlacklist page and then upload it. I promise that I’ll do that sometime tonight 🙂

Entries, Comments and Connections

It’s weird how these things go … Well, not really weird if you stop to think about it for a second but I just like to be amazed … or was that amused? :p I hadn’t received any comments or spam on this site for the longest time and I was beginning to think that my anti-spam measures were finally paying off. Then I made an entry a couple of days ago and then I heard from my friend Edward today – he said that he couldn’t post comments since they don’t appear and that he can’t leave a tag since the tag board tagged anything he posted as spam :p I checked on it and found that there had been a reason for the absence of comments …

Both the comments system and the tagboard have a spam blacklisting feature. Basically, I adapted the WPBlacklist plugin code to work for the tagboard as well. So, they both run off fairly similar blacklists and for some inexplicable reason, the letter "b" had gotten in to the blacklist :p And that meant that you couldn’t post anything with the letter "b" in it without it getting tagged as spam. I fix the problem and what do you know? Barely an hour later, I get my first tagboard spam in ages and I get two comments posted on various entries over the course of the day! I am not even certain if one of the comments is spam or not but the other is not spam. However, it is intriguing that all this happened on the same day because there really were no comments till now for a while – even if the comments had been deleted because they were mis-identified as spam, I would have received a notification, so I know that there were no spam comments 🙂 Anyway, that’s my little bit of wonderment and mystery for the day … Now I’m going back to doing other stuff – like watching the really gory and fairly pointless "Resident Evil: Apocalypse" :p