Where, oh, where, oh, where, is my agent?

I’ve been researching and compiling a long list of agents over the last couple of years in my effort to find somebody to represent "Honest, the Martian Ate Your Dog". I still haven’t found representation, but I certainly have a pretty big list of agents which is not doing me much good till I finish my second novel and start querying them again 🙂

So, I thought, why not set up a page online which lists all these agents and publishers so that somebody else can benefit from my research as well? I originally wanted to add an export feature to WriteTrack (I keep agent information in WriteTrack, my submission management software) so that I could export the database to HTML. But the more I thought about it, the less efficient an HTML page seemed. It would be a long static list of agents/publishers which would take forever to load and would be hard to search.

So I began thinking about a way to quickly code up a nice little interface to access and manage the data and to even provide a way for users to give feedback on the agents/publishers. The first thing I thought of was – Adobe Flex. Basically, it’s a new development environment (as well as a suit of technologies) which uses Flash to provide a spiffy interface which is also extremely fast to design and develop. I had not used Flex before but I had been meaning to and after about a day of mucking around, I came up with this page.

The page works fine but it takes a little bit of time to load and needs Flash 9 to work. While I am happy enough with the interface for the moment, I am considering going the AJAX route and redoing the interface so that it loads even faster and doesn’t depend on third-party technology so much. Since the data will remain the same, you can continue to use this interface till I do something new which will hopefully have even more features and will work even better 🙂

Tagging the mix

OK, I switched back to MovableType a couple of weeks ago. Since then, I’ve been working on getting Blog to do every other little thing I wanted working in conjunction with MovableType. That’s also mostly done now (yes, expect a new Blog beta soon – I just need to test this build for a few days to make sure that everything works fine and to add a few more features for "classic" blog – such as a new <$BlogTags> tag :p)

Now that Blog works the way I want with MovableType, I’ve been turning my attention to the latest features that come with MovableType 3.3 🙂 One of these is the ability to use tagging. What that means (if you’re not up on tagging) is that you can use a set of keywords to tag the subjects (or areas) that a particular entry deals with. I mostly use tagging because I love tag clouds – they give me a more visual idea of the kind of subjects that I talk mostly about and what is important to me on a long term basis :p

Till MovableType 3.3, I was faking the tags. I was actually using categories to give the appearance of tags. The problem with categories is that they are a bit more static than tags. You have to first create a category and then set it up before you can use it. On the other hand, with tags, all you do is simply specify any keyword/tag you want on the tag field and the blogging framework handles all the nitty gritty stuff behind the scenes. So, being utterly lazy, it was no wonder that I opted to use tags :p

The problem was, that I had already set up all my entries in Blog using categories and I really hate to duplicate work. I didn’t want to have to go through hundreds of entries, look at the categories and then set up tags for them. I got around this in Blog by providing an option in the latest beta which asks you at the time of upgrade whether you want to convert categories to tags. But what about MovableType? I could have edited all the existing entries in Blog and have published to MovableType again but that also sounds too much like work :p So I sat down and started working on a new PHP script which would convert existing categories in a MovableType blog to tags.

The result is a new script which seems to work fairly well 🙂 You can find it under the Blogging Scripts section on my site if you need this particular functionality. Just remember that the script works only with MovableType 3.3 or later …

Bugs and betas

Yes, I know, I probably should have another entry in the self-editing series today :p But I decided to take a short breather to write about coding so as to reassure any Blog users sitting on the edges of their seats :p

I’ve finally gotten around the HTML comments being parsed out issue in HTMLEdit 🙂 As I believe I mentioned when I last wrote about it, I had switched to using HTML comments to mark out the position of the More … tag instead of the Blog tag that I’d been originally using. The HTML comments however were being stripped out and there was no help from the HTMLEdit author even after posting on his mailing list. He did release a new build of HTMLEdit a few days ago but there was still no answer to my question and the new build did not resolve the issue.

So, I sat down yesterday with the source code for the new build and Blog and began doing a few systematic runs to isolate the bug. I finally discovered that the issue was that any HTML comments were being saved fine but were stripped out when the entry was loaded again into the HTMLEdit component. The question was, why? So it was back to more debugging. It turned out that the HTMLEdit author was checking to see if a comment tag was in the HTML document’s body and the comment was saved only if it was in the body.

Now my entries, they are not full HTML documents. I simply save the entry as a snippet of HTML and so of course, there is no BODY tag and so the comments did not turn up as being in the document body :p So they were being stripped out. I modified the code to remove the body check and also notified the author of the component via his mailing list. So what this means is that I’m now ready to proceed with the coding that’s needed to get the next beta of Blog ready 🙂

What’s planned? Well, the thing which was holding me back was the conversion of <$BlogMore> tags to <!–more–> comments. Now that I can do that, I will be bumping up a beta version number to do the necessary internal changes. This release will also have tag support since MovableType has tag support via XMLRPC and I want it. I also would like to roll in the changes for the category categorization :p But I’m not sure if I’ll get that bit working in time for the next release. But hopefully, I’ll have a release ready soon …

Site and Blog ….

Well, the site is coming along fairly OK 🙂 So now that the transition seems to be over and MovableType working fairly OK, I’ve been turning my attention to what MT offers and how I can leverage it.

One of the first things to fall under my searching eye was tags :p Now I’ve been talking about tags before and my efforts to implement them in a way that would work with Blog and whatever remote blogging platform that I used. WordPress didn’t actually do much in the way to support this out of the box – at least, not when I last checked. MT on the other hand, supports tags in a big way – especially in their upcoming 3.3 release.

In fact, MT supports tagging via the XMLRPC interface as well! The uploaded entry structure via XMLRPC has allowance for a tag field and so all I’d have to do from Blog would be to simply provide a place to enter the tags in the Blog interface and then upload the added tags with an entry and MT would take care of the rest. What could be more perfect? :p

The problem here is of course that I’d have to make another database change. As I’ve explained before, this is where I’m stuck with Blog at the moment. It looks as if the HTMLEdit author has disappeared again. He does this every once in a while but unfortunately, what this means is that I can’t proceed further with the current work on Blog till either he comes back or I find (and fix) the bug myself. At the moment, it looks as if the latter course of action is the only one available …

Moving the type …

Yes, as some of you might have noticed already, I’m back to using MovableType 🙂 There is less server load, in certain ways it’s much more configurable and (as I keep saying) I do like their anti-spam options much more than I like the WordPress solutions. Sure, there are downsides. I can’t have dynamic stuff like calendars which show the current date :p But overall, I think this is a better solution to go with.

The changeover itself was fairly easy. It took me about a day to modify my old script for moving data from Movable Type to WordPress and to convert it to move data from WordPress to MovableType 🙂 The testing took a bit longer and I had to do some rewriting as I discovered issues. Somewhere along the way, I also found out about MovableType 3.3 beta and had to again adapt the script for the single relevant database change that I found there. I have put up the scripts here if anybody wants to use them.

Of course, once the data was transferred over safely from WordPress, I had to re-do the templates that I used on my site and the creation and tweaking of templates took about another day. Was I ready for posting after that? Oh, no :p I now had to test posting with Blog to see that I could post from my desktop to my new MT based blog using Blog. This resulted in further code changes. Not because Blog wouldn’t publish to MT but due to a curious problem which arose due to this particular set of circumstances.

Blog keeps track of what entries have been published to a remote server. I had already published the entries for my WordPress based blog. But the MT blog has different entry IDs than the WP blog did. So I couldn’t simply switch remote blogs. I could have wiped everything on the server and published all the entries from my desktop via Blog to generate new entry IDs but then I would have lost the comments on the server since Blog doesn’t track comments when publishing via XMLRPC (mostly because none of the APIs provide a way to download comments). So, I had to add new functionality to Blog which would allow me to synchronize existing entries with those on a remote server. Adding that functionality and testing it took some more time.

Now, I’m finally ready to post to the blog again using Blog :p So let’s see how this one goes …

Find that bug …

I’ve been working a lot on Blog the last few days but I’ve hit a brick wall again 🙁 I’ve done a lot of minor interface tweaks that I think will be pretty useful but then I wanted to do one major interface change – have categories be listed under different remote blogs. The reasoning is simple. Currently, the way the category system is set up, all categories are displayed as one big list. The problem is, that when you have multiple remote blogs, after a while it becomes difficult to figure out which category belongs to which remote blog. So I thought that I’d change the category display to a treeview where each branch represented a different remote blog. Unfortunately, things weren’t that simple :p

Or rather, the implementation is indeed that simple but getting there is not. And it has nothing to do with categories. Instead, it has everything to do with version number changes. Blog, as you might know, has been stuck in 8.0 beta for a few years now. I work on it when I can but a version number is bumped up only when a database change is required. At the moment, there are a couple of database changes that need to be made – one to add a table to hold the relationship between remote blogs and categories and another to remove the More column from the entries table because I decided to have the full entry in one database column. It is the second change which is posing a problem.

In order to combine the More column and the Entry column, I was going to use a comment, <!–more–>, which would replace the standard <$BlogMore> tag. This was supposed to be an easy and quick workaround since WordPress at least supports the <!–more–> tag out of the box and treats it as an entry break. However, it turns out that the HTMLEdit component has a strange bug where it strips out HTML comment tags on a second save. Not the first save mind you, but the second save. The comment tag gets saved to the DB fine on the first save and is loaded from the DB fine after that but if you save the entry again, the tag disappears without a trace.

I asked over at the HTMLEdit component author’s newsgroups about this but soon after, his whole site became inaccessible :p So now I’ve been contemplating wading into his source code to see what is going on but it looks like a major job and I’m not certain that I really want to get into it. But I might have no other choice …

Coding again …

I started working on Blog again. Despite all the research that I did into other blogging tools which supported the MetaWeblog API, I just couldn’t find an app which worked exactly the way my rather picky tastes dictate :p Sure, I found a couple which came close but nothing was quite there. If everything else was there, then live spellchecking wasn’t there and I do like knowing if I make a spelling mistake instead of having to check on words over and over to make sure that I wasn’t imagining things 🙂

So back to Blog it was. The first issue I faced was, the question of how to implement the UI so that somebody could select either to publish a post or to save as draft easily. The current menuing system I was using wasn’t very good at displaying toolbar buttons with multiple options :p I had originally given up on the standard Delphi components because I had run into issues with setting up custom menus/toolbars buttons on the fly but by now I had forgotten what the exact issue had been. So I decided to give the standard Delphi components one more try. I ditched the custom menuing system and went back to Delphi components. I’ll know soon enough how that goes …

I did manage to get the posting as draft to a MetaWeblog system working soon after that. All it needed was changing a few lines of code 🙂 I then wanted to get uploading of images via the MetaWeblog API system working. While this involved fairly few code changes to implement, detecting new images and ensuring duplicates are not uploaded proved to be a lot more difficult. So I put that on hold for the moment in favour of another objective – getting custom tag support working with the WYSIWYG editor. Still working on that one …

Writing and rewards

I found myself wondering today if writing is as rewarding as coding is 🙂 I do both but the thing is that recently, I have not had enough time and so I have to split my time between the two. I would work on one app, then write a story and then go back to another app and so on. However, the two do not come with equal rewards.

I code for the challenge. I like solving a problem and finding a way to do something that I had to do manually or finding a way to do something that I couldn’t do before. Most of my software applications were developed out of personal need. Of course, once I released the application, others have found it useful and have suggested features that I have added. Blog is probably the best illustration of this behaviour. Over the years, it has grown to have so many features that I don’t even use. But I like knowing that other people find it useful. However, I guess in the long run, I still develop Blog because I personally find it useful.

There’s this other app of mine Amanuensis, a writer’s editor, that I developed sometime back. I found it extremely useful in writing my first novel. Once I completed my first novel, I put Amanuensis development aside because I took the whole novel into Word and have been using Word since then to edit the novel and to write the several short stories that I have written since then. However, suddenly I find myself wanting to go back to Amanuensis and to finish the app and to add a few new features that I want it to have. Of course, to do that, I’d have to take time off from writing.

Now writing, that’s a lot more personal. I don’t think I write for enjoyment. I come up with the stories because it amuses me, true. But actually putting the story down on paper, stringing the words together – that’s work. So I write mostly because I want to publish and so, unlike with coding, there is no sense of achievement once I complete a story. The sense of achievement will be there when I actually get a story published. But before that, I have to go through the agony of waiting for a story to actually get through the submission process, the dejection of having a story rejected, the innumerable days of writing a new story, the weeks of revising the story and then going through the submission process all over again. I still find myself wondering if it’s really worth it … And I guess I will never know since publication is sort of an elusive dream :p

Sure, I’ve been published hundreds of times here in Sri Lanka and based on my publication history, I could probably get published here again without too much trouble. But now I want to be published outside Sri Lanka but that’s like starting all over again since nobody outside Sri Lanka really cares about your publication history in Sri Lanka :p But what happens once I get published once outside of Sri Lanka? I’ll want to get published again. And I’ll probably want to be published in different countries. And then I’ll want novels as well as short stories published. So it’s a never ending process with ever changing goals. Is it worth all that mental anguish? I really don’t know …

Time to write again …

I’ve gone through all three of my currently active software projects and done releases for all of them, I’ve also got a software project on back burner which I won’t be able to totally test till Office 2007 comes out :p (No, it’s not Office 2007 specific but it’s a Word add-in and I might as well make sure that it works with the latest release of Office since it’s just around the corner – at least the public beta … or so they say :p)

Be that as may be, I think I’ve done enough coding for a while. Now it’s time to get back to the writing. The problem, as always, is lethargy. I seem to need to build momentum to get going on any project. I procrastinated on the next release of Blog for the longest time because I just couldn’t build up the enthusiasm to get started on coding again. Once I got going, I couldn’t stop :p Now that I’m in full-blown coding mode, I find myself unable to put aside coding and go back to writing. But I must. Otherwise, the writing will just atrophy till I get back into it months from now and I don’t want that to happen.

The issue is always getting started. Once things get going – it’s all downhill from that point on. I know what I want to write next. I have a story (or at least a world) in my head. This one is rather a vibrant world as well – I keep getting images about the world, the society that I want to write about. But I keep putting it off, thinking that I should wait for word on the last short story that I submitted or that I should try to get some more submissions going or something. If I would actually do something writing related – even as simple as submitting stuff to agents or publishers – I think it would be fine. It’s this sitting around and waiting that I just can’t abide …

To Analyze and Categorize

Now that I’ve got Blog categories working fine via WordPress (or vice versa :p) and the Heat Map plugin is doing an excellent job of displaying a tag cloud (or at least simulating one), I decided that it was time to do one more thing – go through all my old entries and re-categorize them to reflect the actual subject matter.

Since none of my old entries had categories assigned, it was easy enough. All I had to do was make the default category (General) count zero :p Of course, in order to actually do this, I had to go read all of my old entries, find the categories that they belonged to, create the categories if they weren’t there and then add the categories to the entries. This meant that I basically had to do a lot of reading of my own old entries.

I did learn a few interesting stuff about how I used to do things :p It looks as if I wasn’t as compulsive about entries in the past as I am now. But it also looks as if I was a lot less controlled in what I wrote. There were a lot more introspective entries where I would just talk out loud to clarify my thoughts and there certainly was a lot more about what was going on with me on a day to day basis. And there was a lot more soapboxing too :p

It looks as if I have gotten too busy (or a bit more reserved) nowadays. I hardly ever talk about stuff going on in real life any more and I don’t do that much of reflective writing (at least not the soul searching variety). Even my comments on movies used to be much more reflective whereas now, it’s more descriptive but not does not go into so much detail about the ideas behind the movie. Not sure if the changes are a good thing but they certainly are there 🙂

Of course, now that categories have become that much more important, I have started thinking about how Blog can be improved to utilize categories. One idea is to have a way to navigate entries not just by date but also by category. I’m not exactly sure how to implement this yet – perhaps a tab panel where the calendar currently is so that you can switch between calendar and category views? – but I do want to look into a method to do this. The other thing is to have a hierarchical category view so that you can have a bunch of categories under another category – for instance, movies, books, television and music under entertainment. WordPress supports such a hierarchy but it does not affect how things work beyond the categorization itself. In Blog, I simply want it for ease of organization and for ease of access to different categories. Again not quite sure how to implement it (without breaking any existing functionality that is :p)