Tags, categories and changes

After getting the latest builds of Blog to work, I have decided to remove Ultimate Tag Warrior from the mix here on SM. This is not due to any issues with UTW itself – it’s a great plugin and does the job well – but due to me realizing that UTW was overkill and that for what I was doing, it introduced overhead that wasn’t needed 🙂

Basically, I post all my entries from Blog. I never use the WP admin interface much except to remove spam. I do all my editing, posting and other entry management stuff via Blog itself. The only reason I was looking for a tag cloud mechanism was so that I could show a visual representation of the various topics covered on this site and let somebody interested drill down easily to only the posts that interested them. UTW allows you to do all that but in my case, I was simply converting categories to tags to do this. Since the categories are a built-in feature of WordPress, it occurred to me that I didn’t need the overhead of UTW (which would slow down XMLRPC to an extent) if I had a plugin which displayed a tag cloud based on my categories.

Enter the WordPress Heat Map plugin which does exactly that 🙂 After a brief search of the Net, I actually found a couple of different alternatives. The other one was the Weighted Categories plugin, which apparently inspired the author of the Heat Map Plugin. However, the Heat Map plugin appeared to be better maintained and supported and so, I went with that :p Incidentally, the author of the Weighted Categories plugin is apparently working on a way to make his plugin work with keywords instead of categories. I would be interested in that if it didn’t create too much overhead and worked without any issues with XMLRPC-based posting. But I doubt it … There is another plugin which supports keywords – the Keywords plugin, and the author is apparently working on tag cloud kind of support. Unfortunately, it works basically the same as UTW (just not as well :p) and so, I don’t see the point to pursuing that one either :p

Basically, what I’d like from a keyword tagging plugin for WP, is for the plugin to examine each new post, identify the keywords automatically and then add them as tags. Of course, at the moment, that appears to be too much to ask for :p The biggest problem appears to be identifying the keywords. Yahoo Content Analysis doesn’t do too great a job of it as I’ve mentioned before and while Tagyu does do a much better job, it is abominably slow and tends to end up in my XMLRCP session timing out before the analysis is done :p The best solution would be something that the plugin can do itself instead of contacting an external server but the implementation is the key ….

Midnight fires in the coding forges …

It’s more Blog news again I’m afraid since I haven’t been doing much else :p (Incidentally, I really should get the Blog page re-designed so that it is more consistent with the look of the new site … you know how it goes, so many things to do ..) Anyway, getting back to Blog coding, after doing some extensive coding and light testing, the category feature in Blog works fully and it even publishes via XMLRPC to WordPress :p I was really happy to get that bit working finally since I’ve been working non-stop at that functionality since I got back to working on Blog.

Now that that is done, I have a few more things to fix up in Blog before I can do a release. The internal release versions have been jumping around quite a bit since the last beta release due to the fact that I’ve been sending private builds to a few people who asked 🙂 I need to keep the newer builds compatible with the older builds and so I’m now up to 8.0 beta 5 at the moment. I’m hoping that I can do a public release of beta 5 but I do need to get a few more things fixed up first.

One thing that I have fixed since the category stuff was done was the find feature. It has been broken since at least beta 3 and I finally got around to fixing it :p At the same time, I want to add the option to be able to search the current entry for some given text as well as searching the current blog and all blogs under Blog. I even managed to fix up the search results dialog so that it shows the new WYSIWYG view as well as the old HTML view. However, I just discovered that the HTMLEdit component that I use might not be able to do search and replace of text. This is going to be a bit of an issue with the find feature and the new replace feature that I intend to implement … bummer 🙁 Hopefully, I can find a workaround … There are also the pages of bug reports on the 8.0 betas at the forums that I need to attend to … Work, work, work :p

Mad coding ….

I’ve been working on Blog every spare moment I’ve got :p Most of the coding yesterday was on fixing the categories. The category display in Blog is based on a new Delphi component that I created – it’s a (sort of) data aware check list component which displays values from a database table as choices for you to select. Most data aware components are basically tied to a specific field in a data table. This one is only sort of data aware since it retrieves the information for the list from a table but doesn’t actually store the results in a data column :p Instead, you have to both supply it with the initially checked values and then later retrieve the final check list and store it manually yourself. I don’t know if anybody else has ever needed a component like this but I’ve come across this specific need several times so far. Hence, the creation of the component.

However, since it’s my own component, it had a few bugs left. Plus, I’d done it as a quick-and-dirty component a long time ago and had not gone back to it. So, I spent yesterday tightening up the data-aware-ness of the component a bit more. The category list was supposed to be editable in-place (kind of like WordPress’ Ajax aware categories but I had this in place a long time before that :p). But the editing in-place wasn’t working properly because the component itself wasn’t totally data-aware. So after much searching, debugging and fixing, I was able to get that all working. In the process, I also managed to stumble upon a few optimizations which seems to have made Blog itself load much faster :p All in all, it was good day of coding work though the result was that I went to bed much later than usual and boy am I tired today 🙂 I have to still do all the work related to handling remote categories and posting category selection via XMLRPC to a remote blog but I’m getting there and the goal is ever closer …

Coding again …

Well, some of the internal Blog changes are done 🙂 I finally added a data module to the Blog code, transferred over all the data tables scattered over different forms on to the data module and cleaned up the code to work with the data module. Then of course, there was all the work involved in getting the code to compile again. It is finally done and in the process I fixed another issue – the internal Blog categories were not showing up properly and this is something which has been annoying Laurie :p Turns out that the categories were getting assigned fine but were not displaying due to an issue with the sequence of events that were being carried out.

Now that I’ve got the code working again, I plan to redo the entire Journal Management dialog. Instead of the current single dialog with multiple tabs, I’m going to go for four different dialogs which will allow more flexibility in terms of how each area (servers, blogs, templates and uploads) works. This is mostly to accommodate all the new category management stuff which is going to have to come in for the new release. I’m going to have to add category lists for each remote blog so that each installation of WP (or whatever supported blog) is kept distinct from the internal Blog categories. I will also have to add code to merge in the new remote blog categories with existing Blog categories and keep track of what gets assigned where. Ah, joy :p

On other coding fronts, I still have that add-on plugin I started work on. I still want to finalize the feature where it will automatically create tags for all posts without tags even though I will not use the tag for the main purpose it was created 🙂 So, I don’t know which one I will want to work on today – Blog or the plugin but hopefully, both will get worked on …

Tag, you’re it!

We were out the whole day yesterday and so I didn’t get to make my usual blog entry. Because of this, I now have two blog entries queued. Yes, I could make them both today but I’m OCD-ness just wouldn’t permit me to do that – so here’s yesterday’s entry and today’s entry will appear tomorrow :p

I put sprucing up Blog on hold for a bit to do some investigating day before yesterday. I wanted to see how good Yahoo Content Analysis Web Services was when it came to getting keywords for a blog entry. I didn’t want to write any code of my own and so I settled on the Terms2posts plugin from Semiologic instead. Unfortunately, the plugin wouldn’t work out of the box and I had to do some debugging to first find out what was wrong. I did find what was going on and managed to get the plugin working but was not satisfied with the keywords that Yahoo returned – it had stuff like "unfortunately" and "ugh" (I say "ugh" a lot :p) and that just didn’t look appealing to me 🙂

So I turned my attention towards Tagyu. They have an API that developers can use to get suggested tags for a given post and I wanted to see how well that works. I quickly hacked the Terms2post plugin and added a function into it that would fetch posts from Tagyu by using some code that the Ultimate Tag Warrior plugin (UTW) used. This seemed to be just my day for non-working code because that bit of code wouldn’t work either :p I went through some debugging and discovered why tagyu tag fetching didn’t work, got it working and finally got some suggestions for my test post. I was actually happy with what I got this time. So I decided that Tagyu was the way to go.

Now came the difficult part. I needed a way to automatically tag posts in WordPress when they were published because I wouldn’t have categories working via XMLRPC for a while longer and I’m not known to be a patient man :p So I needed tagging working easily now, damn it! I decided to write a new WP plugin which would be an add-on to UTW and would simply get suggested tags for any new post and add the suggestions automatically to the UTW tables, giving me an automated tagger 🙂 After a couple of hours of tinkering with code, I actually had the plugin working – at least the base functionality. I’ve also added an admin menu which currently does not have any options (the options are there the admin screen just does not display or save any of it – they are hardcoded at the moment :p). The admin menu also has a button to populate all posts without tags in the blog with tags automatically but I haven’t finished this functionality either. I first want to check out the base functionality in a production environment and then I’ll get the rest working. So here goes ….

Back to Blog

Boy, this whole tag cloud thing is conjuring up a cloud of its own :p Yesterday, I got the Ultimate Tag Warrior plugin but it turns out that I have to specify the tags for the plugin in WP for it to work 🙁 Sure, as Lorelle pointed out in the comments yesterday, I can always set the plugin to use categories as tags and that should solve the issue right? Except that I post to my WP installation via Blog and Blog currently has no support for categories 🙁

OK, that’s not entirely true. Blog has some great category support (though Laurie will tell you it is broken in the current betas :p) but those categories don’t transfer over when publishing to WP via XMLRPC. So, I decided to get back to Blog and fix that. Of course, me being me, I can’t work on one particular issue only – I had to go redo a whole lot of other things :p There was one outstanding issue from the early days of Blog – how I handled database tables internally. The way I was doing it was not quite efficient and rather clumsy because that’s how I had started all that time ago in the early days of Blog. Blog was my first Delphi app and it shows. However, I have not fixed this all this time since Blog has gotten increasingly more complicated over the years and fixing the way data tables were linked was a rather cumbersome task.

I began fixing that yesterday :p Yeah, I’m a glutton for punishment. I also added in support via the XMLRPC interface for working with categories. That part at least works now but I realized then that due to the way Blog works, categories were going to be a major pain! Blog can publish to multiple destinations and this means that conceivably, somebody can publish the same blog to two different WP installations. This would mean that each installation would have its own list of categories and Blog would have a third list of categories internally. How do you keep all these different categories distinct. How do you make sure that Blog marks the correct categories on each WP installation? This is going to require a whole new set of tables and logic 🙂 Ah well, the joys of coding …

Can I tag along?

Yesterday’s serendipitous discovery of TagCloud has led to a few other developments since then :p I realized after watching my tag cloud for a while yesterday that it wasn’t changing at all – it was simply displaying the tags for wordsmith.org and nothing at all from my own blog entries. So I decided to do some digging to see what was going on. I still am not sure what exactly is wrong with TagCloud because I had assumed that the Yahoo Content Analysis service basically went through your site and figured out what the relevant tags for your entries were instead of you having to set up tags for each entry. However, nothing seemed to be happening – maybe their indexing service was slow or maybe it wasn’t functional. All I know is that even after 24 hours, my site was not displaying any tags.

So, I decided to look for alternatives tailored specifically towars WordPress 🙂 Or more accurately, I wanted to find something for WP which would generate tags in case I was wrong about TagCloud and they actually needed the tags somehow explicitly specified in the RSS feed. That was when I came across this blog post – now here was a solution (and a total solution at that :p) to what I’d been looking for. So I went over to the Ultimate Tag Warrior page, downloaded the plugin and installed it. I removed my existing TagCloud entry on my blog and added my own personalized tag cloud for my site. But nothing still showed up!

Why not? Because none of my entries had existing tags :p So now I had to find a way to add the tags to get a tag cloud. I spent a little time doing the entries by hand and trying to figure out what tags to assign before giving it up as too time consuming. The Ultimate Tag Warrior plugin does have a button which is supposed to retrieve the suggested tags for each entry from Tagyu but for some reason, the button wouldn’t work – at least at my end. I decided to go through the source and see what was happening and if necessary, write a plugin of my own which would connect to Tagyu and pull suggested tags for each entry at the time of publishing. Perhaps I could adapt the system to also go through all my existing entries and add tags …. So I have a new project to work on … Not that I really wanted one but oh well, need something to keep life interesting, right? :p

Coding, Testing, Writing – Life goes on …

I finally finished editing my new short story, "The Wyrm’s Lair" and sent it out. I haven’t really tried submitting short stories anywhere before (heck, I only started submitting my novel less than a year ago :p) and so am not sure how it will go – keep good thoughts in mind for it 🙂 Of course, I’m not totally accurate in saying I haven’t submitted any short stories before – I have submitted two stories before. One, a long time ago to an e-mag and they didn’t like it and the other one was just before I started work on the short story that I sent out today. The second one was rejected as well but I’m not totally surprised about that one since that didn’t have too much action – it was more of a character/mood piece. This one, I really like and it sort of came together much better even than the first draft once I’d edited it. However, it might not be everybody’s cup of tea :p So now that that’s done, I have to consider what I start working on next. Probably editing "Honest" …

I’m also beta testing AstroSynthesis. AS is probably one of the best space mapping software out there 🙂 I love it because it helps me to map out the universe that I’m writing about. It fleshes things out and helps me keep track of what world is where and how far away each world is and so on. Now if I could take it a step further and I could also map out each planet as far as the land masses and stuff goes, it would be perfect. Actually AS has a sister product, Fractal Mapper which does help you do this but it takes quite a bit of work to create a good looking Fractal Mapper map and I just don’t have the patience :p Besides, I already have enough projects without taking any more on. But back to topic, I am beta testing AS 2.0 and I was really thrilled to be picked to beta test since besides my own apps, I haven’t beta tested anything much. Of course, what with all the other stuff going on, I haven’t been able to do as thorough a beta test as I would have liked and I feel a bit guilty about this since the developer is sort of getting cheated by having me beta test. Hopefully, I can make up for it in the future beta releases.

Now that WriteTrack is out, I have started work on another project – a Word add-in to highlight often used (and misused) words in a document. I’ve already written most of the code for this as a macro and so the logic isn’t too difficult but there is a UI to the add-in and that’s where I’m struggling at the moment. I hope to have the add-in done within a week and then I hope to keep adding more functionality to it as I go on, creating a suite of little utilities that will be helpful to writers. That’s something I see a great need for with Word – it’s not too shabby a word processor but most of the available tools are just as overpriced as Word is :p We need more free stuff so that us poor writers can actually afford to have them 🙂

Releases, bugs and other stuff ….

I released the next beta of WriteTrack yesterday. I’d been working to get the features sorted out for the last couple of days and I was finally satisfied enough with it to do a release. There was one major feature outstanding when I did the last beta – the ability to track income and expenses on your writing projects. I got that sorted out and I also fixed a lot of minor usability issues since I’ve been actively using it during the last couple of days to transfer over all my old tracked submissions from the Excel spreadsheet I was using into WriteTrack. It’s only when you use something that you figure out what doesn’t work and what can be improved 🙂 I still want a few minor features added to the app but I think it is more or less ready for a 1.0 release as long as nobody finds any major bugs in this beta.

Speaking of bugs, there has been a bug which has been irritating me in a major way and unfortunately, it’s not something that I can fix. The bug is with my banking system :p My bank is HSBC and they are probably one of the bigger banks in Sri Lanka, what with being an international bank and all. They have an Internet banking service where I can check balances, do transfers, pay bills etc and I find this very useful since I don’t leave the house much. I usually pay my rent via the Internet banking system but at the end of last month when I tried to make my usual transfer (the same transfer that I’d done for six months before) I was presented with an error message.

I call their customer support service and a Customer Support Rep (CSR) takes down the details of the issue and promises that they’ll call back the next day to let me know the resolution. The next day comes and goes, no call. I try the system again, it still doesn’t work. So I call them again. The same CSR takes the call, says that she’ll talk to the relevant people and they’ll get back to me and the next day comes, nothing again and the system still does not work. I call again. A different CSR this time and she says that the Internet banking department looked at it, that the problem wasn’t in their department and that it has been escalated to networking and that they’d "definitely" get back to me the next day.

We were out most of the day the next day but I didn’t get a call on my mobile nor were there any messages on the answering machine when we got back. It was too late to call by then and it being Friday, I couldn’t do anything over the weekend either. I call on Monday and get another CSR, I tell her the issue and tell her that I had not heard anything from them after repeated promises of calls. She escalates it to a "senior colleague" who is courteous and says she understands and that the matter will be dealt with and that I’d hear from them before 5 o’clock. True to her word, she does call back around 3 o’clock and says that it looks like a technical issue at their end and that it has been escalated to IT and that I’ll hear from them by the next day, which was yesterday.

End of day yesterday, still nothing from HSBC. I try the system again in the hope that they fixed it but didn’t call me. It still doesn’t work. So I call their support line and ask for a manager. I get to a manager explain the situation to her and tell her that I find this rather unacceptable. She makes soothing noises, says that she understands and that I’ll hear back from them the next day – today. I’m hoping that they’ll prove me wrong and fix the issue and call me to say that they did it. But given the track record so far, I’m not taking any bets on it.

I do have two questions though – is it so hard to keep a customer informed if you are taking a long time to resolve the issue? I work in customer support, I know it’s not easy for the CSRs to do this and I don’t expect them to. But if the issue is escalated, shouldn’t whoever it was escalated to take the time to let the customer know? My second question is, what the heck is their IT department doing? It shouldn’t take them over a week to fix a simple issue like this, should it? Come on, even without knowing anything about their system, I think I can fix it within a day :p Ah well, such is life in Sri Lanka …

And to summarize ….

This week’s been rather eventful … and it’s only beginning :p Actually, I guess I’m talking about last week. But given the fact that I only have a tenuous concept of time at any given moment, I might actually be talking about Sunday-after-next and still might not really have an inkling about what I’m really talking about :p Incidentally, I’m one of those people who believes that the week begins on Monday and so, I guess this is a new week. The past week had a lot of things going on and so I thought I’d do a recap here.

I finished my latest short story … either yesterday or day before. Strangely enough, it actually was fun writing it after I got going and wasn’t as hard as I thought it would be :p I kind of like how the world I built in that story turned out but not sure if others will think so. I want to try and submit this story to a couple of places but first, it needs a little bit more polishing. Laurie’s going through it first (and I need a bit of a break from it) and then I will go back to working on it.

I also started work on WriteTrack again. This was actually due to something else I finished this week – a new query letter. I submitted my query letter over at Absolute Write Water Cooler – a good forum for writers BTW – and they (or actually one really helpful person :p) tore it apart and re-built it from the ground up. I like how the new query letter turned out and couldn’t wait to get querying again :p So, I sent out a couple of queries using the new query letter and that meant that I had to track the new queries using WriteTrack :p I got to work originally to fix a few minor annoyances but then it turned into more serious work where I actually added in the last of the missing functionality in WriteTrack, fixed a few bugs, added suggested features by beta-testers and so on. I still have a few minor UI issues to sort out but once that’s done, a new build should be ready to go out … maybe today 🙂