Still working on the web-update component with frequent forays into side jobs :p I thought of a few enhancements for the data mining app I wrote for my friend (I call it Harvester) and implemented that today. I would like to go back to that app at a later time and try to come up with some form of a template based approach which would let somebody choose a web page, define the values they want to extract by highlighting and naming them and have the program extract those values from any page which fits that particular template. Sounds easy enough but it would be a heck of a lot harder to implement than you’d imagine because of the uncertainty factor – the web page layout can change at any time, the page might be dynamically generated and might not have had some information which might appear on another page of the same category etc. But, it is an interesting concept and something that would tie in with quite a few of my other projects.

The only work I was able to do on the web-update component was to test it further. I discovered a few more bugs at the file download stage and fixed those but I still haven’t gotten to a stage where all the selected components are successfully downloaded. The only thing I can be happy about is that I used the kind of test data which does create problems and has helped me identify a few bugs and logical errors which otherwise might not have been identified till much later :p

I didn’t have too much time for coding today but I did go back to the web-update component. I had discovered that the demo for the web-update component had suddenly started a mysterious IDE crash each time I tried to close the demo project. I thought this was due to a new grid component that I had introduced into the demo project to display the update information for all the new components. I tried fixing the problem for a while before I realized that the crash was caused not by the grid component but by the web-update component itself :p So I had to hunt through the code to see what I had changed that would cause the crash – I finally discovered that I had mistakenly moved up the call to the Free method of a string list before some code that used that string list in the destructor for the web-update component. Once I fixed that, it worked fine again 🙂

While I did manage to get the demo finally compiled and running with the new changes to the update component that supports individual component updates, I found that it still didn’t work exactly right. I fixed a few bugs and code glitches which was stopping it from even downloading and parsing the update info file but that’s as far as I got yesterday. So I still need to make sure that the parsed information is displayed properly and that user input is received as to which components are to be downloaded before only those components are downloaded and installed. If all of that works, then everything is set and I can go back to Blog 8.0 and incorporate the web-update component into it.

I did a little bit of work on the web-update component but didn’t get around to testing it since I got involved in another project :p A friend of mine wanted a data-mining application written that would allow him to select some information on a web page and then have the program identify the different elements in the selection and populate a CSV file with the information so that it can probably be imported into a database. Now, I wasn’t planning to write a customizable data-miner where you can specify how each field was to be derived – sort of like Snipper if you recall. Rather, I was going to write a custom data-miner which would work for this particular web page … but I am toying around with the idea of a customizable utility in the back of my mind … just so you know :p

I decided to go with Visual Studio .NET and C# for the app since I’d done all of my previous HTML parsing stuff in Delphi and had always run into one particular problem – being able to parse the HTML in a selection as HTML tags and attributes. I thought this was due to a limitation in Delphi’s support for some of the MS HTML handling interfaces and so decided to go with C# where MS would have made sure that it supported the latest :p The UI itself came together pretty fast but I still prefer Delphi’s RAD IDE – MS just hasn’t learnt enough from their competition. While the Visual Studio .NET IDE is pretty good and makes UI development much faster, they don’t have (or at least *I* couldn’t find ..) some of the elements that make UI development such a pleasure with Delphi – simple things like being able to link the menu component to the statusbar component so that menu hints are displayed on the statusbar … little things like that. Plus, C#’s toolbar stuff is abysmal – you can’t write event code for each toolbar button, you just have to figure out which button was clicked from a generic toolbar button clicked event and then carry out the specific action for that particular button. I hated that … but I digress …

I managed to get the UI going pretty fast and then came the actual data extraction stuff. The UI had a web browser component where the user selected the data that they wanted to extract and the identification of the selected portion on the web browser view was simplicity itself since I’d done it many times before – all I had to to do was get an IHTMLDocument2 interface for the browser component and then get the selection property of the interface to get an IHTMLSelectionObject interface. Now came the tough part, actually parsing the stuff. I’d assumed before (not so sure that’s how it should be done now though …) that if I was able to create a control range using the createRange method for the selection, that I’d be able to get a list which would have the HTML tags and their attributes neatly separated. This was what I’d been trying to do in Delphi a couple of times before but always ended up failing. I thought it would work in C# but what do you know? I failed again :p So, I’m beginning to think that maybe it’s my approach which is flawed.

Anyway, since that approach didn’t work and a few others I tried didn’t either, I decided to go back to manually parsing the selected HTML. I did find something that I had not known before – that the IHTMLTxtRange interface created by createRange had both an HTMLText property as well as a Text property. I’d always been using the Text property before but now I used the HTMLText property since I had found that most of the information I needed was in table cells – so all I had to do was isolate the table cell information and work with that. I wrote a new function to isolate just the text in table cells and then the rest was just a matter of coding … It all works fine now – at least I think so but I’ll know for sure once my friend has had a go at the final app …

The magic of movies …

It’s been a while since I made an entry here but once again, not due to a lack of material to write about but mostly due to a lack of time to do so. Since I try to write about my development work every day in the morning before I set out for work, I find that I’ve usually run out of time by the time I get done with that and need to rush to work. I usually vow to write once I’m at work and have a spare moment but that spare moment never seems to come by since I get pulled into one thing or another and am on the go constantly till I leave work in the evening. But enough of that and on to what I wanted to talk about …

I’ve been watching quite a few movies over the weekends and while I’ve been catching up on some of the movies from last summer that I missed, last weekend I decided to go on a trip down memory lane and watch some of the older movies in my collection that I had not seen due to some reason or other. So it came to be that quite by accident (or maybe not) that I went through a blood-and-gore fest to rival any butcher’s shop on the corner – but I forget that they don’t have those anymore :p I watched "Fair Game" with William Baldwin and Cindy Crawford, "Get Carter" with Sylsvester Stallone and Michael Caine, "L.A. Confidential" with Kevin Spacey, Russel Crowe and Guy Pierce and to top off the trip down memory lane, re-watched "The Big Hit" with Mark Wahlberg and Lou Diamond Phillips … and all this over just two days mind you, though my usual quota is more like four or five movies a day :p

Of what I watched, "Fair Game" was basic fluff – fun, exciting but nothing substantial to it. "Get Carter" was more solid and I really liked Sly’s acting in it but it was a bit predictable – maybe because I knew Michael Caine had played Sly’s role originally and so the story had to turn the way it did at the end … I will say no more in case there are people out there who still have not seen it and want to <g> "L.A. Confidential" was great in that the characters were all *real* in the sense that they weren’t *heroes* who took the moral high ground but rather ordinary people acting as people normally do – lots of shades of grey. It did take me aback quite a few times to see a character do something totally against what I’d want him to do but I could understand it as being human rather than heroic. However, the plot twists were a bit predictable again but I was struck by the fact that all the main leads were mostly unknown (at least to me) at the time the movie originally came out. I think I first noticed Guy Pierce when "Memento" came out and Russel Crowe when "Gladiator" was released .. I’ve no idea when I first noticed Spacey – maybe "The Negotiator" but I certainly did not remember him from "The Usual Suspects" but that was because I wasn’t watching the names of actors then :p

I’d already seen "The Big Hit" – on the big screen at that – but I wanted to watch that again since it seemed to fit right in with the other movies that I’d been watching. When I first watched it, it had been a Friday evening and I’d been stressed from work and I found the movie to be darkly humorous and laughed a lot but I didn’t find it to be that funny this time – I still smiled at a few things, had a chuckle or two and still enjoyed the action sequences but what struck me most was how pathetic Mark Wahlberg’s characters was – how he was being used by everybody that he thought of as friends or lovers (but then again his job and the fact that he has a fiancee and a girl friend on the side is out of character with his whole I-don’t-want-anybody-to-hate-me attitude) and kept on telling him "you idiot! wake up and smell the roses!" :p I guess if you can get that immersed in a movie, it’s still a good movie :p Anyway, I was again struck by the fact that "The Big Hit" had a major Indian/Hong Kong feel to it but that probably isn’t surprising since I believe the director Che-Kirk Wong comes from Hong Kong and this was his first Hollywood movie 🙂

I was able to figure out the solutions to all the minor problems I had with the web-updater component yesterday and the code is finally done, but not tested yet :p I even added in the code to handle optional components that the user can decide not to update at the current moment though I have no such components yet. As far as I can tell, the code came together really well but since I am still putting together a demo application to test the functionality, I don’t know for sure that it works the way I want it to and that it works smoothly. I’ve already started work on a demo application though and should be able to see if it works today. In the meantime, I’ve also thought of a few problem areas and improvements overnight and that too can be incorporated into the component.

I still have no idea how much overhead that the web-updater component will add to Blog and while everybody says that download sizes of a few megabytes are no problem, I still seem to be stuck in the early dial-up days since I feel guilty at each additional kilobyte :p Of course, some of the UI changes I have in mind will add a lot more than a few kilobytes (especially the replacement of all the data grids with something a lot better looking and more functional) but I’m hoping the the web-update component will over set that eventually by the fact that you don’t have to download the full installer each time you want to update but instead can download just the bit(s) that you need. Let’s see how that goes as things progress though since the proof of the pudding is always in the eating :p

In my usual quirky and unpredictable style, I’ve put the CD cataloging application on hold and gone back to the web-updater component :p I tested its functionality yesterday and after some tweaks and bug fixes, was able to get it working with a single file download. Of course, I had to immediately start working on the source to handle multiple files and to implement a mechanism to assign versions to each different component and to check versioning on each component separately and download only those that have been updated. I’m still trying to get all the details into place for that. While the basic idea isn’t much more different than doing a single file, the actual implementation details are bogging me down – such as how do I send update information to the parent component since I can’t do a simple old version number and new version number deal as I would for a single component. I’d have to send a whole list containing the component name and the old version and the new version. Add to this the fact that I also want to go by release date since the same version component might be updated or fixed and re-released with a new release date (yes, I know I can up the version number here but there are instances where this is not practical due to my internal beta numbering system) and the fact that I also want to provide optional update components (like plug-ins) that the user can deselect from the download list, and the complexity grows ever more :p But I’ll work it out …

In the meantime, don’t expect Blog 8.0 to be out soon :p Even the first work-in-progress release containing the web update stuff should take a while since I’ll probably start working on a few other things (probably including the CD cataloger which I currently call DiskMan and maybe even a rebirth of PostMan due to certain other circumstances …) concurrently. BTW, I’ve heard from one user that he loses custom snippet toolbar buttons on restarting Blog. Now this was one of the main fixes I implemented for the 7.1 release and it seems to work just fine over at my end. So I’m not sure why it’s not working for him. Are custom snippet toolbar buttons working fine for everybody else? Or doesn’t anybody use custom snippet toolbar buttons anymore? :p

I’ve been kind of quiet this weekend since I was trying to stop myself from going into rant mode but I guess I’ve got to rant when I’ve got to rant – so here goes :p This guy had posted a review about Blog on BetaNews on the Blog page and he gave Blog as low a rating as he could because Blog had been identified by one of his apps as spy/nuisance ware. Now I was really mad at that time because I make all efforts to provide a good, bug free app and also try to provide good support – and all of this for free. So when my app gets called spyware, I have a tendency to go ballistic :p In hindsight, I realize I can’t blame the guy because he was going by his what his utility told him and for most users, I guess that would be enough – how can a computer program be wrong? :p I just wish he’d contacted me to clarify the matter instead of just slamming Blog.

Anyway, I tried out the utility he used – it’s called PestPatrol … sorry, don’t have a link since I’m at home and offline – and sure enough, it identifies the Blog directory as having malware that could install a Flash applet or something which displays an annoying message. The program it tags? The uninstall.exe from NullSoft! Now, I know tons of people use the NullSoft installer package and if that was malware, I’d think it would have been noticed y now. Plus, I’ve run both AdAware and SpyBot Search & Destroy on my machine many times and neither of them ever identified the uninstall.exe as malware. So I can only conclude that PestPatrol has a slightly faulty database or a faulty identification routine. I should probably write to them and point it out to them – perhaps when I’ve cooled down a bit :p

In other news, I haven’t really felt like testing out the web update component (I was lazy – or just too tired after all that coding) and so haven’t done much work on it since I got the code completed. But I’ve been working on something else. Yeah, another one of those ideas of mine :p Actually, the idea was already there but I just didn’t like the implementation <g>. There was this app called Visual CD which lets you store directory listings for your removable media as well as hard disk folders. It then lets you search these lists so that you can find where a specific file is without having to go through all your removable media one by one. I liked the app and the UI but I hated the fact that while Visual CD provided compressed archive file support so that you can browse an archive file’s contents if the removable media was in the drive, it would not store information about the contents of archives in its file list so that you can search for a file even inside the archives. I wrote to the developer and offered to help with the coding since s/he seemed not to have enough time to do so plus they’d mentioned that they used VB and Delphi for their apps and I was hoping that Visual CD would be done in Delphi. But I never heard back from the developer. So I decided that I would write my own version that had archive support as well :p

So far all I’ve done is to work out the file listing storage format and a method to compress the file listing file so that it would take less space. I still have to add support for adding compressed archive listings to the file listing set, to do the code so that I can access the contents of any folder or archive in the listings quickly to allows directory changes in the UI and then do the UI itself. It is another one of those interesting side projects which most people would have no use for but that I need badly since I have a lot of unlabeled CD’s that I have to sort through every time I need to find a specific file …