Peter's Blog

I'm looking at AideRSS an 'intelligent' RSS service. It has bang whizzo technology to rate articles in RSS feeds so you can choose to look at only the best. I just ran it on this blog (see here) and it thinks that of the last fifteen posts (i.e. an RSS bucketful) 26% are good, 26% are great and 26% are the best. I think these are all the same 26% (i.e. 4 posts) in each category and the ratings are a bit polarised, either 10/10 or 0/10 according to whether or not they have been cited on Bloglines (whatever that means). The results seem arbitrary, some wow posts are rated and not others.

Now I have more reading time on my hands I could do with a nice intelligent rss filtering service. For example, on the whole I like Engadget except for the endless new MP3 players and mobile phones they reel off. Likewise on the whole I like Slashdot which is where I discovered AideRSS but only maybe one in five stories there warrant my attention (e.g. software patents: not a must read for me).

Anyway, I'll try setting it up and seeing whether it filters on what it thinks I like or what everyone else seems to like. There is a difference.

Todo: look at bloglines, see how to cite my own posts.

Google personalised home page has got me back into Slashdot. I got bored with slashdot after using an RSS feed that only showed article headlines as teasers to the content. Ironically Google home page only shows headlines.

On a whim I tried logging into my old slashdot account for the first time in what must be four or five years and it remembered me!

I did some metamoderating (moderating the moderators) as it helps while away the time.

I have tried a few rss reading options for my ipaq. All these allow articles to be downloaded to the pda for reading offline when out of wifi range or with the wifi turned off to save the battery.

• avantgo: an online service that generates feeds suitable for display offline on a pda. It's free if you use less than 2M if data a day. You can chose from specially formatted content such as the guardian newspaper or any rss feed
• feederreader: an rss aggregater that supports enclosures i.e. it can download podcasts. I found this over complicated and fiddly. It is free although donations are encouraged.
• egress: a commercial rss reader but I have found it much better than the others and it isn't that expensive. It also supports enclosures.

I wanted an rss reader that supported enclosures so I could download podcasts directly to the pda where I can listen to them anywhere, no messing with synchronisation. I have even plugged the pda into my hifi with good results.

I first used feederreader for this but I finally abandoned it because it gave the files it downloaded meaningless names containing just numbers. Egress gives them the name the author gave them which better describe the contents. Ok I could launch the playback from within feederreader but I don't want to, I don't want to have too many apps open at once or the pda gets flaky

I then found Egress to be a pretty good way to read rss in itself: the buttons in the pda step through the articles nicely. I haven't bothered with avantgo since I started using egress. I'm still in the trial period but i'm sure I will buy it.

Bought myself a new PDA, an HP Ipaq rx1950 Pocketpc. It was a bit of an impulse buy, a bit of retail therapy I felt in need of a few weeks back. The model I bought was essentially the cheapest I could get locally that had integrated wifi.

I used to use a palm tungsten T2 but it's digitiser does not work properly, it takes about five goes to get past the initial calibration and it's downhill from there. I still had a hankering for a pda, a portable notetaking device to serve as a backup for my failing memory.

Following my experiences with the palm and bluetooth I decided that wifi was pretty much essential, bluetooth was slow and the bluetooth stacks in windows complex and buggy.

The rx1950 runs Windows Mobile 2005, the latest name for Windows CE/pocketpc. It is a stylus driven thing, not my first preference but keyboard models are more expensive and have smaller screens.

Bullet point review:

• first impression when I took it out the box was how light it is, much lighter than my old palm. It's fairly thin as well, maybe half an inch thick. It would fit better in a trouser pocket if not for the carry case that comes with it which is almost as thick again.
• nice solid build quality: HP after all.
• stylus has a tendency to fall out: I put it in the carrying case the wrong way round to restrain it or I would certainly lose it.
• no charging cradle: comes with a USB cable and a mains adapter that plugs into the USB cable: rather awkward and fiddly and annoying that it cannot charge from the USB port. There are aftermarket USB cables available that will charge it.
• synchronises with the pc using activesync and I was amazed to discover that it won't sync over the wifi. Apparently microsoft decided it was a security hole and they couldn't think of a way to plug it so they just removed the feature. This leaves USB or infra-red so I'm using USB when I feel the need. Many pocketpc apps are available as cab files that can be downloaded directly to the device over the wifi and installed. Some producers are not enlightened to this and ship apps with windows installers that use activesync and hence can only be installed at home base.
• the device has 32M of 'program memory' for running programs data and this isn't really enough. The symptoms of this seem to be the o/s terminating apps that are not in the foreground. If you have, say, windows media player playing something then you can only run maybe one more program before something gets randomly zapped. Microsoft have tried to create a paradigm where you don't have separate applications but flip between different modes without worrying about having to close apps down. Apps are closed automatically by the OS as memory runs low and should be designed to save their state such that when they are reopened they are in the same state as when they were shut down. Unfortunately it looks like developers use standard development techniques and applications being suddenly terminated by the OS leaves the user high and dry.
• it has another 32M for storing programs but I bought a 1G SD card and I put everything in that.
• the screen is just about big enough. It is ok for reading without scrolling too much. For most web browsing it is awful unless I use http://skweezercom or google mobile to strip out any fancy stuff.
• the character recognisers that come with it are not much good: the handwriting recognition (transcriber) is slow and inaccurate. I am using something called Tengo which is a bit like predictive text on a phone so involves hitting only six big buttons. It has some clever design features and there is something about it that is kinda fun. I am writing this review with it so you may spot predictive-text style wrong word errors.
• a frequently used feature is the reset button: windows mobile is a typical windows o/s and isn't sophisticated enough to offer robust task management. Applications can lock it hard and banging on an unresponsive plastic screen is particularly fruitless.
• it has speaker and microphone. The speaker is just about loud enough to listen to podcasts. I haven't tried skype on it, the skype site doesn't list the rx1950 as supported and the cpu may not be fast enough.
• has 3.5mm audio jack and the sound to my ears was pretty good. It is a good mp3 player.
• battery life is very good, one charge gives a good day of use.

As a device for browsing rss feeds while watching tv, taking notes, listening to music, watching videos or whatever it is just fine. It is more convenient than a laptop and I can carry it in my pocket so is far more mobile. I can see that it is a quirky platform and will probably have been killed off by mobile phones and mp3 players within two years time.

I'll write about the applications I have installed in it some other time.

I like using a web base aggregator so that I can access it from whereever I may be and it will remember my subscrptions and what articles I have already read. For some time I've been toying with the idea of installing an RSS aggregator on my dedicated server. This would be something to replace bloglines but which I could tweak to my liking. I haven't been in a big hurry to do this as bloglines has been ok thus far. However, the other day I was using it and I realised how slow it can be, waiting for it to download new articles. I thought this was an excellent reason to seek alternatives.

I had a look around for web based aggragators and found the wikipedia list of news aggregators and from this I found Gregarius. Installation was very straightforward:

• Create mysql database, user and password for it
• untar Gregarius into domain directory
• edit config file to tell it details of the database
• access it through web server.

It's pretty standard LAMP stuff.

It's nice to use, very configurable, much more so than bloglines. You can control when the feeds get aggregated (for now I plan to press the refresh button to manually trigger aggregation), how many articles per page, you can read as a 'river of news' (articles from different sources mixed togther by creation date) or read them grouped by feed. Other aggregators only support the 'river of news' format which I don't really like as I don't like to mentally context switch between unrelated topics between each posting: for example, first post is a dilbert cartoon, followed by bbc news about untimely deaths, followed by an engadget piece about a new mp3 player, it's all over the place. I prefer to go through my feeds in the order that is interesting to me at the time (cherry pick, Dilbert first normally).

Gregarius supports a few themes and plugins and so far these have fixed the minor quibbles I had. For example, I wanted to read postings in order of creation rather than newest first which means reading down the page. However, the 'mark all as read and get some more' button was at the top of the page some once I got to the bottom I had to scroll to the top to get more articles. Someone had already had this problem and there was a plugin available to put an extra button at the bottom of the page.

One little hack I had to do was with the 'lilina' theme which supports a button to collapse the left sidebar which will be useful on my Dell d410 with a 12" screen. However, this theme also collapses the articles down to titles by default which I did not want. I fixed this be editing the themes/lilina/item.php file as follows:

    <!--pcw <div class="content" id="c<?php echo rss_item_id(); ?>" style="display:none">-->
    <div class="content" id="c<?php echo rss_item_id(); ?>" style="display:block"

i.e. by default the article contents is display:block rather than display:none.

What I most miss from bloglines is being able to press 'j' to go to the next article.

What I dislike about Gregarius most is that it isn't spelt Gregarious.

ToDo: figure out how to get cron to prune old articles from the database. It can be done explicitly through the admin pages but I don't want more chores.

I had a play with googles online rss reader. It is very fancy but I don't think it would tempt me away from bloglines:

• You only see one article at a time. Bloglines shows you all the articles from a feed in one page which I think is better for spotting the interesting stuff. You can step through the articles with the j button in both bloglines and the google reader (coincidence? Group homage to vi?)
• I don't want to go through a mix of articles from all feeds in chronological order. I like to go through feed by feed. If I open the list of feeds in the google reader it occupies half the screen on my d410.
• It is a bit buggy: at one point the 'loading' splash thing got stuck on. But like most google software it is beta.
• It carefully strips the colour coding and indentation out of syntax highlighted code. Bloglines strips the indentation. I am not sure why they do this, is it the presentation police?

What I am looking for is the mysthical rss aggregator with bayesian filtering that will learn what interests you and only show that. I'd write one if there were only nine days in every week.

Conclusion: google see rss as the future and want to slap adverts all over my blatherings.

If my rss subscribers are getting a flood of seemingly duplicate postings it is because I decided to reformat my posts a tad to emphasise the awtags links below the node bodies. I edited the awtags source to change the word 'tags:' to the more informative 'Related Topics:' and I edited the awTags_TagLinks css style to delineate the links from the node body:

   1  .awTags_TagLinks {
   2      padding: 5px;
   3      margin: 20px 10px 10px 10px;
   4      border-top: 1px solid black;
   5  }
   6  
   7  .sticky .awTags_TagLinks {
   8      visibility: hidden;
   9      display: inline;
  10  }

This also hides awtags from sticky nodes which I use at the top of tag descriptions.

I decided to update my python podcast script ipydder.py. Based on the comments for the old version I ditched the horrible sax parser and used the lovely Beautiful Soup. This has simplified it a great deal, 49 lines of code if I delete all the comments.

   1  #!/usr/bin/python
   2  #
   3  # ipydder mark 2
   4  # Download podcasts
   5  #
   6  
   7  import BeautifulSoup
   8  import os
   9  
  10  def GetPodcast( strFeed, strTargetDir):
  11      #
  12      # Get the rss file.
  13      #
  14      strRSS = os.popen4( 'wget -q -O - "%s"' % strFeed)[1].read()
  15  
  16      #
  17      # Parsing it.
  18      #
  19      oSoup = BeautifulSoup.BeautifulStoneSoup()
  20      oSoup.feed( strRSS)
  21  
  22      #
  23      # Go through the items in the feed.
  24      #
  25      for oItem in oSoup( 'item'):
  26          #
  27          # Look for enclosures in the item.
  28          #
  29          strUrl = oItem.enclosure[ 'url']
  30          if not strUrl:
  31              #
  32              # If no enclosures, see if there is a link
  33              #
  34              strUrl = oItem.link.string
  35  
  36          if strUrl:
  37              #
  38              # Remove whitespace.
  39              #
  40              strUrl = strUrl.strip()
  41  
  42              #
  43              # Look for mp3 enclosures
  44              #
  45              if strUrl[-4:].lower() == '.mp3':
  46                  #
  47                  # Ok there is an mp3 file to be had
  48                  #
  49                  # Determine unique id for this item
  50                  #
  51                  strGuid = oItem.guid.string
  52                  if not strGuid:
  53                      #
  54                      # No guid, lets use the file url.
  55                      # Presumably each file name is unique.
  56                      #
  57                      strGuid = strUrl
  58  
  59                  #
  60                  # See if guid has already been downloaded by searching the database
  61                  #
  62                  strDBFile = strTargetDir + '/.ipydder.db'
  63  
  64                  try:
  65                      oDB = [ strLine.strip() for strLine in open( strDBFile).readlines()]
  66                  except(IOError):
  67                      oDB = []
  68  
  69                  if not strGuid in oDB:
  70                      #
  71                      # try to download the file
  72                      # Use wget as a more robust way to download big mp3 files
  73                      #
  74                      print 'Downloading %s' % strUrl
  75  
  76                      os.chdir( strTargetDir)
  77                      strResults = os.popen4( 'wget -nv "%s"' % strUrl)[1].read()
  78  
  79                      strFileName = strTargetDir + os.path.basename( strUrl)
  80                      print 'Downloaded file %s' % strFileName
  81                      print strResults
  82  
  83                      #
  84                      # Remember that the file has been processed, don't download it again.
  85                      #
  86                      oDB.append( strGuid)
  87                      open( strDBFile, 'wt').write( "\n".join( oDB))
  88  
  89  strHomeDir = os.environ['HOME'] + '/'
  90  
  91  GetPodcast( "http://radio.weblogs.com/0001014/categories/dailySourceCode/rss.xml",
  92                  strHomeDir + "Podcasts/DailySourceCode")
  93  GetPodcast( "http://www.morningcoffeenotes.com/rss.xml",
  94                  strHomeDir + "Podcasts/CoffeeNotes")

rss is a standard for publishing information such that it can be read using a news aggregator. Instead of you visiting a list of sites every day, the aggregator reads the articles from the sites and puts them in one place in a single stream of news.

I use bloglines for my rss aggregation. On it I subscribe to my own rss feeds to reassure myself that they are reaching the outside world. Since I installed the awtags module my postings have all contained links to their tag entries. This doesn't really bother me, it may encourage people to visit my site if they see useful links in the rss feed.

Bloglines tries to deliver articles that have changed and it appears to do this by comparing the contents of the rss file with its previous contents. If there are any small changes bloglines displays the article in the same way as it does for new articles.

I don't like to hastle people when I edit articles to correct spelling mistakes or whatever so I have altered the drupal ping module to only ping (i.e. tell the outside world) if a new posting is created, not if it is modified. However, bloglines appears to poll my feed and so the slightest change will result in articles appearing as if the are new. Today I assigned some tags to some older articles using awtags and this was suficient for the them to be displayed by blogines as new articles as they were still in the rss feed.

This is the long way of apologising to anyone who thinks I a winding them up by republishing old articles with no noticable changes.

I see I have a new subscribed on bloglines. Welcome, hope you find my whitterings interesting. I am getting more visitors since I started using awtags, especially from technorati.