Raking in Drupal posts

Rake is the ruby equivalent of make except it is much better in that you are scripting in ruby and not the nasty primeval make language. Rails uses rake a lot as a simple place to gather together a load of administration tasks such as running tests, running database migrations etc.

When I moved all my drupal blog posts from drupal over to PetersBlogger I decided to use a nice organised rake script rather than do my usual practise of having a directory full of utility scripts.

In my rails installation I created a file called lib/tasks/jobs.rake and in it I put my import code:

   1  task (:import_drupal => :environment) do
   2    print "Importing Drupal\n"
   3    oTagTable = {}
   4    #
   5    # Import all the articles.
   6    #
   7    Drupal::Article.find(:all, :conditions => "type <> 'image'").each do |oArticle|
   8      print "#{oArticle.title}\n"
   9      oTime = Time.at( oArticle.created)
  10      oPost = Post.create( :nid => oArticle.nid,
  11                          :title => oArticle.title,
  12        :body => oArticle.body,
  13        :created => oTime)
  14  
  15      #
  16      # import the tags related to the article.
  17      #
  18      oArticle.tags.each do |oTag|
  19        if oTagTable[oTag.tag]
  20          oNewTag = oTagTable[oTag.tag]
  21        else
  22          oNewTag = Tag.new( :tag=>oTag.tag)
  23          oTagTable[oTag.tag] = oNewTag
  24        end
  25  
  26        oPost.tags << oNewTag
  27      end
  28  
  29      #
  30      # Import the comments for the article.
  31      #
  32      oArticle.comments.each do |oComment|
  33        strWho = oComment.name
  34        strComment = oComment.comment
  35        if strWho == nil or strWho == ""
  36          strWho = "Peter"
  37        end
  38        if strComment == nil or strComment.strip == ""
  39          strComment = "{nothing to say}"
  40        end
  41        oPost.comments << Comment.create( :who => strWho,
  42            :body => strComment,
  43            :created => Time.at( oComment.timestamp), :published => 1)
  44        if not oPost.comments[-1].valid?
  45          p oPost.comments[-1]
  46          p oPost.comments[-1].errors
  47        end
  48      end
  49  
  50      oPost.save!
  51    end
  52  end

This creates a rake task called 'import_drupal' which I can invoke simply by running

rake import_drupal

The task runs with the appropriate rails environment so all my models are in place. I had to create a model to access the records in the drupal database thus:

   1  module Drupal
   2    class Comment < ActiveRecord::Base
   3      establish_connection configurations['drupal']
   4      set_table_name 'comments'
   5    end
   6    class Tag < ActiveRecord::Base
   7      establish_connection configurations['drupal']
   8      set_primary_key 'tid'
   9      has_and_belongs_to_many :articles,
  10        :class_name => 'Drupal::Article',
  11        :join_table => 'awtags_node',
  12        :foreign_key => 'tid',
  13        :association_foreign_key => 'nid'
  14      set_table_name 'awtags'
  15    end
  16    class Article < ActiveRecord::Base
  17      establish_connection configurations['drupal']
  18      set_primary_key 'nid'
  19      self.inheritance_column = 'poopy'
  20      has_many :comments, :table_name => 'comments', :foreign_key => 'nid', :class_name => 'Drupal::Comment'
  21      has_and_belongs_to_many :tags,
  22        :class_name => 'Drupal::Tag',
  23        :join_table => 'awtags_node',
  24        :foreign_key => 'nid',
  25        :association_foreign_key => 'tid'
  26      set_table_name 'node'
  27    end
  28  end

This model handles all the relationships between the records, including the has_and_belongs_to_many relationship between posts and tags (provided by awtags).

I needed an entry in config/database.yml to define the connection to the drupal database:

drupal:
  adapter: mysql
  database: petersblog
  username: secret
  password: doyouthinkidpostithere

and it all works nicely. If I invoke it thusly:

rake import_drupal RAILS_ENV=production

then it will import the posts into the rails production database.

I'm giving away a few of my schema secrets here which pedantic rails developers might leap on, specifically I have used my own fields called 'nid' and 'created' to perform roles that rails already provides 'id' and 'created_on' for. The reason for the duplication was that during development I was having problems setting these fields to the values from the drupal database, rails was insisting on giving them it's own values. By having my own fields I had total control. There are no doubt ways around this but I didn't have time to find them (although I did find other people moaning about the same problem).

And that's how I managed to save 1500 posts from the hell that is php.

Peter's Blog - Nodes for rake

Raking in Drupal posts