Peter's Blog

Redefining the Impossible

WoW: Scraping the Armory in Python


So how does one go about mining the information in the World of Warcraft Armory? Inspired by applications like the Warcraft Signature Generator I had to find out.

Here is a python script to interrogate the armory. This will download an xml document containing all the information about a character.

   1  # Interrogate wow armory
   2  
   3  import urllib2
   4  import xml.dom.minidom
   5  
   6  #
   7  # Character we are nosing into
   8  #
   9  strRealm = 'Aerie Peak'
  10  strCharacter = 'Pookypoo'
  11  
  12  #
  13  # Look the data up in the european armory. Change this for US.
  14  #
  15  strUrl = "http://armory.wow-europe.com/character-sheet.xml?r=%s&n=%s" %
  16            (strRealm.replace( ' ', '+'), strCharacter)
  17  
  18  #
  19  # Open url.
  20  # Need to specify firefox as user agent as this makes the server return an XML file.
  21  # If this is not done we get html.
  22  #
  23  oOpener = urllib2.build_opener()
  24  oOpener.addheaders = [
  25     ('user-agent',
  26      'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4'),
  27     ]
  28  
  29  req = urllib2.Request( strUrl)
  30  strFile = oOpener.open(req).read()
  31  
  32  #
  33  # Now have raw xml file, can print it if interested
  34  #
  35  # print strFile
  36  
  37  #
  38  # Use xml dom to parse the file
  39  #
  40  oDoc = xml.dom.minidom.parseString( strFile)
  41  
  42  #
  43  # Quick hack to display certain attributes of certain elements.
  44  #
  45  strAttributes = (( 'character', 'level'),
  46                      ( 'character', 'guildName'),
  47                      ( 'baseStats', 'agility'))
  48  
  49  for strElement, strAttribute in strAttributes:
  50      oElement = oDoc.getElementsByTagName( strElement)[0]
  51      strValue = oElement.getAttribute( strAttribute)
  52      print strElement, strAttribute, strValue
Toggle Line Numbers

This particular script will display the characters level, guild name and agility. There is much more information in there, it's just a matter of finding a use for it. I might get around to automatically updating the character levels shown on this site as the sig generator that inspired this gets hammered a lot and is not totally reliable. UPDATE: not criticising here, it's just a very popular site with heavy loading. The tools there are very nice and more than my imagination could come up with.


Filed under: games python warcraft wow

3 Comments

Spikeles Says:

over 2 years ago

You might be interested in this site: http://wow.tachyonsix.com/armory/

Peter Says:

over 2 years ago

Yes, this is where the Signature Generator is. Other useful tools too.

Peter

snipe Says:

over 2 years ago

Thanks so much for the XSL tip - I was struggling with an app in PHP getting only HTML output, and the user agent specification fixed it right up. Thanks a million! (I'm building a facebook application where users can post their toon stats)

Sorry but comments on this post are now closed.