Peter's Blog - Nodes for exchange

Backing up Exchange

Microsoft Exchange uses a nasty database to store all the users mailboxes. Picture it as a big black box, email goes in and there it stays in a locked-in proprietary form. If users lose any email then tough: you cannot just browse through a backup of the database and drag and drop the message, you have to jump through very small high hoops: mount a backup of the database and restore the whole mail box. It's a horror unless you invest in third party tools.

BUT I came across a Better Way. Download a tool from Microsoft called ExMerge. This tool allows you to export all the messages in a user's mailbox to a .PST file. This file can easily be opened in outlook and manipulated with drag and drop. Should you be unlucky enough to lose your exchange database you can reimport the .PST files back into Exchange. Should a user mess up their mailbox, give them the PST file and let THEM sort it out!

Exchange backup strategy:

daily dump of all User's accounts to PST files (actually it synchronises the account with the PST files so it copies the differences which is much faster).
put PST files into standard backup/archive strategy. To be safe, do this in addition to backing up the raw exchange database.

This makes it easy to restore a user's email from a particular date: get the PST file from the tape and open it in outlook.

Best of all, ExMerge is a free download.

The main problem running it on the Exchange Server is that the login account it runs under needs to have 'Receive As' and 'Send As' permissions on the mailbox. It is trivial to schedule it to run every day from the windows scheduler.

Outlook vs Thunderbird

I'm back to using Outlook for two reasons:

the journal feature may be useable as a desktop notetaking tool. Being integrated into the email program means fewer apps running and a leaner and meaner system (ha). It does look like a half decent note taking tool, entries can be formatted, pictures pasted in etc.
was reading how hackers can carry packet sniffers on memory keys and became paranoid about my unencrypted password floating around the ether(net). I looked into setting up ssl authentication on the exchange server but it involves messing around setting up an ssl certificate server and I couldn't be bothered.

I've given up on using folders to categorise my email: I leave it all in the inbox and rely in searching to find stuff. I have google desktop search to speed this up but that still thinks I am still using thunderbird and tries using that to display messages. How to persuade it I have vaccilated back to the evil ones?

Related Posts: desktop email exchange google outlook thunderbird

Opera

I'm trying the Opera 9 preview as an alternative (or complement) to Firefox. Opera has been attractive to me in the past but always gave odd rendering problems or crashes that put me off. This version has been ok thus far and has features built in that are quite seductive. Ok firefox has extensions but these all seem to break every time they upgrade it and I have got fed up with reinstalling them (too high maintenance).

Opera goodies, some maybe all available as Firefox extensions but are bundled with opera:

Fast Forward: this button seems to find the 'next' link on your page and clicks it for you. You can go through google results very quickly.
gestures: right click and move the cursor around to control the browser: e.g. right click and move right does the fast forward action so takes you to the next page with no fuss and minimal physical effort.
email client: this supports IMAP and works with Microsoft Exchange. It chucks all email in one folder and gives lots of options for searching through it. For example, you click on a contact name and see all the mail from that contact. Looks ok so far and is appealing as the memory footprint of Opera should be less than Firefox + Thunderbird. Only downside so far: cannot compose html email. Not a showstopper as I rarely bother to format email (like I rarely bother to format blog postings) and it does display html email (as a web browser should be able to).
It supports widgets if you have a need for a huge clock that cannot be resized. There are only 9 of them so far (clock, calculator, calendar etc).

Possibly more cool: Opera Mini. This is a version of Opera written in Java so it works on my Sony-Ericsson K750i phone. This is a proper web browser, it supports http/html, it is not a crippled WAP thing so you can look at real web sites. It displays them in a tiny font that gets a lot of info on the screen. It communicates with a server at Opera which compresses the pages you are viewing and minimises the download: the front page of this site was a 8k! I installed it thusly:

Download the .jar and .jad files for the k750i.
Connect phone to USB
Copy files to \MSSEMC\Media files\other
Unplug phone (stopping USB devices on Windows 2000, not necessary on XP).
On phone, go to 'My Items', Other.
Select the .jar file, right click and Install. It will ask you if you want it installed as an Application or Game: you decide.
For uk vodafone, go to Settings/Connectivity/Internet Settings/Internet Profiles and create a new profile. Give it any name and set the 'Connect Using' field to 'Contract Internet'.
Go to Settings/Connectivity/Internet Settings/Settings for Java(tm) and select here the new profile you created above.
Go to My Items/Applications (or Games) and select Opera Mini. It will do a connectivity test. If it fails the internet profile for java above might be wrong.
Enjoy

Related Posts: exchange firefox imap opera thunderbird

Outlook Users can read each others mail boxes

Panic at work when it was discovered that anyone could read anyone elses mailbox on the Exchange 2000 Server through Outlook.

Examining the permissions in the 'Exchange Advanced' tab of a Users properties in Active Directory, it appeared that the 'Everyone' user had 'Full Mailbox Access' rights.

Found a mskb article which says that in Exchange Administrator this can happen if the 'Everyone' user had been given 'Receive As' and 'Send As' accesses. Denied both of these in Exchange and rebooted it. After this the 'Everyone' user in active directory no longer had 'Full Mailbox Access' and people could no longer look at each other's mail.

Working with Active Directory and Exchange and the horrible plethora of subtly different property dialogs is bewildering. There is one property dialog in exchange with a 'Client permission' button on it that shows different dialogs if you click it with and without holding the CTRL key (mentioned in this article), you wouldn't guess it from the dialog: why not two buttons? Why the hidden option?.

I can think of a few reasons for this complex mess:

it keeps 10,000 software engineers at microsoft busy knocking out complex architectural stuff that doesn't add any real benefits
much money to be made from training courses
creates an army of people with noddy qualifications resistant to change

Using Thunderbird with Microsoft Exchange

I seem to get a lot of people visiting this site looking at how to use thunderbird with Exchange. I don't think I ever described it in detail as I found it quite easy. I just enabled an IMAP service on the exchange server and I connect to that from Thunderbird using the standard procedure (username/password). IMAP has the advantage of keeping all the messages on the server so it is someone elses problem to back them up. Also searching through messages seems to work much faster as the server is good at that kind of thing and Microsofts local message archive format, the .pst file, is pretty poor. It is probably just as easy to set up POP3. Email is sent using authenticated smtp so I am prompted for username/password to send messages.

With this setup I don't get shared calendars or any other executive gimmicks but I don't use them anyway.

Sorry if this suggestion seems inane but as far as I know this is the only way this can be done.

Related Posts: exchange imap thunderbird

Status Report

Servant Salamander: coming to rely on this. Pretty much destined to register it. Today I discovered that I can get it to calculate space used in subdirectorys and then send a report of directory names and sizes, sorted by size, to the clipboard. Try doing that with Windows Explorer. I've stopped using filezilla, Salamander is a useful sftp client. Things I do wish it could do:

copy directly from one sftp site to another. Similarly, since you can browse straight into a zip or tgz file, it would be nice to be able to copy a file from within an archive on one window to other window showing an sftp site. The user interface spoils you, you start thinking these things are possible, you try them and are thwarted.
create tar.gz files. I have actually learned the incantation for this, I am becoming such a command line junkie:
```
tar cvfz blah *
```
It can do zips and rars and stuff.
It sometimes goes a bit crazy with the confirmation dialogs, sometimes giving two or three in a row. I don't have faith enough to disable them, combining them into one dialog would be nice (e.g. yes I want to copy, I want to copy it there, I want to overwrite everything).

Thunderbird: since I fixed the exchange server I have been using this instead of outlook and have not looked back. I use an IMAP connection to the Exchange server and it is working just fine. I used to have problems with it when my network connection was flakier (it kept asking me for my password) but since I installed XP on my pc the network has been ok (one of those software decay things on my old Win2k install). I think outlook was tainted by spambayes which was pretty lethargic in moving spam around (to get around outlook's flaky message filtering it does lots of sleeps to keep out of the way). So you go to look at a message and find that spambayes is still in the process of moving it to the junk folder. Thunderbirds integrated bayesian spam filter is working for me. Another thing I like about thunderbird, the flag thing that pops up and goes away again when new mail arrives. The outlook taskbar thing always used to get stuck on, even after deleting new messages, to the extent that I disabled it.

Related Posts: email exchange imap outlook salamander thunderbird

POP and IMAP on Exchange Server broken.

I wanted to get the POP and IMAP services on our Exchange 2003 server running so:

I could use Thunderbird (or at least had the choice)
Folks could potentially look at their email from their mobile phones.

Clicking on the 'POP3' service in the Exchange manager gave the useless error 'The server is not operational'. Googling for this implied a DNS problem so I looked in the event log and found a number of instances of error event 5789. The details of this were:

Attempt to update DNS Host Name of the computer object in Active Directory failed.
The updated value was 'server.work.co.uk'. The following error occurred:

The parameter is incorrect.

Searching for event 5789 gave this MS knowledge base article. Aparently this error is due to the DNS domain name being different to the Active Directory Domain Name. In the case here, the DNS domain name of the exchange box should have been domain_name.work.co.uk, not work.co.uk. I fixed this in Control Panel/System/Network Identification and did the mandatory reboot.

When it came up, life was sweet, existing POP3 and IMAP services were reborn and Thunderbird is working nicely.

Moral: Never ever assume that working with Microsoft Exchange is easy.

Related Posts: exchange imap thunderbird windows

exchange

Ugly, overblown email server written by about 3,000 software engineers at Microsoft who have never heard the expression 'keep it simple'.

Python Web Applications

I've been contemplating some python web development. This is mainly for the following reasons:

I'd like my blog to have features for threading subjects based on keywords, a kind of related articles list. This would be dynamically updated, old articles could link to newer articles. This is possible in Drupal but the taxonomy system is laborious to configure through the web interface. There is a new keyword feature (folksonomies) but it appears to be a library for other modules to use. There are some modules that promise this kind of thing but they are not what I am looking for. The most important thing is that old articles are displayed with links to new articles, which involves messing around with themeing to add links to a cached page or figuring out how to flush the cache and put articles through a filter that would add the links.
I'd rather spend my limited programming time being creative with technologies I am already familiar with. I don't like php enough to want to learn it any more. I know and like python.
I don't want to spend my time trying to figure out how the Drupal code base works. Ok, I'm lazy and I have a short attention span and I'm not very interested in how it works.
If I do this as a spare time project I want to enjoy it, I don't want it to feel like a maintenance exercise. I can do that at work.
I have found that using librarys etc written by third partys can give you 95% of what you want, then you spend weeks trying to hack them to get the extra 5%. The only exception to this rule I can think of is the python standard library which is way cool.

So I looked at a few web application frameworks. This is a little lightweight and short on detail, I did most of it a few weeks ago and I didn't take notes.

cherrypy: Requires python 2.3 and my hosting service will only give me python 2.2 with cgi. I think generator functions are way cool and I'd love to use them. Do I stand any chance of getting my hosting service to upgrade? I'll email them and ask.

quixote: got the demo going, although I had to fix some of the urls within it as they appeared to be broken. It worked easily enough with cgi, the 1.3 version worked with python 2.2. However, again they have just released version 2 which has a couple of yield statements in it, limiting it python 2.3.

webware: I don't think I could see the advantage of 'servlets' and their web site (presumably powered by webware but it only says 'python powered') is not an impressive display of it's capabilities.

zope/plone: I've toyed with zope in the past (version 2) and found it's learning curve to be almighty. It may be very capable but you have to learn the zope way to do everything. The developers seem to be concentrating on inventing new object-orientated paradigms rather than keeping things simple and I hate huge complex object models. I don't like the web-based source editor (what, no syntax highlighting?: although I could use mozex and edit in vim). Also, non-cgi.

I was most interested in quixote until I came to the python 2.3 stumbling block with the new version 2. I don't want to settle for an old version. This led me on to think about what these frameworks actually give me:

map urls to class/method calls
- big deal
session management
- do I need it for my simple purposes?
templating systems
- Cheetah
interface to cgi/mod_python/fastcgi/whatever
- I can only have cgi anyway
easy form generation
- classes to build the contents of a form. How many forms will I need? Can't templates do this?
already used on the internet and exposed to hackers
- lets not be too ambitious with what we do. Keep drupal in the background for data entry.
learning curves

Which has led me to... straight cgi. Run my own scripts from cgi, using the cgi module in the standard python library to get the url parameters and generate a nice error page when things go wrong. I can get it going easily enough and below, with just a few hours hacking, is something that can display nodes from the drupal database. The output is themed through the Cheetah template engine and gives me the same look as my drupal theme. Given that it is only cgi it is not massively scalable, but I understand every line of it and that means a lot in terms of development time. Given that it generates pages that look like the drupal pages, I can chop and change, leaving some bits to drupal (data entry) and others in python (presentation). It's a lash up but so what?

The main challenge is porting my wilki module to python, particularly the geshi syntax highlighting.

I am not planning to rewrite Drupal and I probably won't release Yet Another Python Web Framework. I may not even finish it. It is something for me to play with.

   1  #!/usr/bin/python
   2  #
   3  import Cheetah.Template
   4  import cgi
   5  import cgitb; cgitb.enable()
   6  import time
   7  import datetime
   8  import os
   9  import pdo
  10  
  11  def ThemeAndOutput( strPageTitle, strContent, oParams={}):
  12      "Theme the site"
  13  
  14      strSideBar = """
  15      <div class="block">
  16      <h2>Reports</h2>
  17      <ul>
  18          <li><a href="Intranet.cgi">Drupal Emulator</a></li>
  19      </ul>
  20      </div>"""
  21  
  22      oDict = {
  23          'Root': '/SMB',
  24          'Url': '/Intranet/Intranet.cgi',
  25          'Title': 'Drupal Emulator',
  26          'Slogan': 'There are no Problems, only Challenges',
  27          'PrimaryLinks': [ '<a href="http://intranet.org">Intranet</a>',
  28                              '<a href="http://www.google.co.uk">Google</a>'],
  29          'PageTitle': strPageTitle,
  30          'SideBar': strSideBar,
  31          'Footer': 'Copyright 2005 Peter Wilkinson'
  32      }
  33  
  34      #
  35      # Read page template and shove in the content.
  36      #
  37      strTemplate = open( 'PageTemplate.html').read() % { 'Content': strContent}
  38  
  39      #
  40      # Run template and content through cheetah in one go.
  41      # Is this faster? Doubt it.
  42      #
  43      oHtml = Cheetah.Template.Template( strTemplate, [oDict, oParams])
  44  
  45      print str(oHtml)
  46  
  47  def MainPage( oForm):
  48      "Main welcome page: tell user what is going on"
  49  
  50      #
  51      # Determine how many links to show.
  52      #
  53      strFrom = oForm.getfirst( 'from')
  54      if strFrom == None:
  55          strLimit = ' LIMIT 10'
  56      else:
  57          try:
  58              nLimit = int(strFrom)
  59              strLimit = ' LIMIT %d, 10' % nLimit
  60          except:
  61              strLimit = ' LIMIT 10'
  62  
  63      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  64      rs = c.open( 'SELECT * FROM node ORDER BY created DESC %s' % strLimit)
  65  
  66      strTitles = []
  67      while rs.next():
  68          strTitles.append( '<a href="Intranet.cgi?node=%s">%s</a>' %
  69                (rs.fields['nid'].value, rs.fields['title'].value))
  70  
  71      strPage = """
  72  <ul>
  73  #for $strTitle in $titles
  74  <li>$strTitle</li>
  75  #end for
  76  </ul>
  77      """
  78  
  79      ThemeAndOutput( "Welcome", strPage, { 'titles': strTitles})
  80  
  81  def NodePage( oForm):
  82      "Display a node"
  83  
  84      strNode = oForm.getfirst( 'node')
  85      if strNode == None:
  86          ErrorPage()
  87          return
  88      else:
  89          try:
  90              nNode = int(strNode)
  91          except:
  92              ErrorPage()
  93              return
  94  
  95      c = pdo.connect( 'Module=MySQLdb;user=drupal;passwd=secret;db=drupal')
  96      rs = c.open( 'SELECT * FROM node WHERE nid = %d' % nNode)
  97  
  98      if rs.next():
  99          strTitle = rs.fields['title'].value
 100          strBody = rs.fields['body'].value
 101      else:
 102          ErrorPage()
 103          return
 104  
 105      strPage = """
 106  <h2>$strTitle</h2>
 107  
 108  <pre>
 109  $strBody
 110  </pre>
 111      """
 112  
 113      ThemeAndOutput( strTitle, strPage,
 114            { 'strTitle': strTitle, 'strBody': strBody})
 115  
 116  def ErrorPage():
 117      strPage = """<p>There has been some kind of navigation error, the link
 118  you just clicked is broken</p>
 119  <p>Better <a href="mailto:someone@somewhere.com">complain about it</a>.</p>
 120      """
 121  
 122      ThemeAndOutput( "Drupal emulator", strPage)
 123  
 124  oForm=cgi.FieldStorage()
 125  
 126  node = oForm.getfirst( 'node')
 127  if node:
 128      NodePage( oForm)
 129  else:
 130      MainPage( oForm)

The page template is kept in a seperate file. It is based on the phptemplate of my drupal theme and was ported over in less than an hour. Theming engines? A specialisation of templates.

   1  Content-Type: text/html
   2  
   3  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   4   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   5  <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
   6  <head>
   7  <title>$Title</title>
   8  <meta http-equiv="Content-Style-Type" content="text/css"/>
   9  <style type="text/css">@import url(style.css);</style>
  10  </head>
  11  <body>
  12      <div id="headerleft">
  13      </div>
  14      <div id="headermain">
  15          <div id="ie6hack">
  16          </div>
  17      </div>
  18      <div id="headertitle">
  19          <h1>
  20              <a href="$Url" title="$Title">
  21              $Title</a>
  22          </h1>
  23      </div>
  24      <div class="slogan">
  25          $Slogan
  26      </div>
  27      <div id="logo">
  28          <a href="$Url" title="$Title">
  29              <img src="ITL.gif"/>
  30          </a>
  31      </div>
  32      <div id="headerright">
  33      </div>
  34  #if len($PrimaryLinks) > 0
  35      <div id="primarylinks">
  36          <ul>
  37  #for $PrimaryLink in $PrimaryLinks
  38              <li>$PrimaryLink</li>
  39  #end for
  40          </ul>
  41      </div>
  42  #end if
  43  
  44      <div id="ie5forcemargininsidehack">
  45          <div id="main">
  46              <div class="tab">
  47                  <div class="tabbackground">
  48                      <div class="tableft">
  49                          <h3>
  50                              $PageTitle
  51                          </h3>
  52                      </div>
  53                  </div>
  54              </div>
  55              <div class="tabbody">
  56                  <div class="ieisbuggy">
  57                  </div>
  58                  <div id="content">
  59                      %(Content)s
  60                  </div>
  61                  <div class="ieisbuggy">
  62                  </div>
  63              </div>
  64          </div>
  65          <div id="footer">
  66              <div class="footerleft">
  67                  <p>
  68                  $Footer
  69                  </p>
  70              </div>
  71          </div>
  72      </div>
  73      <div id="sidebarright">
  74          <div id="sidebar">
  75              <div class="sidebarbox">
  76                  $SideBar
  77              </div>
  78          </div>
  79          <div id="sidebarbottom">
  80          </div>
  81      </div>
  82  </body>

Related Posts: blog cheetah drupal email exchange google php python theme vim wilki

Exchange not Restarting

Had a big problem with the Exchange Server. It stopped working and upon rebooting the system the exchange services would not start. The principal service that was stuck was the "Microsoft Exchange System Attendant" and the other services were waiting for it. Result: no company email for two and a half days.

When the service finally gave up trying to start it would give the following error:

The Server is not operational
Facility: Win 32
ID no: 8007203a
Exchange System Manager

Someone had been altering the configuration of the DNS server over the weekend and had deleted a few zones that did not appear to be used. Looking through the list of computers in the Active Directory on the Domain Server (a different box to the Exchance Server), these all had their URLs specified as being in one of the zones that was deleted. On a hunch I recreated one of the deleted zones with a single host entry for the exchange server and, voila, exchange started and has been fine.

It seems that the basic cause was that Exchange could not get access to the Active Directory on the Domain Server, probably because the Domain Server was doing some kind of validation on the IP address that was failing.

What is weird is that all other Windows computers in the building were in the same deleted zone but everyone was still able to log into the domain server. There may be some extra security in there to get access to Active Directory compared to logging into a domain, who knows.