Nowadays, Linux users can comfortably dwell in the world of Microsoft Exchange, from mails to calendars, from graphical groupware suites to APIs in Python.

1 Context

I once had to cook up a shared calendar which not only had to be publicly queried read-only as an iCalendar, but also edited by several privileged managers. It turns out that's one of the things Microsoft Exchange is rather good at. The problem is, how do you make it so the calendars can be edited with whichever tool you like, on whichever platform, and not just Microsoft Outlook on Microsoft Windows?

2 Exchange from Evolution

Evolution – which used to be called Ximian Evolution, then Novell Evolution – is a groupware suite which operates much like Microsoft Outlook. It's a mail client, it manages calendars, it stores contacts, and does all this in a rather integrated fashion. It didn't take long before it started supporting Exchange, a server technology Microsoft has been running for decades to provide this kind of services. It's built around the MAPI – Messaging Application Programming Interface – protocol, which is said to be poorly documented and not to support calendars. This must be why it's been since Exchange Server 2007 giving way to the SOAP-based EWS – Exchange Web Services. Not that this matters, particularly, as Evolution supports both of them.

But it doesn't mean that getting either of these protocols to work on Evolution is obvious when you don't know how to configure it. Certainly, you'll need to install what Debian-based distributions call the evolution-ews package and then restart Evolution processes. You'll find there's quite a few of them that will keep running even after you think you exited Evolution, which is why some guides suggest you restart your computer after installing the package. As you might have guess, issuing a pkill evolution is good enough, however.

Now comes the interesting part where you have to configure a new account. Let's assume we're at CERN. Note that this procedure works inside as well as outside CERN:

1. File → New → Mail Account. You'll feel quickly that the core of all things Exchange is e-mail. So even if you only plan to use Exchange for working with calendars, what you need to do is configuring a mail account. After all, a calendar is nothing but a peculiar mailbox in a folder.
2. Hit the Next button in the Welcome screen of the Mail Configuration Assistant.
3. In the Identity screen, enter a valid Email Address so the assistant lets you continue. You might as well uncheck the Look up mail server details based on the entered e-mail address at CERN, because the assistant won't be able to make a good guess.

4. If you've successfully installed the evolution-ews package, there should be an Exchange Web Services entry in the Receiving Mail's Server Type menu. Once you select it, the Username will be – again – wrongly guessed from your e-mail address and you should set it to something like CERN\fred. The Host URL, even if it's wrong too, will nonetheless suggest a rather helpful URL ending with EWS/Exchange.asmx. Combine this with what host name the CERN Mail Services documentation will instruct you to use, and you'll soon conclude that the correct Host URL is:

   https://mailstore.cern.ch/EWS/Exchange.asmx

   Both NTLM and Kerberos are supported types of authentication and will effectively work out of the box. Yes, Kerberos. It does put a smile on my face, this does.
5. The Receiving Options screen offers to turn a few more knobs to your taste, but the defaults are probably reasonable.
6. Give a meaningful account name in the Account Summary screen.

7. Finally, hit Apply in the Done screen.

After a while, your mail and folders will appear. Hit the Calendar button and enjoy the sight of all your events which will have made it to Evolution, too.

3 Shared Calendars and Interoperability

Let's say we're working with people who use Microsoft Outlook on the one hand and Evolution on the other hand. When it comes to sharing calendars for editing, there's 4 use cases to consider:

1. Sharing a calendar from Outlook to Outlook users.

   That's the easiest case, as it's probably the only one Microsoft could be bothered to seriously test. Right-click on the calendar you wish to share → Share → Share Calendar. This will open a new message window with as attachment a sharing_metadata.xml file which will tell the recipient's Outlook client to display a button labelled Open this calendar. When clicked, the shared calendar will be added to My Calendars. (Sigh – Microsoft and their obsessions of My This, My That. It doesn't even make sense, here, since the shared calendar I just added is by definition not mine. Anyway.)

2. Sharing a calendar from Evolution to Evolution users.

   Make sure the calendar in question was created in your Exchange account: its type should be named after it in Evolution. Right-click on it → Permissions. Click the Add button and look up the person you want to share it with. Set the Permission level to Editor. That's what Outlook does when you share a calendar with somebody else. Make a note of the Folder ID, which you need to share with your colleague. What I quite like about disclosing it, is that you can choose how to communicate it. It doesn't have to be e-mail, as with Outlook. Although e-mail is probably the easiest mean, in fairness. Confirm by clicking OK.

   Your colleague can now fetch your shared calendar from his Evolution, although it's crucial he does so from the Mail component. Calendars, like many other things, are treated as folders, and for some reason you can't juggle with folders from any other components than the Mail one. Right-click on the account name → Subscribe to folder of other user, enter or look up the calendar creator (just type the username if it's convenient) in the User field and the folder ID in the Folder name field which accepts both names and IDs. Back in the Calendar component, the new calendar will now be listed.

3. Sharing a calendar from Outlook to Evolution users.

   The mail Outlook will send to your colleague won't look much use at first. It comes with a sharing_metadata.xml file which offers three IDs, one of which is a FolderID. Sadly, trying it out as is in Evolution won't work, and it will complain is malformed. The problem is that its format as presented by Outlook's XML message is a Base64-decoded sequence of 2-byte hexadecimal ASCII values. What Evolution expects is what a function like this one would return:

   def dehexify(string):
       return base64.b64encode(''.join([chr(int(string[i:i + 2], 16))
                                        for i in range(0, len(string), 2)]))

   But it still won't work, because the FolderID Outlook sends is too short, missing some information which may presumably be contained somehow in either the EntryId or the MailboxId. Maybe both. But try as I might, I couldn't work out how. Dead-end.

4. Sharing a calendar from Evolution to Outlook users.

   Even knowing the Folder ID from Evolution, I couldn't convince Outlook to use it for adding the shared calendar. Right-clicking on Shared Calendar → Add Calendar → Open Shared Calendar only lets you look people up, and once you select your contact, you don't get to choose which shared calendar you effectively want to add – it's always going to be the main calendar, the one labelled Calendar, with events just marked as Busy. Not much use.

   Would a way to add a shared calendar in Outlook be to forge a sharing_metadata.xml file? Part of me hoped that not all three IDs would be necessary. So I devised a funny way to try and work that out by dissecting it. I sent myself a real invitation from Outlook. I started Mutt and duplicated the mail I received ( Shift C ). I edited ( E ) the duplicate, first to change the subject so I could tell it from the original one, then replace the Base64 block encoding the contents of sharing_metadata.xml with a modified version run through the base64 command which successively omitted each ID. I opened the mail from Outlook and tried clicking the Open Shared Calendar button:
   1. With the EntryId omitted, this caused Outlook to just crash without further ado.
   2. With the FolderId omitted, this caused Outlook to complain that the folder was unavailable.
   3. With the MailboxId omitted, this also caused Outlook to complain that the folder was unavailable.

   Sadly, it appears that all three IDs are necessary. And if I wasn't able to understand what role EntryId and MailboxId played when I tried to reformat the folder ID for Evolution to use it, I was certainly not capable of forging them either. Another dead-end.

   Googling around suggests solutions that all converge towards configuring each party's Outlook client to:
   • Have the recipient add a (read-only) mailbox (File → Account Settings → E-mail tab, double-click your account from the list, More Settings button → Advanced tab → Add, type the name of the sender, a list will be opened is there's any ambiguity, acknowledged all the dialogues that ended up getting themselves opened in the process and restart Outlook. Did you say horrendous? Well, it's not the worse part.
   • The worse part is that the sender has to change their folder permissions such that any recipient can at least list all calendars to find the shared one. For the sake of security, that's not the case by default, nothing is readable by anybody but you.

   This approach is cumbersome, insecure and unnecessary knowing that working with folder IDs as Evolution does it is all it should have taken.

4 A Python API

I've been dreaming for a long time about programmatically working with Exchange. It's only as I wrote those lines that I got my hands dirty and gave the exchangelib Python module a whirl. Now, I didn't need to get my hands dirty at all in the end because exchangelib makes your life remarkably easy. Consider for instance what it takes to exhaustively list all the events of a calendar – a feature which is strangely missing from both Evolution and Outlook which only show parts of the calendar. With exchangelib, this handful of lines will do the business:

#! /usr/bin/env python

import exchangelib

email = 'john.doe@example.com'
creds = exchangelib.Credentials(email, 'mypassword')
account = exchangelib.Account(email, credentials=creds, autodiscover=True)

for event in account.calendar.all():
    print event.subject

It's one piece of software jewellery, this module is. And what we can then also use exchangelib for is retrieve IDs. In particular, the folder_id attribute could come in handy in what we previously attempted to achieve:

print account.calendar.name, account.calendar.folder_id
for calendar in account.calendar.children:
    print calendar.name, print calendar.folder_id

You will see from the output that the IDs are the same Base64-encoded ones used in Evolution. So then, how could this help us in our previously unfulfilled use cases which involved:

3. Sharing a calendar from Outlook to Evolution users.

   Instead of the sharing_metadata.xml file, the sender could instead use exchangelib to work out a folder ID that can be used with Evolution. Of course, the sender could just as well use Evolution.

4. Sharing a calendar from Evolution to Outlook users.

   The folder_id attribute – exactly the same folder ID Evolution uses – is, as we noticed before, a superset of the FolderId element in the sharing_metadata.xml that would normally be sent by Outlook. All there remains to do is to Base64-decode it before turning it into a sequence of 2-byte hexadecimal ASCII values:

   def hexify(string):
       return ''.join(['%02X' % ord(c) for c in base64.b64decode(string)])

   Truncated to the right length, that's one step closer to forging the XML sharing message. Sadly, I still couldn't work out a way with exchangelib to find or otherwise put together an EntryId or a MailboxId. So we're not further forward on that front.

5 Further Interoperability Problems

Once you do manage to share calendars between Outlook and Evolution users, there's other problems that could arise.

For instance, creating all-day events across the daylight saving time change confuses both Evolution and Outlook in some obscure cases. In Evolution, the event will keep losing its all-day flag and make it end one day late. In Outlook, the all-day flag remains and the visual span stops at the right date, but the end time is effectively one day late anyway. Perhaps it's because it's the next day at 12:00 AM – midnight, no time spent on that next day. But that's a bit misleading. Misleading enough to badly confuse Evolution.

Generally speaking, the synchronisation is flimsy. Removing events from Outlook does not necessarily have them removed in Evolution (even though I confirmed with exchangelib that they were indeed gone). Right-clicking on the calendar → Refresh doesn't help. Restarting Evolution doesn't help either. Nor does attempting a pkill evolution. Nor even trying to remove the event again from Evolution, which then loudly complains that it Cannot remove calendar object: The specified object was not found in the store. After all this, why then does it keep displaying it? There's clearly some stubborn cache getting in the way, or Evolution 3.26.1 somehow disagreeing with Exchange 14.3.399.0.

Finally, events aren't perceived the same way in Outlook and Evolution. Making a New Appointment in Outlook, i.e. an event not requiring attendees, causes to create an event seen by Evolution as a Meeting, which does require attendees. So much so that it becomes impossible in Evolution to edit it, remove the attendees since you don't want to send any invitation, make any other changes and then save because it will complain that At least one attendee is required. You may attempt to right-click on the event → Convert to Appointment, then save, but it will have no effect: next time you open it, it will be a Meeting again. You're left with removing the event and recreating it anew. And we've seen before how problematic that can be.

6 Other Ways to Share Calendars

Thinks look pretty grim, then, when it comes to interoperability on Exchange between different clients. I probably shouldn't complain though – until a few days ago, I didn't even know how to configure Evolution to use the CERN Exchange services, let alone realised how easy exchangelib is to work with.

For completeness, I'd like to add that Outlook lets you publicly publish calendars as iCalendar files accessible from a URL. Right-click on the calendar you wish to share → Share → Publish This Calendar. Interestingly, what it does is to open a web browser on a specific web form to configure how you want to publish it. So I wouldn't really call this an Exchange feature Outlook supports and Evolution doesn't. Just find the right URL which, for once, is loosely based on the Base64-encoded, non-hexified folder ID as exchangelib and Evolution would present it, and Bob's your uncle. It's slightly frustrating that you can at most only publish events one year before today and one year after. My first use case would have already needed more than this. The Access level is, to my taste, best left Restricted. When made Public, your full e-mail address will appear in the URL for a calendar which is said to be searchable on the Internet.

7 References

• iCalendar
• Evolution
• MAPI and EWS in the Microsoft Exchange Server
• Outlook 2010 Manual Configuration at CERN
• Talking SOAP With Exchange
• Python client for Microsoft Exchange Web Services (EWS)
• open a mail in outlook #146 gives some inspiration as to how to decode an ID for use in the sharing_metadata.xml file.