Rohland de Charmoy

A while ago I was involved in a project at Clyral where we developed a web based support ticketing system (similar to Zendesk). One of the things we focused on was dealing with the detection of new content in an email conversation. Typically, emails would bounce back and forth between a customer and an agent and we wanted to ensure that our web interface wasn’t cluttered with the full email thread for every reply the customer sent in (the original conversation was always included in outbound emails for convenience).

When we looked at implementing the mechanism which would only extract new content from an inbound email, we found it surprising that there was no official standard for demarcating the beginning and end of an email in an email thread. We took a look at a few mail clients and discovered that Microsoft Outlook is capable of dealing with this problem. If you use one of newer versions of Microsoft Outlook, you’ll notice that the client is capable of detecting the boundaries between emails. Take a look at the screenshot below:

Notice how Outlook has detected the boundaries between each email. Each email has a gradient which assists you to navigate the thread, furthermore, if you hover over a specific email, you can jump to the next/previous email in the thread relatively easily.

While there didn’t seem to be a standard way of achieving this, we managed to get a decent solution in place for our ticketing system. Since we were sending out the emails which customers would reply to, we could standardise the email content such that we could easily detect the boundary when a customer responded. This was not a full proof solution, so in the end, we implemented a heuristic algorithm that could deal with most of the common mail clients out there. Implementing this feature made it infinitely easier to manage conversations with customers. Obviously, we always kept keep the original email as customers would sometimes reply with changes to the original email content (such as answering questions inline).

Last year, we started using FogBugz as our general case and project management system. What we discovered was that FogBugz isn’t as smart as the system we developed in terms of dealing with email conversations. Outbound emails would not include the full email conversation and if you don’t use the web interface to respond, the boundary between the new content and email thread isn’t detected at all. This usually leads to a very cluttered case view where you need to scroll over copious amounts of duplicate text.

To deal with this problem, I implemented a very simple JavaScript customisation which scans over the content in a case and hides any email text which is superfluous. You always have the capability to toggle the content if you do need to inspect it. I’ve included the code for the customisation below. We use Microsoft Outlook (and most of our clients do as well), so the solution works well for us. The code simply scans email for the ‘From: ..’ and splits the email there. It’s not rocket science.

$(function(){
    $('.emailBody').each(function(index, element){
        var body = $(element);
        var edgeIndex = body.html().indexOf('<br>\nFrom: ');
        if (edgeIndex == -1){
        	return;
        }
        var mainBody = body.html().substring(0, edgeIndex);
        var quotedBody = body.html().substring(edgeIndex);
        body.html(mainBody);
        body.append('<div class="showQuote" style="padding-top:5px"><a href="javascript:void(0);" class="dotted" onclick="$(this).parent().parent().find(\'.emailThreadBody\').toggle();">- show quoted text -</a></div>');
        body.append('<div class="emailThreadBody" style="display:none;"></div>');
        body.find('.emailThreadBody').html(quotedBody);
    });
});

Hope this helps. Hopefully, this issue is addressed in the next major version of FogBugz.

Cheers,
Rohland