Our Wish List for Encryption Browser Extensions

PGP encryption is one of the most frequently requested features for Roundcube and for good reasons more and more people start caring about end-to-end encryption in their everyday communication. But unfortunately webmail applications currently can’t fully participate in this game and doing PGP encryption right in web-based applications isn’t a simple task. Although there are ways and even some basic implementations, all of them have their pros and cons. And yet the ultimate solution is still missing.

Browser extensions to the rescue

In our opinion, the way to go is with a browser extension to do the important work and guard the keys. A crucial point is to keep the encryption component under the user’s full control which in the browser and http world can only be provided with a native browser plugin. And the good news is, there are working extensions available today. The most prominent one probably is Mailvelope which detects encrypted message bodies in various webmail applications and also hooks into the message composition to send signed and encrypted email messages with your favorite webmail app. Plus another very promising tool for end-to-end encryption is coming our way: p≡p. A browser extension is at least planned in the longer term. And even Google just started their own project with the recently announced end-to-end Chrome extension.

That’s a good start indeed. However, the encryption capabilities of those extensions only cover the message body but leave out attachments or even pgp/mime messages. Mostly because there extension has limited knowledge about webmail app and there’s no interaction between the web app and the extension. On the other side, the webmail app isn’t aware of the encryption features available in the user’s browser and therefore suppresses certain parts of a message like signatures. A direct interaction between the webmail and the encryption extension could help adding the missing pieces like encrypted attachment upload and message signing. All we need to do is to introduce the two components to each others.

From the webmail developer’s perspective

So here’s a loose list of functionality we’d like to see exposed by an encryption browser extension and which we believe would contribute to an integrated solution for secure emailing.

A global (window.encryption-style) object providing functions to:

  • List of supported encryption technologies (pgp, s/mime)
  • Switch to manual mode (i.e. disabling automatic detection of webmail containers)

For message display:

  • Register message content area (jQuery-like selector)
  • Setters for message headers (e.g. sender, recipient)
  • Decrypt message content (String) directly
  • Validate signature (pass signature as argument)
  • Download and decrypt attachment from a given URL and
    • a) prompt for saving file
    • b) return a FileReader object for inline display
  • Bonus points: support for pgp/mime; implies full support for MIME message structures

For message composition:

  • Setters for message recipients (or recipient text fields)
  • Register message compose text area (jQuery-like selector)
  • … or functions to encrypt and/or sign message contents (String) directly
  • Query the existence of a public key/certificate for a given recipient address
  • File selector/upload with transparent encryption
  • … or an API to encrypt binary data (from a FileReader object into a new FileReader object)

Regarding file upload for attachments to an encrypted messages, some extra challenges exist in an asynchronous client-server web application: attachment encryption requires the final recipients to be known before the (encrypted) file is uploaded to the server. If the list of recipients or encryption settings change, already uploaded attachments are void and need to be re-encrypted and uploaded again.

And presumably that’s just one example of possible pitfalls in this endeavor to add full featured PGP encryption to webmail applications. Thus, dear developers of Mailvelope, p≡p, WebPG and Google, please take the above list as a source of inspiration for your further development. We’d gladly cooperate to add the missing pieces.

Roundcube Next – If we would start over again

Roundcube indeed became a huge success story with tens of thousands of installations worldwide. Something I never expected back in 2005 when I started the project as a fresh alternative to the well established but already aged free webmail packages like SquirrelMail or Horde IMP. And now, some 9 years later, we find ourselves in a similar position as the ones we previously wanted to replace. Although we managed to adapt the Roundcube codebase to the ongoing technological innovations, the core architecture is still ruled by the concepts which seemed to be right back when we started. And we’re talking about building a web app for IE 5 and Netscape 6 when browsers weren’t as capable and performant as they are today and when the term AJAX has not yet been known nor did we have nifty libraries such a jQuery or Backbone.js at hand.

It more often happens that, when discussing the implementation of new features to Roundcube, we find ourselves saying “Oh man, that’s going to be an expensive endeavor to squeeze this into our current architecture! If we could just…”. This doesn’t mean that the entire codebase is crap, not at all! But sometimes you just silently wish to give the core a fresh touch which respects the increased requirements and expectations. And that’s the challenge of every software product that has been around for a while and is still intensively developed.

When looking around, I see inspiring new webmail projects slowly emerging which don’t carry the legacy of a software product designed almost a decade ago. I’m truly happy about this development and I appreciate the efforts of honest coders to create the next generation of free webmail software. On the other hand it also makes me a bit jealous to see others starting from scratch and building fast and responsive webmail clients like Mailpile or RainLoop which make Roundcube look like the old dinosaur. Although they’re not yet as feature rich as Roundcube, the core concepts are very convincing and perfectly fit the technological environment we find ourselves in today.

So what if we could start over and build Roundcube from scratch?

Here are some ideas how I could imagine to build a brand new webmail app with todays tools and a 9 years experience in developing web(mail) applications:

  • Do more stuff client side: the entire rendering of the UI should be done in Javascript and no more PHP composing HTML pages loaded in iframes.
  • The server should only become a thin wrapper for talking to backend services like IMAP, LDAP, etc.
  • Maybe even use a common API for client-server communication like the one suggested by Inbox.
  • Design a proper data model which is used by both the server and the client.
  • Separate the data model from the view and use Backbone.js for rendering.
  • Widget-based UI composition using simple HTML structures with small template snippets.
  • Keep mobile, touch and hi-res devices in mind when building the UI.
  • Do skinning solely through CSS and maybe allow single template snippets to be overridden.
  • More abstraction for storage and caching layers to allow alternative backends like MongoDB or Redis.
  • Separate user auth from IMAP. This would allow other sources or accounts to be pulled into one session.
  • Use more 3rd party libraries like require.js, moment.js, jQuery or PHPMailer, Monolog or Doctrine ORM.
  • Contribute to the 3rd party modules rather than re-inventing the wheel.

While this may now sound like a buzzword bingo from a web developers conference (and the list is certainly not complete), I indeed believe in these very useful and well developed modules that are out there at our service. This is what free software development is all about: share, use and contribute.

But finally, not every part of your current Roundcube codebase is badly outdated and should be replaced. I’d definitely keep our current IMAP, LDAP and HTML sanitizing libraries as well as the plugin system which turned out to be a stable and important component and a major contributor to the Roundcube’s success.

And what keeps us from re-building Roundcube from the ground up? Primarily time and the fear of jeopardizing the Roundcube microcosmos with a somewhat incompatible new version that would require every single plugin to be re-written.

But give use funding for 6 month of intense work and let’s see what happens…

The Kolab story

Today I’d like to share a success story of a picture perfect project collaboration as it only happens in the open source world without any commercial, political or geographical borders. It all started back in 2009 after a short interview about Roundcube was published on a techworld.com blog. Short time after we got an email from Georg Greve, founder of the FSFE and member of the Kolab Groupware project. At that time, Kolab already made its name as a free competitor to Microsoft Exchange and Outlook and they were just about to found a new company to push Kolab to the next level. One thing Kolab definitely needed was a better web client to access all the groupware data from anywhere. And this is where Roundcube seemed to fit in perfectly. Although Roundcube was “just” an email client, the Kolab guys saw great potential in our codebase and the vital community around it. And now, more than three years after, we can all witness the great success of this decision.

Lots of lessons learned

After some first meetings and discussions about a possible collaboration between Kolab and Roundcube, we agreed on a general interest from both sides. For the newly founded company Kolab Systems who (as opposed to the Roundcube project) directly hits customers with real needs and urgent bugs to be fixed, it was important to get direct and reliable access to Roundcube developers in order to be agile enough to establish professional services around the open source software stack that is Kolab.

This is also where we started to learn how to run an open source project in a more professional and sustainable way. They taught us the importance of having release branches with long-term support, a clear road map and well documented changesets. Kolab also helped us to better understand the whole licensing topic and with them getting us some legal consultancy from the FTF we finally made the transition to GPLv3 with the added exceptions for skins and plugins. Escpecially the latter one opened the doors for Roundcube to become more accepted in commercial environments which previously had their concerns about the strict terms of the GPL.

Great boost of development

But let’s also have a look at the product side of the story. All users of Roundcube have been on the receiving end of technical and other benefits starting from the 0.7 release. To give you some background about myself:

When Georg and I were talking for the first time, my day-job left me very little time to work on Roundcube. I hadn’t contributed much code in a while, and was even considering to step back from the project altogether because I fell short of my own expectations. Of course it’s impossible to say what would have happened in an alternate universe, but fact is that Kolab Systems enabled me to get back to work on Roundcube. And because Kolab is fully open source, everything has become available to all who use Roundcube.

Here are just some of the features and plugins we added to Roundcube as part of the Kolab integration work:

  • ACL plugin
  • Calendar plugin
  • Tasks plugin
  • Advanced contact search
  • Savable contact search queries
  • Personal spell check dictionaries
  • ODF document viewer plugin
  • Inline PDF viewer plugin

And these are only the “visible” additions to our software which have primarily been initiated and forced by Kolab. Lots of improvements on the stability and quality of the underlying codebase, namely the IMAP and LDAP libraries, are referable to reports coming from the Kolab community.

The work on the various plugins used to bring full-stack groupware functionality to Roundcube also resulted in several upstream patches to other open source projects such a jQuery UI and the jQuery fullcalendar. And that’s what makes free software development truly awesome!

Quality guaranteed

Another very positive aspect of the collaboration with a mature OSS project such as Kolab with a competitive company in the background is the quality assurance and testing we get from them. While we get a lot of bug reports from our own community that help stabilizing our code, there’s no real QA process set up for Roundcube, mainly due lack of resources. Because Kolab Systems takes responsibility (and money) for the installations of the Kolab Groupware, they have a strong interest in quality assurance. And with Roundcube now being an important component of their suite, we can just ride on the wave of QA management and processing. The only price we pay is to fix bugs within a reasonable time. And again, the results of this are accessible for everybody who regularly updates their Roundcube installation.

On an operative and decision-making level, Kolab Systems and the Kolab project itself strive to be “good citizens” in the open source world the same way as we do. Without having to ask for it, it was made clear that Roundcube would remain its own fully independent and emerging project run by the community. Whenever there is a question of whether a certain feature is in Roundcube’s general interest, whether it should go into Roundcube core, or into the Kolab specific modules, or perhaps even approach differently altogether, that decision is always ours to make with no interference from Kolab.

So we kept Roundcube as the lean, focussed web mailer that I believe it should be, and moved all other functions into their separate modules. From the perspective of Roundcube, this has led to improvements on all aspects that matter: adoption, code, quality, community and allowing both Alec and myself to spend lots of time that benefits all users of Roundcube.

Summarizing all these facts and stories, this “K”ollaboration finally IS nothing but a success story. Two great open source projects came together to become even greater.

Migrating our subversion repository to github

We recently moved our svn repository to git hosted at github. Now I’d like to share my experience doing that migration and explain the steps we took to do so. I pretty much followed the process described by John Goulah in his blogpost. So here are the concrete steps I took:

1. Download the entire subversion repository to the local machine for performance reasons.
2. Create an authors mapping file which looks something like this:

#svnuser = gituser <email>
thomasb = thomascube <thomas@...>

3. To start, initialize the git repository with reference to the old svn repos:

$ git svn init file://<path-to-local-svn-repos> roundcube-git

4. cd into the roundcube-git folder and edit the .git/config file to configure what to migrate. I just added the following blocks:

[svn]
	authorsfile = authors.txt
[svn-remote "svn"]
	url = file://<path-to-local-svn-repos>
	fetch = trunk/roundcubemail:refs/remotes/svn/trunk
	branches = branches/{release-0.6,release-0.7,release-0.8}:refs/remotes/svn/*
	tags = tags/roundcubemail/*:refs/remotes/svn/tags/*

Don’t forget to place the authors.txt file into the destination folder!

5. Now we can start importing the subversion data by executing

$ git svn fetch

Depending on the size of your repository this will take a while to run. Grab yourself a coffee or two.

6. There are a few more tasks to run in order to finish this off. Download the scripts from the git-svn-abandon repository and put them somewhere in your $PATH. Then run

$ git-svn-abandon-fix-refs

7. Now we also want to create a table mapping svn revisions to git commits. This can be used later on to update references to revisions in Trac comments for example. The scripts used to do that can be downloaded from the TRAC-SVN-to-GIT-migration project.

$ git-svn-create-lookup-table.sh > rev-lookuptable.txt

8. After creating the mapping table, we can now remove the comments referring to the old svn revisions from every git commit messages. This is done with

$ git-svn-abandon-cleanup

Done. The git repository is now complete. For Roundcube, we’re using a public github repository so what’s left is to push all the data to that:

$ git remote add origin git@github.com:roundcube/roundcubemail.git
$ git push --all
$ git push --tags

The next challenge is to update our Trac platform to place nice with the github repository.

9. Showing a git repository as source in Trac is pretty easy. Clone the repository somewhere on the machine where trac runs and install the GitPlugin for Trac. Just follow the instructions on the referred wiki page. In addition to that, the github-trac plugins helps to keep the Trac clone updated when new changes are pushed to the remote github repository.

10. Finally, we wanted all references to SVN revisions in tickets and comments to point to the according commits in our new github repository. And we therefore created the mapping table in step 7. But it turned out that the revision mapping table created by git-svn-create-lookup-table.sh wasn’t correct. None of the listed git commits finally existed in the git repository. Maybe the git-svn-abandon-cleanup changed them. So I was forced to build a new one in order to update all references. I achieved that by comparing the logs from both the SVN and GIT repository with this quickly hacked PHP script.

svn log file://<path-to-local-svn-repos> > svn.log
git log > git.log
php git-svn-create-rev-map.php svn.log git.log > rev-lookuptable.txt

Now we could start the (slightly modified) convertTracTickets.php script from the TRAC-SVN-to-GIT-migration tools and change all references in our Trac tickets. But don’t forget to backup your database before that step.

Say hello to the new face of Roundcube

We’re very happy to give you a first preview of the all new skin for Roundcube webmail. It was created by FLINT, the very talented graphic designer who already designed our website. He worked hundreds of hours for free just to make Roundcube the best-looking webmail software on the planet. And the results are overwhelming: the design is very stylish and modern while remaining clear and functional and it makes our current default skin look like Squirrelmail ;-)

The all new mail view of Roundcube

See all the drafts at our Flickr page.

The fake contents of the screens are in German and they include some features which are not yet implemented but it’s nice to have an idea of the future.

“Roundcube = Sensational” – December 2010

So nice… we really appreciate messages like that.


From: N. I.
To: hello@roundcube.net
Date: Tue, Dec 14, 2010 at 16:09
Subject: Roundcube = Sensational

Hi,
I'm a long-time user of many OSS packages and am aware of the limited time you guys have. I just wanted to say that I've installed the stable version of Roundcube onto my Ubuntu Server and it is, quite frankly, absolutely fantastic. I've played aroud with Horde, Squirrelmail etc over the las couple of years and have found absolutely nothing which comes close to the fantastic look and feel and ease of installation and configuration.
I'm looking forward to using Roundcube and hope to contrib code/docs etc in the future.
Once again, well done on a truly excellent job!
yours (via my new Roundcube installation)
N. I.

“Thanks” – August 2010

What would you do?


From: <unreadable>
To: hello@roundcube.net
Date: Fri, Aug 6, 2010 at 10:49
Subject: Thanks

[root@local roundcubemail]# patch -p0 < roundcubemail-0.1.1_chpwd_forward.patch
patching file index.php
Hunk #1 FAILED at 389.
1 out of 1 hunk FAILED -- saving rejects to file index.php.rej
patching file program/js/app.js
Hunk #1 FAILED at 283.
Hunk #2 FAILED at 294.
Hunk #3 FAILED at 988.
3 out of 3 hunks FAILED -- saving rejects to file program/js/app.js.rej
patching file program/localization/en_US/labels.inc
Hunk #1 succeeded at 336 with fuzz 2 (offset 93 lines).
patching file program/localization/en_US/messages.inc
Hunk #1 FAILED at 79.
1 out of 1 hunk FAILED -- saving rejects to file program/localization/en_US/messages.inc.rej
patching file program/steps/settings/func.inc
Hunk #1 FAILED at 262.
1 out of 1 hunk FAILED -- saving rejects to file program/steps/settings/func.inc.rej
patching file program/steps/settings/passwd.inc
patching file program/localization/zh_CN/labels.inc
Hunk #1 succeeded at 264 with fuzz 2 (offset 87 lines).
patching file program/localization/zh_CN/messages.inc
Hunk #1 succeeded at 96 with fuzz 2 (offset 17 lines).
patching file program/steps/settings/forwards.inc
[root@local roundcubemail]# echo $?
1
[root@local roundcubemail]#

Why ? ?

Thanks !

My environment: OpenLDAP + roundcubemail-0.3.1

Follow

Get every new post delivered to your Inbox.

Join 29 other followers