Using Drupal as a collaborative software translation tool

I've intended to announce this development at Drupalcon San Francisco but unfortunately the session on this was merged with a more general i18n session which was coupled by the ash cloud above Europe, so I could not go. Evidently, collaborative software translation is not a mainstream topic. On the other hand, I keep receiving requests of the general applicability of the tools Drupal.org uses about every two weeks, and this interest always amazes me. While the localization server tool used on localize.drupal.org grew out of needs of the Drupal community, the solutions were architected to be useful in a general purpose software translation environment. While the architecture was there, it was lacking useful UI controls to just run it as a generic software translation tool.

Existing non-Drupal users like the Gallery 2/3 project and the Musescore desktop app utilize custom data connector modules, which the localization server nicely supports, allowing for custom code to gather data for translation. Gallery even uses a custom Localization client port for clients to submit translations to the server, even though their software is not Drupal based. However, translating arbitrary software without writing your custom connector code was not possible earlier.

Thanks to recent changes and additions to the 6.x-2.x version of the Localization server however, people can now set up ready to use translation servers for arbitrary software easily. The built-in functionality supports using Gettext Portable Object files as source files to fill up the list of translatable strings and provides a user interface to set up projects and releases as well as upload the source files. Gettext Portable Objects is a well known de-facto standard and many tools exist to cross-convert to this file format. Here is a quick screenshot tour of the settings involved. I've used the neat Curie theme that I've recently released for these screenshots.

1. Grab Drupal 6 and the following modules: Localization server (latest Drupal 6.x-2.x development version6.x-2.x-beta2 or later); the Plural formula configurator which will let you easily configure languages and the jQuery Update module's 6.x-2.x latest development release (alpha 1 might also work).
2. Enable the Localization server and the Localization community UI modules. These will enable Locale and jQuery Update as dependencies. To make it easier to set up plural formulas for your languages, I suggest to enable the plural formula configurator as well.
3. Add any number of languages you'd like to support via Administration » Site configuration » Languages. With the plural configurator, you'll be able to set up plural formulas for languages (the module gives you examples of most common languages and sets up defaults for languages it knows already).
4. There are two connector modules included with the localization server. Enable the Localization server for Gettext files connector. This will let you work with gettext .po(t) files as sources.

Ok, now that we have all the modules set up, configure them. A module can provide multiple connectors with possible configuration, so l10n_server requires you to select which connector formats you'd like enabled. In this case, it is just one connector without configuration, so just mark it enabled. This will let you add projects which use this connector.

To start off translating a project, the server needs to know of that project. since we are using a "by file upload" connector, we can/need to create the project manually. Go to the localization server projects administration page and click on the "Add project" tab. Fill in the short name for the project (which will appear in URLs), the name, and possibly the home link. You'll have the gettext file handler preselected already.

Now that you have a project to work with, you can start adding releases. Projects can later be removed or disabled (temporarily or indefinitely). It is also possible to start over (which will remove all data except the main identifiers of the project that we just entered above).

Click the releases link and you'll see we have no releases for this project yet. Well, we need to add one at least. So go to adding a new release, add the version number of your release you are about to provide translation for and possibly a download link for this release. Like the project home link, this will only be used to show to users, not to actually download anything. Upload a .po or .pot formatted source file with text you have translatable in your software. I've made up a very quick three-piece mysoftware.po for this example. Note that the server will not deal with the .po headers in the file, so I just omitted them for this example:

How you generate .po files from your software depends totally on your API. PHP itself supports the gettext standard function naming conventions, so you can use the standard gettext tools. You can of course translate software written in whatever language with this tool, since .po files are just plain text transporters for string pairs. If you have a list of translatable strings in any format, generating files in the above demonstrated .po format should not be hard.

The server should inform you that all strings are now saved as sources for translation. Similarly to projects, you are able to start over releases (which will delete all related source strings and let you reimport an updated .po file) or delete the release completely.

But instead of destructing what we've already achieved, let's see the translation UI finally. If you go the Translate menu item from the navigation menu, you'll see a quick overview of our one project with one release with 3 translatable strings. I've set up Hungarian as the only language here, but if you have more, you'll see a selection of languages as well. This screen can be spiced up with an instant status report of the translation status, if you go to your general localization server settings and pick mysoftware as the highlighted project. A nice status report with progress bars for each language will show up on the main page.

Once you actualy go to the translation tab for a language, you'll see the list of strings to translate with fields to add new translations. This user interface will allow you to copy source strings and edit translations. The filter on the top of the page can be used to focus on certain sets of translatable strings and is very useful to track down contributions by individuals.

From here, you can use the user interface to translate and collaborate on your software UI. As experience showed this can even be used to take suggestions for changes in original English user interface text. I'd only like to take one last chance to highlight a neat feature of the server which lets you see and compare all outstanding translation suggestions for strings, quickly assessing the difference between them and deciding which one fits with your translation guidelines or style. This fancy live diff works as you type in a new suggestion for a string and also works for all existing suggestions.

There is a lot more to this user interface and the localization server has lots of other cool features. You can connect the server to the Drupal Organic groups module with the Localization Groups sub-module and create separate translation groups for languages with controlled permissions on the group level. You can enable the remote submission API module to allow for remote translation submissions (but as with the Gallery project, you'll need to work on a custom client for this if your software is not Drupal based, because we only have the localization client implementing this API on the client side now). You can track top contributors in the language teams and export the translations as .po files for distribution is whatever form.

These tools proved very useful on localize.drupal.org, and making them useful for the general audience should hopefully help get more of you on board in using and improving it. We have some exciting things on the horizon including actions integration (where stuff happening in the translation flow will inject short status reports to a status feed) and integration with automated translation tools (eg. Google translate) as well as more sophisticated translation memory (TM) tools. There are still missing pieces in our tools, the Drupal-sepcific export options will still show up when you use the gettext connector only for example, but we are getting pretty close.

Your feedback welcome!