Using Drupal as a collaborative software translation tool

I've intended to announce this development at Drupalcon San Francisco but unfortunately the session on this was merged with a more general i18n session which was coupled by the ash cloud above Europe, so I could not go. Evidently, collaborative software translation is not a mainstream topic. On the other hand, I keep receiving requests of the general applicability of the tools Drupal.org uses about every two weeks, and this interest always amazes me. While the localization server tool used on localize.drupal.org grew out of needs of the Drupal community, the solutions were architected to be useful in a general purpose software translation environment. While the architecture was there, it was lacking useful UI controls to just run it as a generic software translation tool.

Existing non-Drupal users like the Gallery 2/3 project and the Musescore desktop app utilize custom data connector modules, which the localization server nicely supports, allowing for custom code to gather data for translation. Gallery even uses a custom Localization client port for clients to submit translations to the server, even though their software is not Drupal based. However, translating arbitrary software without writing your custom connector code was not possible earlier.

Thanks to recent changes and additions to the 6.x-2.x version of the Localization server however, people can now set up ready to use translation servers for arbitrary software easily. The built-in functionality supports using Gettext Portable Object files as source files to fill up the list of translatable strings and provides a user interface to set up projects and releases as well as upload the source files. Gettext Portable Objects is a well known de-facto standard and many tools exist to cross-convert to this file format. Here is a quick screenshot tour of the settings involved. I've used the neat Curie theme that I've recently released for these screenshots.

  1. Grab Drupal 6 and the following modules: Localization server (latest Drupal 6.x-2.x development version6.x-2.x-beta2 or later); the Plural formula configurator which will let you easily configure languages and the jQuery Update module's 6.x-2.x latest development release (alpha 1 might also work).
  2. Enable the Localization server and the Localization community UI modules. These will enable Locale and jQuery Update as dependencies. To make it easier to set up plural formulas for your languages, I suggest to enable the plural formula configurator as well.
  3. Add any number of languages you'd like to support via Administration » Site configuration » Languages. With the plural configurator, you'll be able to set up plural formulas for languages (the module gives you examples of most common languages and sets up defaults for languages it knows already).
  4. There are two connector modules included with the localization server. Enable the Localization server for Gettext files connector. This will let you work with gettext .po(t) files as sources.

Ok, now that we have all the modules set up, configure them. A module can provide multiple connectors with possible configuration, so l10n_server requires you to select which connector formats you'd like enabled. In this case, it is just one connector without configuration, so just mark it enabled. This will let you add projects which use this connector.

To start off translating a project, the server needs to know of that project. since we are using a "by file upload" connector, we can/need to create the project manually. Go to the localization server projects administration page and click on the "Add project" tab. Fill in the short name for the project (which will appear in URLs), the name, and possibly the home link. You'll have the gettext file handler preselected already.

Now that you have a project to work with, you can start adding releases. Projects can later be removed or disabled (temporarily or indefinitely). It is also possible to start over (which will remove all data except the main identifiers of the project that we just entered above).

Click the releases link and you'll see we have no releases for this project yet. Well, we need to add one at least. So go to adding a new release, add the version number of your release you are about to provide translation for and possibly a download link for this release. Like the project home link, this will only be used to show to users, not to actually download anything. Upload a .po or .pot formatted source file with text you have translatable in your software. I've made up a very quick three-piece mysoftware.po for this example. Note that the server will not deal with the .po headers in the file, so I just omitted them for this example:

msgid "My software"
msgstr ""
msgid "Welcome to My software"
msgstr ""
msgid "Download translations"
msgstr ""

How you generate .po files from your software depends totally on your API. PHP itself supports the gettext standard function naming conventions, so you can use the standard gettext tools. You can of course translate software written in whatever language with this tool, since .po files are just plain text transporters for string pairs. If you have a list of translatable strings in any format, generating files in the above demonstrated .po format should not be hard.

The server should inform you that all strings are now saved as sources for translation. Similarly to projects, you are able to start over releases (which will delete all related source strings and let you reimport an updated .po file) or delete the release completely.

But instead of destructing what we've already achieved, let's see the translation UI finally. If you go the Translate menu item from the navigation menu, you'll see a quick overview of our one project with one release with 3 translatable strings. I've set up Hungarian as the only language here, but if you have more, you'll see a selection of languages as well. This screen can be spiced up with an instant status report of the translation status, if you go to your general localization server settings and pick mysoftware as the highlighted project. A nice status report with progress bars for each language will show up on the main page.

Once you actualy go to the translation tab for a language, you'll see the list of strings to translate with fields to add new translations. This user interface will allow you to copy source strings and edit translations. The filter on the top of the page can be used to focus on certain sets of translatable strings and is very useful to track down contributions by individuals.

From here, you can use the user interface to translate and collaborate on your software UI. As experience showed this can even be used to take suggestions for changes in original English user interface text. I'd only like to take one last chance to highlight a neat feature of the server which lets you see and compare all outstanding translation suggestions for strings, quickly assessing the difference between them and deciding which one fits with your translation guidelines or style. This fancy live diff works as you type in a new suggestion for a string and also works for all existing suggestions.

There is a lot more to this user interface and the localization server has lots of other cool features. You can connect the server to the Drupal Organic groups module with the Localization Groups sub-module and create separate translation groups for languages with controlled permissions on the group level. You can enable the remote submission API module to allow for remote translation submissions (but as with the Gallery project, you'll need to work on a custom client for this if your software is not Drupal based, because we only have the localization client implementing this API on the client side now). You can track top contributors in the language teams and export the translations as .po files for distribution is whatever form.

These tools proved very useful on localize.drupal.org, and making them useful for the general audience should hopefully help get more of you on board in using and improving it. We have some exciting things on the horizon including actions integration (where stuff happening in the translation flow will inject short status reports to a status feed) and integration with automated translation tools (eg. Google translate) as well as more sophisticated translation memory (TM) tools. There are still missing pieces in our tools, the Drupal-sepcific export options will still show up when you use the gettext connector only for example, but we are getting pretty close.

Your feedback welcome!

Tags: 

Comments

zephyrus's picture

Voting on suggestions

Hi Gábor,
first of all, big thanks for this project. It's going to save me a lot of work :-). I was looking for a crowdsourced translation tool for our company's website and Drupal Localization Server fulfills almost all our needs. There's just one thing -- we want our users to be able to vote on translation suggestions. It should simplify and speed up the approval process.

There apparently isn't a module for that, is it? After a quick look at the code, I suppose I'll have to modify the server itself. Is there any work in progress on this?

Bryan Klein's picture

Organic Groups for Different Languages

"You can connect the server to the Drupal Organic groups module with the Localization Groups sub-module and create separate translation groups for languages with controlled permissions on the group level. "

Can you provide some details for how this should be done?

I would like each language to have a group, and they can only see and contribute to translations in that language group. Within the group I want two roles, those who can suggest translations and those who can administer translations within the language group.

Guidance on how to connect the dots would be hugely appreciated.

Thank you for your time and this excellent project.
-Bryan

Adam Gerthel's picture

Is it possible to use this

Is it possible to use this for custom modules (both Features and custom site-specific modules) as well?

This is what I'm contemplating:

Extracting po-files from a module (a feature for example) using potx, uploading the .po file to our own i18n_server, translate all strings, and have l10n_update get the .po file from our own i18n_server. Basically the exact same localize.drupal.org works, but for our custom (and site/project specific) modules.

Adam Gerthel's picture

Thank you for the quick reply

Thank you for the quick reply. I'm trying out the server module right now. How are the text groups assigned in .po-files or does it not matter? Since you can now import to, and export from, a certain text group. There doesn't seem to be any relation to this in .po-files?

For example:

Let's say that a module creates a content type with a field upon installation, and the module has been translated to other languages. How does Drupal know in which text group to put the translation? Field labels (for example) aren't translatable unless you install the Field Translation module.

Adam Gerthel's picture

But a single module only uses

But a single module only uses a single .po file. Is the textgroup information stored in the comment for each string?

If so, what happens if you manually import a module's .po file through the UI? You have to specify a single textgroup. If a .po file contains translations for several textgroups wouldn't that clash?

Tim's picture

Unique .po file for a project release ?

Thank you for this module ! I'm really interested in using it. I made some tests with it and it seems that when I create a new project release, I can only upload one unique .po file with "Gettext files connector". Is this correct ? Because I have translations split in different .po files which can be used by different modules. Thanks.

Eugene's picture

Use connectors for custom export

Thank you for your hard work and excellent module!
Have a question about using own connectors to proccessing non-Drupal and non-Gettext files. Could not find anywhere information, how to implement in own connector not only the import feature, but also the export functions (for upload files in the original format)? Or export of data is possible only in the Gettext format , no matter what connector is used for project?

Eugene's picture

Thanks again!

Gábor, thank you for supporting information! Make custom export of course not a problem, so I will do. Thanks again for the useful module!

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.