l10n

Dear Drupal interface translators!

Your valuable work helps Drupal to actual world domination, so we try to support you all ways possible to be able to more efficiently organize your time to translate Drupal projects (the Drupal core system itself, as well as contributed modules, themes and install profiles).

There are big changes planned and in development for Drupal project translations. Make sure to read my Drupal Groups post if you are a Drupal translator or you would like to become one, but the current toolset scares you.

I have been to REMIX 07 today, which was basically a rehash of some of the Microsoft MIX conference topics and presentations for a road show stop in Budapest, Hungary. Although I use much less Microsoft technology then I actually go to the conferences, I mostly enjoy going because it inspires me. I see cool new stuff which of course escalates into cool new stuff I would like to implement with my tool set.

One of the simple ideas that occurred to me today was about in-place interface translation editing. It is so common a request to ask for in-place editing tools for translations in Drupal. Just recently, Boris Mann posted a pointer to SLS, which aims to be a more generic solution for the problem (although I commented there why I don't think it fits Drupal well). Unfortunately we only know in t() that we are working with a localized string, so we could print a span or div with metadata to allow some jQuery to tap in and allow translation editing. Unfortunately in t(), we don't know whether we deal with output for email, SMS, watchdog logs and so on, where the extra div or span would look very unprofessional (in a text medium!). So that rules out the PHP way we could add an in-place translation widget (forget about introducing another wrapper, t() is wrapped deep enough).

This problem is so much on people's mind that Konstantin Käfer did include it in his Google Summer of Code 2006 proposal (but unfortunately did not get around to find a way to deal with it). [Unfortunately, I only found a broken link to that proposal.]

So what can we do? The locale() function in PHP stores a cache of all the text used for translation on the page. So the basic idea is to allow that function to return that cache once the page is done. Then a contributed module could request that in the page footer (once all page content and blocks are done), and export that in JSON. A simple block can be provided to give the user some overview of the strings on the page (translated, untranslated), as well as allowing the user to search for a string used on the page to fix translation for (an autocomplete field would suffice here).

So I thought this is all simple and dandy. Let $string be NULL in locale(), and if so, return the cache. This sound all good, but the cache includes all "short strings" for quick lookup, used on the page or not... Either we need some hidden setting to bypass that if using "in-place interface translation" or we need to track what was actually used from that cache. Any good ideas?

The Hungarian Drupal interface translation team used to use a private Subversion repository to store translations. Our reason for that was that we initially had many people contributing, and it seemed to be difficult to apply for CVS accounts for each of them. It also happened that we had some of our own tools developed and used. Now there is not that many contributors and many of our tools are already migrated to Drupal.org (and the others can be migrated too), so we are moving to Drupal.org. This will most importantly be better for our users, so they can find Hungarian translations in the tarballs downloaded from Drupal.org, without browsing through our own translation repository.

The 'problem' with moving to Drupal.org is that it is quite hard to have an overview of what is happening with Hungarian translations. Although there is a Hungarian translation project, that only hosts the Drupal core files. Module and theme translations are under those module and theme projects. Although there is a move to have these module translations as their own projects (which would result in an explosion of projects there), until that is done, we would still need an overview of what is happening around Hungarian translations.

Yahoo! Pipes to the rescue! We need to watch CVS commit messages, but these commit messages are from all kinds of projects, so we need to watch the general CVS commit page. The patterns we need to watch for are translations/$language, /$language.po and .$language.po. Note, that the last two are delimited at the start so that substring matches are not possible.

I needed to formulate these into Yahoo! Pipes objects and publish the pipe. The language code needed to be a user specified value, so that any translation team can use this pipe. The search strings needed to be built dynamically as a result of this, and the CVS RSS feed was filtered with these patterns. The result is the Drupal translation commits for a given language code pipe.

Finally the only problem with this pipe is that this works 'live' on the given feed, and historical information is not kept. To have an overview of what happened in the past, I have added the resulting RSS feed to the aggregator at drupal.hu, which stores historical data of our commits. A possible problem here is that the refresh interval can only be set as short as 15 minutes, which could be too long, given the frequency of commits at drupal.org. If commits run out of the ten commits long window showed in the RSS feed, we don't see them in Yahoo! Pipes and as a result, we don't aggregate them at drupal.hu. Thankfully aggregator.module can be form_alter()-ed, so we can set a shorter interval if need be.

For now however, we monitor how our new pipe works, and encourage other translation teams to get an overview of their work this way (unless they know something better in which we would be interested too).

Drupal 5 comes out with a nifty new feature (among a lot of others): it only creates database tables and imports CSS files for modules turned on. It is a logical step to do the same with interface translation files. The practice up to Drupal 4.7 was to generate smaller translation template files for translators, so they can better work with strings and collaborate with version tracking tools. These smaller files were merged into one big translation file, which was given to end users to import if they needed the Drupal package work in their language. What should be the new model, and how do we support it? Do I have a working (starter) solution? Yes. Read on!