Drupal 7's new multilingual systems (part 4) - Node translation

In the second part of my article series, before we got on a developer detour, we discussed that Drupal's software interface translation can be pre-provided and collaborated on by the community, but this time we turn to your own content. What's considered content on a Drupal site? Well, in a broad sense, anything that you enter beyond the software user interface translation. For this article, we will limit our discussion to nodes only, and move on to the rest of the structure and page building elements in later pieces.

Language-enabling nodes

Once you enable the built-in Locale module, an interesting small functionality is made available that we did not talk about yet. When you go to edit any content type (at Administration » Structure » Content types), you'll find a Mutilingual support selection widget on the (probably not so obvious) Publishing options vertical fieldset item. This setting really relates to submission options, not publishing options, but that is where it is now (see the related core issue).

But what does enabling multilingual support on a content type means? Well, you'll be able to assign a language to each node for enabled node types. This is useful if you'd like to run a personal multilingual blog for example, where you ocassionaly post in alternate languages, but you have no intention of translating the posts to different languages, just mark them as such.

Once you language enable a content type, a language selection dropdown will appear on node submission. This will include all enabled languages and a special item called Language neutral (which is made the default). The intent of this option is to let you submit nodes without relation to specific languages. Think of nodes you use for images for example, like photos of landscapes that belong to a category of taxonomy terms. The listing or explanations for the gallery would contain translatable text, but the photos could be just the pictures themselves, without translatable content, so you'd use language neutral nodes. (Using entire nodes for image galleries might not be best practice anymore, see the Media project that works with custom entity types).

Language neutral uses the language code 'und', which means undefined and is specified to be used for such cases by the ISO 639 standard. You'll see this language code in Drupal source code all around, so good to know.

Once you language enable a content type, conceptually you should be able to use Views to limit listings per language for example. Drupal core node listings are not language-aware, so merely enabling language support will not limit your front page content listings to certain language nodes only as you switch languages. Unfortunately Views will not let you filter for language in this stage either due to a bug that it bundles the language field filter with node translation. Once you have node translation (see below), views will provide language filtering options.

Translation-enabling nodes

Drupal core carries on the built-in functionality from Drupal 6 to handle translations for nodes in basic ways. Once you enable the core bundled Content translation module on your site, you'll be able to go back to Administration » Structure » Content types and choose Enabled, with translation for multilingual support.

This will in itself not change anything on the node submission user interface. However, once you submit a node in a specific language with that type, a Translation tab will appear on the node edit form. This tab will let you get an overview of all languages on the site, and whether translations of the node are available in those. You'll also be able to add translations for the node or edit any translation. When you choose to translate the base node into a different language, some default fields and the language field will be prefilled for you.

When you edit the original node afterwards, you'll find a new Flag translations as outdated option, that will mark all translations as outdated. This is to be used when substantial editing is made to the base node. This is used by core and contributed modules to highlight outdated translations and help content providers fix up their translations. When a translation is updated the This translation needs to be updated checkbox on the translation should be unchecked. This flag allows translations to be edited without the requirement to also incorporate all source updates at once as well (which would be the case, if we'd only be able to compare last update timestamps).

The built in node translation does not do much else. It also alters the language switcher block links to point to different nodes when switching languages where translated variants are available and it displays links by the nodes themselves for other language versions. However, there were a whole set of modules built around this model in Drupal 6 including the nifty Translation overview module that is still being worked on for Drupal 7 at the time of this writing.

It is important to understand the concept behind this translation feature. The built-in translation module forms translation sets of nodes, where each set has a base node that is translated to the other languages. So the general idea is that you submit nodes in the source language first and translations come afterwards. Translation set relations are very simply managed in the database, but I have not yet seen contributed modules to expose advanced editing of this data (such as assigning existing nodes to existing translation sets, or switching to a different base node for the translation set). It should be fairly easy to write such a module though. In summary, the general concept is that translation sets are formed on top of nodes.

Key advtanges of this approach include well built support for node level content handling all across Drupal core and contrib. The node listing in Drupal allows for language filtering, Views has built-in support, rich node level access checking can be employed to limit permissions on translations, workflows and rules can be set up for nodes, and so on.

Key disadvantages include the above mentioned missing support for advanced translation set handling (which could be implemented in contributed modules) and that sharing data between nodes is cumbersome (and requires more contributed modules). Think of an image gallery again, but this time with captions. The images should be the same, but translations should be made available for titles, captions.

Translation-enabling fields

With the effort to try to solve some shortcomings of node level language support, Drupal 7 now supports languages on the field level! With a huge set of CCK functionality now built into Drupal core, many things on nodes, users, taxonomy terms, comemnts, etc. are fields. Drupal core has built-in support for field languages, you'll find the body field value in code via $node->body['und'][0]['value'] or for custom fields such as $node->field_name['und'][0]['value'] (for nodes in undefined languages). Field API lets developers to mark translatable fields, as is done for the default body field, however, no built-in user interface is provided for translatable fields.

Let's compare the concept of field translation to node translation first! As explained above, node translation sets up a set of nodes as a translation set, so all node level tools can be reused for individual translations. Field translation pushes translations inside the node, which allows for handling the whole set including all translations as one single node, and is more applicable when our use case calls for that approach. However, not all pieces of node information are fields.

To provide a user interface on top of field translation, the contributed Translation module suite was built. The suite includes a translation_node.module which replicates the core functionality, and a translation_upgrade.module providing a migration path from node translations to field translations. In a possibly confusing way, instead of leaving the core translation module alone, the base translation.module in the suite replaces the core translation module once enabled (due to using the same filename). That also means you need to run update.php (if you ran core translation.module before) to actually install the module, because technically there is no new module enabled from Drupal's perspective, it just 'moves' to a different place. There is a discussion about renaming the base module.

Now a separate content language detection option appears in Administration » Configuration » Regional and language » Languages, that you can just set to Interface for now, so it uses the same language as the interface. (I'd also suggest you use the URL method for interface as a start, and experiment with the rest later).

You'll also find new options titled Content translation settings under Administration » Configuration » Regional and language. You'll be able to translation-enable certain entity types in Drupal core here. Unlike Drupal 6's CCK, Drupal 7 comes with fields support for beyond nodes, and instead defines a generic base concept called entities, of which nodes are just one type of. We'll limit our discussion to nodes here, so just enable translation for nodes for now.

Remember, above I said Drupal core translation module lets you choose Enabled, with translation for node types. Now, the contributed content translation module allows you to choose Enabled, with content translation and Enabled, with node translation (if the translation_node.module is enabled). Pick the former. Once you content-translation enable a content type, fields will have a Users may translate this field switch, which lets you specify field translation per field. This is the major value-add of the module, you can cherry-pick the fields you'll need to have different in translations.

For a configured content type, adding a base node looks like as before, even the translation tab on the node will look familiar, but once you go to add or edit a translation, you are not actually submitting another node. The translatable fields of the node will be displayed but not most of the other values. The module lets you assign a separate URL alias for the translation (Drupal supports different URL aliases for the same path per language), but you'll not be able to edit the author of the translation, provide a submission date or do other administrative editing on the translation.

Once you have translations submitted, the language switcher block can be used to switch between the translations. If you chose to have different criteria for content and interface language, both will have their respective blocks. You'll notice that your node URL keeps being node/X, but the language of the node displayed will change based on the identified content language.

I'd explain the concept behind this method of translating content is that it creates kind of "language based subentities" under the node entity type. There are "the Hungarian translation of node 8" and "the German translation of node 8" all under node 8. Some non-field properties will be possible to make different in translations because they have specific implementations in the module, but not all.

There are other limitations and characteristics of the module as implemented currently which might change / improve over time. It does not yet support revisioning of translations. Because translations are second class citizens here, when you view the translated node, hitting the edit tab goes to edit the original node, not the translation.

The module keeps track of the submitter user, creation date, last update date, publication status, etc. of the translations in its own database table, and the original node retains the based node values of course. I looked for but could not find Views integration for these translations. I think Views possibly needs to have an entire "translatable node" type to know about, which takes into account the language code and the node number as a compound identifier when making listings, so it could do lists with the individual translations. Currently I could not find ways to list different translations of the same node in a view.

Nodes also have a pretty strong toolset for permissions, which fields are far from matching. Contributed modules as well as custom code can be used to fine-tune node level access for viewing, editing and deleting, but the same is not true for fields (yet?). For nodes, workflows, rules exist, so you can set up different translation nodes in different translation states and fire actions on transitions. These again need to take into account that node identifiers plus the language codes form a compound identifier and to be considered separate to achieve the level of feature set the node level translations offer.

Finally there are also relations of nodes to other things which are neither fields, neither duplicated by translation module. There is some special support for displaying only comments in the same language as the content language but the last comment timestamp is maintained as one single value for all languages. Same holds for menu items. The node form has means to assign the node to a menu item, but given this is a single node, it only has one menu item relation, so translations need to participate in the same menu too.

My understanding is that lots of tools need to be re-imagined to work with language variants under nodes, versus languages assigned to nodes. And it affects functionality all across the contributed module space. Many tools were made translation-set friendly based on the Drupal 6 translation feature set earlier, and translatable fields turns the model upside-down to introduce a completely different methodology to which they still need to adapt. And once/if they fully adapt, translated versions of nodes will need to replicate all of node's functionality to compete with the feature set, so they could have been nodes to begin with.

Keep the limitations of the two systems in mind when choosing your way of doing translations on your new Drupal 7 sites, and keep an eye on updates to tools to support either method better. To recap: node level translation comes with hard to share fields and hard to manage translation sets and field translation comes with missing functionality across existing common tools.

Third option: shared fields would be the ultimate solution?

My understanding is to take advantage of all existing tools (views, access checking, workflows, rules, etc), it would make sense to work on shared field instances, so that we can use the listing, access, workflow, editing, etc tools, and just share the data on the field level between entities for the types of fields we'd like to share data, instead of trying to go the (as far as I see) entirely too separate node translation way or the (in my view) way too integrated field translation method that is still in its infancy.

I do not have resources to work on this unfortunately. I wish I would have.

Site settings and layout up next

Huh, this was a lengthy piece wasn't it? I hope it provided you with good insight into where things stand with node translation. I'm sorry there is no golden answer yet. As we move forward there are more and more moving parts.

In the next piece, I'm planning to look at site settings and layout (blocks and friends). There are lots of interesting things there, I can assure you.

In the meantime, you can still read part 3 and check out part 2 and part 1 of my series.

Tags: 

Comments

plach's picture

Differences

There a couple of subtle but substantial differences: the language switcher has now a better UX as missing translations are showed as disabled links instead of not being present. Moreover, and this is the big news, you can create nodes in disabled languages, this allows to make an entire site translation available at once.

Reinier Maarschall's picture

Shared field instances

I just read all your 4 posts of the Drupal 7 Multilangual system. I must say it clarifies a lot about the new possibilities within Drupal 7.

It is good to see that Drupal 7 introduces the field translation functionality compared to the node translation system that is used in Drupal 6. I think this is a big step forward. Unfortunately it isn't as complete as i hoped it was.

I was catched about your thoughts of shared field instances. Could you explain this concept in a bit more detail? I would be happy to spend some time on resolving a solution like you described.

catch's picture

I started work on this a long

I started work on this a long time ago in this issue - http://drupal.org/node/132075

It's theoretically possible to make a 'translation set' entity in contrib, attach fields to this, then render them in node forms and when viewing, but in practice it is likely to be a lot of work. The patch in that issue is very out of date, but the basic concept there would get things started.

sun's picture

Translatable fields' real purpose

While I can understand your early-adopter warning, and the few examples you gave are correct, you didn't concisely explain the underlying, fundamental idea of translatable fields.

The main purpose of translatable fields is to have a single entity that actually means the same, regardless of language.

An entity should be a canonical resource. If you read node/8 in English, then someone who reads node/8 in German should read the same. Because it is the same resource.

Now, let's bend your mind (and the spoon) a bit, and think Drupal 7.

You absolutely should be able to just point everyone to your blog post, image, news article, video, page, user account, taxonomy term, comment, book page, or perhaps even e-commerce product, or any other possible entity by linking to the location of the resource. Whether the resource is going to be displayed in an enforced language depends on whether your link already contains language information or not. If it does not, Drupal's language system will automatically negotiate the most appropriate language for every visitor who is following your link.

In an emerging world in which links to unique resources are playing increasingly important roles (RDFa, FOAF, etc), this fundamental concept is key for integration and enhancement.

The shared field instances suggestion still follows the obsolete idea that you'd need a separate resource in order to provide the resource in different languages. This is wrong, because the problem space boils down to data selection and display. Duplicating the resource and hence, its storage is fundamentally the wrong point of attacking the problem space.

However, thanks for this great blog post series! :)

plach's picture

Relationships

Sun's point is strong and cannot be addressed by simply saying "it's a matter of definitions". URLs are resource identifiers and different nodes have different URLs (leaving aliases alone).

IMO this is a key factor to keep in mind when comparing the two translation models: in-entity translation actually provides a different version of the same resource for each language, this introduces a strict relationship between translations; node translation instead provides a looser relationship between two distinct resources.

In your example of a news item appearing in the english front page but not in the french one, IMO we are dealing with two distinct resources living their own distinct lives. I'd like to cite another example: think of an international e-commerce site selling in US, Canada, UK and France; we might have a product (a single resource) which is available only on the american/canadian market and a product variant (another resource), differing, say, by an electric adapter, available only on the british/french market. Both resources need to be translated, by they also need a relationship because most of their textual content is the same and so translators might simply want to reuse almost all their work.

IMO this is a clear use case of both approaches coexisting, but I think this also points out, that this kind of loose relationship is something that perhaps goes beyond the "translation" meaning, in fact we might have that the two products were originally created in english (en_US and en_GB), but they would be related anyway.

I don't know how much what I'm going to propose is feasible, but perhaps we should try to introduce a solid entity relationship API helping us to deal with different, related, resources ("I want a language switcher that makes me navigate between different articles describing the same event in different languages") and instead focus on making in-entity translation workflow fully functional and matching what can be now accomplished with node translation.

plach's picture

Makes sense

Ok, your point makes sense and I stand corrected about URLs, but what about the e-commerce example I cited? Don't you think a generalization of the relationship concept would make sense too?

Obviously I'm thiniking in the long run, since most of the code currently available supports translation sets (and I know how much painful it has been to reach the current level of support).

fietserwin's picture

shared fields

If I understand correctly, shared fields are what you would do with i18nsync (part of i18n) in D6.

For each field (core and CCK) of a node you can set whether it is synchronized for all translations of a node or if it can have a separate value per language specific node. If a field is synchronised and I change it in one language it will be changed in all other languages as well.

Olivier Jacquet's picture

I've never quite understood

I've never quite understood the excitement about translatable fields. Node translation with i18n-sync was quite nice. If we could replace i18n-sync with actual sharing that would be perfect.

bojanz's picture

Great

Great article (or to be more exact, series of articles).
Just wanted to point out something regarding field translation, Views now supports it, and there's a language setting in Query options that does the same thing that the translation filter does for node translations.

Gabriel's picture

This is just a great article!

This is just a great article! I've searched all the web for informations on the system of translations in Drupal 7 and this one is simply the best, covering all the pros and cons of the different methods (Actually I wasn't aware of the diferences).
I hope you publish it on the official Drupal website or anywhere else, so that people can find it better.
Congratulations!

Alex Railean's picture

Struggling to enable content translation

Gábor, thank you for the series of articles; this last article was exactly what I was looking for.

I've never used Drupal, and I set up v7 on a server to create a simple site with 4 pages, each of them being available in 3 different languages.

I've explored the configurations and enabled everything that sounded like a language-related feature; despite that - I still don't see how I can translate a page.

The first screenshot in this article - I don't have that.

Here's what I did:
- enable the 'locale' module
- enable the 'content translation' module
- added the languages via Configuration»Regional and language»Languages
- I am logged on as the admin, so I believe I have the necessary rights to translate content; but just to be sure - I checked the respective options in the permissions for 'authenticated users'

If I go to Administration»Structure»Content types, there is no 'Enabled, with translation for multilingual support'.

I'm using the regular flavour of Drupal 7, downloaded from the site; nothing was patched, tweaked, customized. From my perspective, this is a 100% untouched.

I've double-checked everything, yet I must've done something wrong, since it doesn't work.

Where else should I look?

Alex Railean's picture

Aaah, I did find the option

Aaah, I did find the option now. It is strange - I was using the same brain and the same pair of eyes, but the option became visible only after you read my comment. This must be an example of a "spooky action at a distance" :-)

I admit, it was a user error (you can't trust these idiots :-)

Thank you for the swift response.

jbucks's picture

Node / user references

Thanks for the great set of posts about Drupal's multilingual settings, like other commenters have noted, it's certainly given me an insight into the different options available.

One thing, though; how do translation sets work with node references? If a user of a multilingual site creates a node in one language, and selects another node in a node reference field (say someone chooses an Organization while creating a node of type Department), then the node reference does not seem to apply to the translation set, only to one of the nodes within the set.

What's the best way to handle node references with translation sets to prevent this? Or perhaps the approach where a single node is given fields translations a bit better in this situation?

d6's picture

Currently the best solution

Currently the best solution is keep using Drupal 6 for this usecase. Unfortunatly node/entity reference are under development on the D7 side and the support for i18n is not available... Probably Drupal 8 could be shipped with a stronger internationalization support :-)

One year after's picture

One year after the launch of

One year after the launch of Drupal 7, if you need a working translatable CMS you still have to use Drupal 6 with the help of modules like node_relationships (and maybe with some patches) in order to cover a lot of use cases :-(

Drupal 7 is not ready yet... and probably it never will be. Maybe Drupal 8 could provide a better i18n experience, but we'll have to wait at least 18 months :-(

Pasqualle's picture

und

$node->field_name['und']['0']['value']

vs
$node->field_name[LANGUAGE_NONE][0]['value']

instead of 'und' use the LANGUAGE_NONE constant,
and numbers do not need apostrophes. (it is a zero not a character o, the formatting of the number 0 is "very interesting" on this blog.)

rantebi's picture

Translate selectlist option values in drupal 7

I already posted this on drupal but there was no response, and here seems like the right place for this question.
I'm using the node translation on drupal 7.

I've added some CCK seleclist fields to my site.
They have the following allowed values:

0|Attorney
1|Associate
2|Intern

How can I translate the textual values?
In drupal 6 you had the option to write PHP code for allowed values, so in that way translation was available (For more details see http://drupal.org/node/182884#comment-280925).
But I don't see an equivalent field in drupal 7...
Neither can i find any other issues that solves this problem.

Gábor? Anybody?

Anonymous's picture

I think it is better to write

I think it is better to write a small module to install the fields. Then a function can give you the options for your list.

Example Fewo Module:

File fewo.install

<?php
function fewo_install() {
   
_create_fewo_node_fields();
   
}

function
_create_fewo_node_fields() {
 
$type = 'fewo';
 
 
//fewo_equipment
 
$field = field_info_field('field_fewo_equipment');
 
$instance = field_info_instance('node', 'field_fewo_equipment', $type);
 
  if (empty(
$field)) {
     
$field = array(
     
'field_name' => 'field_fewo_equipment',
       
'translatable' => TRUE,
     
'type' => 'list_integer',
       
'module' => 'list',
       
'cardinality' => -1,
       
'settings' => array(
         
'allowed_values_function' => '_allowed_values_field_fewo_equipment',
        ),
    );
   
field_create_field($field);
  }
 
  if (empty(
$instance)) {
   
$instance = array(
     
'field_name' => 'field_fewo_equipment',
     
'entity_type' => 'node',
     
'bundle' => $type,
     
'label' => t('Equipment'),
     
'required' => 0,
     
'widget' => array(
       
'type' => 'options_buttons',
       
'module' => 'options',
       
'weight' => -21,
      ),
     
'display' => array(
       
'default' => array(
         
'label' => 'hidden',
         
'type' => 'list_default',
        ),
       
'teaser' => array(
         
'type' => 'hidden',
        ),
      ),
    );
   
field_create_instance($instance);
  }
}
?>

Now in the fewo.module file do this:

<?php
function _allowed_values_field_fewo_equipment() {
   
   
$allowed_values = array(
    
1 => t('Dishwasher'),
    
2 => t('Cooker'),
    
3 => t('Oven'),
    
4 => t('Microwave'),
    
5 => t('Toaster'),
    
6 => t('Fridge'),
    
7 => t('Freezer'),
    
8 => t('Holiday Apartment'),
    );
   
    return
$allowed_values;
}
?>

It is better to use a function because you can get your options from different place like a database table.

neil's picture

Teaser Translation

Thanks for these posts Gábor...

Following them I have single node translation for 3 other (than en), languages working well - is it possible to translate the front page teaser content? (no teaser displays for each of the non-en translations unless content language detection is set to Default (en))

many thanks :)

neil's picture

Teaser Translation

Thanks Gabor,

With multiple post teasers it would take a view because, as you indicate, the node's path is language specific and the front page's (ie the default content feed) is not?

So going back from a translated node to Home uses a language path nothing has been published to whereas using taxonomy for example, with language specific terms, the teasers are translated

Aleksa's picture

Enabled?

I only need to mix two different content type with: "Enabled" and "Enabled, with translation". But it's not supported in a good way. I have this options

Multilingual support:
-Enabled
-Enabled, with translation
If enabled, a language selection field will be added to the editing form, allowing you to select from one of the enabled languages. You can also turn on translation for this content type, which lets you have content translated to any of the installed languages.

But whan I "Enabled", to have two Forums, seems evrything is fine, but I create Forum topic, "My great topic" and url is: http://example.com/forum/my-great-topic, but whan I go to secend language, like (german) I get http://.example.com/german/node/1.
But I will like to have only one url. Because, I want to build multilingual forum in sub-categories. This is impossible.
Also, if I like to have publications type, but I want to allow people to add books, but I will not like to translate every single node. Example: If you do not know Chinese and node have abstarct with pdf file version of chinese book, why you will download this book, and why I will tanslate this node to Hungarian. So you will like to find publication in Hungarian with pdf extension, am I right?
"Enabled, with translation" working fine, but this is secend options. "Enabled" This should mean you can have content only in this language and didn't get ugly url:... /node/1???
This is, also, issue of duplicate content in Google, yes?
http://drupal.org/project/translation404 Translation404 improve translations in D6. Little module do great things. It's allow you to tray to change language but give you 404 error, with translated interface, so you can continue seek new content to read in your native language.
So do you have any solutions?
Thank you

Tom's picture

Import/link content translations with nodes

Hi everyone, I need to find solution how to import translated content into nodes from file. I can easly import nodes with Feed Import module but I don't know how to link this nodes with translations.
My english is bad so here you go an example:

Basic article with 'title' and 'body' in Polish.
Now I've got translated basic article in .po or .csv file in English
It's posible to link/import translation?

Thanks for any help.

Bohus's picture

required language

Hello,
I'm using node translation. Is it possible to require e.g. English language version first before enabling translation to the other languages? I wasn't not able to find answer anywhere.
Thanks

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.