Drupal 8 multilingual tidbits 14: intro to content and configuration

Up to date as of October 16th, 2015.

The first eight parts of this extensive series explained several great improvements in the base language system in Drupal 8, then we had five articles explaining all the great new software translation features. We are far from done yet! We still have two major categories of improvements to cover, namely content and configuration language and translation. Before we move on to the details, I wanted to post an introduction because there are very similar motivations and guiding principles around the improvements we made and this is going to make your life way easier compared to Drupal 7!

The glue module maze in Drupal 7

Drupal core in Drupal 7 provides translation capabilities for nodes, but no other content or configuration. Want to translate your blocks or taxonomy terms or menus? No luck. Unfortunately the base system does not understand or support multilingual content and configuration for these, so the rest of the modules cannot depend on that or know what to assume.

In Drupal 7 taxonomy terms are entities, but you have two choices to translate them. Either use the entity_translation contributed module (and the title module) or the i18n module suite. Both attempt to augment the system from the outside to apply their translation scheme on taxonomy terms, and other modules are not really aware of this. Display caching or taxonomy term lookup or autocomplete would not be naturally aware that there may be language concerns involved.

In Drupal 7 menus or blocks are not even entities, so they map to the generic i18n_string service provided by i18n module with their own respective mapping modules (i18n_menu and i18n_block).

Site settings like your site name or user account emails are not exempt from the need for glue modules. The variable suite of base modules are needed to provide the generic API that i18n_variable module will use to provide translation for these variables.

Once you get into contrib land, eg. translating views or webforms, yet more glue modules are needed. For Views, you need i18nviews, for webform, you need webform_localization (note the lack of consistent naming scheme).

These modules provide different levels of integrations and varying user interfaces to support your translation needs. The combination of the original modules as well as i18n, title and variable and the glue modules for each contrib module to make them work with this system ends up in a pretty sizable solution. It works in the end, but this is not what we aim for in Drupal 8.

Drupal 8 has general base solutions in core

Instead in Drupal 8 the configuration and content system is (almost) all encompassing, so you mostly either work with a configuration (entity) or a content (entity). Content entities all have fields. They at least have base fields, but most support configurable fields, so you can customize fields on blocks or taxonomy terms. Both base fields and configurable fields come with native support for multilingual storage, editing and rendering. When you deal with a node or custom block, you can equally have the fields set translatable.

This was made possible by the incredible effort of the configuration management initiative to get a generic configuration system in core and apply it to almost everything that cannot be considered content combined with the huge work of the entity system maintainers and application of the content entity system to much wider systems in core.

When common base systems are used, we don't need specialized mapper modules (and a separate system to map to) to maintain translation information and even contributed modules will be aware of the possibility of language variance and how to deal with it. The common base systems are also great when you need to write code against contact categories or block placements, you can use the same APIs and expect identical behavior.

Best of all, contributed modules applying the same base systems will not stand alone, they would use best practices built in core and work automatically with translations for configuration and content. No need for specialized systems and glue modules!

The extent of multilingual support for content and config

You'll read a lot more about these details in later tidbits. If you don't understand these, that is normal, just wanted to give a quick summary and heads up for what is coming.

The configuration system has full support for multilingual override values, so all elements stored in configuration may have language variants. There is also a contextual access system for configuration, so you can load values with specific language overrides, etc. These overrides are stored with the configuration and are fully deployment friendly as well. Default (shipped) configuration is translatable with the interface/software translation system. Such as shipped Views, content types, fields, etc. This works for all your in-house modules as well as contributed modules. Finally, there is a configuration translation user interface in core that provides a translation user interface almost fully automatically for your configuration (given the developer provided configuration schemas). Just covering these details will take up several tidbits so hold on for the details!

The content entity system also became really impressive. With several elements converted to content entities (menu items, contact forms, custom blocks, etc.) and fieldability available almost universally, storing multilingual content with content entities is natural. Fields provide native multilingual storage and core has a new content translation module that serves a user interface for this functionality applying to all content entity types.

So what is content and configuration then?

I made the following figure to illustrate how the entity system as well as configuration and content relate.

Basically in Drupal 8, there are some configuration elements that are global, but most that you will encounter are itemized and therefore use the configuration entity system. Then almost everything else is based off of the content system. The good news is multilingual coverage is going to be fully available for all of the configuration pieces and content elements. So long as your contributed module or custom code is outside of these APIs, you'll be on your own for multilingual coverage. (Core has path aliases for example which have a custom implementation, but that retains multilingual coverage as a custom implementation.)

Embrace the improvements

For your multilingual needs, the emergence of a generally used entity/field system and a unified configuration system are a bliss. To build new code against these will require less learning and more widely accessible knowledge, and to support foreign language and multilingual sites will be natural due to the base systems doing the heavy lifting. This is just not possible with Drupal 7 however hard we try.

Read on with the upcoming tidbits for more details and insights!

Switch - how to change things when change is hard - for open source contributors

This book has been sitting on my desk for so long precisely so I make sure I share my enthusiasm for it with more of you. I've been suggesting this book to several friends and so far the feedback from them was also overwhelmingly positive.

While Chip and Dan Heath do not deal much with software development and absolutely not open source in this book, I'd boldly say this is one of the greatest books I've read for those who want to get something done in an open source development environment. There are countless great stories in this book from all areas of life. From getting children with cancer to take their pills through saving species to attracting more customers to your carwash, it deals with situations when you seemingly don't have any directing power over the change you want to see in behavior/direction from your peers. You are not a boss, you don't pay these people, maybe you don't even know who they are, but you want to see a positive change happen.

Check out this video for example with a short summary of one of the tips that feels very relevant to the state of Drupal 8:

The authors built the structure of this book around a metaphor from University of Virginia psychologist Jonathan Haidt, saying our emotional side is an Elephant and our rational side is its Rider. The whole book is structured around tips to direct the rider (eg. provide specific guidance for the critical moves), motivate the elephant (eg. shrink the change or show how its already underway) and shape the path (eg. build habits to get there in a natural way).

I'd highly recommend Switch - how to change things when change is hard to anybody who want to get something done in an open source community and would not plan to go it alone.

Drupal 8 multilingual tidbits 13: much improved software translation UI

Up to date as of October 16th, 2015.

Although the basic underlying building block of the interface translation system in Drupal 8 that is used to translate the software itself uses the same concepts as in Drupal 7 and before, most user facing features changed. We covered how it downloads translations automatically now from the community and how you can deviate from the source with customised translations as needed. We also discussed how English can be a translation target as well. We did not yet look at the user interface to actually make these customisations on your site, but you can bet that was improved a great deal too!

In Drupal 7 and before, the built-in user interface to translate Drupal is pretty awkward. First you need to go through an intermediary page that shows you the list of languages with their status, then the translation page itself is a filtered list with source strings and status information.

You cannot actually translate anything on this screen, you need to go in and edit translations for particular strings one by one. Then you get a list of big textareas for each language. If your target language is down on the page, you need to scroll down so much you cannot see the source string anymore.

We believe this model of translating one string to multiple languages at once is not very realistic. The more common workflow instead is for one translator to go through multiple strings for one language.

So what did we do in Drupal 8 about this? First of all, we removed the intermediary page. If you are interested in translation status for your software translations, that is now built into the main language list table (screenshot in previous tidbit). The separate intermediary page is gone. Second, we made the source string table much more useful by making the target string editable for one specific language on the form. This makes us loose the overview of which languages each string is translated to, but is way quicker for the actual translation job. Finally, we also integrated singular/plural translation support (and this of course works with languages with any number of plural variants as well). In Drupal 7, the translation interface cannot even display the singular/plural pairings due to how the data is stored in the database.

In summary, we cut out two unnecessary steps and brought in a very quick translation experience instead which even shows you which translations you changed as you go along (depicted with asterisk and yellow colouring). If you did not even know about this translation user interface in Drupal 7, there is a very good chance that was due to how hard it was to use. Drupal 8 gets you a much better solution in core! And finally, with good Drupal 8 tradition this administration interface is also responsive (adapts to varying screen sizes, even small mobile screens) and is also accessible.

Issues to work on

  • All strings entered on this user interface are now recorded as customised translations. Maybe you want to fix missing strings in community translations without needing to protect those additions from possibly better updates from the community later. Discuss this in https://drupal.org/node/2069867

Drupal 8 multilingual tidbits 12: English can now be translated to

Up to date as of October 16th, 2015.

There are several improvements around English in Drupal 8. As we have seen in tidbit #3, English is now optional. You don't need to have it configured. If you do have it configured, it used to be a big problem however to handle it as a "translatable language". That is, to do text replacement in English text to satisfy your project requirements. For example where Drupal says "Log in" and "Log out", clients may request to see "Sign in" and "Sign out".

Using Drupal core alone, this was possible to achieve in Drupal 7 by configuring a "Custom English" language on the site with a language code different from "en", which then allowed you to replace text with your choosing. This quickly gets confusing though because then both the old "en" English language and your new "Custom English" will show up in language selectors, etc. If you disable the "en" English, then your old content saved with that langcode will not display properly. Its a whole can of worms. You may also use String overrides, but if you are using interface translation from core anyway, using one more module for the same task for a specific language is overkill.

Drupal 8 provides a simple and effective solution for this. By default, if you have English configured, it is not available as a translation target. However, you are only one checkbox away from enabling that feature.

Go edit English and make it available for translation. This will make English behave as any other language and from then on, you can "translate" to English as well.

All strings will still fall back on the built-in English text (eg. if you also have Hungarian on the site, untranslated strings in Hungarian will fall back on Drupal built-in English text). However when English is used for display, all your overrides will be effective. (Cross-language fallbacks can be implemented in contributed modules now with Drupal 8's service architecture for string lookups).

Issues to work on

  • DONE! Even if you don't have English configured on your site, if you go edit configuration that has a language selector, it should not suddenly switch language. English needs to be present (and selected by default) in these cases. That is not currently the case. See and help with http://drupal.org/node/1936216 as you can.
  • DONE! With good tradition, I also found a (minor UI) bug while writing this tidbit. Submitted that at https://drupal.org/node/2069271, hopefully the proposed fix will quickly land.

Drupal 8 multilingual tidbits 11: string customizations tracked

Up to date as of October 16th, 2015.

From previous tidbits, we know the Localization update module is now built-in in Drupal 8 and it is included in a deployment friendly way. We also covered the new context specific text translation APIs. There are a lot more user interface related changes and new functionality included, so let's dive into one of the more interesting ones!

We discussed how Drupal now downloads translations and updates your software translations from the Drupal community (thanks to the translation teams on https://localize.drupal.org/). However we know that not everybody is happy with the community translations proper. So another key feature we included in Drupal 8 is custom translation tracking. How does that work?

When you add or change a translation, we keep track that you did it for yourself, and when you import a translation file in the Gettext .po format (which is still possible manually), you can also specify if the translations in the file are your customisations or from the community. Although we do not store multiple translations of the same string, we keep track on each string whether it was provided locally or fetched from the community.

Why is this useful? Well, first of all, we can protect your translations from overwrites from the community. Now that translations can be automatically updated, your customisations would get lost if not properly tracked and protected. Automated updates protect your customisations using this tracking system. Second, you can identify these strings and work with them specifically. The translation interface includes filtering for customised translations and you can export only your customisations as well (see above), so you can reuse the same customisations on other sites. Nice, isn't it?

Issues to work on

Nothing I know at this time. Feel free to point out issues in the comments.