Drupal 7

I remember how skeptical I was looking at some presenters traveling around to multiple conferences with "the same" presentation a decade or so ago. Having been a course instructor for years and being a presenter for even longer, it looks completely different now. It's not that the topics you cover under the same looking umbrella can be quite different, you also find much better ways to express whatever you want to tell your audience as you experience feedback.

Of course the best would be to present your story crystal clear from the start, but despite being an enthusiastic follower of Garr Reynolds and Nancy Duarte, you'll undoubtedly need lots of time anyway to take a relaxed look on your story and distill to the level needed to form a great presentation. I've actually found it quite hard to refine my slides without actually showing/presenting them to an audience. The faces, questions, smiles and sometimes plain staring expressions you get tell you how you'd done and you can derive ways of how can you improve.

Two interesting examples are my slides on Drupal 7 and localize.drupal.org.

Things been quite quiet lately here on my personal blog. This does not mean that my days were not action packed though. I was quiet due mainly to being busy with working on all the Drupal 7 usability exceptions and blogging directly on the localize.drupal.org news blog.

swag

It is time to share the love and show these to more people around my region. So I'll go to Prague and show what Drupal 7 has on offer and how can people translate Drupal easier then ever before. With two sessions on the schedule, I'll be busy this weekend to show all the awesomeness going on these areas.

The Prague event still has 70 free (as in beer) seats open as of this writing, so it is not late to register! My colleague Robert Douglass will also be there to present sexy search and Acquia. There will be a pretty heavy Hungarian contingent going presenting about Druplash, Services and Amazon EC2 integration plus our Hungarian in honor, Kristof van Tomme presenting knowledge management integration in Open Atrium.

There are plenty reasons to come, so don't miss out!

Mark Boulton created the new Drupal wordmark as part of the Drupal.org redesign efforts. As highlighted by Jeff Traynor of Raincity Studios, the newly relaunched Drupal.com now uses this new wordmark as identification for Drupal and the wordmark is also destined to land on Drupal.org as part of the redesign (implementation unfortunately still underway).

So why not include the wordmark in shipping software as well? A logical step would be to have it on the installation and update (also known as maintenance screens). The patch for using the new Seven administration theme for installation and updates does just that. Because it is only a first approximation of how this should look, I'd love to get help cleaning it up and polishing things like lining up items, correct font sizes and fixing graphics - so we get a really polished experience for people's first Drupal encounter as well.

Moshe Weitzman recently started the D7CX movement to rally people to get Drupal 7 contributed modules upgraded and released on time for the Drupal 7 release. This would be a great boon to the Drupal 7 release, which shapes up to be a huge improvement over Drupal 6 already. He also suggests a contributed module release manager who can help with (among other things) ensuring that tools are available to help people upgrade.

Me being the guy, who maintains modules important for the translation of Drupal interfaces, such as l10n_client, potx and l10n_server, I looked at this movement from my angle. While my modules are not among the highly popular top 40 modules Moshe highlights, these tools are used to translate them and reach many parts of the globe. So in part on Acquia sponsored time and my free time, I went ahead and ported both the Localization client and Translation template extractor to Drupal 7. Both had their own challenges. The client port is just a direct functionality port, so does not include real string context support yet (a feature new to Drupal 7). The template extractor however got full context support to match what Drupal 7 supports currently. It also got its coder_review integration updated. Both modules now have their 7.x-1.x development snapshots downloadable.

Thanks to these updates, I hope people working on the D7CX movement will not be in the dark about localization API changes and usage limitations of the new API. You can run the template extractor on your code or run the code review from Coder review to get error reports when the localization API was not properly used. While Coder review includes some rules on its own to test for some common errors, only using the actual tool translators use can tell you all the errors you can make while writing your code.

Future plans include backporting the Drupal 7 parsing support from the template extractor to Drupal 6 (which would be easy except a little API change required), so when integrated with the localization server, contexts would be properly stored. Which would require an update to the localization server too. The goal is to have a Localization server system which can be used to translate Drupal 7 core and modules, so translators can work on their part before the release too. Also, the localization client would pass on the context information too to the server, so people can keep using that to translate and share their work right away.

People keen on magic names can just refer to this effort as D7TX, the translator experience ;) Have fun using these tools, and as always report bugs and provide patches please!

Those not following the implementation of concepts from http://d7ux.org/ here is a quick summary of the three main areas:

  • The header toolbar from http://www.d7ux.org/header/ is in Drupal 7 already (in the form of toolbar module). Icons are missing (still to be discussed, see http://drupal.org/node/13911#comment-1701664) and customizable shortcuts (http://drupal.org/node/511286) are still under discussion. You can override, revoke permissions and disable the header if you don't want to use/see it, but it is enabled in the default profile by design. There are also some other smaller follow up issues you can find and contribute to from http://drupal.org/node/484820
  • We are hard at work on the overlay. You should be able to use it without the admin theme and toolbar (or with a different admin theme and/or toolbar), but it is aware of the admin toolbar. There seems to be mostly some positioning issues to overcome here, see http://drupal.org/node/517688
  • We are working on the admin theme as well, which Young Hahn christened the Slate theme. The looks of the theme when not in the overlay is still to be refined, but themer reviews are showing good results. Again, this would be the default admin theme for Drupal 7 but will be configurable. See http://drupal.org/node/484860

There are lots of things going on improving user experience even in relatively large scales additionally to these three, but given how close the code freeze is, I am trying to get attention to these three, and especially the overlay and the admin theme, so we can get them in sooner then later.

By now, the Drupal contributed repository grew so big that there are likely multiple solutions to the same problems there. Maybe these solutions solve the tasks with different angles, have slightly different features, but at their core, they aim at solving the same issue. So it is getting harder for site builders to pick the modules they want to use (and drupal.org is planned to offer more tools in helping with the selection). When it comes to core inclusion though, one needs to very clearly define the requirements and meet core goals, such as lean and mean implementation, maintainability and reusability.

With Mark Boulton's and Leisa Reichelt's suggestion to have most of the Drupal administration interface show up in an overlay on top of the public facing webpage, we are again at such technology crossroads. Drupal 6 at least has two contributed modules to implement a similar-looking solution: Popups API and Modal Frame API. Both got their pros when I researched for possible overlay implementations, and frankly, when there are already two modules implemented for the same complex task, why would we start from scratch?

The goals for the core overlay (purposely not called a modal dialog) is to take up most of the window with a dim background on the original page content. The overlay would have a close button on its right side and possibly tabs on top to switch to different subpages (loaded via Ajax or will already be on the same page, but hidden). We don't need the overlay to be moveable or resizable. We don't need to show multiples at once, since it only makes sense to show one. However, the overlay should work well with the header so that when an option is selected, it keeps being active in the header, and the header should work so one can pick another area to work on without first closing the overlay.

This is how the overlay would look:

With different tabs or without tabs but otherwise consistent look elsewhere:

For easier evaluation of how we can meet these goals, I've ported both above mentioned modules to Drupal 7 and implemented Drupal 7 user experience "skins" for them. In the case of Popups module, it was a skin which the module already supports via an internal API, with Modal Frame API, it was an actual glue module which mapped elements marked up for being displayed in the overlay to a style similar to what Mark and Leisa suggests. Neither of them is pixel perfect to what is to be done, but the initial goal was to have D7 versions to evaluate and discuss the implementation internals, so we can fix up styling once we picked either one or a third way.

You can find my Popups module port patch at http://drupal.org/node/466732#comment-1681554 and the Modal Frame API port patch at http://drupal.org/node/491224#comment-1703366 but to make all these easier to test with the D7UX header in progress, all this code is available with the glue modules and custom skins in the D7UX code repository. You can try out both by installing with the d7ux install profile and then either turning on the "Popups API" module or the "D7UX overlay look" module. Make sure to only have one of them enabled at once and let Drupal install their dependencies. Switch between the two implementations by switching between the two modules.

Overlay implemented via popus module:

Overlay implemented via modal frame API module (the close button is not yet themed to look like on the mockups, but think of that as a minor detail):

Let's go beyond the UI and compare the modules:

  Popups API Modal Frame API
Started December 2007 May 2009
Maintainer starbow: http://drupal.org/user/33290 markus_petrux: http://drupal.org/user/39593
Contents Popups API, popups admin (admin mapper) and popups test modules (for human testing), skins Modal Frame API, Example glue module for testing
Based on Lots of custom code based on jQuery Small amount of custom code based on jQuery UI and jQuery
Codebase Light on PHP code, heavy on JS Light on both PHP and JS code, jQuery UI makes up for it
Overlay rendering Ajax request for a JSON object which contains the rendered content of the page, JS and CSS files to be loaded for the page and Drupal messages Ajax request for a fully rendered HTML page which is themed by the Modal Frame API module to only include the main region and some wrapper code.
Overlay display Merges rendered overlay into main HTML document; merges in CSS and JS files (possible ID collision) iframe element added to the page (no requirement for CSS and JS merging)
Stacked popups Supported, only one visible at a time Not supported
Dirty forms (unsaved edited form warnings) support Supported via custom code Supported via http://drupal.org/project/dirtyforms
Form submission in popups Reloads originating page (via Ajax or full reload) or runs custom callback Overrides form submit redirection and closes overlay

We are obviously at crossroads with picking our ways. The Popups module approach already has numerous core patches which were suggested earlier, while Modal Frame API leverages jQuery UI to lessen custom code burden and iframes to avoid colissions in merging HTML documents, which might be more attractive.

What did I miss in my comparison? Did I miss another alternative? What do you think a core solution should look like? jQuery UI to core? Just a targeted custom implementation for our own needs? IFrames or HTML merging? JSON output or overlay specific theming? These are general conceptual questions, so I choose to write a blog post to try and start a conversation instead of using the issue queue, which would be useful for more targeted questions. However, answers to some of the questions infer answers to others. What do you think?

Seeing how Young Hahn, David Rothstein, Paul Lovvik, Charlie Gordon, Daniel F. Kudwien, myself and others work on parts of the Drupal 7 user experience implementation already, it is getting harder to apply all the patches, keep up with their development and get them integrated to work together with various efforts.

So to ease our work, I've sat down and set up a third party code repository for the D7UX effort. Using a third party repository for core development is not anything new. For example, the Fields API work used a third party repository and the Drupal 6 internationalization work also did that. When multiple people work on projects which need integration and multiple patches to apply at once, it is easier to experiment in a third party repository and roll core patches from there. Once core patches are accepted or declined, and code freeze comes, we stop using this repository, since it's purpose is going to be over.

The D7UX repository is hosted on Google Code at http://code.google.com/p/d7ux/ (thanks for Google's support for Open Source development) and is already used by Young Hahn and myself with others to hopefully follow. You can check out the work anonymously based on instructions from http://code.google.com/p/d7ux/source/checkout

Currently, it hosts the header patch created by Young and two alternative versions for the overlay implementation by myself (blog post coming up). It also includes a D7UX install profile for testing. Our work is planned to be integrated in Drupal core itself, so this install profile is a temporary measure to keep the number of our core patches low, but let people quickly try out our work. We manage a list of the patches we applied at http://code.google.com/p/d7ux/source/browse/trunk/patches/patches.txt

Both Young and myself are administrators of the repository, and we are open to collaborators, so reach to us if you'd like to collaborate.

My previous blog post entitled On a mission to improve page regions in Drupal 7 outlined how special casing things like custom help settings, the mission statement and footer message make understanding and using Drupal harder and general assumptions not being applicable to set text formats or visibility on such items. Having the somewhat special content region was also another example. After more work in this area, I'd figure I post a status update for those who'd like to have a bigger picture overview:

  • System help text is now a block in Drupal 7. You can offer your users to customize it (turn it on/off themselves) or hide excessive help text on certain pages. Since the help block is placed in a help region, you can add custom help to any pages with custom visibility settings tailored to your needs. You can use full text format support for these blocks. No more special user registration guidelines or contact page help settings. It is all general now and way more powerful. I was alerted that content type help is another special case which is not yet migrated. The issue used to track that remaining task is at http://drupal.org/node/448784 (but does not have a patch yet, contributions welcome!)
  • I've suggested migrating the site mission statement to a block. Since that was overloaded both as a (badly implemented) markup aware site mission and an RSS site description, the patch migrates the mission statement to a block and adds a standalone setting for the RSS site description. Similar migration path to the help blocks. Patch to review and test at http://drupal.org/node/428800
  • New since my last blog post: site settings in Drupal 6 and before also includes the footer message. This is even easier to eliminate then the site mission, since it is not overloaded for any other reasons. It should be migrated to a footer message block (which the user can then retitle based on their previous use of the block to "Copyright", "Affiliate links" or whatever). Patch for review and testing at http://drupal.org/node/453080
  • Finally, I've been working more on the "main content region as a block" functionality, which lets you reposition your main content and put blocks above and below it without having special _top and _bottom block regions to get your desired result. This patch makes all Drupal 7 themes required to have a "content" region to ensure that users cannot easily break their sites. You'll not be able to enable themes which don't implement this region. Also, the main page content block is required to show up in some region for the same reason. Feedback is more then welcome at http://drupal.org/node/428744

I hope we can get through these soon, since I have even more ideas on eliminating funky special cases in site building and theming making site setup and theme authoring even easier.

Last spring, I've read a Drupal theming book and was amazed by how things like the search and menu features for themes need special settings and theming while their parallel implementation exist as blocks. I was also deep in improving input formats (which was since then taken on by others) and did a comparison of what is not input format enabled in Drupal. That uncovered even more parallel implementations of things which should basically be blocks.

At Acquia, we are (among other cool things) hard at work to improve the user experience for Drupal 7 and as part of that effort, I am focusing on regions and blocks so that page building can be more unified then ever. You should know where to look to position your content, disable items on the page, set up custom text for pages, etc.

Help settings

I started off with the help settings. Drupal themes have a $help variable and that help text is usually generated by one or more modules. There are two independent user provided settings for help however. One can put admin provided help on top of the user registration and the contact pages. These are implemented as custom settings which add this help to the page in custom ways.

I realized that by making the $help item a region, we can gain certain cool features:

  1. You could add custom help to any page you want. Cut out the two one-off settings and let the admin put custom help on any page they want.
  2. Use your favorite input format to enter the text with your favorite WYSIWYG editor for it (or without that obviously).
  3. The admin could completely disable display of help. Display of help could even be disabled for higher roles (site editors), for whom help might be in the way. Different custom help could be provided for different roles.
  4. Admins could provide users with user customization possibility for the help display. This is already supported by Drupal for any block.
  5. Custom set help text could be displayed on more then one page (eg. in a site area).

All this without coding, loosing two custom settings and making the help area a region. Pretty cool, heh? Check out the patch which needs a reroll here: http://drupal.org/node/240873

Content region

In Drupal, the main page content is output by appending the blocks put into the content region in this order. There is no way to put stuff above the main page content by default. Many themes worked around this in Drupal 5 and 6 by adding a content_top and a content_bottom region instead, so you can put stuff above and below the main page content. I believe "top" and "bottom" or "above" and "below" are keywords we handle inside the regions system. We move stuff around inside one region to order them. So I suggested we should have one region for content and make the "main page content" a regular looking block instead.

There are obviously some improvements needed to ensure that all themes can accommodate this block and that people cannot easily turn it off. These are being discussed. Patch needing review at http://drupal.org/node/428744

Site mission

This case provided the title hint for the blog post. Mission statement is a setting in Drupal at least in the past half decade. It is configurable on a plain textarea on the site settings page (no input format selection) and is hardwired in themes to show on the front page of the site. Or not at all if the theme customization options have it unchecked.

If we go a bit farther, we see we have a way to put text into a region of the website on one specific page here. Wow, sounds like a highly simplified version of what blocks can do ripped of all the flexibility (even input format selection).

Having mission as a region would allow us cool things like:

  1. We can have a mission statement displayed on more then the front page (e.g also on the company team page).
  2. We can have the mission statement only displayed somewhere else, but not on the front page (eg. only on the company team page).
  3. We can set whatever input format we want so we can even display complex maps for example.
  4. We can put more then one block into that region, so dynamic data can be output there via module defined blocks.

This all sounds too good so obviously there are some small drawbacks as well. Discuss in the issue queue: http://drupal.org/node/428800

Even more blocks, even more regions?

If you look at the site and theme settings, a few remaining items pop into your face, which will need attention once we have the above covered:

  1. The footer message. This should just be one block among the others in the footer. No reason to make it special whatsoever.
  2. Special search feature of themes. This is just a search block themed differently. Sites using this most often not use the regular search block provided by search module. As shown in Mark's and Leisa's video on themes, having a search setting on themes, while search module is not enabled, so you cannot even enable it is disconcerting. If the theme has a special place for the search block, it can expose a search region, where people can put their search block, and that would be themed specially in that region. All block visibility and other settings would apply to this one nicely and it would only ever show up once search module is enabled. Plus it would allow to put alternate search boxes into that region. Blocks exposed by ApacheSolr or any other search module.
  3. Special main and secondary menu display of themes. Again, these are *already blocks*. Themes can just theme them differently when they are in their designated regions. Again, less theme settings, more visibility control, being able to swap menus in sections of the sites via other menu blocks, and so on.

I hope I managed to find some people who agree that special settings in site settings and themes should finally go and regions and blocks should take over the stage. They are already well developed and have all the features we need to implement the custom stuff we remove, but at the same time allow for amazingly more. Please help on the above three improvements and then we can move on to even more goodness.

Ps. Many people suggested that relying more on blocks is not good now that blocks module is optional in Drupal 7. Things like Panels are supposed to be able to take over the UI and set the layout for the page in more versatile ways, hence blocks module became optional. I believe that this does not mean that we should keep custom one-off settings around and broken theme settings alive. Also, Panels exposes all the blocks for the page layout setup, so people can just as well use them to set up their page.

One of my first new year's readings was Dries' reflection post on 2008 which includes predictions for 2009. One of the predictions is that he sees Drupal 7 to be released in the last quarter of 2009. He predicts pain but a strong outcome.

I predict that Drupal 7 will be released in the fourth quarter of 2009. The two most exciting features in Drupal 7 core will be custom content types and radical improvements in usability. To reduce the risk of important modules falling behind in support or update path, a significant portion of the Content Construction Kit (CCK) related modules will move to core and we'll pave the way for the Views modules. The same holds true for other important contributed modules, including token module, path auto module, and image handling functionality. In 2009, core becomes bigger, not smaller. The Drupal 7 code freeze will be longer than expected regardless our new continuous test framework, and the upgrade path to Drupal 7 will be more painful than hoped for. But like always, we'll come out stronger than before...

Two key takeaways to spread from my mind are that:

  • You should seriously move to Drupal 6 instead of staying on older versions, in anticipation of a new major Drupal release around the corner. The key modules are already out, and they are amazingly more useful then their Drupal 5 or earlier counterparts. The action happens in the contributed modules area!
  • The doors for your work on Drupal 7 are wide open! You should not hold back big ideas on the grounds of a shortly coming Drupal 7 release. We still have time to design and discuss bigger changes. No wonder the fields in core effort needs time to stabilize and achieve "CCK in core" on higher levels that we anticipated before.

Happy upgrading Drupal 5 sites, working on Drupal 6 modules and contributing to Drupal 7 core in 2009!