Drupal 7

Moshe Weitzman recently started the D7CX movement to rally people to get Drupal 7 contributed modules upgraded and released on time for the Drupal 7 release. This would be a great boon to the Drupal 7 release, which shapes up to be a huge improvement over Drupal 6 already. He also suggests a contributed module release manager who can help with (among other things) ensuring that tools are available to help people upgrade.

Those not following the implementation of concepts from http://d7ux.org/ here is a quick summary of the three main areas:

By now, the Drupal contributed repository grew so big that there are likely multiple solutions to the same problems there. Maybe these solutions solve the tasks with different angles, have slightly different features, but at their core, they aim at solving the same issue. So it is getting harder for site builders to pick the modules they want to use (and drupal.org is planned to offer more tools in helping with the selection). When it comes to core inclusion though, one needs to very clearly define the requirements and meet core goals, such as lean and mean implementation, maintainability and reusability.

With Mark Boulton's and Leisa Reichelt's suggestion to have most of the Drupal administration interface show up in an overlay on top of the public facing webpage, we are again at such technology crossroads. Drupal 6 at least has two contributed modules to implement a similar-looking solution: Popups API and Modal Frame API. Both got their pros when I researched for possible overlay implementations, and frankly, when there are already two modules implemented for the same complex task, why would we start from scratch?

The goals for the core overlay (purposely not called a modal dialog) is to take up most of the window with a dim background on the original page content. The overlay would have a close button on its right side and possibly tabs on top to switch to different subpages (loaded via Ajax or will already be on the same page, but hidden). We don't need the overlay to be moveable or resizable. We don't need to show multiples at once, since it only makes sense to show one. However, the overlay should work well with the header so that when an option is selected, it keeps being active in the header, and the header should work so one can pick another area to work on without first closing the overlay.

For easier evaluation of how we can meet these goals, I've ported both above mentioned modules to Drupal 7 and implemented Drupal 7 user experience "skins" for them. In the case of Popups module, it was a skin which the module already supports via an internal API, with Modal Frame API, it was an actual glue module which mapped elements marked up for being displayed in the overlay to a style similar to what Mark and Leisa suggests. Neither of them is pixel perfect to what is to be done, but the initial goal was to have D7 versions to evaluate and discuss the implementation internals, so we can fix up styling once we picked either one or a third way.

You can find my Popups module port patch at http://drupal.org/node/466732#comment-1681554 and the Modal Frame API port patch at http://drupal.org/node/491224#comment-1703366 but to make all these easier to test with the D7UX header in progress, all this code is available with the glue modules and custom skins in the D7UX code repository. You can try out both by installing with the d7ux install profile and then either turning on the "Popups API" module or the "D7UX overlay look" module. Make sure to only have one of them enabled at once and let Drupal install their dependencies. Switch between the two implementations by switching between the two modules.

Let's go beyond the UI and compare the modules:

We are obviously at crossroads with picking our ways. The Popups module approach already has numerous core patches which were suggested earlier, while Modal Frame API leverages jQuery UI to lessen custom code burden and iframes to avoid colissions in merging HTML documents, which might be more attractive.

What did I miss in my comparison? Did I miss another alternative? What do you think a core solution should look like? jQuery UI to core? Just a targeted custom implementation for our own needs? IFrames or HTML merging? JSON output or overlay specific theming? These are general conceptual questions, so I choose to write a blog post to try and start a conversation instead of using the issue queue, which would be useful for more targeted questions. However, answers to some of the questions infer answers to others. What do you think?

Seeing how Young Hahn, David Rothstein, Paul Lovvik, Charlie Gordon, Daniel F. Kudwien, myself and others work on parts of the Drupal 7 user experience implementation already, it is getting harder to apply all the patches, keep up with their development and get them integrated to work together with various efforts.

My previous blog post entitled On a mission to improve page regions in Drupal 7 outlined how special casing things like custom help settings, the mission statement and footer message make understanding and using Drupal harder and general assumptions not being applicable to set text formats or visibility on such items. Having the somewhat special content region was also another example.

Last spring, I've read a Drupal theming book and was amazed by how things like the search and menu features for themes need special settings and theming while their parallel implementation exist as blocks. I was also deep in improving input formats (which was since then taken on by others) and did a comparison of what is not input format enabled in Drupal. That uncovered even more parallel implementations of things which should basically be blocks.

At Acquia, we are (among other cool things) hard at work to improve the user experience for Drupal 7 and as part of that effort, I am focusing on regions and blocks so that page building can be more unified then ever. You should know where to look to position your content, disable items on the page, set up custom text for pages, etc.

Help settings

I started off with the help settings. Drupal themes have a $help variable and that help text is usually generated by one or more modules. There are two independent user provided settings for help however. One can put admin provided help on top of the user registration and the contact pages. These are implemented as custom settings which add this help to the page in custom ways.

I realized that by making the $help item a region, we can gain certain cool features:

1. You could add custom help to any page you want. Cut out the two one-off settings and let the admin put custom help on any page they want.
2. Use your favorite input format to enter the text with your favorite WYSIWYG editor for it (or without that obviously).
3. The admin could completely disable display of help. Display of help could even be disabled for higher roles (site editors), for whom help might be in the way. Different custom help could be provided for different roles.
4. Admins could provide users with user customization possibility for the help display. This is already supported by Drupal for any block.
5. Custom set help text could be displayed on more then one page (eg. in a site area).

All this without coding, loosing two custom settings and making the help area a region. Pretty cool, heh? Check out the patch which needs a reroll here: http://drupal.org/node/240873

Content region

In Drupal, the main page content is output by appending the blocks put into the content region in this order. There is no way to put stuff above the main page content by default. Many themes worked around this in Drupal 5 and 6 by adding a content_top and a content_bottom region instead, so you can put stuff above and below the main page content. I believe "top" and "bottom" or "above" and "below" are keywords we handle inside the regions system. We move stuff around inside one region to order them. So I suggested we should have one region for content and make the "main page content" a regular looking block instead.

There are obviously some improvements needed to ensure that all themes can accommodate this block and that people cannot easily turn it off. These are being discussed. Patch needing review at http://drupal.org/node/428744

Site mission

This case provided the title hint for the blog post. Mission statement is a setting in Drupal at least in the past half decade. It is configurable on a plain textarea on the site settings page (no input format selection) and is hardwired in themes to show on the front page of the site. Or not at all if the theme customization options have it unchecked.

If we go a bit farther, we see we have a way to put text into a region of the website on one specific page here. Wow, sounds like a highly simplified version of what blocks can do ripped of all the flexibility (even input format selection).

Having mission as a region would allow us cool things like:

1. We can have a mission statement displayed on more then the front page (e.g also on the company team page).
2. We can have the mission statement only displayed somewhere else, but not on the front page (eg. only on the company team page).
3. We can set whatever input format we want so we can even display complex maps for example.
4. We can put more then one block into that region, so dynamic data can be output there via module defined blocks.

This all sounds too good so obviously there are some small drawbacks as well. Discuss in the issue queue: http://drupal.org/node/428800

Even more blocks, even more regions?

If you look at the site and theme settings, a few remaining items pop into your face, which will need attention once we have the above covered:

1. The footer message. This should just be one block among the others in the footer. No reason to make it special whatsoever.
2. Special search feature of themes. This is just a search block themed differently. Sites using this most often not use the regular search block provided by search module. As shown in Mark's and Leisa's video on themes, having a search setting on themes, while search module is not enabled, so you cannot even enable it is disconcerting. If the theme has a special place for the search block, it can expose a search region, where people can put their search block, and that would be themed specially in that region. All block visibility and other settings would apply to this one nicely and it would only ever show up once search module is enabled. Plus it would allow to put alternate search boxes into that region. Blocks exposed by ApacheSolr or any other search module.
3. Special main and secondary menu display of themes. Again, these are *already blocks*. Themes can just theme them differently when they are in their designated regions. Again, less theme settings, more visibility control, being able to swap menus in sections of the sites via other menu blocks, and so on.

I hope I managed to find some people who agree that special settings in site settings and themes should finally go and regions and blocks should take over the stage. They are already well developed and have all the features we need to implement the custom stuff we remove, but at the same time allow for amazingly more. Please help on the above three improvements and then we can move on to even more goodness.

Ps. Many people suggested that relying more on blocks is not good now that blocks module is optional in Drupal 7. Things like Panels are supposed to be able to take over the UI and set the layout for the page in more versatile ways, hence blocks module became optional. I believe that this does not mean that we should keep custom one-off settings around and broken theme settings alive. Also, Panels exposes all the blocks for the page layout setup, so people can just as well use them to set up their page.

One of my first new year's readings was Dries' reflection post on 2008 which includes predictions for 2009. One of the predictions is that he sees Drupal 7 to be released in the last quarter of 2009. He predicts pain but a strong outcome.

I predict that Drupal 7 will be released in the fourth quarter of 2009. The two most exciting features in Drupal 7 core will be custom content types and radical improvements in usability. To reduce the risk of important modules falling behind in support or update path, a significant portion of the Content Construction Kit (CCK) related modules will move to core and we'll pave the way for the Views modules. The same holds true for other important contributed modules, including token module, path auto module, and image handling functionality. In 2009, core becomes bigger, not smaller. The Drupal 7 code freeze will be longer than expected regardless our new continuous test framework, and the upgrade path to Drupal 7 will be more painful than hoped for. But like always, we'll come out stronger than before...

Two key takeaways to spread from my mind are that:

• You should seriously move to Drupal 6 instead of staying on older versions, in anticipation of a new major Drupal release around the corner. The key modules are already out, and they are amazingly more useful then their Drupal 5 or earlier counterparts. The action happens in the contributed modules area!
• The doors for your work on Drupal 7 are wide open! You should not hold back big ideas on the grounds of a shortly coming Drupal 7 release. We still have time to design and discuss bigger changes. No wonder the fields in core effort needs time to stabilize and achieve "CCK in core" on higher levels that we anticipated before.

Happy upgrading Drupal 5 sites, working on Drupal 6 modules and contributing to Drupal 7 core in 2009!

Dries said (again) in his State of Drupal keynote this past week that a whole lot of people want WYSIWYG editors in Drupal. He also highlighted FCKeditor as the most popular one used for this job in the Drupal community (based on call-home data from Update Status module). So you'd say this is just too easy, move FCKeditor to core and we are done. Well, going the simple and quick way would just alienate those from RTEs (another nice acronym for these editors), because these editors are just too aggressive when it comes to pushing themselves to your face.

The assumption in these editors is to treat all textareas as fields with HTML input by default, so you'll obviously need the HTML editor on them. But this is not the case. In previous times, if you installed an RTE, and went to a block's configuration page, the textarea where you can provide a list of paths on where the block will show/hide was taken over by the RTE. Through time, the RTE authors realized this is a problem and built in tools to limit their reach. If you look FCKeditor's profile editing interface (image on the right), you'll find settings to exclude certain form item IDs from FCKeditor's work. But who knows the form IDs? Well, FCKeditor authors built a tool which tells you about the form IDs on forms, eg. on the block configuration page, it says: The ID for excluding or including this element is: edit-pages - the path is: admin/build/block/configure/user/1. Then if you need to modify your configuration on this field, you need to remember these values, go over to the linked "excluding or including" page and alter the list of items there.

There are a couple of flaws with this approach:

• End users need to deal with (internal) form item IDs to configure their site for forms which are not covered by the default settings.
• What guarantees that "edit-pages" will not be used as a form ID for HTML input in another form? Nothing. If we block it out, we block it out in all forms.
• Blacklisting keeps textareas open to FCKeditor by default, so if a contrib module comes requiring text input in a field, there is no way to tell RTEs not to process that field, the user will face the pushy RTE and submit a bug against the contrib module such as http://drupal.org/node/293502 (and will get users hack as hell).
• If this all was not enough, FCKeditor also has a simplified toolbar view, which is used for text input using the filtered XSS admin processor (when input format is not selected, but some HTML is still allowed). Again end users set up which textareas are using this method (beyond some defaults included). How they get to know that? Well, trial and error, or they need to look into the code.

All these ugly hassles for the end user, while we programmers know exactly that input from a textarea will be plain text (eg. email text, a list of paths), passed through filter XSS admin or filtered via one of the input filters set up. Not only that, but we know if a textarea is passed through input filters and the particular input filter used escapes HTML, it is pointless to display an RTE for HTML. So it is not only the page the textarea was displayed on, it is not only the ID of the textarea, it is also the properties of the selected input format for that textarea (when applicable) which defines whether FCKeditor should be there or not.

These are all things programmers can tell Drupal about without any user interaction, completely removing these settings and making WYSIWYG editing really seamless for the user. How do we tell Drupal what type of input is on the textarea? Well, we tell it to use a specific format. There are two competing proposals on how this should work, both in the issue: Textarea with input format attached. If you care about WYSIWYG editors in Drupal core (for Drupal 7), please help do quality reviews and provide feedback on the approaches proposed. This is a prerequisite to making RTEs integrate seamlessly to Drupal.

As I explained in an earlier blog post, I was tasked by Acquia (specifically Dries) to identify and propose fixes to issues around integrating visual content editors (WYSIWYG = What You See Is What You Get or RTE = Rich Text Editor). Instead of rushing to select an RTE, the focus of my roadmap is to solidify input format support in Drupal so that integrating a visual editor becomes a peace of cake at the end hopefully.

Textarea and format identification

Drupal is very flexible and this makes integrating an RTE harder compared to one-purpose systems. First I tried to deliver a solution for RTEs to identify the form fields to attach to. For this to work, we need input format metadata attached to form elements, which is implemented in issue #125315 with the new #input_format Form API key. This suffers from some parenting issues still (does not support multiple format selectors in one form), which we need to solve before being able to commit. On the same angle, the RTE should know whether that format has anything to do with the format supported by the RTE, be it HTML, wiki markup or bbcode for example. Drupal uses HTML by default, and administrators need to explicitly disallow HTML. This setting however was tucked down in the "Filter HTML" filter, which made it non-intuitive to identify. So issue #24988 deals with breaking that out to its own filter for easy non-HTML format identification. This way we can easily disable HTML RTEs on such formats.

Putting permissions to place

Simplifying input format setup is quite important. Right now, there is a disconcert between setting up general permissions and input format access settings. Although the later configuration maps input formats to user roles, it is implemented in its own custom permission system. By leveraging the core permissions system for this in #110242, just like the node permissions are implemented with their dynamic names, we clean up the user interface, bring in consistency to permission setup and also simplify the API.

More fun with blocks

Finally, there are quite a few textarea based input interfaces without format support in Drupal core still. For some site settings, like the contact page help or the user registration help, I think it makes a lot of sense to just use blocks. By making the help area an official theme region, we can put the help text there as a block, and allow users to add any number of blocks to the help region to add more custom help to any page. The one-off solutions for contact page help and user registration help can be easily generalized to an admin-configurable custom help system in issue #240873. By using blocks in the help region, admins can limit and share help between different pages, provide different help for different roles and use any format for their help input. If this sounds a lot like the helpedit module or somewhat like the helptip module, that's not a coincidence, this simple change would move some of the features of these modules to core.

Many of you know that I am employed by Acquia to work on Drupal 6 full time right now, but since Drupal 6 is nearing a release, you might have wondered what I am going to work on next.