Drupal.org

We need to admit, these days, people are more reliant on contributed modules then Drupal core. What is in Drupal core is taken as a given and some requested new feature would ever come in a contributed module in a reasonable timeframe for a Drupal site builder. People are not waiting for Drupal 7 to get a feature, people are active in the issue queues to get things done sooner. Drupal core is supposed to lay the foundation and be stable for long, while contributed modules can experiment, change, and add new features on a much more active pace. So as a result, Drupal core can keep using the "will be released sometime" approach, and it does well with Drupal 7 allowing for more improvements by not keeping the "every 12 months" release cycle intact. With contributed modules on the other hand, much more frequent releases are anticipated with new features introduced through a shorter timeframe.

What's common in both, is that for longer lasting development, overseeing progress on major areas like the file API, the database layer and so on is important, while on shorter release cycles, knowing what is required for the release to be fixed, and whether the project is on track or not could be key to get contributors. So having (A) overviews of project progress and (B) checklists of things to be done is increasingly needed on drupal.org. There are a few things people did about these needs recently, and all have their advantages and disadvantages. There are some reasonably new approaches, so I decided to highlight the ways I see people approach project planning on drupal.org.

Drupal.org runs the infamous project* module family to help micro (and not so micro) projects run around the Drupal core. These include modules, themes and translations as well as installation profiles. Currently, drupal.org hosts more then four thousand (yes, more then 4000) projects, with support for a CVS space for hosting the code with branching and tagging, as well as a release system tied to those CVS concepts. Projects can also have an issue queue, which people can submit issues with, covering bug fix requests and suggestions as well as new feature requests. So a project's issue queue can be quite busy, but it is the only "core" way to track what needs to be done, who is assigned to these tasks, who is working on them, what is the current status and so on.

So eventually people started to experiment building new tools on top of project issues to help manage project planning, overviews and checklists. Here is a hopefully comprehensive list of what tools people built on top:

I've helped recently to unfork the Bluebeach theme (the theme used on Drupal.org and subsites) which was used with different code on drupal.org and groups.drupal.org. So now both sites can use the same source code for their theming. I've also ported the changes to the Drupal 6 port of the theme which was done by Earl Miles earlier, based on the Drupal 5 version. These were all in anticipation of doing upgrades of drupal.org and subsites with the existing theme before going to the next step with Mark Boulton's redesign.

Now Niel Drumm stepped ahead and put the Drupal 6 version of Bluebeach into testing with his port and improvements of the API module, which is used to run api.drupal.org. The new testing site at d6-api.drupal.org is the first Drupal.org subsite to be upgraded from Drupal 5 / to run Drupal 6 (the Drupalcon Washington site was launched on Drupal 6, but it runs under drupalcon.org not drupal.org).

Niel is asking for feedback on a groups.drupal.org post about the upgraded theme and module as well as the new API comparison feature (which I like very much). Since this is the first test site running the theme, helping test just that ensures a smoother upgrade of Drupal.org itself which paves the way for the redesign implementation. Take the new site for a test drive, check out the new features!

In first of a series of posts, I'd like to go ahead and talk about project handling functionality, one of the most important tools behind Drupal.org. At this moment, Drupal.org is running Drupal 5, and a big chunk of modules which don't have a Drupal 6 version to migrate to on Drupal.org is the project module family: project (also includes project_release and project_usage), project_issue, pift_server, cvslog and even comment_upload.

Except the comment_upload module (which allows file uploads on comments in a general way), maintenance for these modules are headed by Derek Wright and Chad Phillips. An outstanding thing about these modules is that they keep improving and being adjusted to user needs. Automated testing integration tools were developed and keep improving, so patches submitted against Drupal 7 get automated testing. This is just plain great. However, all this huge amount of motion is going on in the Drupal 5 version of the module. And given that Drupal.org needs a stable environment, it takes considerable effort to maintain a stable Drupal 5 branch with all these feature improvements and changes coming in.

While these modules do not even have a Drupal 6 branch yet, Adam Light went ahead and worked on a Drupal 6 port for project module. He hosts this in his own private Subversion repository (see http://drupal.org/node/157694#comment-891587 and the rest of the comments there). Since he started off long ago from a then current version of the module and implemented Views integration (instead of the one-off SQL based pages in the Drupal 5 version), the Drupal 6 port has a largely refactored codebase and does not carry the improvements made to the Drupal 5 version since then.

The lead maintainers however are at this point more interested in working on a new stable release for Drupal 5, given that some bigger changes they are planning to make would be easier to manage on their own instead of as part of a bigger porting and migration work to a new Drupal version and to a Views based backend. This gets us to a message of "please help with a new stable Drupal 5 release of project module before Drupal 6 work can be considered". While these patches are relatively big, they are far from how big of a monster patch is the Drupal 6 upgrade. All-in-all the possibly awkward conclusion is that maintainers look for help with the Drupal 5 version before Drupal 6 work can be started.

For concrete action items, Adam Light summarizes it well:

Let me explain the situation a little here:

There are still some things that we'd like to get fixed in the Drupal 5.x branch of the project and project issue modules before we branch for Drupal 6. See http://groups.drupal.org/node/10865 and http://groups.drupal.org/node/16069. One big issue that affects all of project* land is #98278: project* namespace bugs in $node. The project* maintainers have agreed that it would be best to fix this problem *before* branching, because if we try to fix it during the port that will make testing the port even more difficult than it will be.

Creating a D6 branch itself will not really unblock things. Hunmonk or I could also do that ourselves. However, the ported code that is currently in my SVN repository contains a *lot* of changes, and all of us agree that those changes should not just blindly be checked into the project cvs repository without at least some review.

Yes, we all realize that this is a non-ideal situation, and that the port is moving much slower than anyone of us would like. The best way anyone could help move the port forward would be by helping to write or review patches for any issue not yet finished in the list of things to do before the next 5.x release. Mostly, that means reviewing the $node namespace patches in the issue I linked to above. The unfortunate part of this issue is that it is huge, but really boring. And none of the sites run by the project* maintainers use a combination of modules that actually causes this bug to reveal itself. But at the same time, we realize that lots of project* users *do* use such a combination of modules (eg. they use pathauto with project), and so we need to fix this bug soon.

So there are numerous big issues affecting Drupal.org which will be solved as part of the Drupal 6 port, but the main issue holding back the port from even starting is an issue which does not even affect drupal.org (and therefore is not going forward on any reasonable speed).

In summary, there are difficulties in how improvements on drupal.org are expected by some people right now instead of after an upgrade, and the maintainers are taking on work on these items; and issues not affecting drupal.org holding back our most important upgrade ever. So you can help at least three ways with the project module upgrade:

• Put away some of your cool feature ideas for project modules on drupal.org for now. Let's focus on porting and bugfixing or we are not going to get over new feature requests anytime soon.
• Help test patches against the Drupal 5 version of project modules to fix long standing bugs. See http://groups.drupal.org/node/10865 and http://groups.drupal.org/node/16069
• Help test and fix existing issues in the Drupal 6 port of project module. It at least has Views integration issues coming from the RC2 API changes in Views. See my pending patch at http://drupal.org/node/157694#comment-1069892 which still needs work.

We definitely need your help in many ways. Let's do good for the drupal.org upgrade / redesign!

Drupal.org (and its whole site family) is being redesigned, thanks to heroic efforts by the Drupal Association. The site family grew quite big (and is still growing) so there is a lot of stuff to do. You probably have your own gripes on how it should work, so now is the time to get involved! Just as you'd expect from good architects when redesigning your home, Leisa and Mark are running a series of blog posts (just watch Drupal Planet to see them) to discuss details of the redesign, and understand our needs. Anyone can put in their opinions, and it is their job to filter it down and produce useful results in a relatively short time.

I started contributing with a one-on-one interview with Leisa at Drupalcon Szeged 2008. This event was a good opportunity to interview lots of different type of people on how they use the site, what are their problems and happy moments with it. If you have not been to Szeged, or have not been able to get an interview, there is no reason to step back. You can also contribute with comments on various blog posts (just as I do), taking part in the online card sort and submitting your own wireframe suggestions.

I've just decided today to sit down and make sure my opinion gets into the pipe, so I submitted this simplistic wireframe (click on it and get to Flickr to see the notes):

My goal with the suggestions is to get rid of the blog-look for the home page and get prime-time for more important stuff by highlighting them by topic area. We don't need such a long explanation on what Drupal is, if people can instantly see, that Drupal events are happening near them, they can buy books to hold in their hands, people build cool sites with Drupal and they can earn money with Drupal as well, within a thriving community, which takes software and security seriously. If you break this sentence down, several boxes come up for the homepage to highlight security bulletins, showcases, the Drupal Planet, events, and so on. These all describe Drupal's several aspects themselves, leaving the intro itself to a short explanation. I believe this kind of hub homepage would finally get us to a state where we can tell people to just go to the drupal.org homepage and get a decent overview of Drupal.

Do you think you can do better then me or have other ideas to get highlighted in the redesign process? Why not participate? Come wireframe with Leisa.