I've had various deep discussions with contributed module maintainers recently about their process to update code to Drupal 9 and one point struck me. We are so attached to "Make it ready for Drupal 9" that a key point of the message may be lost. Check out this section of the State of Drupal keynote from DrupalCon Amsterdam 2019 where Dries Buytaert showcases Johanna's relatively simple site that she prepares for the Drupal 9 upgrade entirely in Drupal 8. Notice that she does all the steps in Drupal 8 other than the final Drupal 9 upgrade itself:
This is the base principle of the process towards Drupal 9, making your Drupal 8 site better and more prepared, so the move to Drupal 9 itself at the end is a relatively small step and you got a better Drupal 8 site in the meantime. You are not jumping over the fence all at once, but in gradual steps. I thought a comparison with Drupal 6 to 7, 7 to 8 and 8 to 9 would help, since people may have assumptions or prior experiences with those, so its worth looking at how our new process compares to the two previous transitions.
Concentrating on the high level steps to do, this is how the three processes compare:
- From Drupal 6 to 7, there was a long list of changes, but the general API of Drupal stayed mostly the same. You would need to update some things in your custom code. Even though it may be a few things, your new code would only work with the new major core version. When you went to update from Drupal 6 to 7, you needed to keep your database in the background, use all the new Drupal 7 compatible code, and run update.php to carry over your database.
- From Drupal 7 to 8, the whole API paradigm of Drupal changed, so you needed a heavy rewrite of your custom code. There was absolutely no way custom Drupal 7 code would work with Drupal 8. The database also changed a lot, so instead of running an update.php script that would update your core and contributed database data, you would do a big database migration.
- The Drupal 8 to 9 process is entirely different again, but in a good way. First of all we are back at you running update.php on your codebase (no data migration!). So we went back to what we already knew worked well earlier. But the code updates also got quite different. This time, we are introducing deprecated APIs in minor Drupal 8 releases and add the new APIs to use instead. So we are gradually making Drupal 9 available in Drupal 8 as much as possible within our own control.
The benefits are huge! This means that custom code and drupal.org projects can be gradually updated within Drupal 8 on the way Drupal 9 step by step. Making improvements on the Drupal 8 site by updating contributed modules and adopting the new APIs in custom code like Johanna did in Dries' demo is making the Drupal 8 site itself better and closing the gap to Drupal 9 at the same time. All the new APIs get more use and thus get more battle tested in the process, all before Drupal 9 is even released. So all there will be left for Drupal 9.0.0 itself is to remove the old APIs and update third party dependencies.
Enough of the marketing, let's talk about the fine print where some contributed modules may not be kept Drupal 8 compatible and become Drupal 9 compatible at the same time.
The 12% size wrench in the system
As I posted in July, 76% of deprecated API use was possible to fix already at the time. I kind of brushed off the "Fixable later" category because it was not yet a pressing need then. But let's look at how we arrive at what can be fixed now vs. later.
In 2018 we extended security coverage of Drupal core minor versions to 12 months. That means that there is a 6 month overlap of when a new minor release comes out but we are still supporting the two versions prior. For example, when Drupal 8.8.0 comes out on December 4, 2019, we'll stop supporting Drupal 8.6.x and still keep supporting Drupal 8.7.x until June 3, 2020, the date of our next minor release.
For custom code written on a site, as Drupal core is updated, the custom code can keep up to date with the latest and greatest APIs. However for contributed modules, the choice is up to the module maintainer. There is no point in supporting core versions which are out of security coverage. But if maintainers keep updating to the latest core minor APIs, their contributed module's latest versions would not be compatible with sites running perfectly supported core versions. This becomes a problem especially if the contributed module in question needs to make a security release, when it needs to create a new branch retroactively for an old core version unless they want to force users of Drupal core to update their security-supported versions.
So this is where the "Fix now" vs. "Fixable later" differentiation comes in. Here is more fine-grained breakdown of the up to date data from October 22. Still 75% is fixable now. Breaking down the "Fixable later" category, another 6% is fixable once Drupal 8.6.x goes unsupported on December 4, 2019. A good 8% cannot be automatically categorized unfortunately. That leaves 12% that are deprecations introduced in Drupal 8.8. This includes some that have replacements in prior Drupal versions, so the data could use some cleaning up. But nonetheless there will be a measurable amount of deprecated APIs that one cannot fix right now and still be compatible with Drupal 8's all supported branches and Drupal 9.0.0 on day one.
Some contributed project maintainers feel the campaign to update for Drupal 9 makes them look bad, because this appears to be a catch 22 situation that puts them in a difficult corner.
What can you do in this case?
Since you are serious about supporting Drupal 8's supported branches (huge thanks for that!), there are the following options:
- Add temporary workarounds to the deprecated APIs, such as conditional code to only use the deprecated APIs when they are still there. Mike Lutz and Sascha Grossenbacher (Berdir) worked a lot in this area and I believe they are both working on posts with tips. (Will link them in when available.)
- Create a new major branch of your project for Drupal 8.9.0/9.0.0 support specifically. Or if you are also affected by some dependency update, such as Symfony 4.4 API changes, branch for Drupal 9.0.0+ support in particular and keep your existing branch for Drupal 8 / Symfony 3.4. One thing to note here is Drupal.org does not allow for 9.x releases anymore, so even after you created a 9.x branch, you will not be able to make releases out of it. This is in preparation for the introduction of semantic versioning for contributed projects, which will do away with the 8.x/9.x prefixes entirely. The workaround in the meantime is to create a new 8.x branch and use the right core_version_requirement value.
- Assume for the best case scenario, and drop support for Drupal 8.7 earlier. If you need to release a critical bug fix or a security release, you can still branch two new minor versions of your project, one from the prior minor version that was still 8.7 compatible and one from the new code that is not. Include the fixes in both. Add composer and info.yml dependency information to make it impossible to install the incompatible versions on 8.7.x. (Hat tip to Sascha Grossenbacher (Berdir) for this idea).
It may not be possible to attain 100% Drupal 9 compatibility in each contributed module on day one of the Drupal 9.0.0 release, and there will be some where the update is not that simple. I firmly believe our process is still way better then what we had before, and we are learning a lot from the mistakes we make. That said, I'll keep advocating to prepare now because 75% of the work can be done anytime. Assuming the percentages stay the same, this raises to 81% very soon in early December (and may be even higher based on some of the uncategorized items falling into this field). That is a lot of preparation that we can do already! If instead of asking is this entirely Drupal 9 compatible, we switch our minds to gradually improve within Drupal 8 and get much closer to Drupal 9, then we can use our time quite flexibly and make the existing Drupal 8 sites better. Only a relatively small gap remains at the end then, which was the goal to begin with.
Further reading
- If you use Upgrade Status, it includes the same categorization and can be used to focus on what is actionable now.
- I also covered this topic in my open source State of Drupal 9 slides that you can present at your own meetup, company, conference, etc.
- The official Drupal 9 documentation also suggests keeping supported core branches in mind.