Composer

Testing Composer 2 RC1 with Drupal 9: huge memory and time savings; and what does it mean for Drupal

Drupal 8 and 9 core were already made compatible with Composer 2 back in May. Last week the Drupal package repository (packages.drupal.org) rolled out full support for Composer 2 as well. While Michael Anello did Drupal vs. Composer 2-alpha2 benchmarks in July and Malabya also did an even more detailed benchmark in July, various things have been improved in Composer 2 since July and the packages.drupal.org improvement should also show.

Inspired by numbers posted by Konstantinos Skarlatos from a simple scenario to the #composer drupal.org/slack channel today, I went ahead and attempted to produce a reproducible scenario you can also try at home. I included all commands to copy-paste. There are no external dependencies other than a PHP executable on your command line and curl to download files as used. For reference, my PHP is as follows (definitely not the latest):

$ php -v
PHP 7.3.11 (cli) (built: Jun  5 2020 23:50:40) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.11, Copyright (c) 1998-2018 Zend Technologies

Set up the latest Composer 1 and 2 locally

First let's grab the latest Composer versions locally. This will avoid any confusion as to which version is used when.

$ curl https://getcomposer.org/composer-1.phar --output composer-1.phar
$ php composer-1.phar -V
Composer version 1.10.13 2020-09-09 11:46:34

$ curl https://getcomposer.org/composer-2.phar --output composer-2.phar
$ php composer-2.phar -V
Composer version 2.0.0-RC1 2020-09-10 15:39:45

Install Drupal 9 with create-project

Make sure that we are not using cache and ask composer to profile itself. This got me two Drupal 9.0.6 copies, again to separate our testing environments properly.

$ php composer-1.phar --no-cache --profile create-project drupal/recommended-project composer1site
[...]
[9.3MiB/41.02s] Memory usage: 9.35MiB (peak: 197.23MiB), time: 41.02s

$ php composer-2.phar --no-cache --profile create-project drupal/recommended-project composer2site
[...]
[12.5MiB/18.29s] Memory usage: 12.48MiB (peak: 13.71MiB), time: 18.29s

The time is cut in half but the peak memory usage had an even bigger drop to less than 7% of the Composer 1 memory peak with Composer 2.

Adding and removing a Drupal module

You only do site creation once for a site. Let's add and remove a Drupal module! Konstantinos tested with Metatag which sounds like a good one to do. While Metatag only has Drupal project dependencies, the savings are likely similar for projects with 3rd party dependencies.

$ cd composer1site
$ php ../composer-1.phar --no-cache --profile require drupal/metatag
[...]
[629.4MiB/65.35s] Memory usage: 629.36MiB (peak: 1557.29MiB), time: 65.35s

$ php ../composer-1.phar --no-cache --profile remove drupal/metatag
[...]
[450.9MiB/47.98s] Memory usage: 450.85MiB (peak: 1377.67MiB), time: 47.98s

$ cd ..

$ cd composer2site
$ php ../composer-2.phar --no-cache --profile require drupal/metatag
[...]
[13.7MiB/3.13s] Memory usage: 13.67MiB (peak: 14.23MiB), time: 3.13s

$ php ../composer-2.phar --no-cache --profile remove drupal/metatag
[12.1MiB/0.18s] Memory usage: 12.1MiB (peak: 12.49MiB), time: 0.18s

Ok, that is pretty jaw dropping. Adding Metatag goes from 65 seconds down to 3 seconds. Removing Metatag goes from 48 seconds to 0.18(!) seconds. Memory usage goes down to 1% (one percent!), from a gigabyte and a half to 12-14 megabytes.

What does this mean for Drupal?

Given the explicit support on drupal.org, you should be able to use Composer 2 for your Drupal site builds whenever you are comfortable, maybe already in the RC stage or later on as it gets stable. Assuming any composer plugins you need are compatible. Drupal sites are often using cweagans/composer-patches, which is Composer 2 compatible in the dev version, but did not have a release yet. You can change your composer.json to have "cweagans/composer-patches": "1.x-dev" and do a composer update to get the development version that is already compatible.

Many are not using composer at all to build their sites. However, an increasing number of drupal.org projects require external Composer dependencies to function. Based on last week's data from Ryan Aslett from the Drupal Association, 12% of drupal.org projects directly have a third-party Composer dependency. Another 6% have a Drupal dependency that in turn has a third-party Composer dependency. So even if we only look at direct and one-level indirect dependencies, 18% of Drupal.org projects require Composer to be installed properly. I also looked at various segments of projects and the ones requiring Composer are very well distributed. 15.2% of the top 500 projects (by usage) require Composer while 16.2% of the top 5000 do. So if you are not yet using Composer to build your Drupal sites, it seems to be only a matter of time. The improvements in Composer 2 should help a lot with its adoption I think.

These improvements will also be crucial for the Automated Updates initiative which has Composer 2 compatibility on their roadmap as well. Signature verification is only possible with Composer 2 and the lower memory and time requirements allow it to be run from web requests.

Finally, the downloadable tarballs of Drupal core are built with Composer 1 currently to make it easy to transition from a tarball based site to a Composer site. Allow Drupal 9 (including dev builds) to use Composer 2, part 2 is open to move that to Composer 2. Unfortunately a sizable blocker to that is Composer's "prefer-stable" setting cannot be relied on to produce a stable release. That would need more discussion and a solid resolution to move forward. There is less than 3 weeks before Drupal 9.1 is locked down, so your feedback there would be more than welcome!