Fix the installation process hanging

Axel Beckert wrote:
> > I tried to reproduce it with sysvinit and still failed :(
>
> I just was able to reproduce it by installing matrix-synapse on a
> Raspberry Pi running Debian Sid with sysvinit as init system and it
> hangs at the configuration stage after "Config is missing
> macaroon_secret_key". (This warning is though a red herring as
> upstream withholds the fact that it will automatically generate a key
> in this case.)
[...]
> It now hangs there for about an hour.

Contents of /var/log/matrix-synapse/homeserver.log:

2019-06-01 13:01:28,548 - root - 209 - WARNING - None- ***** STARTING SERVER *****
2019-06-01 13:01:28,745 - root - 212 - WARNING - None- Server /usr/lib/python3/dist-packages/synapse/app/homeserver.py version 0.99.2
2019-06-01 13:01:28,748 - root - 214 - INFO - None- Server hostname: c3pio.deuxchevaux.org
2019-06-01 13:01:29,523 - synapse.app.homeserver - 242 - WARNING - None- Starting daemon.
2019-06-01 13:01:29,548 - twisted - 240 - WARNING - - /usr/lib/python3/dist-packages/synapse/config/tls.py:300: builtins.UserWarning: Self-signed TLS certificates will not be accepted by Synapse 1.0. Please either provide a valid certificate, or use Synapse's ACME support to provision one.

The time stamps are all from the time of installation.

From htop:

 9500 root       20   0  7684  2608  1352 S  0.0  0.3 25:57.98 ├─ SCREEN -RdU
 9501 root       20   0  7264  2512  1772 S  0.0  0.3  0:00.65 │  ├─ bash
 2658 root       20   0  178M  125M 49080 S  0.0 13.6  1:24.26 │  │  └─ aptitude -u
 4501 root       20   0 12616  7496  1684 S  0.0  0.8  0:00.40 │  │     ├─ dpkg --status-fd 66 --configure --pending
 4970 root       20   0 19836 13500  3604 S  0.0  1.4  0:01.32 │  │     │  ├─ perl -w /usr/share/debconf/frontend /var/lib/dpkg/info/matrix-synapse.postinst configure
 4502 root       20   0  1468   380   332 S  0.0  0.0  0:00.00 │  │     │  └─ sh -c (test -x /usr/lib/needrestart/dpkg-status && /usr/lib/needrestart/dpkg-status || cat > /dev/null)
 4503 root       20   0  1468    68     0 S  0.0  0.0  0:00.00 │  │     │     └─ sh -c (test -x /usr/lib/needrestart/dpkg-status && /usr/lib/needrestart/dpkg-status || cat > /dev/null)
 4504 root       20   0  1468   380   332 S  0.0  0.0  0:00.02 │  │     │        └─ sh /usr/lib/needrestart/dpkg-status
 3499 root       20   0  178M  125M 49080 S  0.0 13.6  0:00.00 │  │     └─ aptitude -u
[...]
 5066 matrix-sy  20   0 69700 46988  6964 S  0.0  5.0  0:19.14 ├─ python3 -m synapse.app.homeserver --config-path /etc/matrix-synapse/homeserver.yaml --config-path /etc/matrix-synapse/conf.d/ --daemonize
 5067 matrix-sy  20   0 69700 46988  6964 S  0.0  5.0  0:02.13 │  └─ python3 -m synapse.app.homeserver --config-path /etc/matrix-synapse/homeserver.yaml --config-path /etc/matrix-synapse/conf.d/ --daemonize

So synapse has already forked, the init script seems to have exited
already and even the postinst script has exited already. The question
is what causes debconf to wait?

"ps auxwwwf | fgrep -1 matrix" shows that there's a zombie process
leftover as (former) child of the debconf process:

root      4970  0.0  1.4  19836 13504 pts/0    S+   13:00   0:01  |           \_ /usr/bin/perl -w /usr/share/debconf/frontend /var/lib/dpkg/info/matrix-synapse.postinst configure
root      4977  0.0  0.0      0     0 pts/0    Z+   13:00   0:00  |               \_ [matrix-synapse.] <defunct>

I suspect — based on the name part "matrix-synapse." that this zombie
was eitherr the matrix-synapse.postinst script or the
matrix-synapse.config script.

Looking with lsof what's still opened by the debconf frontend process
(pid 4970), I noticed these two pipes:

frontend 4970 root    7w  FIFO    0,8      0t0 1899008614 pipe
frontend 4970 root    8r  FIFO    0,8      0t0 1899008615 pipe

Grepping through the whole process space, I noticed that at least one
of these pipes is connected to the python3 process running
matrix-synapse which looks odd to me:

frontend   4970                           root    7w     FIFO        0,8      0t0 1899008614 pipe
frontend   4970                           root    8r     FIFO        0,8      0t0 1899008615 pipe
python3    5066                 matrix-synapse    3w     FIFO        0,8      0t0 1899008615 pipe
python3    5066  5067 python3   matrix-synapse    3w     FIFO        0,8      0t0 1899008615 pipe

And indeed, calling "service matrix-synapse stop" on another console
makes the hanging install process to continue.

So it seems that matrix-synapse's --daemonize doesn't properly close
file descriptors upon forking — which is likely an upstream issue.

With systemd, systemd probably covers that. With init scripts we have
start-stop-daemon to fix that, so let's do that.

Closes: #920339

0 files changed, 0 insertions, 0 deletions