Avoid running apache2ctl graceful which may restart apache, instead use the service command

Following on from 27927bfd630dfb09fed9d27b2c9ed37e90c0eaa3, I reproduced
the problem again and it looked like this:

   CGroup: /
           ├─init.scope
           │ └─1 /lib/systemd/systemd --system --deserialize 25
           └─system.slice
             ├─ssh.service
             │ ├─  640 /usr/sbin/apache2 -k graceful
             │ ├─  816 /usr/sbin/apache2 -k graceful
             │ ├─ 1133 /usr/sbin/apache2 -k graceful
             │ ├─ 1140 /usr/sbin/apache2 -k graceful
             │ ├─ 1298 /usr/sbin/apache2 -k graceful
             │ ├─ 1599 /usr/sbin/apache2 -k graceful

There were dozens of the processes which had all been started in a small amount
of time. I was upgrading ikiwiki-hosting at the time, so it seems likely that
ikisite ran it for each of the dozens of sites. Seems to point the finger at
apache2ctl graceful, but I don't reproduce the problem running it manually.

My guess is that, in some circumstances, apache2ctl graceful fails to
talk to the currently running apache daemon, and so I guess proceeds to
kill it and starts a new one, thus escaping the systemd cgroup.

If so, this change should avoid that, while keeping the graceful detection of
broken apache configs.

(cherry picked from commit 49956083ada2efe59ddf497182e7e1d2717b2a49)

Gbp-Pq: Name Avoid-running-apache2ctl-graceful-which-may-restart-apach.patch

2 files changed, 30 insertions, 7 deletions

diff --git a/CHANGELOG b/CHANGELOG
index a1b3358..612c0b4 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,8 @@
 ikiwiki-hosting (0.20180720) UNRELEASED; urgency=medium
 
   * Avoid directly running init scripts, instead use the service command.
+  * Avoid running apache2ctl graceful which may restart apache, instead use
+    the service command.
 
  -- Joey Hess <id@joeyh.name>  Wed, 17 Apr 2019 11:03:52 -0400
 
diff --git a/ikisite b/ikisite
index 6407289..aa47342 100755
--- a/ikisite
+++ b/ikisite
@@ -1689,8 +1689,7 @@ sub enable {
 	}
 
 	# reload apache config
-	eval { shell("apache2ctl", "graceful") };
-	if ($@) {
+	if (! apache_reload_graceful()) {
 		# avoid leaving apache in a broken state
 		foreach my $site (keys %setup) {
 			if (apache_before_2_4()) {
@@ -1700,9 +1699,6 @@ sub enable {
 				shell("a2dissite", $site);
 			}
 		}
-
-		shell("apache2ctl", "graceful");
-		error "apache2ctl graceful failed";
 	}
 	
 	enabledns($hostname);
@@ -1850,7 +1846,7 @@ sub disable {
 			$reload=1;
 		}
 	}
-	shell("apache2ctl", "graceful") if $reload && ! $options{temporary};
+	apache_reload_graceful() if $reload && ! $options{temporary};
 
 	return 1;
 }
@@ -2008,7 +2004,7 @@ sub maintaincerts {
 	# own.
 	if ($inuse && ! -e "/etc/cron.d/certbot") {
 		eval { shell("certbot", "renew", "--non-interactive", "--quiet") };
-		eval { shell("apache2ctl", "graceful") };
+		apache_reload_graceful();
 	}
 
 	return 1;
@@ -3308,6 +3304,31 @@ sub nsupdate {
 	chdir "/";
 }
 
+# checks apache config, only reloading apache if the config is legal to
+# prevent a bad config leaving it not running
+#
+# returns false if the config is bad; the caller should then revert
+# whatever changes it made
+sub apache_reload_graceful {
+	if (shell_exitcode("apache2ctl", "configtest") == 0) {
+		# Use service rather than letting apache2ctl graceful
+		# reload apache, because the latter has been observed
+		# sometimes stopping apache and starting it again under a
+		# different cgroup than the one systemd keeps apache in.
+		eval { shell("service", "apache2", "reload") };
+		if ($@) {
+			return 0;
+		}
+		else {
+			return 1;
+		}
+	}
+	else {
+		return 0;
+	}
+}
+
+
 sub loadsetup_safe {
 	my $setupfile=shift;