core: add new logic for services to store file descriptors in PID 1

With this change it is possible to send file descriptors to PID 1, via
sd_pid_notify_with_fds() which PID 1 will store individually for each
service, and pass via the usual fd passing logic on next invocation.
This is useful for enable daemon reload schemes where daemons serialize
their state to /run, push their fds into PID 1 and terminate, restoring
their state on next start from the data in /run and passed in from PID
1.

The fds are kept by PID 1 as long as no POLLHUP or POLLERR is seen on
them, and the service they belong to are either not dead or failed, or
have a job queued.

3 files changed, 161 insertions, 11 deletions

diff --git a/man/sd_listen_fds.xml b/man/sd_listen_fds.xml
index 6999db980..437774563 100644
--- a/man/sd_listen_fds.xml
+++ b/man/sd_listen_fds.xml
@@ -73,7 +73,7 @@
                 <para>If the <parameter>unset_environment</parameter>
                 parameter is non-zero,
                 <function>sd_listen_fds()</function> will unset the
-                <varname>$LISTEN_FDS</varname>/<varname>$LISTEN_PID</varname>
+                <varname>$LISTEN_FDS</varname> and <varname>$LISTEN_PID</varname>
                 environment variables before returning (regardless of
                 whether the function call itself succeeded or
                 not). Further calls to
@@ -83,10 +83,11 @@
 
                 <para>If a daemon receives more than one file
                 descriptor, they will be passed in the same order as
-                configured in the systemd socket definition
-                file. Nonetheless, it is recommended to verify the
-                correct socket types before using them. To simplify
-                this checking, the functions
+                configured in the systemd socket unit file (see
+                <citerefentry><refentrytitle>systemd.socket</refentrytitle><manvolnum>5</manvolnum></citerefentry>
+                for details). Nonetheless, it is recommended to verify
+                the correct socket types before using them. To
+                simplify this checking, the functions
                 <citerefentry><refentrytitle>sd_is_fifo</refentrytitle><manvolnum>3</manvolnum></citerefentry>,
                 <citerefentry><refentrytitle>sd_is_socket</refentrytitle><manvolnum>3</manvolnum></citerefentry>,
                 <citerefentry><refentrytitle>sd_is_socket_inet</refentrytitle><manvolnum>3</manvolnum></citerefentry>,
@@ -103,6 +104,16 @@
                 <para>This function call will set the FD_CLOEXEC flag
                 for all passed file descriptors to avoid further
                 inheritance to children of the calling process.</para>
+
+                <para>If multiple socket units activate the same
+                service the order of the file descriptors passed to
+                its main process is undefined. If additional file
+                descriptors have been passed to the service manager
+                using
+                <citerefentry><refentrytitle>sd_pid_notify_with_fds</refentrytitle><manvolnum>3</manvolnum></citerefentry>'s
+                <literal>FDSTORE=1</literal> messages, these file
+                descriptors are passed last, in arbitrary order, and
+                with duplicates removed.</para>
         </refsect1>
 
         <refsect1>
diff --git a/man/sd_notify.xml b/man/sd_notify.xml
index 35f6f71ab..2bf3383c0 100644
--- a/man/sd_notify.xml
+++ b/man/sd_notify.xml
@@ -46,6 +46,9 @@
         <refnamediv>
                 <refname>sd_notify</refname>
                 <refname>sd_notifyf</refname>
+                <refname>sd_pid_notify</refname>
+                <refname>sd_pid_notifyf</refname>
+                <refname>sd_pid_notify_with_fds</refname>
                 <refpurpose>Notify service manager about start-up completion and other service status changes</refpurpose>
         </refnamediv>
 
@@ -65,6 +68,30 @@
                                 <paramdef>const char *<parameter>format</parameter></paramdef>
                                 <paramdef>...</paramdef>
                         </funcprototype>
+
+                        <funcprototype>
+                                <funcdef>int <function>sd_pid_notify</function></funcdef>
+                                <paramdef>pid_t <parameter>pid</parameter></paramdef>
+                                <paramdef>int <parameter>unset_environment</parameter></paramdef>
+                                <paramdef>const char *<parameter>state</parameter></paramdef>
+                        </funcprototype>
+
+                        <funcprototype>
+                                <funcdef>int <function>sd_pid_notifyf</function></funcdef>
+                                <paramdef>pid_t <parameter>pid</parameter></paramdef>
+                                <paramdef>int <parameter>unset_environment</parameter></paramdef>
+                                <paramdef>const char *<parameter>format</parameter></paramdef>
+                                <paramdef>...</paramdef>
+                        </funcprototype>
+
+                        <funcprototype>
+                                <funcdef>int <function>sd_pid_notify_with_fds</function></funcdef>
+                                <paramdef>pid_t <parameter>pid</parameter></paramdef>
+                                <paramdef>int <parameter>unset_environment</parameter></paramdef>
+                                <paramdef>const char *<parameter>state</parameter></paramdef>
+                                <paramdef>const int *<parameter>fds</parameter></paramdef>
+                                <paramdef>unsigned <parameter>n_fds</parameter></paramdef>
+                        </funcprototype>
                 </funcsynopsis>
         </refsynopsisdiv>
 
@@ -175,7 +202,7 @@
                         <varlistentry>
                                 <term>MAINPID=...</term>
 
-                                <listitem><para>The main pid of the
+                                <listitem><para>The main process ID (PID) of the
                                 service, in case the service manager did
                                 not fork off the process
                                 itself. Example:
@@ -185,7 +212,7 @@
                         <varlistentry>
                                 <term>WATCHDOG=1</term>
 
-                                <listitem><para>Tells systemd to
+                                <listitem><para>Tells the service manager to
                                 update the watchdog timestamp. This is
                                 the keep-alive ping that services need
                                 to issue in regular intervals if
@@ -199,12 +226,53 @@
                                 check if the the watchdog is enabled.
                                 </para></listitem>
                         </varlistentry>
+
+
+                        <varlistentry>
+                                <term>FDSTORE=1</term>
+
+                                <listitem><para>Stores additional file
+                                descriptors in the service
+                                manager. File descriptors sent this
+                                way will be maintained per-service by
+                                the service manager and be passed
+                                again using the usual file descriptor
+                                passing logic on the next invocation
+                                of the service (see
+                                <citerefentry><refentrytitle>sd_listen_fds</refentrytitle><manvolnum>3</manvolnum></citerefentry>). This
+                                is useful for implementing service
+                                restart schemes where services
+                                serialize their state to
+                                <filename>/run</filename>, push their
+                                file descriptors to the system
+                                manager, and are then restarted,
+                                retrieving their state again via
+                                socket passing and
+                                <filename>/run</filename>. Note that
+                                the service manager will accept
+                                messages for a service only if
+                                <varname>FileDescriptorStoreMax=</varname>
+                                is set to non-zero for it (defaults to
+                                zero). See
+                                <citerefentry><refentrytitle>systemd.service</refentrytitle><manvolnum>5</manvolnum></citerefentry>
+                                for details. Multiple arrays of file
+                                descriptors may be sent in seperate
+                                messages, in which case the arrays are
+                                combined. Note that the service
+                                manager removes duplicate file
+                                descriptors before passing them to the
+                                service. Use
+                                <function>sd_pid_notify_with_fds()</function>
+                                to send messages with
+                                <literal>FDSTORE=1</literal>, see
+                                below.</para></listitem>
+                        </varlistentry>
+
                 </variablelist>
 
                 <para>It is recommended to prefix variable names that
-                are not shown in the list above with
-                <varname>X_</varname> to avoid namespace
-                clashes.</para>
+                are not listed above with <varname>X_</varname> to
+                avoid namespace clashes.</para>
 
                 <para>Note that systemd will accept status data sent
                 from a service only if the
@@ -217,6 +285,36 @@
                 <function>sd_notify()</function> but takes a
                 <function>printf()</function>-like format string plus
                 arguments.</para>
+
+                <para><function>sd_pid_notify()</function> and
+                <function>sd_pid_notifyf()</function> are similar to
+                <function>sd_notify()</function> and
+                <function>sd_notifyf()</function> but take a process
+                ID (PID) to use as originating PID for the message as
+                first argument. This is useful to send notification
+                messages on behalf of other processes, provided the
+                appropriate privileges are available. If the PID
+                argument is specified as 0 the process ID of the
+                calling process is used, in which case the calls are
+                fully equivalent to <function>sd_notify()</function>
+                and <function>sd_notifyf()</function>.</para>
+
+                <para><function>sd_pid_notify_with_fds()</function> is
+                similar to <function>sd_pid_notify()</function> but
+                takes an additional array of file descriptors. These
+                file descriptors are sent along the notification
+                message to the service manager. This is particularly
+                useful for sending <literal>FDSTORE=1</literal>
+                messages, as described above. The additional arguments
+                are a pointer to the file descriptor array plus the
+                number of file descriptors in the array. If the number
+                of file descriptors is passed as 0, the call is fully
+                equivalent to <function>sd_pid_notify()</function>,
+                i.e. no file descriptors are passed. Note that sending
+                file descriptors to the service manager on messages
+                that do not expect them (i.e. without
+                <literal>FDSTORE=1</literal>) they are immediately
+                closed on reception.</para>
         </refsect1>
 
         <refsect1>
@@ -295,13 +393,25 @@
                 <example>
                         <title>Error Cause Notification</title>
 
-                        <para>A service could send the following shortly before exiting, on failure</para>
+                        <para>A service could send the following shortly before exiting, on failure:</para>
 
                         <programlisting>sd_notifyf(0, "STATUS=Failed to start up: %s\n"
               "ERRNO=%i",
               strerror(errno),
               errno);</programlisting>
                 </example>
+
+                <example>
+                        <title>Store a File Descriptor in the Service Manager</title>
+
+                        <para>To store an open file descriptor in the
+                        service manager, in order to continue
+                        operation after a service restart without
+                        losing state use
+                        <literal>FDSTORE=1</literal>:</para>
+
+                        <programlisting>sd_pid_notify_with_fds(0, 0, "FDSTORE=1", &amp;fd, 1);</programlisting>
+                </example>
         </refsect1>
 
         <refsect1>
diff --git a/man/systemd.service.xml b/man/systemd.service.xml
index 0b68aa089..4c890dfb7 100644
--- a/man/systemd.service.xml
+++ b/man/systemd.service.xml
@@ -1117,6 +1117,35 @@
                                 command.</para></listitem>
                         </varlistentry>
 
+                        <varlistentry>
+                                <term><varname>FileDescriptorStoreMax=</varname></term>
+                                <listitem><para>Configure how many
+                                file descriptors may be stored in the
+                                service manager for the service using
+                                <citerefentry><refentrytitle>sd_pid_notify_with_fds</refentrytitle><manvolnum>3</manvolnum></citerefentry>'s
+                                <literal>FDSTORE=1</literal>
+                                messages. This is useful for
+                                implementing service restart schemes
+                                where the state is serialized to
+                                <filename>/run</filename> and the file
+                                descriptors passed to the service
+                                manager, to allow restarts without
+                                losing state. Defaults to 0, i.e. no
+                                file descriptors may be stored in the
+                                service manager by default. All file
+                                descriptors passed to the service
+                                manager from a specific service are
+                                passed back to the service's main
+                                process on the next service
+                                restart. Any file descriptors passed
+                                to the service manager are
+                                automatically closed when POLLHUP or
+                                POLLERR is seen on them, or when the
+                                service is fully stopped and no job
+                                queued or being executed for
+                                it.</para></listitem>
+                        </varlistentry>
+
                 </variablelist>
 
                 <para>Check