Using multiple fetchmail instances for instant gratification

For various reasons I have 10 email accounts spread across various servers for the family and me, and have always used an instance of fetchmail to round-robin connect to these and download all emails locally (see my previous post about cloud email). But this means I might have to wait 5-10 minutes to get new emails! But no longer…

As it happens, all the mail servers I use support IMAP (as they should), and fetchmail supports the IMAP IDLE command, which lets you connect and then get instant notifications to changes to the mail folders (in this case, fetchmail instantly slurps new emails down to your box).

However, it can only do this if only one email account is configured, and it is not trivial to run multiple fetchmail instances (each with a different config. file pointing to a separate email account).

But, it’s also not that tricky, and essentially boils down to running each one with a unique value for the environment variable FETCHMAILHOME so that it has somewhere unique to put its PID file.

So, I wrote the following /usr/sbin/fetchmail-service script (a modified version of the one-shot /etc/init.d/ version I’ve been running for years).  It creates a separate fetchmail instance for each /etc/fetchmail.conf and /etc/fetchmail.d/*.conf it finds, so each of these can have a single account with IMAP IDLE set.

For now, it still uses /etc/init.d functions;  I’ll tidy that up at some point.

#!/bin/sh
#
# fetchmail This shell script takes care of starting and stopping
# the fetchmail DAEMON.
#
#
# chkconfig: 2345 95 5
# description: fetchmail fetchs mail from pop3/imap mail servers
 
fetchmailuser=mail
fetchmailgroup=mail
fetchmailddir=/var/lib/fetchmail
 
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
[ -f /usr/bin/fetchmail ] || exit 0
 
# See how we were called.
case "$1" in
    start)
        # Start daemons.
        echo -n "Starting fetchmail: "
        update_boot_stage 'Starting email fetching service'
	conffile=/etc/fetchmail.conf
	[ -f "$conffile" ]  &&  FETCHMAILHOME=$fetchmailddir daemon --user=$fetchmailuser /usr/bin/fetchmail --daemon 60 --syslog --fetchmailrc "$conffile"
	if [ ! -d "$fetchmailddir" ];  then
		mkdir "$fetchmailddir"
		chown $fetchmailuser:$fetchmailgroup "$fetchmailddir"
		chmod 700 "$fetchmailddir"
	fi
        for conffile in /etc/fetchmail.conf.d/*.conf;  do
		bname=$(basename "$conffile")
		fetchmailddirsub=$fetchmailddir/$bname
		if [ ! -d "$fetchmailddirsub" ];  then
			mkdir "$fetchmailddirsub"
			chown $fetchmailuser:$fetchmailgroup "$fetchmailddirsub"
			chmod 700 "$fetchmailddirsub"
		fi
		[ -f "$conffile" ]  &&  FETCHMAILHOME=$fetchmailddirsub daemon --user=$fetchmailuser /usr/bin/fetchmail --daemon 60 --syslog --fetchmailrc "$conffile"
        done
        touch /var/lock/subsys/fetchmail
        success "fetchmail started"
        echo
        ;;
 
    stop)
        # Stop daemons.
        echo -n "Shutting down fetchmail: "
	for instance in $fetchmailddir/*/*.pid;  do
		fetchmailddirsub=$(dirname $instance)
		if [ -f "$instance" ];  then
			pid=$(head -1 "$instance")
			FETCHMAILHOME=$fetchmailddirsub su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
		fi
		rmdir $fetchmailddirsub
	done
	for instance in $fetchmailddir/*.pid;  do
		if [ -f "$instance" ];  then
			su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
		fi
	done
        rm -f /var/lock/subsys/fetchmail
        success "fetchmail stopped"
        echo
        ;;
 
    restart)
        $0 stop
        $0 start
        ;;
 
    status)
        status fetchmail
        ;;
 
    *)
        echo "Usage: fetchmail {start|stop|restart|status}"
        exit 1
esac
exit 0

Cross fingers, it’s only been running with the lot for quarter of an hour, but it seems to be doing the job!

Finally, for some reason the Fedora fetchmail RPM has never come with service control scripts (hence my writing the above one).  I’ve now moved the init.d version to /usr/sbin as mentioned, and written a functional systemd config. file for it that will cleanly manage all instances:

[Unit]
Description=A remote mail retrieval and forwarding utility
After=sendmail.service

[Service]
ExecStart=/usr/sbin/fetchmail-service start
ExecReload=/usr/sbin/fetchmail-service restart
ExecStop=/usr/sbin/fetchmail-service stop
Type=forking
GuessMainPID=no

[Install]
WantedBy=multi-user.target

Posted

in

by

Comments

11 responses to “Using multiple fetchmail instances for instant gratification”

  1. andrew Avatar

    Thanks for posting your code. I adapted it for use with debian:


    #!/bin/sh
    #
    # Fetchmail2 init script
    # Latest change: Oct 14 2012
    #
    ### BEGIN INIT INFO
    # Provides: fetchmail
    # Required-Start: $network $local_fs $remote_fs $syslog
    # Required-Stop: $remote_fs
    # Should-Start: $mail-transport-agent exim4 $named
    # Should-Stop: $mail-transport-agent exim4
    # Default-Start: 2 3 4 5
    # Default-Stop:
    # Short-Description: init-Script for system wide fetchmail daemon
    ### END INIT INFO
    #
    # A fetchmailrc file containg hosts and passwords for all local users should be
    # placed in /etc/fetchmailrc. Remember to make the /etc/fetchmailrc mode 600
    # to avoid disclosing the users' passwords.
    #
    # This is a modified version designed to start multiple instances of fetchmail
    # Creates an instance for each conf file in /etc/fetchmail.conf.d
    #

    set -e

    # Defaults
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    DAEMON=/usr/bin/fetchmail
    fetchmailuser=fetchmail
    fetchmailgroup=nogroup
    OPTIONS="-v "

    fetchmailddir="/var/run/fetchmail"
    UIDL="/var/lib/fetchmail/.fetchmail-UIDL-cache"
    START_DAEMON="no"

    . /lib/lsb/init-functions

    if [ -r /etc/default/fetchmail ]; then
    . /etc/default/fetchmail
    fi

    if [ ! "x$START_DAEMON" = "xyes" -a ! "$1" = "status" ]; then
    echo "Edit /etc/default/fetchmail to start/stop fetchmail"
    exit 0
    fi

    test -f $DAEMON || exit 0

    case "$1" in
    start)
    for conffile in /etc/fetchmail.conf.d/*.conf; do
    bname=$(basename "$conffile")
    fetchmailddirsub=$fetchmailddir/$bname
    if [ ! -d "$fetchmailddirsub" ]; then
    mkdir -p "$fetchmailddirsub"
    chown $fetchmailuser:$fetchmailgroup "$fetchmailddirsub"
    chmod 700 "$fetchmailddirsub"
    fi
    PIDFILE=$fetchmailddirsub/fetchmail.pid
    OPTIONS2="$OPTIONS --fetchmailrc $conffile"
    echo "start $conffile"
    FETCHMAILHOME=$fetchmailddirsub start-stop-daemon -S -o -q -p $PIDFILE -x $DAEMON -u $fetchmailuser -c $fetchmailuser -- $OPTIONS2
    done
    ;;
    stop)
    for instance in $fetchmailddir/*/*.pid; do
    fetchmailddirsub=$(dirname $instance)
    if [ -f "$instance" ]; then
    pid=$(head -1 "$instance")
    PIDFILE=$fetchmailddirsub/fetchmail.pid
    FETCHMAILHOME=$fetchmailddirsub start-stop-daemon -K -o -p $instance -x $DAEMON -u $fetchmailuser
    # FETCHMAILHOME=$fetchmailddirsub su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
    fi
    rmdir $fetchmailddirsub
    done
    for instance in $fetchmailddir/*.pid; do
    if [ -f "$instance" ]; then
    su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
    fi
    done
    rm -f /var/lock/subsys/fetchmail
    success "fetchmail stopped"
    echo
    ;;
    force-reload|restart)
    log_begin_msg "Restarting mail retriever agent:" "fetchmail"
    if ! start-stop-daemon -K -o -q -p $PIDFILE -x $DAEMON -u $USER; then
    log_end_msg 1
    exit 1
    fi
    sleep 1
    if start-stop-daemon -S -q -p $PIDFILE -x $DAEMON -u $USER -c $USER -- $OPTIONS; then
    log_end_msg 0
    else
    log_end_msg 1
    exit 1
    fi
    ;;
    try-restart)
    if test -e $PIDFILE ; then
    pid=`cat $PIDFILE | sed -e 's/\s.*//'|head -n1`
    PIDDIR=/proc/$pid
    if [ -d ${PIDDIR} -a "$(readlink -f ${PIDDIR}/exe)" = "${DAEMON}" ]; then
    $0 restart
    exit 0
    fi
    fi
    test -f /etc/rc`/sbin/runlevel | cut -d' ' -f2`.d/S*fetchmail* && $0 start
    ;;
    awaken)
    log_begin_msg "Awakening mail retriever agent:" "fetchmail"
    if [ -s $PIDFILE ]; then
    start-stop-daemon -K -s 10 -q -p $PIDFILE -x $DAEMON
    log_end_msg 0
    exit 0
    else
    log_end_msg 1
    exit 1
    fi
    ;;
    debug-run)
    echo "$0: Initiating debug run of system-wide fetchmail service..." 1>&2
    echo "$0: script will be run in debug mode, all output to forced to" 1>&2
    echo "$0: stdout. This is not enough to debug failures that only" 1>&2
    echo "$0: happen in daemon mode." 1>&2
    echo "$0: You might want to direct output to a file, and tail -f it." 1>&2
    if [ "$2" = "strace" ]; then
    echo "$0: (running debug mode under strace. See strace(1) for options)" 1>&2
    echo "$0: WARNING: strace output may contain security-sensitive info, such as" 1>&2
    echo "$0: passwords; please clobber them before sending the strace file to a" 1>&2
    echo "$0: public bug tracking system, such as Debian's." 1>&2
    fi
    echo "$0: Stopping the service..." 1>&2
    "$0" stop
    echo "$0: exit status of service stop was: $?"
    echo "$0: RUNUSER is $USER"
    echo "$0: OPTIONS would be $OPTIONS"
    echo "$0: Starting service in nodetach mode, hit ^C (SIGINT/intr) to finish run..." 1>&2
    if [ "$2" = "strace" ] ; then
    shift
    shift
    [ $# -ne 0 ] && echo "$0: (strace options are: -tt $@)" 1>&2
    su -s /bin/sh -c "/usr/bin/strace -tt $* $DAEMON $OPTIONS --nosyslog --nodetach -v -v" $USER &1
    else
    su -s /bin/sh -c "$DAEMON $OPTIONS --nosyslog --nodetach -v -v" $USER &1
    fi
    echo "$0: End of service run. Exit status was: $?"
    exit 0
    ;;
    *)
    log_warning_msg "Usage: /etc/init.d/fetchmail {start|stop|restart|force-reload|awaken|debug-run}"
    log_warning_msg " start - starts system-wide fetchmail service"
    log_warning_msg " stop - stops system-wide fetchmail service"
    log_warning_msg " restart, force-reload - starts a new system-wide fetchmail service"
    log_warning_msg " awaken - tell system-wide fetchmail to start a poll cycle immediately"
    log_warning_msg " debug-run [strace [strace options...]] - start a debug run of the"
    log_warning_msg " system-wide fetchmail service, optionally running it under strace"
    exit 1
    ;;
    esac

    exit 0

  2. fnx Avatar

    Thanks for that; unfortunately, it looks like formatting in comments is a bit hit & miss!

  3. Adam Williamson Avatar

    Thanks for this, I just set it up on my server (F19) and it’s working fine. I had to twiddle SELinux a bit to make it work with SELinux enabled, though (allow fetchmail to write to /var/lib/fetchmail, essentially). Do post somewhere if you update this…

  4. fnx Avatar

    I’ve yet to brave enabling SElinux, and am also still on F16 (long story, should be updating before too long). I plan to use selinux=permissive once the new update is stable, so I’d be interested in what you had to do.

    I have never had to look into selinux stuff in anger (beyond following popup instructions to fix stuff on my laptop and doing some simple file context changes on a server I access), so I’m blissfully unaware of the details of that whole area.

  5. fnx Avatar

    Oh yes, and I’ve since found that the pseudo service scripts go somewhere else now (some libexec dir. IIRC), as opposed to the /usr/sbin/*-service name I currently use. Not that it really matters, I guess.

  6. Volker Wysk Avatar

    Hi!

    I’ve tried your script. Alas, it failed, because I don’t have /etc/rc.d/init.d/functions and /etc/sysconfig/network. I run a (K)Ubuntu system.

    bye
    Volker

  7. fnx Avatar

    I did some tidying up a while ago, and removing at least one of those dependencies was something I did. Possibly both refs. are now gone. I’ve copied the latest versions of both files to:

    http://www.fnxweb.com/data/fetchmail-service
    http://www.fnxweb.com/data/fetchmail.service

    Note the new, more correct, location of the fetchmail-service script is /usr/libexec. I don’t know if *buntu uses the sysconfig architecture (for the customisation file).

  8. Volker Wysk Avatar

    I succeeded with Andrew’s debianized version above, and your unit file.

    Thnx!

  9. Thomas Avatar
    Thomas

    Is this variable “$fetchmailddirs” a typo as it is nowhere assigned a value?

    1. fnx Avatar

      Yep, that’s a typo, long fixed on my local copy (which is now on its third major iteration [#2 was systemd] as I run it ATM directly on my QNAP NAS using an opkg environment which is a slightly modified init.d style.

      The typo line is to cater for a “core“ (normal) way of running fetchmail, but I don’t actually use that I only have entries in my subdirs so I must’ve not noticed that error for a while, good catch!

      I’ll be moving back to systemd at some point, as I’m slowly migrating it back to Fedora, albeit in a container.

Leave a Reply to fnx Cancel reply

Your email address will not be published. Required fields are marked *

*

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close