Using multiple fetchmail instances for instant gratification

For various reasons I have 10 email accounts spread across various servers for the family and me, and have always used an instance of fetchmail to round-robin connect to these and download all emails locally (see my previous post about cloud email). But this means I might have to wait 5-10 minutes to get new emails! But no longer...

As it happens, all the mail servers I use support IMAP (as they should), and fetchmail supports the IMAP IDLE command, which lets you connect and then get instant notifications to changes to the mail folders (in this case, fetchmail instantly slurps new emails down to your box).

However, it can only do this if only one email account is configured, and it is not trivial to run multiple fetchmail instances (each with a different config. file pointing to a separate email account).

But, it's also not that tricky, and essentially boils down to running each one with a unique value for the environment variable FETCHMAILHOME so that it has somewhere unique to put its PID file.

So, I wrote the following /usr/sbin/fetchmail-service script (a modified version of the one-shot /etc/init.d/ version I've been running for years).  It creates a separate fetchmail instance for each /etc/fetchmail.conf and /etc/fetchmail.d/*.conf it finds, so each of these can have a single account with IMAP IDLE set.

For now, it still uses /etc/init.d functions;  I'll tidy that up at some point.

  1. #!/bin/sh
  2. #
  3. # fetchmail This shell script takes care of starting and stopping
  4. # the fetchmail DAEMON.
  5. #
  6. #
  7. # chkconfig: 2345 95 5
  8. # description: fetchmail fetchs mail from pop3/imap mail servers
  9.  
  10. fetchmailuser=mail
  11. fetchmailgroup=mail
  12. fetchmailddir=/var/lib/fetchmail
  13.  
  14. # Source function library.
  15. . /etc/rc.d/init.d/functions
  16. # Source networking configuration.
  17. . /etc/sysconfig/network
  18. # Check that networking is up.
  19. [ ${NETWORKING} = "no" ] && exit 0
  20. [ -f /usr/bin/fetchmail ] || exit 0
  21.  
  22. # See how we were called.
  23. case "$1" in
  24. start)
  25. # Start daemons.
  26. echo -n "Starting fetchmail: "
  27. update_boot_stage 'Starting email fetching service'
  28. conffile=/etc/fetchmail.conf
  29. [ -f "$conffile" ] && FETCHMAILHOME=$fetchmailddirs daemon --user=$fetchmailuser /usr/bin/fetchmail --daemon 60 --syslog --fetchmailrc "$conffile"
  30. if [ ! -d "$fetchmailddir" ]; then
  31. mkdir "$fetchmailddir"
  32. chown $fetchmailuser:$fetchmailgroup "$fetchmailddir"
  33. chmod 700 "$fetchmailddir"
  34. fi
  35. for conffile in /etc/fetchmail.conf.d/*.conf; do
  36. bname=$(basename "$conffile")
  37. fetchmailddirsub=$fetchmailddir/$bname
  38. if [ ! -d "$fetchmailddirsub" ]; then
  39. mkdir "$fetchmailddirsub"
  40. chown $fetchmailuser:$fetchmailgroup "$fetchmailddirsub"
  41. chmod 700 "$fetchmailddirsub"
  42. fi
  43. [ -f "$conffile" ] && FETCHMAILHOME=$fetchmailddirsub daemon --user=$fetchmailuser /usr/bin/fetchmail --daemon 60 --syslog --fetchmailrc "$conffile"
  44. done
  45. touch /var/lock/subsys/fetchmail
  46. success "fetchmail started"
  47. echo
  48. ;;
  49.  
  50. stop)
  51. # Stop daemons.
  52. echo -n "Shutting down fetchmail: "
  53. for instance in $fetchmailddir/*/*.pid; do
  54. fetchmailddirsub=$(dirname $instance)
  55. if [ -f "$instance" ]; then
  56. pid=$(head -1 "$instance")
  57. FETCHMAILHOME=$fetchmailddirsub su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
  58. fi
  59. rmdir $fetchmailddirsub
  60. done
  61. for instance in $fetchmailddir/*.pid; do
  62. if [ -f "$instance" ]; then
  63. su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
  64. fi
  65. done
  66. rm -f /var/lock/subsys/fetchmail
  67. success "fetchmail stopped"
  68. echo
  69. ;;
  70.  
  71. restart)
  72. $0 stop
  73. $0 start
  74. ;;
  75.  
  76. status)
  77. status fetchmail
  78. ;;
  79.  
  80. *)
  81. echo "Usage: fetchmail {start|stop|restart|status}"
  82. exit 1
  83. esac
  84. exit 0
  85.  

Cross fingers, it's only been running with the lot for quarter of an hour, but it seems to be doing the job!

Finally, for some reason the Fedora fetchmail RPM has never come with service control scripts (hence my writing the above one).  I've now moved the init.d version to /usr/sbin as mentioned, and written a functional systemd config. file for it that will cleanly manage all instances:

[Unit]
Description=A remote mail retrieval and forwarding utility
After=sendmail.service

[Service]
ExecStart=/usr/sbin/fetchmail-service start
ExecReload=/usr/sbin/fetchmail-service restart
ExecStop=/usr/sbin/fetchmail-service stop
Type=forking
GuessMainPID=no

[Install]
WantedBy=multi-user.target


Advertisement


About fnx

PSN: the_phoenix

This entry was posted in Software and tagged , , , , , , . Bookmark the permalink.

9 Responses to Using multiple fetchmail instances for instant gratification

  1. andrew says:

    Thanks for posting your code. I adapted it for use with debian:


    #!/bin/sh
    #
    # Fetchmail2 init script
    # Latest change: Oct 14 2012
    #
    ### BEGIN INIT INFO
    # Provides: fetchmail
    # Required-Start: $network $local_fs $remote_fs $syslog
    # Required-Stop: $remote_fs
    # Should-Start: $mail-transport-agent exim4 $named
    # Should-Stop: $mail-transport-agent exim4
    # Default-Start: 2 3 4 5
    # Default-Stop:
    # Short-Description: init-Script for system wide fetchmail daemon
    ### END INIT INFO
    #
    # A fetchmailrc file containg hosts and passwords for all local users should be
    # placed in /etc/fetchmailrc. Remember to make the /etc/fetchmailrc mode 600
    # to avoid disclosing the users' passwords.
    #
    # This is a modified version designed to start multiple instances of fetchmail
    # Creates an instance for each conf file in /etc/fetchmail.conf.d
    #

    set -e

    # Defaults
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    DAEMON=/usr/bin/fetchmail
    fetchmailuser=fetchmail
    fetchmailgroup=nogroup
    OPTIONS="-v "

    fetchmailddir="/var/run/fetchmail"
    UIDL="/var/lib/fetchmail/.fetchmail-UIDL-cache"
    START_DAEMON="no"

    . /lib/lsb/init-functions

    if [ -r /etc/default/fetchmail ]; then
    . /etc/default/fetchmail
    fi

    if [ ! "x$START_DAEMON" = "xyes" -a ! "$1" = "status" ]; then
    echo "Edit /etc/default/fetchmail to start/stop fetchmail"
    exit 0
    fi

    test -f $DAEMON || exit 0

    case "$1" in
    start)
    for conffile in /etc/fetchmail.conf.d/*.conf; do
    bname=$(basename "$conffile")
    fetchmailddirsub=$fetchmailddir/$bname
    if [ ! -d "$fetchmailddirsub" ]; then
    mkdir -p "$fetchmailddirsub"
    chown $fetchmailuser:$fetchmailgroup "$fetchmailddirsub"
    chmod 700 "$fetchmailddirsub"
    fi
    PIDFILE=$fetchmailddirsub/fetchmail.pid
    OPTIONS2="$OPTIONS --fetchmailrc $conffile"
    echo "start $conffile"
    FETCHMAILHOME=$fetchmailddirsub start-stop-daemon -S -o -q -p $PIDFILE -x $DAEMON -u $fetchmailuser -c $fetchmailuser -- $OPTIONS2
    done
    ;;
    stop)
    for instance in $fetchmailddir/*/*.pid; do
    fetchmailddirsub=$(dirname $instance)
    if [ -f "$instance" ]; then
    pid=$(head -1 "$instance")
    PIDFILE=$fetchmailddirsub/fetchmail.pid
    FETCHMAILHOME=$fetchmailddirsub start-stop-daemon -K -o -p $instance -x $DAEMON -u $fetchmailuser
    # FETCHMAILHOME=$fetchmailddirsub su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
    fi
    rmdir $fetchmailddirsub
    done
    for instance in $fetchmailddir/*.pid; do
    if [ -f "$instance" ]; then
    su $fetchmailuser -c '/usr/bin/fetchmail --pidfile "$instance" --quit' > /dev/null 2>&1
    fi
    done
    rm -f /var/lock/subsys/fetchmail
    success "fetchmail stopped"
    echo
    ;;
    force-reload|restart)
    log_begin_msg "Restarting mail retriever agent:" "fetchmail"
    if ! start-stop-daemon -K -o -q -p $PIDFILE -x $DAEMON -u $USER; then
    log_end_msg 1
    exit 1
    fi
    sleep 1
    if start-stop-daemon -S -q -p $PIDFILE -x $DAEMON -u $USER -c $USER -- $OPTIONS; then
    log_end_msg 0
    else
    log_end_msg 1
    exit 1
    fi
    ;;
    try-restart)
    if test -e $PIDFILE ; then
    pid=`cat $PIDFILE | sed -e 's/\s.*//'|head -n1`
    PIDDIR=/proc/$pid
    if [ -d ${PIDDIR} -a "$(readlink -f ${PIDDIR}/exe)" = "${DAEMON}" ]; then
    $0 restart
    exit 0
    fi
    fi
    test -f /etc/rc`/sbin/runlevel | cut -d' ' -f2`.d/S*fetchmail* && $0 start
    ;;
    awaken)
    log_begin_msg "Awakening mail retriever agent:" "fetchmail"
    if [ -s $PIDFILE ]; then
    start-stop-daemon -K -s 10 -q -p $PIDFILE -x $DAEMON
    log_end_msg 0
    exit 0
    else
    log_end_msg 1
    exit 1
    fi
    ;;
    debug-run)
    echo "$0: Initiating debug run of system-wide fetchmail service..." 1>&2
    echo "$0: script will be run in debug mode, all output to forced to" 1>&2
    echo "$0: stdout. This is not enough to debug failures that only" 1>&2
    echo "$0: happen in daemon mode." 1>&2
    echo "$0: You might want to direct output to a file, and tail -f it." 1>&2
    if [ "$2" = "strace" ]; then
    echo "$0: (running debug mode under strace. See strace(1) for options)" 1>&2
    echo "$0: WARNING: strace output may contain security-sensitive info, such as" 1>&2
    echo "$0: passwords; please clobber them before sending the strace file to a" 1>&2
    echo "$0: public bug tracking system, such as Debian's." 1>&2
    fi
    echo "$0: Stopping the service..." 1>&2
    "$0" stop
    echo "$0: exit status of service stop was: $?"
    echo "$0: RUNUSER is $USER"
    echo "$0: OPTIONS would be $OPTIONS"
    echo "$0: Starting service in nodetach mode, hit ^C (SIGINT/intr) to finish run..." 1>&2
    if [ "$2" = "strace" ] ; then
    shift
    shift
    [ $# -ne 0 ] && echo "$0: (strace options are: -tt $@)" 1>&2
    su -s /bin/sh -c "/usr/bin/strace -tt $* $DAEMON $OPTIONS --nosyslog --nodetach -v -v" $USER &1
    else
    su -s /bin/sh -c "$DAEMON $OPTIONS --nosyslog --nodetach -v -v" $USER &1
    fi
    echo "$0: End of service run. Exit status was: $?"
    exit 0
    ;;
    *)
    log_warning_msg "Usage: /etc/init.d/fetchmail {start|stop|restart|force-reload|awaken|debug-run}"
    log_warning_msg " start - starts system-wide fetchmail service"
    log_warning_msg " stop - stops system-wide fetchmail service"
    log_warning_msg " restart, force-reload - starts a new system-wide fetchmail service"
    log_warning_msg " awaken - tell system-wide fetchmail to start a poll cycle immediately"
    log_warning_msg " debug-run [strace [strace options...]] - start a debug run of the"
    log_warning_msg " system-wide fetchmail service, optionally running it under strace"
    exit 1
    ;;
    esac

    exit 0

  2. fnx says:

    Thanks for that; unfortunately, it looks like formatting in comments is a bit hit & miss!

  3. Thanks for this, I just set it up on my server (F19) and it’s working fine. I had to twiddle SELinux a bit to make it work with SELinux enabled, though (allow fetchmail to write to /var/lib/fetchmail, essentially). Do post somewhere if you update this…

  4. fnx says:

    I’ve yet to brave enabling SElinux, and am also still on F16 (long story, should be updating before too long). I plan to use selinux=permissive once the new update is stable, so I’d be interested in what you had to do.

    I have never had to look into selinux stuff in anger (beyond following popup instructions to fix stuff on my laptop and doing some simple file context changes on a server I access), so I’m blissfully unaware of the details of that whole area.

  5. fnx says:

    Oh yes, and I’ve since found that the pseudo service scripts go somewhere else now (some libexec dir. IIRC), as opposed to the /usr/sbin/*-service name I currently use. Not that it really matters, I guess.

  6. Pingback: Unix/Linux:Use /etc/init.d/fetchmail with the personnal ~/.fetchmailrc – Unix Questions

  7. Volker Wysk says:

    Hi!

    I’ve tried your script. Alas, it failed, because I don’t have /etc/rc.d/init.d/functions and /etc/sysconfig/network. I run a (K)Ubuntu system.

    bye
    Volker

  8. fnx says:

    I did some tidying up a while ago, and removing at least one of those dependencies was something I did. Possibly both refs. are now gone. I’ve copied the latest versions of both files to:

    http://www.fnxweb.com/data/fetchmail-service
    http://www.fnxweb.com/data/fetchmail.service

    Note the new, more correct, location of the fetchmail-service script is /usr/libexec. I don’t know if *buntu uses the sysconfig architecture (for the customisation file).

  9. Volker Wysk says:

    I succeeded with Andrew’s debianized version above, and your unit file.

    Thnx!

Leave a Reply

Your email address will not be published. Required fields are marked *

*