Push notifications on linux with cronjobs and smartmontools

Up until very recently, I was having smartmontools (and other cron processes such as backups) email me when a hard drive was failing. For some reason my already-finicky mail-forwarding setup ended up failing and I couldn’t get it to consistently send again, so I thought to myself: What other ways can I have my server notify me in an obvious way? The answer was Push Notifications!

Now, I wouldn’t attest to knowing the details behind post-notifications; my quest was for a quick and easy solution for Android: Simplepush.io.

Pros of Simplepush:

• Super easy to set up
• “Just works”
• Supports end-to-end encryption
• Works for android

Cons:

• Costs $4 $10/yr
• Doesn’t seem to work for iOS (works for iOS now)
• Isn’t a self-hosted solution, if that matters to you.

I did find a free solution that would also work with iOS using Telegram, and maybe I’ll do that if simplepush.io ever disappears: Telegram API: Send Message – Personal Notification Bot

Anyway, let’s get into how I did this!

The scripts

First, I used the script freely available by simplepush: simplepush/send-encrypted

I saved this in /usr/local/bin

Next, I created a new script in /usr/local/bin/ called “simplypush_wrapper.sh” with the following content:

#!/bin/bash

# Edit this part to contain your PSWD='PASSWORD'
# Change PASSWORD to correct password (include single-quotes)
SALT='SALT'   # Change SALT to correct salt
SPKEY='KEY'    # Change KEY to the simplepush key generated by the app.

# No need to edit anything below.
# Set MSG, TITLE, and EVENTs based on arguments, if they exist.
if [ -z "$1" ]; then MSG='empty-msg'; else MSG="$1"; fi
if [ -z "$2" ]; then TITLE='Push-notification'; else TITLE="$2"; fi
if [ -z "$3" ]; then EVENT='misc'; else EVENT="$3"; fi

# Usage "/usr/local/bin/simplepush_wrapper.sh 'message' 'title' 'event'
/usr/local/bin/send-encrypted.sh -k "$SPKEY" -p "$PSWD" -s "$SALT" -m "$MSG" -t "$TITLE" -e "$EVENT"

(be sure to replace PASSWORD, SALT, and KEY with your own password, salt, and key from the app!)

Now, I can call this wrapper script and it will always send a push notification to my phone like so:

ajadmin@nazz ~ $ /usr/local/bin/simplepush_wrapper.sh "A message" "A title" "Event."

This is the result:

Now I can start rewriting my cron jobs to utilize this wrapper. Most of these are simple. For example, my array uses btrfs for RAID10. Every Sunday at 9am, I run a scrub on the RAID10 in my NAS. The result (regardless of how it completes) are emailed to me:

# Run every sunday at 9pm, email me the results
0 21 * * 0 btrfs scrub start -c3 -B /mnt/RAID10

The simplest way I thought to rewrite this would just to capture the output of this command and send it in as an argument to my wrapper. Thankfully we have “$()” for that!

# Run every sunday at 9pm, push the results
0 21 * * 0 /usr/local/bin/simplepush_wrapper.sh "$(btrfs scrub start -c3 -B /mnt/RAID10)" 'btrfs-scrub' 'status'

Simple, right? Let’s try something a little more complicated.

Every hour, I check to see if there’s any errors being reported to btrfs with the following crontab entry:

@hourly /sbin/btrfs device stats /mnt/RAID10 | grep -vE ' 0$'

Since the command /sbin/btrfs device stats /mnt/RAID10 normally ends in “ 0” if there is no error for that disk, this command will normally yield no output. For my cronjob, it means I would not get an email. The moment the array starts to spit out errors, I would get an email from my server on the hour, indicating which drive is failing. Fun!

While I may be able to be just as hacky with my wrapper, I figured I’d just put it in a one-line if statement:

# Check btrfs- email me on drive failure
@hourly if /sbin/btrfs device stats /mnt/RAID10 | grep -vE ' 0$'; then /usr/local/bin/simplepush_wrapper.sh "$(/sbin/btrfs device stats /mnt/RAID10)" 'btrfs error' 'failure'; fi

Of course, I’m going to have to modify some of my backup scripts to be more nicely formatted, instead of perhaps dumping everything under the sun to me like it used to, but you get the point.

What about smartmontools?

While it’s nice that I’m getting push notifications from crontab, smartmontools has a bit of a different way of handling things. By default, it will email you the alerts when something is going FUBAR but you can actually change this in the /etc/smartd.conf file. Originally, I had something along the lines of this:

/dev/sdX -a -o on -S on -s (S/../.././16|L/../../6/00) -W 4,35,40 -m email@example.com -M daily

I found that smartmontools actually has an argument, -M exec script that allows you to execute a specific script while passing a variable $SMARTD_MESSAGE instead of sending an email. So I created /usr/local/bin/smartdnotify.sh with the following content:

/dev/sdX -a -o on -S on -s (S/../.././16|L/../../6/00) -W 4,35,40 -m root -M exec /usr/local/bin/smartdnotify.sh -M daily

With that out of the way, I modified /etc/smartd.conf with the following:

/dev/sdX -a -o on -S on -s (S/../.././16|L/../../6/00) -W 4,35,40 -m root -M exec /usr/local/bin/smartdnotify.sh -M daily

A note on security

Since crontab is touching these scripts in /usr/local/bin, I would ensure that they’re only editable by root, but executable by anyone:

ajadmin@nazz ~ $ sudo chown root:root -R /usr/local/bin && sudo chmod 755 -R /usr/local/bin
ajadmin@nazz ~ $ ls -lh /usr/local/bin
total 24K
-rwxr-xr-x 1 root root 4.2K Aug  5  2017 rsnapreport.pl
-rwxr-xr-x 1 root root 1.5K Aug 25 00:02 send-encrypted.sh
-rwxr-xr-x 1 root root  736 Aug 31 18:18 simplepush_wrapper.sh
-rwxr-xr-x 1 root root  136 Sep  1 18:06 smartdnotify.sh
ajadmin@nazz ~ $ 

That should be about it! Did I miss anything? What do you think? Let me know in the comments!