Categories
Scripts Tutorials

ZFS health check in Grafana / Telegraf

I recently set up a ZFS mirror on my home server and found myself needing a way to be alerted if something went wrong. That same server runs Grafana and InfluxDB, and collects various metrics from my other machines (and itself) via Telegraf. Since I already have email alerts set up with that stack, it felt simplest to use it for this solution.

A really simple script

#!/bin/sh

# Compares the expected zpool status output with the actual status.
# Copy to a global location such as /usr/local/bin so it's accessible to Telegraf.
# Note: This can provide a false-postive if the output of the command changes, which is not guaranteed to be stable.

# Returns 0 for "false" (not healthy), returns 1 for "true" (healthy)
# Chose using integers over booleans due to how Grafana handles alerts.

OUTPUT="zfs_status,host=[HOSTNAME HERE] healthy="

if [ "$(zpool status -x)" != "all pools are healthy" ]; then
	OUTPUT=${OUTPUT}"0i"
else
	OUTPUT=${OUTPUT}"1i"
fi

echo $OUTPUT

There are similar scripts floating around on the Internet so I used those for inspiration. The only difference with mine is that it outputs the InfluxDB Line Protocol.

host= is just a convenient tag where you could put your box’s hostname (or call the hostname command and interpolate it).

Everything else should be explained by the script, including the possibility of false positives. Feel free to rename the zfs_status field to anything you wish. In my instance I use tws_zfs_status to differentiate custom fields I’ve created and possibly prevent namespace conflicts.

The Telegraf side

Telegraf has a super handy exec input where you can run arbitrary commands, so that’s what we use:

[[inputs.exec]]
  commands = ["sh /usr/local/bin/zfs_check.sh"]
  timeout = "5s"
  data_format = "influx"

When Telegraf collects data from its inputs it will write a one or a zero for its zfs_status field.

And finally, Grafana

The basic setup in Grafana is:

  • Stat type
  • Grab the last value
  • Map 1 to “Healthy” / green, 0 to “Unhealthy” / red
  • Set up an alert for when the value is less than 1

That’s basically it! It’s a really good idea to test it by temporarily tweaking the script to output a 0 and waiting for an email to arrive. 🙂

Categories
Apple Scripts

A dog timer Shortcut

Shortcuts User Guide - Apple Support

I’ve played with the Shortcuts app a few times and although their power is impressive, I haven’t found a need for any in my day-to-day workflow. This Shortcut may finally change that.

Here’s the problem

Our dog literally can’t handle the heat. She’s a Miniature Schnauzer born with a microtrachea so she struggles to regulate her body temperature. If she gets overheated it’s bad news.

I typically let her out multiple times a day while working from home. All I have to do is open up our back door, let her out into her fenced doggie area, and let her back in when it’s time.

That last item is trickier than it sounds – if I don’t set a timer I risk getting into the zone at work and losing track of the world. That could be disastrous, so up until now I’ve set a timer and altered the duration depending on the temperature outside. I wondered if I could improve that with a Shortcut.

Categories
Coding Projects Scripts

Scraping an Arris cable modem status page

Screenshot of the modem status page with arrows pointing to a screenshot of the data's final form in Grafana.
This project’s purpose is to start with a status page and end with Grafana graphs and alerts.

It felt good to complete this project that’s been on my list for quite some time. The main goal was to scrape the values from my modem’s status page and pipe them into InfluxDB, which feeds Grafana. Not only could I look at data trends, but I could receive alerts if certain values exceeded an acceptable threshold.

Overall this is a straightforward process:

  1. Pull in the HTML from the status page (which happens to not need any authentication, making it even easier)
  2. Parse the tables we care about (Downstream and Upstream) using XPaths
  3. Munge the data into something suitable for InfluxDB
  4. Insert the data into InfluxDB
  5. Query the InfluxDB data from Grafana

I knew I wanted to use Python for the project, so I first looked into Scrapy. After wrapping my mind around it (somewhat) I gave it a go and actually had a working solution… but it felt way over engineered and at times inflexible for what I wanted. I threw 90% of that solution away and went with a simpler script.

What I landed on was something that’s custom and lightweight, but extendable in case someone has a different status page or wants to use an alternative to InfluxDB.

Grafana screenshot showing that a fluctuation in downstream power around 10:00 a.m. caused the "Correcteds" values to spike.
I’ve had it running for a day and I’m already seeing interesting data!

See the repo on GitHub!

Categories
Scripts

I bookmark. A lot.

I’ve been working on a Python script to help me parse (and make sense) of all the bookmarks I consume on a daily basis. I was interested in just how many I accumulate daily, monthly, and yearly. It turns out, a lot.Output

Categories
Arduino Coding Projects Scripts Video

Destroying the web with a plasma ball

I plan to do a more thorough write-up on my plasma-ball project, but for now here’s the video, some pictures, and a link to the repo.

Github: https://github.com/twstokes/arduino-plasma-ball

It made Boing Boing – woot!

Categories
Coding Projects Raspberry Pi Scripts Video

Raspberry Pi + Garage door opener: Part 2

In Part 1 of my ‘Raspberry Pi + Garage door’ series, I showed a super simple way to control a garage door with a script that could potentially be ran from the Internet.

This part expands on that and tackles the issue: ‘How do I know the state of my garage door if I’m not at home?’

Because the code operates no differently than someone pressing a single button on the remote control, you would normally have to look with your own eyes to see if you were closing, stopping, or opening the garage door. This can be an issue when your eyes are nowhere close to it.

I wanted to come up with a solution that didn’t involve running new equipment such as a switch to detect the door’s orientation. I decided to utilize what I already had in the garage: A camera. Namely, this one: Foscam FI8910W

The idea is to use the camera to grab an image, pipe that image into OpenCV to detect known objects, and then declare the door open or closed based off of those results.

I whipped up a couple of shapes in Photoshop to stick on the inside of my door:

shapes_web

 

videostream
Shapes taped to the inside of the garage door

I then cropped out the shapes from the above picture to make templates for OpenCV to match.

Templates for OpenCV

The basic algorithm is this:

  1. Get the latest image from the camera
  2. Look for our templates with OpenCV
  3. If all objects (templates) were detected, the door is closed – otherwise it’s open

1423729418.65
Shapes successfully detected

 

To help make step 3 more accurate, I added a horizontal threshold value which is defined in the configuration file. Basically, we’re using this to make sure we didn’t get a false positive – if the objects we detect are horizontally aligned, we can be pretty certain we have the right ones.

I was happy to find that the shapes worked well in low-light conditions. This may be due to the fact that my garage isn’t very deep so the IR range is sufficient, as well as the high contrast of black shapes on white paper.

Currently I have some experimental code in the project for detecting state changes. This will not only provide more information (e.g. the door is opening because we detect the pentagon has gone up x pixels), but is good for events (e.g. when the alarm system is on, let me know if the door has any state change).

I’ve tested running this on the Raspberry Pi and it works fine, though it can be a good bit slower than a full-blown machine. I have a Raspberry Pi 2 on order and it’ll be interesting to see the difference. Since this code doesn’t need anything specific to the Raspberry Pi, someone may prefer to run it on a faster box to get more info in the short time span it takes for a door to open or close.

I’ve created a video to demo the script in action!

GitHub: https://github.com/twstokes/door_detect

Categories
Scripts

Detecting Smith numbers

After learning about Smith numbers today I cooked up this script fairly quickly. There are many places where it could be optimized and cleaned up but that wasn’t really the point – I just needed something that was accurate, and I’m pretty sure this is.

It also helped me to learn about the ulimit command after running it with a HUGE number that had taken up about 7GB of swap before I had to kill the process remotely via SSH…


#!/usr/bin/python
# smith number detector - TWS 1-12-13

def detectSmith(num):
  if(sumFactors(num) == sumDigits(num)):
    return True
  else:
    return False

# calculate the sum of the factors
def sumFactors(num):
  factorList = factors(num)
  sum = 0

  for factor in factorList:
    if factor > 9:
      # if a number (base ten) has multiple digits it has to be broken down
      sum = sum + sumDigits(factor)
    else:
      sum = sum + factor

  return sum

# sum the digits in a number with two or more
def sumDigits(num):
  sum = 0  

  if num > 9:
    while num > 9:
      sum = sum + num % 10
      num = num / 10
    sum = sum + num
    return sum
  else:
    # should never reach this
    return num

def factors(num):
  primeFactorList = []

  factor = lowestPrimeFactor(num)
  currentNum = num

  while factor != -1:
    primeFactorList.append(factor)
    currentNum = currentNum / factor
    factor = lowestPrimeFactor(currentNum)

  if(currentNum != num):
    # preventing returning the number passed in
    primeFactorList.append(currentNum)

  return primeFactorList

def lowestPrimeFactor(num):
  for x in range(2, (num / 2) + 1):
    if(num % x == 0):
      return x
  return -1

for number in range(1, 1000):
  if(detectSmith(number)):
    print "The number", number, "is a Smith number."
Categories
Scripts

A simple Linux content search script

A simple script I use for work to search the contents of files. It just wraps up ‘find’ and ‘grep’ to recursively search and display the location of the file.

#!/bin/bash
# Tanner Stokes - 8-23-12
# Search contents of files

if [ -z "$1" ] || [ -z "$2" ]
then
echo "Usage: csearch \"filetype\" \"content\""
echo "ex: csearch \"*.php\" \"my_function()\""
exit 1
fi

find . -name "$1" -exec grep -Hn "$2" {} \;
Categories
Scripts

Herp Derp YouTube comments

Herp Derp now has a dedicated page!

Categories
Scripts

Changing an LCD string on a Dell server remotely with Python

Here’s a little Python script I cooked up for work. We often have to change the custom LCD string on boxes if the machine is given a new name. Before, we’d have to restart a machine to access the DRAC configuration to change the name. With this script, all you have to do is enable IPMI through the DRAC remotely (if it isn’t already enabled), and know the correct credentials.

If the LCD isn’t set to view the custom string, you should be able to change it from the front of the box. As you can see, the function that I have commented out at the bottom is supposed to do this for you, but I never got it to work.

2023 update: Check out Pierre’s solution which properly sets the mode!

You may have to change ‘/usr/sbin/ipmitool’ to your appropriate path.

#!/usr/bin/python
import os

# Tanner Stokes - tannr.com - 2-26-10
# This script changes the LCD user string on Dell machines that conform to IPMI 2.0

sp_hostname = raw_input ("\nEnter DNS or IP of SP: ");
user_string = raw_input("Enter LCD string: ")

hex_string = ""

for x in user_string:
 	hex_string += hex(ord(x))
	# add space between each hex output
	hex_string += " "

print '\nTrying to change LCD string on '+sp_hostname+'...'

return_val = os.system('/usr/sbin/ipmitool -H '+sp_hostname+' -I lan -U root raw 0x6 0x58 193 0 0 '+str(len(user_string))+' '+hex_string)

if (return_val == 0):
	print 'LCD string changed successfully.\n'
else:
	print '\nNon-zero return value, something went wrong.'
	print 'Make sure IPMI is enabled on the remote host and the DNS or IP is correct.\n'

# this function supposedly sets the user string to show on the LCD, but never got it to work
# this can be changed from the front of the box anyway
# os.system('/usr/sbin/ipmitool -H '+sp_dns+' -I lan -U root raw 0x6 0x58 194 0')