Sensu Epic

Epic Goal:   My goal is to just really figure out how to use Sensu. 
Some of the TODOs on my list are.

• Send message securely between Sensu Master and Clients
• Set up a Sensu HA system (more than one master)
• Write a few checks
• Write a few handlers

10/21/2014: 

The first post of this epic shows simply how to set up Sensu and get a simple check working

10/24/2014: 

Setting up a Sensu Client on a Separate Server.

10/27/2014: 

Setting up a Second Sensu Client on a Separate Server.

10/27/2014: 

Setting up a Sensu Client to send messages via SSL.

10/28/2014: 

Setting up a Sensu Client to send messages via SSL, outside of your network.

11/19/2014: 

Getting Sensu to talk to HipChat

11/25/2014: 

Getting Sensu to talk too trigger a Pagerduty alert.  This is a really fun and useful one to get working.

1/13/2015: 

I wrote a simple check that would warn X days before a date and issue a critical X days before a date.  Since I made this check my team argued with me a bit and had me remove the critical portion (not it only warns).   Here is the argument they had.

Warning on arbitrary information is valid, but a critical is not.  A critical alert has to be non-arbitrary, and queryable somehow.   If I were handed a date for when a license expired (Our situation) I should only issue a warning.  If I had someway to query a system when the license was expiring then I could issue a critical.

In the first case if we paid for an extension to the license and forgot to update the Sensu check we would only get a warning.  In the second case if we extended the license the system we queried would automatically have the new date and no alert would be triggered until the new date came up.

Its a Sensu check Philosophy I agree with, I left my example as is, but to "fix" it just remove its ability to issue a critical/

2/20/2015: 

We had a, potential, need on our team to view multiple teams Sensu boards from the same dashboard.  Uchiwa can easily do that and even allow you to filter the results by teams easily.  Way to go Uchiwa team.

3/3/2015: 

Trying to support an old legacy system that emails out.... We needed to alert when we got X numbers of emails over Y time.  This is what I came up with.