Pagerduty getting started

Posted on Wednesday, August 27, 2014

This guide will get you through signing up for a test account and setting up a rotating schedule with escalation policies

What is pagerduty ?

Pagerduty is a great tool for alerting your team of incidents in your system.  Its located at [1].  There services can tie into several monitoring services, and when triggered can contact the proper on duty person on your team.

Signing up for a free trial

Currently, 8/2014, pagerduty has a free 30 day trial.

Click on Sign Up Now

Enter your relevant information and click continue.

The sign in screen should open.  Enter your password and click Sign in.

And now you are logged in.

If you look on the upper right hand you will see this

I have 30 days left of my free trial.

Surprise I have not had to enter  credit card number yet.  Way to go pagerduty!  

Now let's do something

Let's follow the pager duty suggested steps

1.      Set your contact Info
2.      Add Users
3.      Create an on-call schedule
4.      Add services

1. Set your contact Info

Click ON "Set Your Contact Info"

Hey its smart enough to grab my gravatar photo,  which reminds me it needs to be updated to my newer one.

This is much better.  Not so serious and blank faced, almost angry, look.

To fix the time zone, click on Edit Profile in the upper right

Select your time zone.

Click Save Changes

Click Add Phone Number

Enter in your phone number set it to mobile (if it's your cell) and click save

Do the same thing for SMS

If you have an iPhone, iPad, or android device you can download pagerduty  app. The iOS app is at [2]

Pagerduty iOS app

Enter the email address you use for pagerduty and click search

Enter your password and click Sign In.

Click OK to allow pagerduty to push notifications

Head back to the pagerduty web page and refresh

Your iOS device is registered

As a test click  "Send Test Notification"

You should get this pop up

Click View

2. Add Users

It's time to add another user

It looks like pagerduty is smart enough to figure out we passed phase one.  Click on Add Users to start phase two.  Remember each user will cost you $19/mo when you finally go live so think about that before you add everyone and their brother.

Enter your team members info, name and email.  Set them as a User.  Then click Add.

Nice, it keeps this prompt open and allows you to keep adding team members.


When you are done click Send Invites

The Users interface opens and you can see the users you have added and if they have accepted the invite.

Your new users should receive an email with an accept link. 

When your user accepts the invite they are presented with this screen.  They need to enter in a password and click Sign Up.

The new user needs to click on their email address in the upper right and select My Profile. 

This page will allow them to add their contact methods.

After the user has accepted the invite the resend invite button should go away.

3. Create an On-Call Schedule

Click Create an On-Call Schedule.

Click Create your first on-call schedule

Give the On-Call Schedule a name and set the desired Time Zone to use.

Click Add User

A user pull down will appear,  Select a user and click Add User

The user has been added.

Add another user and click Done.

Now we need to set up the rotation between who is on call.
For this test I am going to set up daily rotation that runs from 8:00 AM to 5:00 PM Monday through Saturday (no Sunday). 

From the pull down select Daily.

The schedule now appears

Click the checkbox here to restrict the shift times.

This Restrictions dialog will open.

Select "Restrict on-call duty to specific times of the day"
And set the from and to times

Click Apply

The calendar updates with the restrictions.

But that is not the schedule I want its 8:00 AM to 5:00 PM every day of the week….

Click edit to change the schedule

Select the second restrict radio button and set to Monday from 0:800 to 17:00, then click the plus button.

This will add another range.

Set this range for Tuesday from 8:00 AM to Tuesday 5:00 PM/

Keep doing this until you have the 6 days a week set up.

Click Apply

That gives me the schedule I want.

When you are all done click Save Changes

From here I can click on Calendar view to see a nice Calendar of the on duty schedule.

4. Add Service

Now we need to add a service.  Click Add Services.

Click + Add New service

Give the service a name and, for now, select default as the escalation type.  I will go over this later.

Now you need to select how you are going to integrate with this tool.

Look at the pull down menu, there are a lot of options.

For now keep it simple.  Select Integrate via email.
An email address will be shown (you can change its name here if you wish)

Click Add service.

The service screen will now display this service (which is not quite done).

Give it a test run

Send an email out to the integration email of the service.

My iPad was notified via the pagerduty iOS software.

On the pager duty dashboard I can see its been triggered but not acknowledge.

But now phone call?  What is wrong?

I will acknowledge this alarm and start, figure out how to fix the phone call and start over.

checkbox the incident and click Acknowledge

On the iOS device I can see that it was acknowledged and by whom.

From the web site I can see that it was acknowledged.  Click Resolve to resolve the issue.

Open up My profile.

Here is my problem.  I need to add a Notification rule.  Currently its only set up to email and push to my iPad.

Click Add Notification Rule

Select your phone and click Save.  Now it will call this number 0 minutes after an incident occurs.

Send another email out to trigger an alarm.

Here is the audio I got back.

It read the email subject line, but not the email contents. Keep this in mind when creating emails to send.

As a side note pagerduty calls always come from 1-415-349-5750

Looking at the pagerduty site I can see the incident and that it was acknowledged and who acknowledged it.

Click on Details

You can see all the details sent, including the body of the e-mail message sent.

Click the Incident number to open its detailed page

Perfect!  This is the page I was interested in.

This has

·         Time the Incident was opened
·         Who is assigned to it
·         What time the incident was acknowledged
·         What time the incident was resolved
·         Any notes added to the Incident (scroll down to the bottom)

What's this?

… In 2 minutes I will find out.

It called me again.  I acknowledged it and did not resolve it, and as a result it called me again.  This is the default behavior.

In my case I do not want it to call back after an acknowledge and I do not want it to ever auto resolve.  A solution could take ½ a day and I want to record that fact.

Setting the default auto resolve and acknowledge call back

Click On Services

From the controls pull down on the service select edit.

            Incident Ack Timeout
            And Incident Auto-Resolution

Save Changes

Run another test

Perfect I acknowledged the call and it's not going to call me back at all.

The missing Escalation Policy

What I think I am missing at this point is the escalation policy.

Open the Escalation Policies

Add a new Escalation Policy.

Here is a new blank escalation policy

Give the escalation policy a name.

Click in the "Alert the following user section and select a schedule.  I selected the Emergency! Schedule.

You are free to add more people to the list or schedules.  You could have it always call you no matter who is on schedule.

I changed the min to wait until next rule kicks in to 20 minutes.  If there is no acknowledgement from the first alert the next rule will be applied.

For the second alert rule I am having it call me.

By checking the box and entering in 9 I am setting it to repeat this sequence up to 9 times if there is no acknowledgement.

Click Save.

Go back to services

Edit the service you want to add the escalation policy to.

Select the new Escalation Policy

Scroll to the bottom and Save Changes.

Running another test

I sent an email again.  It correctly called the first person on the duty.
They acknowledged it and it did not continue onto the next person.  Then I tried to trigger another incident, but I had not resolved the last one.  Instead of triggering another event the email was absorbed and documented within the prior event.

After this I "resolved" the issue and sent a new email.  This time I let the first person ignore it and let it go onto the next person.  This worked like it should 20 minutes after the first person ignored the call, the second one received a call and acknowledged it.

That should do it!  There are the basics to get going with

[1]        pagerduty web site

                Accessed 08/2014

  This post is a part of and epic, the pagerduty and twilio epic.

Epic Goal:   Set up a phone number, via twillio, that when called will set off a pagerduty event.

No comments:

Post a Comment