Monitor and Respond to My Own Custom Condition

Instructions to have PacketWise automatically monitor a condition you define based on a PacketWise metric, and optionally respond with notification or corrective actions

This recommendation makes use of the adaptive response feature and its ability to allow custom definition of the condition to monitor. It's for a fairly advanced user.

Used as an example throughout this recommendation, consider the following scenario.
As the application administrator for your organization, you have tried to ensure prompt performance for your group's most business-critical application — SAP. Already, you have:

• Created traffic classes for inbound and outbound SAP traffic (Create a Traffic Class)
• Assigned partitions giving SAP traffic 20 percent of your capacity (Create a Partition)
• Defined a rate policy for each SAP class with a high priority (Set a Rate Policy)
• Defined "good performance" as an average response time of 1800 ms or better, for at least 95 percent of the time. (Set Thresholds)
• Checked performance by viewing the Service-Level Compliance graph and the table of response-time statistics (View Response-Time Graphs and Monitor Response Times)

Now, you'd like to be able to stop checking SAP's performance manually. Your goal is to have PacketWise automatically monitor SAP's service-level compliance. When performance is unacceptable, you'd like to be notified via email, and you'd like the size of SAP's partition to grow to 30 percent of capacity to address the problem while you figure out what to do for the long term.

For background on the adaptive response feature, including the definitions for agents, action files, red/yellow/green status categories, and more, see Adaptive Response Overview.

Phase One: Plan Your Adaptive Response Agent

You will be adding an adaptive response agent to monitor your custom-defined condition, one based on a PacketWise measurement variable. First, you'll attend to planning.

1. Make sure there is not already an agent template that monitors for the condition you have in mind. If there is, use the recommendation called Monitor and Respond to a Condition of Interest Using Agent Templates instead of this one.
   
   For our example, there is no suitable agent template.
   
   
2. Peruse the available class, partition, and link measurement variables. Choose the metric that assesses the condition you want to monitor. Start a page of planning notes, and note your variable's name and whether it is a class, partition, or a link variable. If it is a common variable, note whether you want to monitor the status of a class, partition, or the whole link.
   
   For our example, you could monitor several of the response-time metrics. But for our purposes, we'll choose service-level%, a class variable. It represents the percentage of transactions that satisfied their performance requirement, in our case the percentage of SAP transactions that finished within 1800 ms.
   
   
3. Determine the threshold values you'd like to have in your agent.
   
   What condition indicates there is a problem or a need to respond? This is the agent's red threshold.
   
   What indicates that things are okay? This is the agent's green threshold.
   
   How frequently should PacketWise check your condition? This is the agent's evaluation interval.
   
   For our example, you consider good performance to be when SAP transactions complete promptly at least 95 percent of the time (the green threshold). You decide that you want PacketWise to interpret a percentage below 90 as a problem (the red threshold). For our purposes, checking every five minutes should be sufficient.
   
   
4. What actions do you want to take when your problem gets bad (crosses the red threshold)?
   
   Do you want to notify somebody? Do you want to execute some other commands to document or even correct the problem?
   
   An action file is essentially a script that contains the actions you decide are appropriate. An action file can be any sequence of PacketWise CLI commands. PacketWise automatically executes the commands in your action file when your threshold is crossed. Keep in mind that you don't necessarily need to take any action. Many times, it is sufficient just to be able to check a condition's status.
   
   Each agent has a set of variables whose values are passed along at run time to any triggered action file. The action file can use any of those variables' values, or not (your choice). You might think of the variables as parameters to the action file to avoid confusion with PacketWise measurement variables. Consult the Action Files Overview to see which variables (or parameters) your action makes available.
   
   For our example, if the service-level percentage drops below 90, you want an email to yourself and you want to increase SAP partitions' sizes.
   
   
5. Decide on an appropriate action or notification (if any) for when your problem returns to normal levels and stops being an issue. Again, the action file can be any sequence of PacketWise CLI commands and can use any of the variable values made available by the agent.
   
   For our example, you decide you don't really want to do anything automatically in response to a return to green status. You don't want another email because you would already be checking SAP performance if its status had been red, making sure you fixed the problem. And you plan to attend to the partitions' sizes manually the first few times SAP performance drops. You might configure an automatic action to decrease the partitions' sizes later, once you gain confidence in how things go.
   
   
6. Make a note of any other details or tasks to which you must attend, if any, to complete your plan.
   
   
7. With your plan notes intact, proceed to the next phase.
   

Phase Two: Create and Test your Action Files

You can create your agent and your action files in either order. You'll create your action files next here because it's convenient to have your action-file names when you define your agent.

1. You can use three styles of notification (email, SNMP trap, syslog) in your action files. Each requires some setup before the corresponding CLI command will succeed. If you chose any notification to be included in an action file, attend to the required setup, as follows:
   
   
   • If you want to be notified via SNMP traps, configure PacketWise for SNMP support and define the SNMP trap destination(s).
   • If you want to be notified via email, set up email notification by configuring an SMTP server and defining email recipients.
   • If you want a message to be sent to a syslog server, set up the syslog feature.
   
   
2. If your plan calls for a red action, you must create the action file that will be executed automatically when your red threshold is crossed.
   
   In addition, you can consult Create a Command File, Action Files Overview, and PacketWise CLI commands for extra information you might need to create your action file.
   
   For our example, you would do the following:
   • Using the Info tab's File Browser, navigate to 9.258/Agent/Cmd.
   • Click new cmd file and enter a name in the File Name field: RedSAP.
   • Enter your script in the Contents area of the Command File Editor. It would look like this:
     #Title: Monitor Sap's performance. Boost partition sizes when poor.
     send email <YourEmailAddress> "SAP Performance Dropped" "Performance for $class-id dropped to $me-value"
     partition apply inbound/sap 30% none
     partition apply outbound/sap 30% none
     
     
3. If your plan calls for a green action, create the action file that will be executed automatically when your green threshold is crossed (from a yellow or red status).
   
   Consult Create a Command File, Action Files Overview, and PacketWise CLI commands for extra information you might need to create your action file.
   
   For our example, you would not create a green action file.
   
   
4. Test your action files with the run button, as described in Create a Command File, supplying sample values for any $variables you used in your script of CLI commands. PacketWise automatically prompts you for sample values and, once entered, displays the output in the results area of the page.
   
   
5. Note the names and spellings of your two action files on your plan notes. You will need them later when editing your agent.

Phase Three: Add Your Agent

1. Consulting your plan, add your agent.
   
   Select Link ME Variables, Partition ME Variables, or Class ME Variables for the agent template, according to your plan.
   Supply a name for your new agent that is descriptive of its purpose.
   Adjust the Evaulation Interval according to your plan.
   Enter the names of your red and green action files, if any.
   
   For our example, you would:
   Select Class ME Variables for the agent template.
   Enter Monitor SAP Performance for the agent's name.
   Change the Evaluation Interval to 5 minutes.
   Enter RedSAP.cmd for the red action file's name.
   
   Note: It can be prudent and less risky to first define and test your agent without the names of your action files (so that actions won't be automatically triggered before you are confident of your agent). Test your agent and your action files separately, then you can add the action file names to the agent later.
   
   
2. Click OK and edit parms. The Edit Agent Entry window opens.
   
   
3. Edit the agent with field values from your plan notes.
   
   Enter the red and green thresholds, the traffic class name you wish to monitor, the name of the PacketWise measurement variable you noted, and so on, for all the fields in the window and described in Edit an Agent.
   
   For our example, you would fill in the fields to look like the following:
   
   
   
   
   
4. Save your new agent's definition.
   
   
5. Check your agent's status occasionally and check its reports.