Script to monitor performance of Oracle VM Server.

Most of the time the load average of VM server indicates the future disaster that might occur in the vm environment.

Its always recommended to implement precautionary major to avoid such situation.

xentop - display real-time information about an xVM system and domains
SYNOPSIS
xentop [-h] [-V] [-d=seconds] [-n] [-r] [-v] [-b] [-i=iterations]
DESCRIPTION
The xentop command displays information about the Solaris xVM system
and domains in a continually-updating way. It is analogous to the BSD
UNIX top command.
OPTIONS
The following options are supported:
. . .
-b, --batch
Output data to stdout in batch mode.
-i, --iterations=iterations
Maximum number of iterations xentop should display before terminat-ing.

Executing
xentop --batch --iteration=1
giving me 0% cpu utilization.

There fore executing following command which somehow giving me acceptable values for cpu utilization.
xentop -d=10 --batch --iteration=4


I m thinking of getting 3 4 entries in such a way and calculate the average.
One more issue is Hostname is displayed only 10 character long.

any way i will post the end result in some time.




Script:

#!/bin/bash

fn_main()
{
rm -rf /tmp/Load_Average_Check.log > /dev/null 2>&1
echo "Target Name = $1" >> /tmp/Load_Average_Check.log
echo "Occurred At = `date` " >> /tmp/Load_Average_Check.log
echo "Message = Load Average on $1 is $2 & has crossed the Critical threshold " >> /tmp/Load_Average_Check.log
echo "Metric = Load Average on $1 " >> /tmp/Load_Average_Check.log
echo "Metric value = $2 " >> /tmp/Load_Average_Check.log
echo "Severity = Critical " >> /tmp/Load_Average_Check.log
echo "Acknowledged = No " >> /tmp/Load_Average_Check.log
echo >> /tmp/Load_Average_Check.log

echo "*******************************************************" >> /tmp/Load_Average_Check.log
################################################################################################################
echo "High Cpu Utilisation (%)in $1" >> /tmp/Load_Average_Check.log
#ITERATION indicates the number of time vm servers are checked for cpu utilization.
ITERATION=4
ssh -Tqn root@$1 xentop -b -d 2 -i $ITERATION > xentopresult.txt
#generate hostname file to be concatenated with monitoring log.
while [ $ITERATION -gt 0 ]
do
ssh -Tqn root@$1 xm list | awk '{print $1}' >> mno.txt
ITERATION=`expr $ITERATION - 1`
done

#merge hostname file to output.
paste mno.txt xentopresult.txt > final.txt
#To print Top 10 instances of maximum cpu utilization.
cat final.txt |grep -vE "NAME"| awk '{print $1,$5}'| sort -nr -k 2 | head -13 >> /tmp/Load_Average_Check.log
cat final.txt |grep -vE "NAME|Domain-0"| awk '{print $1,$5}'| sort -nr -k 2 | head -10 > vmwithguest.txt
#echo "Content of vmwithguest."
#cat vmwithguest.txt
echo "*******************************************************" >> /tmp/Load_Average_Check.log

cat vmwithguest.txt | awk '{print $1}'| sort -u > check.txt
#echo "Content after sorting vmwithguest"
#cat check.txt

while read line
do
MACHINENAME1=`echo $line | awk '{print $1}'`
match=0
CPU=0
CPUTOTAL=0
CPUAVERAGE=0
while read line
do
MACHINENAME2=`echo $line | awk '{print $1}'`
if [ "$MACHINENAME1" == "$MACHINENAME2" ]
then
match=`expr $match + 1`
CPU=`echo $line | awk '{print $2}' |awk -F "." '{print $1}'`
CPUTOTAL=`expr $CPU + $CPUTOTAL`
fi
done < cpuaverage="`expr">> /tmp/Load_Average_Check.log
echo " " >> /tmp/Load_Average_Check.log
fi
done <>> /tmp/Load_Average_Check.log
echo "NOTE: Make sure Load Average is always below 4% for better Performance" >> /tmp/Load_Average_Check.log

mail -s "Load Average on $1 is $2" Username@xyz.com < /tmp/Load_Average_Check.log
mail -s "Load Average on $1 is $2" Username2@xyz.com < /tmp/Load_Average_Check.log
mail -s "Load Average on $1 is $2" Username3@xyz.com < /tmp/Load_Average_Check.log
}


while read line
do
#Calculate Load on remote machine.
LOAD=`ssh -T -q -n $line w | grep average | awk -F "e:" '{print $2}' | awk -F "," '{print $1}' | awk -F "." '{print $1}'`
if [ $LOAD -gt 7 ]
then
fn_main "$line" "$LOAD"
fi
done < SERVERNAMES