Monday 10 February 2014

Shell script to get Email Notification for every new occurence of java.lang.OutOfMemoryError in logs

blogger
Shell script to get Email Notification for every new occurrence of java.lang.OutOfMemoryError in logs

This script will send the Email Notification whenever there is new occurrence of java.lang.OutOfMemoryError in logs with latest Timestamp that will help you in diagnosing the issue.

Benefit: This script is helpful in monitoring the Environment and will let the administrator know about the issue ASAP so that he can perform the required solution and can prevent unbearable delay.As it sends the notification it also takes the backup of log file.

Below are steps you need to follow:

Step 1) Create the directory

cd /shared/fmw/build/myscript

Step 2) Create AutocheckOOM.sh file in the current directory with below content:

#!/bin/bash
##########
ENVNAME="Env:"
COUNTER="0"
WORKDIR="/shared/fmw/build/myscript"
msName="MngdSvr1"
Log_LOC="logfile_location"

LogFiles=( MngdSvr1.log )

# email notifications will be sent via mail
EMAIL="mail_id"
# CC list in the notification mail
CCList="ccmail_id"
# From email address in the notification Email
FromAdd="frommail_id"



#Functions

OOM() {
for logfile in ${LogFiles[@]} ;do
Count=`grep "java.lang.OutOfMemoryError" $Log_LOC/$logfile | wc -l`
COUNTER=$[$COUNTER + $Count]
export COUNTER
done
}


BackupLogs() {
for logfile in ${LogFiles[@]} ;do
if [ -f $Log_LOC/$logfile ]; then
tar -czf $Log_LOC/$logfile.tar.gz $Log_LOC/$logfile
fi
done
}

Main() {
OOM
#ProcCheck
if [[ "$COUNTER" != "0" ]] ; then
echo "`date` :Out Of Memory Condition Detected.."
echo "`date` :Backing up logs for future reference.."
BackupLogs
cat $Log_LOC/NGS_MngdSvr1.log | grep java.lang.OutOfMemoryError | grep "####<" > temp.txt
output=$(tail -1 temp.txt | awk -F'>' '{print $1}' | awk -F'<' '{print $2}')
echo ''>> a.txt
test=$(cat a.txt | grep "$output")
if [ "$test" == "" ]; then
echo $output >> a.txt
#echo $output
echo "$ENVNAME OutOfMemory Error Detected at $output" | mail -s "$(echo -e "Auto-Msg: $ENVNAME : OutOfMemory Error\nContent-Type: text/html")" $EMAIL $CCList
 $FromAdd
fi
#CleanLogs

else
echo "`date` :Exiting, No Out of Memory found...";
exit 0
fi
}
Main


Note : change the location of log according to your Env.
           change the email_id accordingly.
          change the name of managed server accordingly

After this you just need to run the shell script or better way is to set this as a job in cron.

sh AutocheckOOM.sh

Thanks a lot for your patience!!!!

Regards
-Ashish




Sunday 2 February 2014

SQL Scripts for Monitoring Transactions

blogger
                                           SQL Scripts for Monitoring Transactions

There are some useful scripts that are helpful in monitoring the instances or transactions for particular composites deployed in weblogic Environment.

Benefit: These scripts are helpful for techies working in production support that will help them in analyzing
the load coming to there environment for particular services or for all the services deployed in weblogic Environment.

Below are the scripts:

Average, minimum, and maximum duration of components

SELECT DOMAIN_NAME,
COMPONENT_NAME,
DECODE(STATE,'5','COMPLETE','9','STALE','10','FAULTED') STATE,
TO_CHAR(MIN((TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),12,2))*60*60) +
(TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),15,2))*60) +
TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),18,4))),'999990.000') MIN,
TO_CHAR(MAX((TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),12,2))*60*60) +
(TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),15,2))*60) +
TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),18,4))),'999990.000') MAX,
TO_CHAR(AVG((TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),12,2))*60*60) +
(TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),15,2))*60) +
TO_NUMBER(SUBSTR(TO_CHAR(MODIFY_DATE-CREATION_DATE),18,4))),'999990.000') AVG,
COUNT(1) COUNT
FROM CUBE_INSTANCE
WHERE CREATION_DATE >= SYSDATE-1
--AND COMPONENT_NAME LIKE '%%'
AND COMPOSITE_NAME LIKE '%%'
GROUP BY DOMAIN_NAME, COMPONENT_NAME, STATE
ORDER BY COMPONENT_NAME, STATE


Note: Enter the name of the component or composite name accordingly.


Number of instance in every hour (load query)


SELECT inner_tab.hour_time,count(*) no_of_incidents  FROM ( select to_number(to_char(created_time, 'HH24')) hour_time  from COMPOSITE_INSTANCE
where created_time BETWEEN to_date('23-09-2013 19:00:00','DD-MM-YYYY HH24:MI:SS')
AND to_date('24-09-2013 00:00:00','DD-MM-YYYY HH24:MI:SS')
)inner_tab GROUP BY inner_tab.hour_time order by inner_tab.hour_time


Note: change the date accordingly.


Running instances of any particular composite in last one hour


select compin.id, substr(compin.composite_DN, 0, instr(compin.composite_DN, '!')-1) Composite_name, compin.source_name, compin.conversation_id
, to_char(compin.created_time, 'MM/DD/YY-HH:MI:SS')
from composite_instance compin
where
compin.state = '0'
and compin.id not in (select cmpst_id from cube_instance cubein)
and compin.created_time > sysdate - 1/24
and substr(compin.composite_DN, 0, instr(compin.composite_DN, '!')-1) IN('composite_name1','composite_name2');


Note: change the name of the composites accordingly.


Instance processing times


SELECT create_cluster_node_id, cikey, conversation_id, parent_id, ecid, title, state, status, domain_name, composite_name, cmpst_id, TO_CHAR
(creation_date,'YYYY-MM-DD HH24:MI:SS') cdate, TO_CHAR(modify_date,'YYYY-MM-DD HH24:MI:SS') mdate,
extract (day from (modify_date - creation_date))*24*60*60 +
extract (hour from (modify_date - creation_date))*60*60 +
extract (minute from (modify_date - creation_date))*60 +
extract (second from (modify_date - creation_date))
FROM   cube_instance
WHERE  TO_CHAR(creation_date, 'YYYY-MM-DD HH24:MI') >= '2013-05-06 11:00'
AND    TO_CHAR(creation_date, 'YYYY-MM-DD HH24:MI') <= '2013-05-06 18:00'
ORDER BY cdate;


Note: change the date and time accordingly.


Number of long Running(More than 7 days) instances for any particular composite.


select compin.id, substr(compin.composite_DN, 0, instr(compin.composite_DN, '!')-1) Composite_name, compin.source_name, compin.conversation_id
, to_char(compin.created_time, 'MM/DD/YY-HH:MI:SS')
from composite_instance compin
where
compin.state = '0'
and compin.id not in (select cmpst_id from cube_instance cubein)
and compin.created_time < sysdate - 7
and substr(compin.composite_DN, 0, instr(compin.composite_DN, '!')-1) IN('Partition/composite_name');


Note : change the partition and composite name accordingly.

Thanks a lot for your patience!!!!

Regards
-Ashish