slp_srvrtop进程，cluster.log大量日志

服务器

一个数据库主机/var目录很高

成都创新互联专注为客户提供全方位的互联网综合服务，包含不限于网站制作、成都网站建设、巴马网络推广、成都微信小程序、巴马网络营销、巴马企业策划、巴马品牌公关、搜索引擎seo、人物专访、企业宣传片、企业代运营等，从售前售中售后，我们都将竭诚为您服务，您的肯定，是我们大的嘉奖；成都创新互联为所有大学生创业者提供巴马建站搭建服务，24小时服务热线：13518219792，官方网址：www.cdcxhl.com

[biprd1]root#df -g

Filesystem GB blocks Free %Used Iused %Iused Mounted on

/dev/hd4 0.50 0.26 49% 10920 15% /

/dev/hd2 7.00 3.17 55% 56403 8% /usr

/dev/hd9var 5.00 0.31 94% 7284 8% /var

/dev/hd3 1.50 0.88 42% 20596 9% /tmp

/dev/hd1 0.12 0.07 42% 187 2% /home

/dev/hd11admin 0.12 0.12 1% 5 1% /admin

/proc - - - - - /proc

/dev/hd10opt 6.00 3.09 49% 22598 4% /opt

/dev/livedump 0.25 0.25 1% 5 1% /var/adm/ras/livedump

/dev/u01_fs 30.00 11.93 61% 235400 8% /u01

/dev/tsmlog 0.25 0.25 1% 9 1% /tsmlog

/dev/fslv00 60.00 44.19 27% 12 1% /bibak

/dev/nmonlv 0.50 0.31 38% 39 1% /nmon

/dev/backup_fs 40.00 13.00 68% 76 1% /backup

/dev/u02_fs 409.00 167.11 60% 398 1% /u02

top进程主要是两个，cpu IDLE 7%：

Topas Monitor for host: biprd1 EVENTS/QUEUES FILE/TTY

Tue Mar 26 15:29:49 2019 Interval: 2 Cswitch 912 Readch 1750.2K

Syscall 494.2K Writech 2225.9K

CPU User% Kern% Wait% Idle% Physc Entc Reads 481 Rawin 0

ALL 45.4 47.7 0.0 6.8 0.50 99.8 Writes 80508 Ttyout 576

Forks 4 Igets 0

Network KBPS I-Pack O-Pack KB-In KB-Out Execs 4 Namei 14864

Total 2.5 20.0 5.6 1.4 1.2 Runqueue 7.2 Dirblk 0

Waitqueue 0.0

Disk Busy% KBPS TPS KB-Read KB-Writ MEMORY

Total 0.4 1024.0 8.0 0.0 1024.0 PAGING Real,MB 12544

Faults 2558 % Comp 77

FileSystem KBPS TPS KB-Read KB-Writ Steals 0 % Noncomp 5

Total 2.7K 8.3K 1.7K 1.0K PgspIn 0 % Client 5

PgspOut 0

Name PID CPU% PgSp Owner PageIn 0 PAGING SPACE

slp_srvr 3735686 50.0 128.6 root PageOut 256 Size,MB 8192

syslogd 4718742 50.0 5.4 root Sios 256 % Used 13

topas 29294718 0.0 4.6 root % Free 87

sleep 31719550 0.0 0.2 root NFS (calls/sec)

sshd 36831360 0.0 4.1 root SerV2 0 WPAR Activ 0

oracle 19136702 0.0 15.7 oracle CliV2 0 WPAR Total 0

oracle 44433504 0.0 13.7 oracle SerV3 0 Press: "h"-help

oracle 27721982 0.0 13.7 oracle CliV3 0 "q"-quit

oracle 14745714 0.0 13.9 oracle

vmmd 458766 0.0 1.2 root

oracle 10682470 0.0 18.3 oracle

lrud 262152 0.0 0.5 root

oracle 11337728 0.0 20.1 oracle

oracle 8978578 0.0 13.9 oracle

oracle 8192170 0.0 13.8 oracle

syncd 2228416 0.0 0.6 root

clcomd 8519684 0.0 4.9 root

EvMgrC 5570622 0.0 10.0 root

nfsd 2949318 0.0 1.8 root

ksh 7012396 0.0 0.5 root

报错信息如下：

[biprd1]root#cat /var/hacmp/adm/cluster.log

.......

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Mar 26 15:19:41 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

.....

通过重启服务问题依旧，日志文件大约每三分钟就增长100M

stopsrc -s syslogd

startsrc -s syslogd

于是kill slp_srvr进程

[biprd1]root#ps -ef|grep slp

root 2818092 1 0 Sep 24 - 0:52 /opt/ibm/director/cimom/bin/tier1slp

root 3735686 1 66 Sep 24 - 2136:34 ./slp_srvreg -D

root 16646316 16253072 0 16:40:51 pts/0 0:00 grep slp

[biprd1]root#kill -9 3735686

[biprd1]root#

马上停止生成日志，cpu也降了下来，再将slp_srvr进程起来

cd /usr/sbin

./slp_srvreg -D

[biprd1]root#df -g

Filesystem GB blocks Free %Used Iused %Iused Mounted on

/dev/hd4 0.50 0.26 49% 10921 15% /

/dev/hd2 7.00 3.17 55% 56403 8% /usr

/dev/hd9var 5.00 1.58 69% 7296 2% /var

/dev/hd3 1.50 0.88 42% 20596 9% /tmp

/dev/hd1 0.12 0.07 42% 187 2% /home

/dev/hd11admin 0.12 0.12 1% 5 1% /admin

/proc - - - - - /proc

/dev/hd10opt 6.00 3.09 49% 22645 4% /opt

/dev/livedump 0.25 0.25 1% 5 1% /var/adm/ras/livedump

/dev/u01_fs 30.00 11.93 61% 235774 8% /u01

/dev/tsmlog 0.25 0.25 1% 9 1% /tsmlog

/dev/fslv00 60.00 44.19 27% 12 1% /bibak

/dev/nmonlv 0.50 0.31 38% 39 1% /nmon

/dev/backup_fs 40.00 13.00 68% 76 1% /backup

/dev/u02_fs 409.00 166.94 60% 400 1% /u02

[biprd1]root#iostat 1 4

System configuration: lcpu=8 drives=5 ent=0.50 paths=10 vdisks=6

tty: tin tout avg-cpu: % user % sys % idle % iowait physc % entc

0.0 108.6 0.5 3.3 95.9 0.2 0.0 7.0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn

hdisk0 0.0 0.0 0.0 0 0

hdisk5 1.0 25.5 6.4 16 0

hdisk1 0.0 0.0 0.0 0 0

hdisk6 0.0 0.0 0.0 0 0

hdisk7 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait physc % entc

0.0 865.0 18.5 8.7 72.9 0.0 0.2 36.0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn

hdisk0 0.0 0.0 0.0 0 0

hdisk5 0.0 19.0 4.8 0 12

hdisk1 0.0 0.0 0.0 0 0

hdisk6 0.0 76.0 4.8 0 48

hdisk7 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait physc % entc

0.0 873.6 0.4 2.8 96.8 0.0 0.0 5.9

Disks: % tm_act Kbps tps Kb_read Kb_wrtn

hdisk0 0.0 0.0 0.0 0 0

hdisk5 0.0 0.0 0.0 0 0

hdisk1 0.0 0.0 0.0 0 0

hdisk6 0.0 0.0 0.0 0 0

hdisk7 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait physc % entc

0.0 873.6 0.4 2.9 96.8 0.0 0.0 6.0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn

hdisk0 0.0 0.0 0.0 0 0

hdisk5 0.0 0.0 0.0 0 0

hdisk1 0.0 0.0 0.0 0 0

hdisk6 0.0 0.0 0.0 0 0

hdisk7 0.0 0.0 0.0 0 0

[biprd1]root#ls -l

total 2099800

-rw-r--r-- 1 root system 0 Nov 28 2011 clavan.log

-rw-r--r-- 1 root system 74978079 Apr 02 09:04 cluster.log

-rw-r--r-- 1 root system 100000040 Apr 01 16:39 cluster.log.0

-rw-r--r-- 1 root system 100000078 Apr 01 16:36 cluster.log.1

-rw-r--r-- 1 root system 100000002 Apr 01 16:34 cluster.log.2

-rw-r--r-- 1 root system 100000002 Apr 01 16:31 cluster.log.3

-rw-r--r-- 1 root system 100000002 Apr 01 16:28 cluster.log.4

-rw-r--r-- 1 root system 100000040 Apr 01 16:26 cluster.log.5

-rw-r--r-- 1 root system 100000116 Apr 01 16:23 cluster.log.6

-rw-r--r-- 1 root system 100000078 Apr 01 16:21 cluster.log.7

-rw-r--r-- 1 root system 100000078 Apr 01 16:18 cluster.log.8

-rw-r--r-- 1 root system 100000078 Apr 01 16:15 cluster.log.9

drwxr-xr-x 2 root system 12288 Mar 07 2018 history

drwxr-xr-x 2 root system 256 Sep 24 2016 wpar

可见日志不再大量生产

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-031 [804399472] Calloc call failed. errno = 804399468.

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 16:41:10 biprd1 daemon:err|error syslog: slp: 0660-080 [804399792] SLPServiceListener -- The SA failed to compute received message: 潇 (-21).

Apr 1 17:04:50 biprd1 daemon:err|error syslog: slp: 0660-065 [804398544] Impossible to parse attribute |^L^Am.

Apr 1 17:04:50 biprd1 daemon:err|error syslog: slp: [804398928] decode_srvreg -- __srv_reg_local failed with rc = 804398924.

Apr 1 17:04:50 biprd1 daemon:err|error syslog: slp: 0660-084 [804399280] The SA failed to decode and compute received message: 醊繼Am (-2).

Apr 1 18:04:50 biprd1 daemon:err|error syslog: slp: 0660-065 [804398544] Impossible to parse attribute |^L^Am.

Apr 1 18:04:50 biprd1 daemon:err|error syslog: slp: [804398928] decode_srvreg -- __srv_reg_local failed with rc = 804398924.

Apr 1 18:04:50 biprd1 daemon:err|error syslog: slp: 0660-084 [804399280] The SA failed to decode and compute received message: 醊繼Am (-2).

Apr 1 19:04:50 biprd1 daemon:err|error syslog: slp: 0660-065 [804398544] Impossible to parse attribute |^L^Am.

Apr 1 19:04:50 biprd1 daemon:err|error syslog: slp: [804398928] decode_srvreg -- __srv_reg_local failed with rc = 804398924.

Apr 1 19:04:50 biprd1 daemon:err|error syslog: slp: 0660-084 [804399280] The SA failed to decode and compute received message: 醊繼Am (-2).

网页题目：slp_srvrtop进程，cluster.log大量日志
网页URL：http://cqcxhl.com/article/chechd.html