Solaris: Netbackup: bpcd issues

Solaris and Netbackup, Issues with the bpcd process

Very often I run into the issue that the bpcd process is getting hung or having similar issues.

Errors might be like:

# netstat -a |grep -i bpcd
reports CLOSE_WAIT bpcd is not responding

One Solution is to restart the process:

Solaris up to 9:

restart by using
# /etc/init.d/inetsvc stop; /etc/init.d/inetsvc start

to restart the full inetd process make sure you use the full line else you might drop the network.

Solaris 10:

# svcs -a |grep bpcd to see the process and state.

# svcadm restart svc:/network/inetd:default           to restart the demon
# svcadm restart svc:/network/bpcd/tcp:default  to restart the demon

if its in maintence mode after those commands:

# svcs |grep bpcd
maintenance    10:28:52 svc:/network/bpcd/tcp:default

Then use
# svcadm clear svc:/network/bpcd/tcp:default

# svcs |grep bpcd       shall report
online          9:25:05 svc:/network/bpcd/tcp:default

reference:    http://seer.entsupport.symantec.com/docs/256685.htm
Sometimes it happens that bpcd appears to be ‘listening’ but does not respond to connection attempts.
The netstat -a command shows bpcd listening:

netstat -a | grep bpcd
*.bpcd               *.*                0      0 24576      0 LISTEN

But bpcd does not respond to connection attempts, and there is no log produced in the bpcd log directory.

This could be because of some problem with the OS or the inetd process. This may occur when there is a security wrapper in the inetd.conf configuration file. This is an OS problem to be solved by the system administrator. For a quick fix they can remove the security wrapper for bpcd, if they are using one.

Check the /etc/inetd.conf file. You might see a line for bpcd that looks something like:

bpcd    stream  tcp     nowait  root    /usr/local/sbin/tcpd    /usr/openv/netbackup/bin/bpcd

Comment out the line with the ‘wrapper’ and add the line as configured by NetBackup:

# bpcd    stream  tcp     nowait  root    /usr/local/sbin/tcpd    /usr/openv/netbackup/bin/bpcd
bpcd    stream  tcp     nowait  root    /usr/openv/netbackup/bin/bpcd bpcd

Stop all the VERITAS NetBackup ™ processes:

/usr/openv/netbackup/bin/goodies/bp.kill_all

Re-read the inetd.conf file:

ps -ef | grep inetd
kill -HUP {inetd_pid}

Restart NetBackup. Try to connect to the bpcd daemon, or try a backup.

telnet {hostname} bpcd

If the connection to bpcd still cannot be established (or there is no security wrapper), the bpcd process can be run in ‘standalone’ mode as a workaround.

You might not be able to start the bpcd process because it cannot bind to port 13782. In that case, you will get a message:

bpcd -standalone
bind(13782) failed: 125

The following steps will allow bpcd to be started in standalone mode.

Edit the /etc/inetd.conf file by commenting out the bpcd line:

# bpcd    stream  tcp     nowait  root    /usr/openv/netbackup/bin/bpcd bpcd

Stop all the NetBackup processes:

/usr/openv/netbackup/bin/goodies/bp.kill_all

Re-read the inetd.conf file:

ps -ef | grep inetd
kill -HUP {inetd_pid}

Now bpcd will not be listening. You will be able to start it in standalone mode with:

bpcd -standalone

Restart NetBackup. Try to connect to the bpcd daemon, or try a backup.

telnet {hostname} bpcd

The system administrator will have to fix the inetd function for a permanent fix. But this will get backups running.