Veritas Cluster Server (VCS) Commands

LLT and GRAB

VCS uses two components, LLT and GAB to share data over the private networks among systems.

These components provide the performance and reliability required by VCS.

LLT LLT (Low Latency Transport) provides fast, kernel-to-kernel comms and monitors network connections. The system admin configures the LLT by creating a configuration file (llttab) that describes the systems in the cluster and private network links among them. The LLT runs in layer 2 of the network stack

GAB GAB (Group membership and Atomic Broadcast) provides the global message order required to maintain a synchronised state among the systems, and monitors disk comms such as that required by the VCS heartbeat utility. The system admin configures GAB driver by creating a configuration file ( gabtab).

LLT and GAB files

/etc/llthosts --> The file is a database, containing one entry per system, that links the LLT system ID with the hosts name. The file is identical on each server in the cluster.

/etc/llttab --> The file contains information that is derived during installation and is used by the utility lltconfig.

/etc/gabtab -->The file contains the information needed to configure the GAB driver. This file is used by the gabconfig utility.

VCS Configuration File:

/etc/VRTSvcs/conf/config/main.cf --> The VCS configuration file. The file contains the information that defines the cluster and its systems.

GAB & LLT Cluster Commands:

#lltstat -n --> Each system has two links and each system is in the OPEN state. The asterisk (*) denotes the system on which the command is typed.

LLT node information:

       Node      State     Links
      *0 hanode1 OPEN      2
       1 hanode2 OPEN      2

#lltstat -nvv --> It Gives more Detailed Output.

Node State Link Status Address

*0 hanode1 OPEN
                      net4 UP       08:00:20:93:0E:34
                      net6 UP       08:00:20:93:0E:35
1 hanode2 OPEN
                      net4 UP       08:00:20:8F:D1:F2
                      net6 UP      08:00:20:8F:D1:F8
2           CONNWAIT
                      net7 DOWN
                      net8 DOWN
             CONNWAIT
                      net9 DOWN
                      net10 DOWN
.
1           CONNWAIT
                      netx DOWN
                      netx DOWN

#lltstat -p --> It verifies the Ports information.

LLT port information:
    Port   Usage     Cookie
      0    gab       0x0
        opens:       0 2 3 4 5 6 7 8 9 10 11 … 28 29 30 31
        connects:    0 1
      7    gab       0x7
           opens:       0 2 3 4 5 6 7 8 9 10 11 … 28 29 30 31
           connects:    0 1
         31   gab       0x1F
           opens:       0 2 3 4 5 6 7 8 9 10 11 … 28 29 30 31
           connects:    0 1

#lltstat -c --> LLT configuration Values it will display

#lltstat -l --> lists information about each configured LLT link

#lltconfig -a list --> List all MAC addresses in the cluster

#lltconfig -U --> Stopping the LLT

#lltconfig -c --> Starting the LLT

GAB:

#gabconfig -a --> List Membership

VCS Daemons:

had --> High Availability Daemon

hashadow --> High Availability Daemon's Back-up Daemon

Veritas Cluster Servers Configuration and Log Files:

/etc/VRTSvcs/conf/config/main.cf --> This is the Main Veritas Cluster server configuration file, that stores all cluster nodes' information.

/var/VRTSvcs/log --> This is the Main folder for all VCS-related logs will be stored.

/var/VRTSvcs/log/engine_A.log --> This is the Mail Log file for VCS, all cluster and service-related logs are stored here.

Starting and Stopping the cluster

"-stale" instructs the engine to treat the local config as stale

"-force" instructs the engine to treat a stale config as a valid one

#hastart [-stale|-force]

Bring the cluster into running mode from a stale state using the configuration file from a particular server

#hasys -force <server_name>

stop the cluster on the local server but leave the application/s running, do not failover the application/s

#hastop -local

stop cluster on local server but evacuate (failover) the application/s to another node within the cluster

#hastop -local -evacuate

stop the cluster on all nodes but leave the application/s running

#hastop -all -force

Cluster Status

display cluster summary

#hastatus -summary

continually monitor cluster

#hastatus

verify the cluster is operating

#hasys -display

Cluster Details

information about a cluster

#haclus -display

value for a specific cluster attribute

#haclus -value <attribute>

modify a cluster attribute

#haclus -modify <attribute name> <new>

Enable LinkMonitoring

#haclus -enable LinkMonitoring

Disable LinkMonitoring

#haclus -disable LinkMonitoring

Users

add a user

#hauser -add <username>

modify a user

#hauser -update <username>

delete a user

#hauser -delete <username>

display all users

#hauser -display

System Operations

add a system to the cluster

#hasys -add <sys>

delete a system from the cluster

#hasys -delete <sys>

Modify a system attributes

#hasys -modify <sys> <modify options>

list a system state

#hasys -state

Force a system to start

#hasys -force

Display the systems attributes

#hasys -display [-sys]

List all the systems in the cluster

#hasys -list

Change the load attribute of a system

#hasys -load <system> <value>

Display the value of a systems nodeid (/etc/llthosts)

#hasys -nodeid

Freeze a system (No offlining system, No groups onlining)

#hasys -freeze [-persistent][-evacuate]

Note: main.cf must be in write mode

Unfreeze a system ( reenable groups and resource back online)

#hasys -unfreeze [-persistent]

Note: main.cf must be in write mode

Dynamic Configuration

The VCS configuration must be in read/write mode in order to make changes. When in read/write mode the

configuration becomes stale, a .stale file is created in $VCS_CONF/conf/config. When the configuration is put

back into read only mode the .stale file is removed.

Change configuration to read/write mode

#haconf -makerw

Change configuration to read-only mode

#haconf -dump -makero

Check what mode cluster is running in

#haclus -display |grep -i 'readonly'

0 = write mode

1 = read only mode

Check the configuration file

#hacf -verify /etc/VRTSvcs/conf/config

Note: you can point to any directory as long as it has main.cf and types.cf

convert a main.cf file into cluster commands

#hacf -cftocmd /etc/VRTSvcs/conf/config -dest /tmp

convert a command file into a main.cf file

#hacf -cmdtocf /tmp -dest /etc/VRTSvcs/conf/config

Service Groups

Add a service group

#haconf -makerw

#hagrp -add groupw

#hagrp -modify groupw SystemList sun1 1 sun2 2

#hagrp -autoenable groupw -sys sun1

#haconf -dump -makero

delete a service group

#haconf -makerw

#hagrp -delete groupw

#haconf -dump -makero

change a service group

#haconf -makerw

#hagrp -modify groupw SystemList sun1 1 sun2 2 sun3 3

#haconf -dump -makero

Note: use the "hagrp -display <group>" to list attributes

List the service groups

#hagrp -list

list the groups dependencies

#hagrp -dep <group>

list the parameters of a group

#hagrp -display <group>

display a service group's resource

#hagrp -resources <group>

display the current state of the service group

#hagrp -state <group>

clear a faulted non-persistent resource in a specific grp

#hagrp -clear <group> [-sys] <host> <sys>

Remove the host

#hagrp -modify grp_zlnrssd SystemList -delete <hostname>

Add the new host (don't forget to state its position)

#hagrp -modify grp_zlnrssd SystemList -add <hostname> 1

Update the autostart list

hagrp -modify grp_zlnrssd AutoStartList <host> <host>

Service Group Operations

Start a service group and bring its resources online

#hagrp -online <group> -sys <sys>

Stop a service group and takes its resources offline

#hagrp -offline <group> -sys <sys>

Switch a service group from system to another

#hagrp -switch <group> to <sys>

Enable all the resources in a group

#hagrp -enableresources <group>

Disable all the resources in a group

#hagrp -disableresources <group>

Freeze a service group (disable onlining and offlining)

#hagrp -freeze <group> [-persistent]

note: use the following to check "hagrp -display <group> | grep TFrozen"

Unfreeze a service group (enable onlining and offlining)

#hagrp -unfreeze <group> [-persistent]

note: use the following to check "hagrp -display <group> | grep TFrozen"

Enable a service group. Enabled groups can only be brought online

#haconf -makerw

#hagrp -enable <group> [-sys]

#haconf -dump -makero

Note to check run the following command "hagrp -display | grep Enabled"

Disable a service group. Stop from bringing online

#haconf -makerw

#hagrp -disable <group> [-sys]

#haconf -dump -makero

Note to check run the following command "hagrp -display | grep Enabled"

Flush a service group and enable corrective action.

#hagrp -flush <group> -sys <system>

Resources

Add a resource

#haconf -makerw

#hares -add appDG DiskGroup groupw

#hares -modify appDG Enabled 1

#hares -modify appDG DiskGroup appdg

#hares -modify appDG StartVolumes 0

#haconf -dump -makero

delete a resource

#haconf -makerw

#hares -delete <resource>

#haconf -dump -makero

change a resource

#haconf -makerw

#hares -modify appDG Enabled 1

#haconf -dump -makero

Note: list parameters "hares -display <resource>"

Change a resource attribute to be globally wide

#hares -global <resource> <attribute> <value>

change a resource attribute to be locally wide

#hares -local <resource> <attribute> <value>

list the parameters of a resource

#hares -display <resource>

list the resources

#hares -list

list the resource dependencies

#hares -dep

Resource Operations

Online a resource

#hares -online <resource> [-sys]

Offline a resource

#hares -offline <resource> [-sys]

display the state of a resource( offline, online, etc)

#hares -state

display the parameters of a resource

#hares -display <resource>

Offline a resource and propagate the command to its children

#hares -offprop <resource> -sys <sys>

Cause a resource agent to immediately monitor the resource

#hares -probe <resource> -sys <sys>

Clearing a resource (automatically initiates the onlining)

#hares -clear <resource> [-sys]

Resource Types

Add a resource type

#hatype -add <type>

Remove a resource type

#hatype -delete <type>

List all resource types

#hatype -list

Display a resource type

#hatype -display <type>

List a partitcular resource type

#hatype -resources <type>

Change a particular resource types attributes

#hatype -value <type> <attr>

Resource Agents

Add a agent

#pkgadd -d . <agent package>

remove a agent

#pkgrm <agent package>

change a agent n/a

list all ha agents

#haagent -list

Display agents run-time information i.e has it started, is it running ?

#haagent -display <agent_name>

Display agents faults

#haagent -display |grep Faults

Resource Agent Operations

Start an agent

#haagent -start <agent_name>[-sys]

Stop an agent

#haagent -stop <agent_name>[-sys]

IT Infra University