vSphere Data Protection (VDP) Troubleshooting Commands

vdp-vsphere-data-protectionIf the VMware vSphere Data Protection (VDP) appliance has a problem, the GUI might not give enough information to resolve the issue. The command-line also gives you useful information that are not visible in the vSphere Web Client.

This post describes commands I use frequently to troubleshoot VDP appliances or gather additional information like capacity reports.

You have to connect to the VDP appliance with SSH as root (password was set during initial configuration)

status.dpn
Display VDP status information

root@vdp01:~/#:  status.dpn
Mon Feb  2 12:30:03 CET 2015  [vdp01.virten.local] Mon Feb  2 11:30:03 2015 UTC (Initialized Wed May 21 14:11:24 2014 UTC)
Node   IP Address     Version   State   Runlevel  Srvr+Root+User Dis Suspend Load UsedMB Errlen  %Full   Percent Full and Stripe Status by Disk
0.0   192.168.222.7  7.0.82-99  ONLINE fullaccess mhpu+0hpu+0hpu   2 false   2.51 7497 46413720  53.6%  53%(onl:2596) 53%(onl:2597) 53%(onl:2592)
Srvr+Root+User Modes = migrate + hfswriteable + persistwriteable + useraccntwriteable

All reported states=(ONLINE), runlevels=(fullaccess), modes=(mhpu+0hpu+0hpu)
System-Status: ok
Access-Status: full

Last checkpoint: cp.20150202100327 finished Mon Feb  2 11:04:54 2015 after 01m 27s (OK)
Last GC: finished Mon Feb  2 10:03:27 2015 after 02h 01m >> recovered 8.86 GB (OK)
Hfscheck in progress: started Mon Feb  2 11:04:55 2015 >> checked 4025 of 6872 stripes (hfscheck)

Maintenance windows scheduler capacity profile is active.
  The maintenance window is currently running.
  Currently running task(s): hfscheck
  Next backup window start time: Tue Feb  3 01:00:00 2015 CET
  Next maintenance window start time: Tue Feb  3 08:00:00 2015 CET
root@vdp01:~/#:


dpnctl status
Display service status information

root@vdp01:~/#: dpnctl status
Identity added: /home/dpn/.ssh/dpnid (/home/dpn/.ssh/dpnid)
dpnctl: INFO: gsan status: up
dpnctl: INFO: MCS status: up.
dpnctl: INFO: Backup scheduler status: up.
dpnctl: INFO: axionfs status: up.
dpnctl: INFO: Maintenance windows scheduler status: enabled.
dpnctl: INFO: Unattended startup status: enabled.

capacity.sh
Analyse space consumption from the last 30 backup jobs. Displays the amount of new data and how much space the garbage collection has recovered.

root@vdp01:~/#: capacity.sh
Date          New Data #BU       Removed #GC    Net Change
----------  ---------- -----  ---------- -----  ----------
2015-01-03     9815 mb 19       -2502 mb 1         7313 mb
[...]
2015-01-30     8730 mb 19       -5924 mb 1         2806 mb
2015-01-31     3403 mb 19        -980 mb 1         2423 mb
2015-02-01     7426 mb 19       -4111 mb 1        -6684 mb
2015-02-02     7982 mb 19       -1012 mb 1        -3029 mb
----------  ---------- -----  ---------- -----  ----------
Average       49650 mb         -50386 mb           -735 mb


Top 3 High Change Clients:
--------------------------
Total for all clients                      489530 mb  100.0%
 vcsa.virten.local_UBeWvXdhIMvDS5GCDS44A    30317 mb   20.4%
 vma.virten.local_UBeWgasvD9bcKVsfas3gfg    24213 mb   16.3%
 db01.virten.local_UBf7fhPBspcCvAZy0FH5pg   14829 mb   10.0%

df -h
Display free partition space. This is not an equivalent to the free space displayed in the GUI but can reveal issues if partitions are full.

root@vdp01:~/#: df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             7.9G  4.8G  2.8G  64% /
devtmpfs              3.9G  160K  3.9G   1% /dev
tmpfs                 3.9G     0  3.9G   0% /dev/shm
/dev/sda1             130M   38M   86M  31% /boot
/dev/sda7             1.5G  385M 1011M  28% /var
/dev/sda9              77G  6.1G   67G   9% /space
/dev/sdb1             256G  146G  111G  57% /data01
/dev/sdc1             256G  141G  116G  55% /data02
/dev/sdd1             256G  141G  116G  56% /data03

cplist
Display Checkpoint status

root@vdp01:~/#: cplist
cp.20150201101307 Sun Feb  1 11:13:07 2015   valid rol ---  nodes   1/1 stripes   7785
cp.20150201120607 Sun Feb  1 13:06:07 2015   valid --- ---  nodes   1/1 stripes   7785
cp.20150202100327 Mon Feb  2 11:03:27 2015   valid --- ---  nodes   1/1 stripes   7785

mccli server show-prop
Display VDP appliance properties. This is an equivalent to the information shown in the vSphere Web Client.

root@vdp01:~/#: mccli server show-prop
0,23000,CLI command completed successfully.
Attribute                        Value
-------------------------------- ----------------------------
State                            Full Access
Active sessions                  0
Total capacity                   1.2 TB
Capacity used                    551.3 GB
Server utilization               41.2%
Bytes protected                  1.8 TB
Bytes protected quota            Not configured
License expiration               Never
Time since Server initialization 192 days 01h:46m
Last checkpoint                  2015-02-02 10:09:06 CET
Last validated checkpoint        2015-02-02 10:00:56 CET
System Name                      vdp01.virten.local
System ID                        154432225@00:50:56:43:44:AB
HFSAddr                          vdp01.virten.local
HFSPort                          27000
IP address                       192.168.222.7:26000
Number of nodes                  1
Nodes Online                     1
Nodes Offline                    0
Nodes Read-only                  0
Nodes Timed-out                  0

mccli activity show
Display backup jobs information. Each activity is a backup job from a single virtual machine. If you have one daily backup job with 10 VMs configured in VDP, you will see 10 activities per day.

root@vdp01:~/#: mccli activity show
0,23000,CLI command completed successfully.
ID               Status    Error Code Start Time           Elapsed     End Time             Type             Progress Bytes  New Bytes
---------------- --------- ---------- -------------------- ----------- -------------------- ---------------- --------------- ---------
9142283520009409 Completed 0          2015-02-02 01:00 CET 00h:28m:45s 2015-02-02 01:48 CET Scheduled Backup 103,371,665,938 0.5%
9142266240012709 Completed 0          2015-01-31 03:33 CET 00h:12m:17s 2015-01-31 03:35 CET Scheduled Backup 57,244,772,623  0.2%
9142274880012709 Completed 0          2015-02-01 01:03 CET 01h:12m:55s 2015-02-01 02:58 CET Scheduled Backup 124,014,703,210 1.8%
[...]

mccli activity get-log --id=<ID>
Get the activity log from a backup job. If a backup job failed, you might find useful information here. Produces lots of information, so it's better to pipe it to a file:

root@vdp01:~/#: mccli activity get-log --id=9142274880012209 > 9142274880012209.txt
root@vdp01:~/#: less 9142274880012209.txt

mccli activity show --name=/<VCENTER>/VirtualMachines/<VM>
Display backup jobs information from a single Virtual Machine

root@vdp01:~/#: mccli activity show --name=/vcsa.virten.local/VirtualMachines/vma

 

  1. Regarding "capacity.sh". If you would like to list more than the "Top 3" changers, simply edit capacity.sh. For VDP 6 the file is located /usr/local/avamar/bin.
    Scroll to the bottom of the file and edit the section that starts:(line 134?)tot=$.
    Change "Top 3" to your preferred number. And change "head -3" to "head -"yourpreferrednumber".

    Save the file then run the command. Your new output will display.

  2. Hello, Working with VDP.

    How can i select only non completed bacp using this command ?

    oot@vdp01:~/#: mccli activity show
    0,23000,CLI command completed successfully.
    ID Status Error Code Start Time Elapsed End Time Type Progress Bytes New Bytes
    ---------------- --------- ---------- -------------------- ----------- -------------------- ---------------- --------------- ---------
    9142283520009409 Completed 0 2015-02-02 01:00 CET 00h:28m:45s 2015-02-02 01:48 CET Scheduled Backup 103,371,665,938 0.5%

    • :~/#: mccli activity show --help
      activity show: Show activity
      [--active=Boolean(false)]: Show active jobs
      [--completed=Boolean(false)]: Show completed jobs

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>