HP Proliant Gen8 Agentless Management floods ESXi and vCenter Logs

Hewlett-Packard introduced a new component called HP Agentless Management within their Gen8 Series. This feature is extensible though an agent running inside the operating system, in that case the ESXi Host. The package (hp-ams) is included in all customized VMware images provided by HP and the HP ESXi Offline Bundles. Unfortunately this package create some issues caused by excessive local logins. This might pointlessly blow up the database and logfiles. You can check that problem by opening the Event-Log provided by the ESXi Host which is flooded with the following messages:

User root@127.0.0.1 logged in as
User root@ logged out (login time: , number of API invocations: , user agent: )
User root@127.0.0.1 logged in as
User root@ logged out (login time: , number of API invocations: , user agent: )

hp-ams

In this post i am going to show how to workaround this issue and remove the hp-ams from you ESXi Host, and how to build a new image without that package.

What is this Service good for?
Prior to remove that service you should know what the hp-ams package is good for and why you would need a contradictory thing like a "Agentless Managent Agent". Without this agent, there a missing some minor features: OS and driver information passed to the iLO and a feature to log iLO events to Guest OS logs. I do not really need that features, so disabling the service is not a problem. To get more information about that service check this document.


Manually disable hp-ams Service
You can easily stop the hp-ams Service to test the impact. This change is active immediately and does not persist a reboot. You do not have to enable Maintenance Mode:

root@esx01.virten.local:~ $ /etc/init.d/hp-ams.sh stop

Remove hp-ams Package
To make this change persistent across reboots you have to remove the hp-ams package. This change requires a reboot:

root@esx01.virten.local:~ $ esxcli software vib remove -n hp-ams
Removal Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed:
   VIBs Removed: Hewlett-Packard_bootbank_hp-ams_500.9.3.5-02.434156
   VIBs Skipped:


Create an ESXi Image and Depot without hp-ams with PowerCLI

You have to download HP Customized ESXi Image (Choose the .zip Depot File).

Add-EsxSoftwareDepot C:\tmp\VMware-ESXi-5.1.0-Update1-1065491-HP-5.50.26-depot.zip

New-EsxImageProfile -CloneProfile HP-ESXi-5.1.0-standard -Name "HP-ESXi-5.1.0-U1-no_ams"

Remove-EsxSoftwarePackage -ImageProfile HP-ESXi-5.1.0-U1-no_ams -SoftwarePackage hp-ams

Compare-EsxImageProfile HP-ESXi-5.1.0-standard HP-ESXi-5.1.0-U1-no_ams

Export-EsxImageProfile -ExportToIso -ImageProfile HP-ESXi-5.1.0-U1-no_ams -FilePath c:\tmp\HP-ESXi-5.1.0-U1-no_ams.iso
Export-EsxImageProfile -ExportToBundle -ImageProfile HP-ESXi-5.1.0-U1-no_ams -FilePath c:\tmp\HP-ESXi-5.1.0-U1-no_ams.zip


Find ESXi Hosts with hp-ams by using PowerCLI

This small PowerCLI Script checks the vCenter for ESXi Hosts with the hp-ams package installed:

Get-VMHost | Sort Name | Foreach {
  $ESXCLI = Get-EsxCli -VMHost $_ -ErrorAction SilentlyContinue
  $ESXCLI.software.vib.list() | Where { $_.Name -like "hp-ams"} | Select @{N="VMHost";E={$ESXCLI.VMHost}}, Name, Version
}
Leave a comment ?

18 Comments.

  1. I have the same problem on the same servers. And tip "Manually disable hp-ams Service" donť work :(

  2. I have this same issue with my Gen8s, and build them with HP OEM ISO VMware-ESXi-5.1.0-Update1-1065491-HP-5.50.26.iso, and cannot even find hp-ams. Yet my events screen in vCenter fills with these useless entries:
    User root@ logged out (login time: , number of API invocations: , user agent: )
    info
    14/03/2014 16:16:58
    hostname
    root

  3. I have 6 hosts in this cluster and only one is throwing this nuisance event now...and I did find hp-ams.sh in /etc/init.d for this host, so I have stopped it. Still scratching my head wondering why it is not on the other hosts. I built them but perhaps one of my colleagues is already on top of this issue :roll:

    /etc/init.d # ./hp-ams.sh stop
    Stopping process 11163 11159 ...
    /etc/init.d # esxcli software vib remove -n hp-ams
    Removal Result
    Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
    Reboot Required: true
    VIBs Installed:
    VIBs Removed: Hewlett-Packard_bootbank_hp-ams_500.9.3.5-02.434156
    VIBs Skipped:
    /etc/init.d #

  4. This bug blow up a customers vCenter DB to >50GB.

    Small bug, huge impact :mad:

    Cheers,
    Mario

  5. I have this issue on a dell r620, none of the suggested fixes work

  6. Jeff, if Dell servers interacts with hp-ams.sh, you got a bigger problems than all of us. :wink:

  7. instated of connecting to ESXi Host directly.

    when we have to remove a vib from more than 100+ esxi host .. We have to connect all server servers to remove the module? it too loaded. Is there any way that we can connect to vCenter using powercli and then read the esxcli of the host. and remove the vib 

    • Should be possible. Something like:

      Get-VMHost | Foreach {
      $ESXCLI = Get-EsxCli -VMHost $_ -ErrorAction SilentlyContinue
      $esxcli.software.vib.remove($false, $false, $false, $false, "hp-ams")
      }

      Not sure about the parameters that are part of $esxcli.software.vib.remove(boolean dryrun, boolean force, boolean maintenancemode, boolean noliveinstall, string[] vibname)

      I do not have a system to test that at the moment.

      • I tried below.. no luck

        PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
        PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like "*scsi-aacraid*"}
        PS C:\Users\Administrator>
        PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
        Missing required parameter --vibname
        At line:1 char:48
        + $hpams | ForEach { $esxcli.software.vib.remove <<<< ($false, $false, $false, $false, $_.Name)}
        + CategoryInfo : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : MethodInvocationException

        • That totally worked for me:

          PowerCLI C:\Users\fgr> $esxcli = Get-VMHost | Get-EsxCli
          PowerCLI C:\Users\fgr> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like “*scsi-megaraid-sas*”}
          PowerCLI C:\Users\fgr> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}

          Message : The update completed successfully, but the system needs to be rebooted for the changes to be
          effective.

          Is $hpams filled correctely? ("PowerCLI C:\Users\fgr> $hpams" output?)

          I've seen older esxcli versions with other options. Can you do this and post the output?

          PowerCLI C:\Users\fgr> $esxcli = Get-VMHost | Get-EsxCli
          PowerCLI C:\Users\fgr> $esxcli.software.vib.remove

          TypeNameOfValue : VMware.VimAutomation.ViCore.Util10Ps.EsxCliExtensionMethod
          OverloadDefinitions : {vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
          maintenancemode, boolean noliveinstall, string[] vibname)}
          MemberType : CodeMethod
          Value : vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
          maintenancemode, boolean noliveinstall, string[] vibname)
          Name : remove
          IsInstance : True

          • below is the output

            PS C:\Users\Administrator\Desktop> $esxcli.software.vib.remove

            TypeNameOfValue : VMware.VimAutomation.ViCore.Util10Ps.EsxCliExtensionMethod
            OverloadDefinitions : {vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
            maintenancemode, boolean noliveinstall, string[] vibname)}
            MemberType : CodeMethod
            Value : vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean m
            aintenancemode, boolean noliveinstall, string[] vibname)
            Name : remove
            IsInstance : True

          • PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
            PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like “*scsi-megaraid-sas*”}
            PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
            Missing required parameter –vibname
            At line:1 char:47
            + $hpams | ForEach { $esxcli.software.vib.remove <<< $hpams
            AcceptanceLevel : VMwareCertified
            ID : VMware_bootbank_scsi-megaraid-sas_4.32-1vmw.500.0.0.469512
            InstallDate : 2015-01-11
            Name : scsi-megaraid-sas
            ReleaseDate : 2011-08-19
            Status :
            Vendor : VMware
            Version : 4.32-1vmw.500.0.0.469512

          • its wired for you how it is working ...

            same commands i am able to do if i am connected individual esxi host using root id Connect-VIServer ... But when i connected with vcenter using administrator (Connect-VIServer it not working getting above error message .. .

  8. PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
    PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like "*scsi-megaraid-sas*"}
    PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
    Missing required parameter --vibname
    At line:1 char:47
    + $hpams | ForEach { $esxcli.software.vib.remove <<< $hpams

    AcceptanceLevel : VMwareCertified
    ID : VMware_bootbank_scsi-megaraid-sas_4.32-1vmw.500.0.0.469512
    InstallDate : 2015-01-11
    Name : scsi-megaraid-sas
    ReleaseDate : 2011-08-19
    Status :
    Vendor : VMware
    Version : 4.32-1vmw.500.0.0.469512

  9. esxcli software vib list
    try this command and show the list
    next unnistall all modules non HP example: Lenovo, IBM , Dell and others non hp and vmware modules

    it works
    in my case lenovo modules generates many logs

    • I would not recommend to generally remove all vendor packages/drivers. Only specific unnecessary packages as explained above.

Leave a Comment

NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Trackbacks and Pingbacks: