Hewlett-Packard introduced a new component called HP Agentless Management within their Gen8 Series. This feature is extensible though an agent running inside the operating system, in that case the ESXi Host. The package (hp-ams) is included in all customized VMware images provided by HP and the HP ESXi Offline Bundles. Unfortunately this package create some issues caused by excessive local logins. This might pointlessly blow up the database and logfiles. You can check that problem by opening the Event-Log provided by the ESXi Host which is flooded with the following messages:
User root@127.0.0.1 logged in as User root@ logged out (login time: , number of API invocations: , user agent: ) User root@127.0.0.1 logged in as User root@ logged out (login time: , number of API invocations: , user agent: )
In this post i am going to show how to workaround this issue and remove the hp-ams from you ESXi Host, and how to build a new image without that package.
What is this Service good for?
Prior to remove that service you should know what the hp-ams package is good for and why you would need a contradictory thing like a "Agentless Managent Agent". Without this agent, there a missing some minor features: OS and driver information passed to the iLO and a feature to log iLO events to Guest OS logs. I do not really need that features, so disabling the service is not a problem. To get more information about that service check this document.
Manually disable hp-ams Service
You can easily stop the hp-ams Service to test the impact. This change is active immediately and does not persist a reboot. You do not have to enable Maintenance Mode:
root@esx01.virten.local:~ $ /etc/init.d/hp-ams.sh stop
Remove hp-ams Package
To make this change persistent across reboots you have to remove the hp-ams package. This change requires a reboot:
root@esx01.virten.local:~ $ esxcli software vib remove -n hp-ams Removal Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: VIBs Removed: Hewlett-Packard_bootbank_hp-ams_500.9.3.5-02.434156 VIBs Skipped:
Create an ESXi Image and Depot without hp-ams with PowerCLI
You have to download HP Customized ESXi Image (Choose the .zip Depot File).
Add-EsxSoftwareDepot C:\tmp\VMware-ESXi-5.1.0-Update1-1065491-HP-5.50.26-depot.zip New-EsxImageProfile -CloneProfile HP-ESXi-5.1.0-standard -Name "HP-ESXi-5.1.0-U1-no_ams" Remove-EsxSoftwarePackage -ImageProfile HP-ESXi-5.1.0-U1-no_ams -SoftwarePackage hp-ams Compare-EsxImageProfile HP-ESXi-5.1.0-standard HP-ESXi-5.1.0-U1-no_ams Export-EsxImageProfile -ExportToIso -ImageProfile HP-ESXi-5.1.0-U1-no_ams -FilePath c:\tmp\HP-ESXi-5.1.0-U1-no_ams.iso Export-EsxImageProfile -ExportToBundle -ImageProfile HP-ESXi-5.1.0-U1-no_ams -FilePath c:\tmp\HP-ESXi-5.1.0-U1-no_ams.zip
Find ESXi Hosts with hp-ams by using PowerCLI
This small PowerCLI Script checks the vCenter for ESXi Hosts with the hp-ams package installed:
Get-VMHost | Sort Name | Foreach { $ESXCLI = Get-EsxCli -VMHost $_ -ErrorAction SilentlyContinue $ESXCLI.software.vib.list() | Where { $_.Name -like "hp-ams"} | Select @{N="VMHost";E={$ESXCLI.VMHost}}, Name, Version }
I have the same problem on the same servers. And tip "Manually disable hp-ams Service" donť work :(
Any error messages?
Does removing the package work?
Pingback: Unable to connect to the MKS: Connection terminated by server on ESXi 5.5 |
I have this same issue with my Gen8s, and build them with HP OEM ISO VMware-ESXi-5.1.0-Update1-1065491-HP-5.50.26.iso, and cannot even find hp-ams. Yet my events screen in vCenter fills with these useless entries:
User root@ logged out (login time: , number of API invocations: , user agent: )
info
14/03/2014 16:16:58
hostname
root
I have 6 hosts in this cluster and only one is throwing this nuisance event now...and I did find hp-ams.sh in /etc/init.d for this host, so I have stopped it. Still scratching my head wondering why it is not on the other hosts. I built them but perhaps one of my colleagues is already on top of this issue :roll:
/etc/init.d # ./hp-ams.sh stop
Stopping process 11163 11159 ...
/etc/init.d # esxcli software vib remove -n hp-ams
Removal Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed:
VIBs Removed: Hewlett-Packard_bootbank_hp-ams_500.9.3.5-02.434156
VIBs Skipped:
/etc/init.d #
This bug blow up a customers vCenter DB to >50GB.
Small bug, huge impact :mad:
Cheers,
Mario
I have this issue on a dell r620, none of the suggested fixes work
Jeff, if Dell servers interacts with hp-ams.sh, you got a bigger problems than all of us. :wink:
instated of connecting to ESXi Host directly.
when we have to remove a vib from more than 100+ esxi host .. We have to connect all server servers to remove the module? it too loaded. Is there any way that we can connect to vCenter using powercli and then read the esxcli of the host. and remove the vib
Should be possible. Something like:
Get-VMHost | Foreach {
$ESXCLI = Get-EsxCli -VMHost $_ -ErrorAction SilentlyContinue
$esxcli.software.vib.remove($false, $false, $false, $false, "hp-ams")
}
Not sure about the parameters that are part of $esxcli.software.vib.remove(boolean dryrun, boolean force, boolean maintenancemode, boolean noliveinstall, string[] vibname)
I do not have a system to test that at the moment.
I tried below.. no luck
PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like "*scsi-aacraid*"}
PS C:\Users\Administrator>
PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
Missing required parameter --vibname
At line:1 char:48
+ $hpams | ForEach { $esxcli.software.vib.remove <<<< ($false, $false, $false, $false, $_.Name)}
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : MethodInvocationException
That totally worked for me:
PowerCLI C:\Users\fgr> $esxcli = Get-VMHost | Get-EsxCli
PowerCLI C:\Users\fgr> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like “*scsi-megaraid-sas*”}
PowerCLI C:\Users\fgr> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
Message : The update completed successfully, but the system needs to be rebooted for the changes to be
effective.
Is $hpams filled correctely? ("PowerCLI C:\Users\fgr> $hpams" output?)
I've seen older esxcli versions with other options. Can you do this and post the output?
PowerCLI C:\Users\fgr> $esxcli = Get-VMHost | Get-EsxCli
PowerCLI C:\Users\fgr> $esxcli.software.vib.remove
TypeNameOfValue : VMware.VimAutomation.ViCore.Util10Ps.EsxCliExtensionMethod
OverloadDefinitions : {vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
maintenancemode, boolean noliveinstall, string[] vibname)}
MemberType : CodeMethod
Value : vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
maintenancemode, boolean noliveinstall, string[] vibname)
Name : remove
IsInstance : True
below is the output
PS C:\Users\Administrator\Desktop> $esxcli.software.vib.remove
TypeNameOfValue : VMware.VimAutomation.ViCore.Util10Ps.EsxCliExtensionMethod
OverloadDefinitions : {vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean
maintenancemode, boolean noliveinstall, string[] vibname)}
MemberType : CodeMethod
Value : vim.EsxCLI.software.vib.remove.InstallationResult remove(boolean dryrun, boolean force, boolean m
aintenancemode, boolean noliveinstall, string[] vibname)
Name : remove
IsInstance : True
PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like “*scsi-megaraid-sas*”}
PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
Missing required parameter –vibname
At line:1 char:47
+ $hpams | ForEach { $esxcli.software.vib.remove <<< $hpams
AcceptanceLevel : VMwareCertified
ID : VMware_bootbank_scsi-megaraid-sas_4.32-1vmw.500.0.0.469512
InstallDate : 2015-01-11
Name : scsi-megaraid-sas
ReleaseDate : 2011-08-19
Status :
Vendor : VMware
Version : 4.32-1vmw.500.0.0.469512
its wired for you how it is working ...
same commands i am able to do if i am connected individual esxi host using root id Connect-VIServer ... But when i connected with vcenter using administrator (Connect-VIServer it not working getting above error message .. .
PS C:\Users\Administrator> $esxcli = Get-VMHost | Get-EsxCli
PS C:\Users\Administrator> $hpams=$esxcli.software.vib.list() | Where { $_.Name -like "*scsi-megaraid-sas*"}
PS C:\Users\Administrator> $hpams | ForEach { $esxcli.software.vib.remove($false, $false, $false, $false, $_.Name)}
Missing required parameter --vibname
At line:1 char:47
+ $hpams | ForEach { $esxcli.software.vib.remove <<< $hpams
AcceptanceLevel : VMwareCertified
ID : VMware_bootbank_scsi-megaraid-sas_4.32-1vmw.500.0.0.469512
InstallDate : 2015-01-11
Name : scsi-megaraid-sas
ReleaseDate : 2011-08-19
Status :
Vendor : VMware
Version : 4.32-1vmw.500.0.0.469512
esxcli software vib list
try this command and show the list
next unnistall all modules non HP example: Lenovo, IBM , Dell and others non hp and vmware modules
it works
in my case lenovo modules generates many logs
I would not recommend to generally remove all vendor packages/drivers. Only specific unnecessary packages as explained above.