Check root password with PowerCLI (Multi-threading!)

Keeping your root password similar in all of your ESXi hosts, is one of the virtual environment key methods to control and maintain large environments. It will make it easier to connect directly to a host in case of vCenter failure, access SSH for troubleshooting and control it from DCUI.

There is a great one-liner by Kelvin Wong, that allows you to get a list of all of the VMhosts that have different password than the standard one.

In this post, I’ll try to:

  1. Make it simpler for PowerCLI beginners to use this script
  2. Provide advanced users with methods of multi-threading in PowerShell 2.0

First, the script for getting a list of hosts with non standard password :

$vCenterName = ""  # ChangeME
$ExportFileLocation = "F:\Scripts\Harel_CheckrootPassword\PasswordNotMatch.txt" # ChangeME
$rootpassword = Read-host -assecurestring -prompt "Please enter local root password"
if (!(get-pssnapin -name VMware.VimAutomation.Core -erroraction silentlycontinue)) {
    add-pssnapin VMware.VimAutomation.Core}
Connect-VIServer $vCenterName

$vmhostsView = get-view -ViewType HostSystem -Property Name,Summary.runtime.ConnectionState `
| Where {$_.Summary.runtime.ConnectionState -eq "connected"} | %{$_.Name}

if (Test-Path  ($ExportFileLocation) -pathtype leaf)
{Remove-Item $ExportFileLocation -Confirm:$False}

$vmhostsView | %{ $err = @() ; 
connect-viserver $_ -user root -password $rootpassword -EA silentlycontinue -EV err ; 
if ($err.count -gt 0) { $_ | out-file $ExportFileLocation -append }
else {disconnect-viserver $_ -force -confirm:$false} }

The file created by this script, contains a list of hosts with root password different than the one typed as input.

BUT, the script will test connection to each of the hosts one-by-one, which may take a while if you have more than 10 hosts. In an environment of 74 hosts, for example, it took the script 10:30 min to run (calculated with measure-command of course):


We will reduce the run time of the script, by using parallelism of the hosts check. Multi-threading in PowerShell.

The script will start the same way as the one above –

$vCenterName = ""  # ChangeME
$ExportFileLocation = "F:\Scripts\Harel_CheckrootPassword\PasswordNotMatch.txt" # ChangeME
$rootpassword = Read-host -assecurestring -prompt "Please enter local root password"
if (!(get-pssnapin -name VMware.VimAutomation.Core -erroraction silentlycontinue)) {
    add-pssnapin VMware.VimAutomation.Core}
Connect-VIServer $vCenterName

$vmhostsView = get-view -ViewType HostSystem -Property Name,Summary.runtime.ConnectionState `
| Where {$_.Summary.runtime.ConnectionState -eq "connected"} | %{$_.Name}

if (Test-Path  ($ExportFileLocation) -pathtype leaf)
{Remove-Item $ExportFileLocation -Confirm:$False}

And now, adding the interesting part –

function CanWeAddJob(){
$Alljobs=(Get-Job -State "Running" | measure-Object).count
if ($alljobs -lt 10) {return $true}
else {return $false}

Foreach ($vmhostname in $vmhostsView){
Do {Start-Sleep -Milliseconds 500}
while (!(CanWeAddJob))

$args = ($vmhostname,$credentials,$ExportFileLocation)

Start-Job -Name $vmhostname -ArgumentList $args -InitializationScript {Add-PSSnapin VMware.VimAutomation.Core} –Scriptblock {$err = @() ;
Connect-VIServer $args[0] -Credential $args[1] -EA silentlycontinue -EV err ; 
if ($err.count -gt 0) { $args[0] | out-file $args[2] -append } 
else {disconnect-viserver $_ -force -confirm:$false} } -RunAs32


Function CanWeAddJob() is checking what is the running job count in your PowerShell session, and determine whether to add more job, or not. In this example, it does it to a maximum of 10 parallel jobs.

Foreach is here to split the long VMHosts list to many separated tasks. It will only add task to the running job queue, if the queue have less than 10 jobs.
PSSnapin was added to each of the new powershell.exe instances created, with RunAs32 parameter, to make it take less RAM of your server / workstation.

While the script is running, you should see something like this in your task manager:


Can you guess what was the run time of the multi-threaded script?


2:42 min, which saved me 75% of the original run time.

Detach All Unused Devices in ESXi Cluster

Removing storage devices (Datastores + RDMs) can be tricky task for the VI admin. He will have to:

  1. Map the datastores / RDMs in use (maybe with PowerCLI scripts, EMC VSI, or other storage plugins)
  2. Delete / Remove the specific devices from virtual environment side (Delete datastore, Remove RDM from VM)
  3. Detach the devices from ESXi hosts
  4. Let the storage admin know this devices can be removed from storage side

I’m here to help you with automation for the 3rd task. The amount of steps required for detaching 1 device off 10 hosts cluster, is 10 steps (or 10 detach operations). Detaching 20 devices off this cluster will result in 200 steps, which can be very tedious task.

This script is written in PowerCLI, VMware KB shows more ways of detaching disks.

Script initialization (Environment names, Snapin, log location, time function):

$vCenterName = ""  # ChangeME
$ClusterNameToDetach = "SQL-Cluster-A" # ChangeME

if (!(get-pssnapin -name VMware.VimAutomation.Core -erroraction silentlycontinue)) {
    add-pssnapin VMware.VimAutomation.Core}

Connect-VIServer $vCenterName

cd F:\Scripts\Serv_DetachClusterDevices\
$dslog_file = ".\log\"+$ClusterNameToDetach+"_log.txt" 

Function Givemetime(){
$mydate = Get-Date
return $mydate = $mydate.ToShortDateString() +" "+ $mydate.ToLongTimeString() +" "

Adding some of the detach disk functions:

function Detach-Disk{
    $storSys = Get-View $VMHost.Extensiondata.ConfigManager.StorageSystem
    $lunUuid = (Get-ScsiLun -VmHost $VMHost | 
      where {$_.CanonicalName -eq $CanonicalName}).ExtensionData.Uuid

Function Detach-RDMs ($naa){
Foreach ($vmhost in $vmhosts){
Write-Host "Starting with"$vmhost.Name
"$(Givemetime)" + "Starting with " +$vmhost.Name | out-file $dslog_file -append -Force
if (($vmhost |  Get-ScsiLun| Where {($_).ExtensionData.OperationalState -eq "ok"} | %{$_.CanonicalName}) -contains ($naa) ){

Write-Host "Detaching device"$naa
"$(Givemetime)" + "Detaching device "+$naa | out-file $dslog_file -append -Force
Detach-Disk -VMHost $vmhost -CanonicalName $naa
Write-Host "Done"
"$(Givemetime)" + "Done" | out-file $dslog_file -append -Force

The power of the script comes here – the ability to map the devices in the cluster, and separate them to few groups:
$ClusterDSDevices – Devices that are used as datastores in the cluster
$ClusterRDMDevices – Devices that are used as RDMs in the cluster
$ClusterVMAXDevices – All Devices that are visible by the cluster, from a specific vendor. In this example, EMC VMAX devices will presented
$AllUseddevices – Both $ClusterDSDevices and $ClusterRDMDevices variables combined

Variables will contain a list of naa devices, and it is your job to determine what should be declared for detach by the end of the script.
In the end of the mapping script I added an example I used – filter for:All VMAX devices (both DS and RDM), that are not in use. It will list the required devices in $DevicesForDetach variable.

function MapDevices (){
$global:DevicesForDetach = $null
$global:vmhosts = Get-Cluster $ClusterNameToDetach | Get-VMHost | Sort Name

$global:ClusterDSDevices = $null
$global:ClusterDSDevices =  Get-VMHost $vmhosts | Get-Datastore | Where-Object {$_.ExtensionData.Info.GetType().Name -eq "VmfsDatastoreInfo"} | %{$[0].DiskName}

$global:ClusterRDMDevices = $null
$global:ClusterRDMDevices = Get-VMHost $vmhosts | Get-VM | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | %{$_.ScsiCanonicalName}

$global:ClusterVMAXDevices = $null
$global:ClusterVMAXDevices = Get-VMHost $vmhosts | Get-ScsiLun | Where {($_).ExtensionData.OperationalState -eq "ok"} | %{$_.CanonicalName} | Where {$_ -like "naa.600009*"} | select -Unique

$global:AllUseddevices = $ClusterDSDevices + $ClusterRDMDevices
$global:AllUseddevices = $AllUseddevices | Where {$_ -like "naa.600009*"} | select -Unique

$global:AllVMAXDevices = $ClusterVMAXDevices

## Change it to your reference:
If ($AllUseddevices){
$global:DevicesForDetach = (Compare-Object -DifferenceObject $AllVMAXDevices -ReferenceObject $AllUseddevices -IncludeEqual | Where {$_.SideIndicator -eq "=>"}) | %{$_.InputObject}
Else {$global:DevicesForDetach = $global:AllVMAXDevices}
## End of change

It is now time to use the list we created, ans start detaching disks, all tasks will be exported to a log.

Write-Host "Device Mapping started..."
"$(Givemetime)" + "Device Mapping started..." | out-file $dslog_file -append -Force
Write-Host "Device Mapping finished"
"$(Givemetime)" + "Device Mapping finished" | out-file $dslog_file -append -Force
if (!($DevicesForDetach)) {
Write-Host "No devices to detach"
"$(Givemetime)" + "No devices to detach on cluster" | out-file $dslog_file -append -Force
else {
Foreach ($device in $DevicesForDetach){
Detach-RDMs ($device)

Log file will look like that in the end of the process:


In this example, 19 devices were detached from 9 hosts, resulted in 171 detach commands to the ESXi hosts in the cluster. The script took less than an hour to complete it.