Help.tests.ps1 silently tests every command on the machine when the module is not loaded

[tablackburn]

tests/Help.tests.ps1 silently degrades from "test help for this module's commands" to "test
help for every command installed on the machine" whenever $env:BHProjectName is not set.

The defect

tests/Help.tests.ps1:22-28:

$params = @{
    Module      = (Get-Module $env:BHProjectName)
    CommandType = [System.Management.Automation.CommandTypes[]]'Cmdlet, Function' # Not alias
}
if ($PSVersionTable.PSVersion.Major -lt 6) {
    $params.CommandType[0] += 'Workflow'
}
$commands = Get-Command @params

With $env:BHProjectName empty, Get-Module writes a non-terminating
ParameterArgumentValidationError, $params.Module becomes $null, and Get-Command -Module $null
applies no filter at all. Reproduced on a developer machine:

BHProjectName = ''
Get-Module : Cannot validate argument on parameter 'Name'. The argument is null or empty.
Get-Module $env:BHProjectName  -> count=1  null=True
Get-Command @params            -> 4780 commands

Describe "Test help for <_.Name>" -ForEach $commands then generates a Describe per command, and
lines 43 and 55 each call Get-Help $command.Name — two lookups per command, across 4,780
commands.

Why it is slow rather than merely wrong

Get-Help falls back to a machine-wide search whenever the name does not resolve exactly.
HelpSystem.GetHelp tries ExactMatchHelp first and calls SearchHelp when that finds nothing;
CommandHelpProvider.SearchHelp then decorates the name as *name* and walks every installed
module through ModuleUtils.GetMatchingCommands, emitting the progress record
Searching Help for {0} .... Microsoft documents the behavior:

When using Get-Help to locate commands, it initially performs a wildcard search for command
names based on your input. If that doesn't find any matches, it conducts a comprehensive
full-text search across all PowerShell help articles on your system.

Crucially the {0} is the name that was passed, not a command being scanned past — so
progress reading Searching Help for Add-ClusteriSCSITargetServerRole means the suite really did
call Get-Help Add-ClusteriSCSITargetServerRole.

Measured on this machine:

lookup	cost
resolvable name	2-4 ms
unresolvable name	~2.6-3.5 s, not cached (3 identical calls: 5025 / 3415 / 3299 ms)

Why Windows PowerShell 5.1 is hit far harder

Not because Get-Help is slower there — the scan is actually slower on 7. It is the number of
names that cannot resolve:

PS 5.1.26100.9168: enumerated=3987  unresolvable=653
PS 7.6.5:          enumerated=3929  unresolvable=83

Modules visible to module analysis but unimportable under 5.1 — CompatiblePSEditions = 'Core',
PS7-only, or dependent on an absent OS feature — miss exact match permanently and pay the full
scan on every call. 653 x 2 calls x ~3 s is roughly 65 minutes; an observed run took 39.

Detection

A full local run of the suite under Windows PowerShell 5.1 takes tens of minutes and floods the
console with Searching Help for ... progress for commands that have nothing to do with this
repository. Under psake the environment variables are set by BuildHelpers, so CI never sees it —
the 5.1 leg runs the same files in about 161 seconds. It only bites when Pester is invoked
directly, which is exactly what a developer does when iterating on one test file.

Fix

Fail loudly instead of degrading. The module failing to resolve must not silently become "no
filter":

• -ErrorAction Stop on the Get-Module call, or resolve the module explicitly and throw when it
  is absent, and
• assert $commands.Count is sane — for example that it matches the manifest's
  FunctionsToExport — so this can never again pass with the wrong 4,780 commands.

Guarding each lookup with Get-Command before Get-Help is worthwhile defence in depth (56 ms
versus 2596 ms for an unresolvable name), but on its own it would only reduce the run to a few
minutes of testing the wrong commands. The real defect is the silent widening.

Not the fix: setting $ProgressPreference = 'SilentlyContinue'. Measured, it buys about 5%
(2655/2665/2678 ms versus 2754/3105/2858 ms) — it silences the console and leaves the scan intact.

Notes

• Pre-existing, and inherited from the widely copied
  juneb/PesterTDD Module.Help.Tests.ps1,
  so other repositories using that pattern are likely affected too. No public issue appears to
  describe this.
• dbatools independently hardened the same pattern by filtering the command set explicitly, and
  its file comments record an adjacent discovery-versus-runtime bug that produced zero tests for
  years.

Done when

Running Invoke-Pester -Path ./tests directly, without the psake build having set the BuildHelpers
environment variables, either tests only this module's exported commands or fails with a clear
message — and never enumerates the whole machine.

[tablackburn]

Correction: the trigger in this issue is wrong

The title and the analysis above say the widening happens "when BHProjectName is unset". That is
not the trigger, and the repro in the body does not hold — I could not reproduce it on either Pester
major.

What actually happens. The widening is caused by Get-Module <name> returning nothing, whatever
the reason. BHProjectName being empty is one way to get there; it is not the common one, and with
it empty the file dies earlier anyway.

Measured on this machine, with the build environment set correctly:

BHProjectName = 'PowerShellBuild'
PowerShellBuild loaded in this session: False
Get-Module $env:BHProjectName -> 0 module(s)
Module value is null: True
Get-Command @params -> 3110 commands

So: the variable is right, the module simply is not loaded in that session, Get-Module returns
nothing, $params.Module becomes $null, and Get-Command -Module $null applies no filter. 3,110
commands rather than 12.

Why the original repro does not hold. With BHProjectName empty the file fails at
Import-PowerShellDataFile -Path $env:BHPSModuleManifest, before ever reaching Get-Command.
Confirmed both ways: discovery-only runs of the pre-fix file produce zero Describe blocks
under Pester 5.7.1 on Windows PowerShell 5.1, and Pester 6.1.0 stops at the same line because
discovery runs with ErrorActionPreference = 'Stop'.

What this does not change. The consequence, the cost, and the fix are all unaffected. Get-Help
still falls back to a machine-wide search for every unresolvable name, and a session that resolves
3,110 commands instead of 12 still pays it. The 39-minute run that prompted this issue was real.

The fix in #179 addresses the real trigger rather than the one described here — it resolves the
module explicitly and throws when it is absent, and asserts the discovered command set matches the
manifest's FunctionsToExport. Both catch the case measured above, where the environment variables
are perfectly fine.

I filed this from a mechanism I had reasoned out rather than reproduced end to end. The
reproduction above is the corrected one.