Requirements for a PowerShell module manager
I write a lot of PowerShell scripts. Many of these get transformed into modules for reuse and occasionally published for others. However making these reusable modules available wherever my scripts may run and propagating updates to these modules is not easily managed.
As a solution to this I’ve been designing a PowerShell module to enable scripts and modules to easily reference and use other modules without requiring these modules to be installed first. The idea is very similar to RubyGems or Nuget but I feel there are some important criteria that need to be met by a PowerShell module manager.
Here is a list of requirements from the perspective of a module-consumer:
- PowerShell already defines standard install locations for modules so a module manager should make it easy to use these by default but still allow them to be overridden.
- Scripts will often run in a least privilege environment so a module manager should not require any special permissions to install a module.
- PowerShell’s Execution Policy depends on scripts that were downloaded from the Internet to be marked as such, a module manager should honour this by marking installed modules with the appropriate Zone Identifier.
- PowerShell allows multiple versions of a module to exist side-by-side and allows scripts to reference a specific version, a module manager must support this behaviour.
- A script should be able to use the module manager to install other modules even when the module manager itself hasn’t been installed yet (eg using a secure bootstrap)
- A script should be able to reference a module by name and optional version number only.
- The module manager should operate on a clean operating system with only PowerShell installed and not have any other pre-requisites.
As a module author, there is also a set of expectations from a module manager:
- A format for modules and their metadata is already defined by PowerShell itself (a PSD1 file with a collection of PSM1, PS1, and DLL files etc in a folder) so a module manager should leverage this instead of defining a new format. The format for transporting a module over the wire is an implementation concern of the module manager and not a concern of the module author.
- Publishing a new or updated module for others to consume should be a PowerShell one-line command that can consume any locally installed module.
- There should be a standard public repository for published modules but private repositories should also be supported for publishing modules intended for personal or enterprise use only.
There will inevitably be some impacts on how authors develop their modules to support the above requirements:
- To support the common PowerShell Execution Policy setting of RemoteSigned, published modules should be signed with a valid Code Signing Certificate issued by a recognised authority. As cheap as US$75.
- Modules must assume they will be executing in a least privilege environment and only operate with the minimum permissions required to perform the task the module is designed to do.
- Related to least privilege, modules should not require a “first-use” process to register or configure the module that would require elevation.
- Modules must not assume they will be installed to a specific location and instead should use the $PSScriptRoot variable for module-relative paths.
There are already some existing solutions for PowerShell module management, some meet most of the requirements above:
- Joel Bennett‘s PoshCode module – Been around the longest but currently only supports single-file modules (ie PSM1 files)
- Andrew Nurse‘s PS–Get – Almost a direct port of Nuget to PowerShell but requires PowerShell v2 to be configured to run under .NET 4.
- Mike Chaliy‘s PSGet – Probably the closest to my ideal but currently it requires a GitHub pull-request to list a module and the module author must also find somewhere to host their module.
I think adding my own solution to this list would only add to the module management problem, especially when you consider the other PowerShell module management solutions that exist but I’m not aware of. Instead it would make more sense to have these existing solutions converge by having them each support the installation of modules published by the others and move toward meeting as many of the requirements above as feasible.
Jason Stangroome 
Hi Jason,
I would love to see a solution to this problem. The thing about the solution is that its not a solution until it has broad adoption. We saw this with package management in .NET, there were a few solutions kicking around but it wasn’t until Microsoft got behind it that it really took off.
Unfortunately in the MS technology space they add the air of legitimacy. Everything to date would need to be considered an experiment. I would be interesting in hearing what Microsoft is planning here. They could definitely help the community out with this one and not have to wait for another PS release.
>> PowerShell’s Execution Policy depends on scripts that were downloaded from the Internet to be marked as such, a module manager should honour this by marking installed modules with the appropriate Zone Identifier.
IMHO this is flawed by design. It’s too easy to write malicious script that will remove zone identification. And this will not require any admin right’s or something. Also I didn’t saw normal implementation. None (Manven, Nuget, gem) of central repositories uses such aspect of security. Central repository is already authority. And this why all central repositories requires strong author athority and have tools for moderation.
>> PowerShell allows multiple versions of a module to exist side-by-side and allows scripts to reference a specific version, a module manager must support this behaviour.
How so? Even if you import module by file name, latest module will override elder one. Can you provide more details on this? Support modules side by side is one of the requested features of the PsGet.
>> A format for modules and their metadata is already defined by PowerShell itself
Agree
>> Andrew Nurse‘s PS-Get – Almost a direct port of Nuget to PowerShell but
It’s not a port. It’s actually NuGet.
>> Mike Chaliy‘s PSGet – Probably the closest to my ideal
:)
>> but currrently requires a GitHub pull-request and to list a module that the module author must also upload somewhere else.
I am working on this. Next week or two it will not require pull-request. At the same time it will require to host module somewhere on trusted source. This is kind of author authority that I am sure is very good to have. Also if module has it’s own repository this is good place for docs, updates and so on.
>> I think adding my own solution to this list and any others I am not aware of would only add to the module management problem.
Welcome :). But note that this means you will need to do same dirty job as every other solution already did. As far I know all solutions you outlined are open source, so may be it’s better to join.
P.S. Another solution you might ovelooked is http://chocolatey.org/ also provides facility to install powershell modules.
Microsoft btw already have central repository http://gallery.technet.microsoft.com/scriptcenter
Waiting for your solution to this Problem!
I have found another one:
Tome Tanasovski wrote a how to, to centralized PowerShell Modules in a Corporate PowerShell ModuleRepository.
http://powertoe.wordpress.com/2010/08/10/corporate-powershell-module-repository-part-1-design-and-infrastructure/
Jeffery Hicks wrote the famous whitepaper PowerShell in the Enterprise
http://jdhitsolutions.com/blog/2010/03/powershell-in-the-enterprise/
For me these are a the missing chapter in a PowerShell Book. Strategies and cook recipes how to use PowerShell in the Enterprise. Because deploying and maintaining Snapins or Modules on all Workstations is not the best solution.
For my Corporate Module Repository, I came up with a slight different solution. I ended with this Folder Structure:
\\Server\PoshRepo\PSModuleVersions\PSCX.8.1\Pscx\pscx.psd1
\\Server\ PoshRepo \PSModuleVersions\PSCX\1.0.0.1\Pscx\pscx.psd1
\\Server\ PoshRepo \PSModuleVersions\PSCX\2.0.0.1\Pscx\pscx.psd1
In the fact that $env:PSModulePath does not scale very good. I had the idea, to create a root folder for all Modules, with Junction Points or Symlinks inside, that are Pointing to each Folder of the latest Module version.
And then to add only this root folder to the $env:PSModulePath. This doesn’t work, because the junctions / symlinks are not as transparent as expected.
So I wrote a Script-cmdlet which simulates the behavior of Get-Module–ListAvailable. It crawls the folder structure and returns a PSModuleInfo Object for each ModuleManifest file .psd1 (latest Version only), by take use of the Test-ModuleManifest cmdlet.
Example to list All Modules from Repository (latest Version of each Module):
. \\Server\PoshRepo\PAMKModuleAvailable.ps1Load only PSCX Module from Repository
Example to load PSCX Module from Repository:
. \\Server\PoshRepo\PAMKModuleAvailable.ps1 -Name PSCX| Import-Module
Even for CASPOL.EXE, Tome wrote a Batch. I have done this as a set of PowerShellScript-Functions to trust or untrust a Share easily.
See: http://gallery.technet.microsoft.com/PowerShell-Caspol-f2c61fb1
You can download the full solution from my German Blog incl. the PSCX Module and the Quest QAD Snapin converted to a Module (35MB)
http://www.admin-source.de/Downloads/Modul_Repository.zip
If you add one rule, then many points are solved easily:
• All Modules must have a Module-Manifest (.psd1) and must lay into a Folder with the same Name as the .psd1 File.
So you are able to turn every .psd1 file into a PSModuleInfo Object, with the help of the Test-ModuleManifest cmdlet. With this PSModuleInfo Object you can see Version Information (Load by Version) and so on…
And you can ZIP and unzip the Folder to transport and install the Module.
• The module manager itself doesn’t have to be a binary File (module) and must be a Script Module. So you have to not hassle around with .NET Code Access Security (CAS) or other additionally security rules.
• Users who were using PowerShell as a shell expected to have the most recent version of a module loaded when using Import-Module
—————————————–
>> PowerShell’s Execution Policy depends on scripts that were downloaded from the Internet to be marked as such, a module manager should honor this by marking installed modules with the appropriate Zone Identifier.
– Full ACK with Mike Chaliy!
See: http://andyarismendi.blogspot.com/2012/02/unblocking-files-with-powershell.html
>> PowerShell allows multiple versions of a module to exist side-by-side and allows scripts to reference a specific version, a module manager must support this behavior.
– can be solved by adding .psd1 file (see above)!
– the –Version Argument from Import-Module are nearly needless see:
http://powertoe.wordpress.com/2010/08/10/corporate-powershell-module-repository-part-1-design-and-infrastructure/
>> A script should be able to use the module manager to install other modules even when the module manager itself hasn’t been installed yet (e.g. using a secure bootstrap)
– Modules needed to be available GLOBALLY by all workstations and servers! Perhaps from the Cloud (Nuget.org, github, any URL pattern)
>> The format for transporting a module over the wire is an implementation concern of the module manager and not a concern of the module author.
– I suggest to support ZIP and the NuGet format (which is even Zip)
>> Publishing a new or updated module for others to consume should be a PowerShell one-line command that can consume any locally installed module.
– just Zip and upload…
>> There should be a standard public repository for published modules but private repositories should also be supported for publishing modules intended for personal or enterprise use only.
– What about commercial Modules ?
>> To support the common PowerShell Execution Policy setting of RemoteSigned, published modules should be signed with a valid Code Signing Certificate issued by a recognized authority. As cheap as US$75.
– this will prevent many Free Module developers from using this module manager, they spend their work not money ;-)
————————————————
See also:
http://scottmuc.com/blog/development/the-madness-must-stop-powershell-package-management/
http://powertoe.wordpress.com/2010/08/10/corporate-powershell-module-repository-part-1-design-and-infrastructure/
@Mike the Microsoft Script Center repository enforces a particular licence (the MS-LPL) on all scripts and modules hosted there which would not be suitable for many module authors.