Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions apps/rush/UPGRADING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,50 @@
# Upgrade notes for @microsoft/rush

### PNPM 11.6.0 and newer: migrate project `.npmrc` credentials

PNPM 11.5.3 stopped expanding environment variables in registry credentials and request destinations
from a project or workspace `.npmrc`. The
`provideNpmrcCredentialsViaEnvironment` Rush experiment provides a compatibility workaround for PNPM
11.5.3 through versions earlier than 11.6.0, but PNPM 11.6.0 introduced a safer native replacement.

Before upgrading to PNPM 11.6.0 or newer, replace committed credential settings such as:

```ini
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
```

with one of PNPM's trusted configuration mechanisms. The direct, file-free replacement is an
environment variable whose name includes the registry:

```text
pnpm_config_//registry.npmjs.org/:_authToken=<token>
```

The `/`, `:`, and `.` characters are part of the environment variable name. Operating-system child
process environments, including Windows environments, can carry these names, but many shells reject
them as assignment identifiers. On POSIX systems, use `env` rather than `export`:

```sh
env "pnpm_config_//registry.npmjs.org/:_authToken=$NPM_TOKEN" rush install
```

CI systems may also provide an environment configuration interface that accepts arbitrary names. Rush
preserves the exact casing of URL-scoped `pnpm_config_//...` names on Windows because registry paths can
be case-sensitive.

If the shell or CI system restricts environment variable names, use one of PNPM's other supported
approaches:

- Write the credential to the user-level PNPM auth configuration before invoking Rush, for example
`pnpm config set "//registry.npmjs.org/:_authToken" "$NPM_TOKEN"`.
- Put the `${NPM_TOKEN}` setting in the user's `~/.npmrc` or a file selected by `npmrcAuthFile`.
- In CI that exclusively builds trusted repositories, set `PNPM_CONFIG_NPMRC_AUTH_FILE=.npmrc` to
explicitly treat the generated project `.npmrc` as trusted. This disables PNPM's repository
protection for that checkout.

Dynamic registry and proxy URLs must also move out of the project `.npmrc` and into trusted user,
global, CLI, or environment configuration.

### Rush 5.135.0

This release of Rush deprecates the `rush-project.json`'s `operationSettings.sharding.shardOperationSettings`
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"changes": [
{
"packageName": "@microsoft/rush",
"comment": "Add a new `provideNpmrcCredentialsViaEnvironment` experiment for PNPM 10.34.2 through 10.x and PNPM 11.5.3 through versions earlier than 11.6.0. For these versions, Rush expands `${VAR}` tokens from the generated `.npmrc`, passing credentials using `npm_config_*` environment variables instead of writing them to disk. PNPM 11.6.0 and newer should instead receive URL-scoped `pnpm_config_//...` credentials directly from CI.",
"type": "minor"
}
]
}
1 change: 1 addition & 0 deletions common/reviews/api/rush-lib.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,7 @@ export interface IExperimentsJson {
omitAppleDoubleFilesFromBuildCache?: boolean;
omitImportersFromPreventManualShrinkwrapChanges?: boolean;
printEventHooksOutputToConsole?: boolean;
provideNpmrcCredentialsViaEnvironment?: boolean;
rushAlerts?: boolean;
strictChangefileValidation?: boolean;
trimRushEnvironmentVariablesForOperations?: boolean;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,5 +150,20 @@
* help prevent operation scripts from accidentally depending on Rush's own internal environment
* variables.
*/
/*[LINE "HYPOTHETICAL"]*/ "trimRushEnvironmentVariablesForOperations": true
/*[LINE "HYPOTHETICAL"]*/ "trimRushEnvironmentVariablesForOperations": true,

/**
* PNPM 10.34.2 through 10.x and PNPM 11.5.3 through versions earlier than 11.6.0 ignore "${VAR}"
* tokens that appear in credentials and registry URLs in a project or workspace .npmrc file. If
* true for those versions, Rush expands the tokens itself: credentials are passed using
* "npm_config_*" environment variables instead of being written to the generated .npmrc file, and
* non-secret settings such as registry URLs are written with their values already expanded. PNPM
* 11.6.0 and newer support URL-scoped "pnpm_config_//..." environment variables, which should
* instead be supplied directly by CI so the trusted environment binds each credential to its
* registry. For example, supply an environment variable named
* "pnpm_config_//registry.npmjs.org/:_authToken" whose value is the registry token. Dynamic
* registry and proxy settings must likewise be supplied through trusted user, global, CLI, or
* environment configuration rather than a project .npmrc.
*/
/*[LINE "HYPOTHETICAL"]*/ "provideNpmrcCredentialsViaEnvironment": true
}
15 changes: 15 additions & 0 deletions libraries/rush-lib/src/api/ExperimentsConfiguration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,21 @@ export interface IExperimentsJson {
* variables.
*/
trimRushEnvironmentVariablesForOperations?: boolean;

/**
* If true, when using PNPM, Rush resolves the `${VAR}` tokens that appear in credentials and
* registry URLs in the `.npmrc` file, instead of relying on PNPM to expand them. Credentials are
* passed to PNPM using `npm_config_*` environment variables and are not written to the generated
* `.npmrc` file.
*
* @remarks
* This compatibility workaround applies to PNPM 10.34.2 through 10.x and PNPM 11.5.3 through
* versions earlier than 11.6.0. PNPM 11.6.0 and newer support URL-scoped `pnpm_config_//...`
* environment variables, which should be supplied directly by CI so the trusted environment binds
* each credential to its registry. Dynamic registry and proxy settings must likewise be supplied
* through trusted user, global, CLI, or environment configuration rather than a project `.npmrc`.
*/
provideNpmrcCredentialsViaEnvironment?: boolean;
}

const _EXPERIMENTS_JSON_SCHEMA: JsonSchema = JsonSchema.fromLoadedObject(schemaJson);
Expand Down
14 changes: 14 additions & 0 deletions libraries/rush-lib/src/cli/RushPnpmCommandLineParser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ import type { IBuiltInPluginConfiguration } from '../pluginFramework/PluginLoade
import type { BaseInstallManager } from '../logic/base/BaseInstallManager';
import type { IInstallManagerOptions } from '../logic/base/BaseInstallManagerTypes';
import { Utilities } from '../utilities/Utilities';
import { getNpmrcEnvironmentVariables } from '../utilities/npmrcUtilities';
import { InstallHelpers } from '../logic/installManager/InstallHelpers';
import type { Subspace } from '../api/Subspace';
import type { PnpmOptionsConfiguration } from '../logic/pnpm/PnpmOptionsConfiguration';
import { PnpmWorkspaceFile } from '../logic/pnpm/PnpmWorkspaceFile';
Expand Down Expand Up @@ -476,6 +478,18 @@ export class RushPnpmCommandLineParser {
}
}

// Provide any credentials that "rush install" moved out of the generated .npmrc file.
// See the "provideNpmrcCredentialsViaEnvironment" experiment.
if (InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(rushConfiguration)) {
const npmrcEnvironmentVariables: Record<string, string> | undefined = getNpmrcEnvironmentVariables({
npmrcFolder: workspaceFolder,
supportEnvVarFallbackSyntax: rushConfiguration.isPnpm
});
for (const [envKey, envValue] of Object.entries(npmrcEnvironmentVariables ?? {})) {
pnpmEnvironmentMap.set(envKey, envValue);
}
}

let onStdoutStreamChunk: ((chunk: string) => string | void) | undefined;
switch (this._commandName) {
case 'patch': {
Expand Down
30 changes: 28 additions & 2 deletions libraries/rush-lib/src/logic/Autoinstaller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { Colorize } from '@rushstack/terminal';

import { AsyncRecycler } from '../utilities/AsyncRecycler';
import { Utilities } from '../utilities/Utilities';
import { getNpmrcEnvironmentVariables } from '../utilities/npmrcUtilities';
import type { RushConfiguration } from '../api/RushConfiguration';
import { PackageJsonEditor } from '../api/PackageJsonEditor';
import { InstallHelpers } from './installManager/InstallHelpers';
Expand Down Expand Up @@ -143,7 +144,10 @@ export class Autoinstaller {
Utilities.syncNpmrc({
sourceNpmrcFolder: this._rushConfiguration.commonRushConfigFolder,
targetNpmrcFolder: autoinstallerFullPath,
supportEnvVarFallbackSyntax: this._rushConfiguration.isPnpm
supportEnvVarFallbackSyntax: this._rushConfiguration.isPnpm,
moveSensitiveSettingsToEnvironment: InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(
this._rushConfiguration
)
});

this._logIfConsoleOutputIsNotRestricted(
Expand All @@ -154,6 +158,7 @@ export class Autoinstaller {
command: this._rushConfiguration.packageManagerToolFilename,
args: ['install', '--frozen-lockfile'],
workingDirectory: autoinstallerFullPath,
environment: this._getPackageManagerEnvironment(autoinstallerFullPath),
keepEnvironment: true
});

Expand Down Expand Up @@ -229,13 +234,17 @@ export class Autoinstaller {
Utilities.syncNpmrc({
sourceNpmrcFolder: this._rushConfiguration.commonRushConfigFolder,
targetNpmrcFolder: this.folderFullPath,
supportEnvVarFallbackSyntax: this._rushConfiguration.isPnpm
supportEnvVarFallbackSyntax: this._rushConfiguration.isPnpm,
moveSensitiveSettingsToEnvironment: InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(
this._rushConfiguration
)
});

await Utilities.executeCommandAsync({
command: this._rushConfiguration.packageManagerToolFilename,
args: ['install'],
workingDirectory: this.folderFullPath,
environment: this._getPackageManagerEnvironment(this.folderFullPath),
keepEnvironment: true
});

Expand Down Expand Up @@ -278,4 +287,21 @@ export class Autoinstaller {
console.log(message ?? '');
}
}

/**
* Returns the environment to invoke the package manager with, or `undefined` to inherit this
* process's environment. See the `provideNpmrcCredentialsViaEnvironment` experiment.
*/
private _getPackageManagerEnvironment(npmrcFolder: string): NodeJS.ProcessEnv | undefined {
if (!InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(this._rushConfiguration)) {
return undefined;
}

const npmrcEnvironmentVariables: Record<string, string> | undefined = getNpmrcEnvironmentVariables({
npmrcFolder,
supportEnvVarFallbackSyntax: this._rushConfiguration.isPnpm
});

return npmrcEnvironmentVariables && { ...process.env, ...npmrcEnvironmentVariables };
}
}
28 changes: 27 additions & 1 deletion libraries/rush-lib/src/logic/base/BaseInstallManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -554,15 +554,41 @@ export abstract class BaseInstallManager {
// Also copy down the committed .npmrc file, if there is one
// "common\config\rush\.npmrc" --> "common\temp\.npmrc"
// Also ensure that we remove any old one that may be hanging around
const {
isPnpm,
packageManagerToolVersion,
experimentsConfiguration: {
configuration: { provideNpmrcCredentialsViaEnvironment }
}
} = this.rushConfiguration;
const shouldWarnAboutIgnoredEnvironmentVariables: boolean | undefined =
isPnpm && provideNpmrcCredentialsViaEnvironment && semver.gte(packageManagerToolVersion, '11.6.0');
const environmentVariableSettingNames: Set<string> | undefined =
shouldWarnAboutIgnoredEnvironmentVariables ? new Set() : undefined;
const npmrcText: string | undefined = Utilities.syncNpmrc({
sourceNpmrcFolder: subspace.getSubspaceConfigFolderPath(),
targetNpmrcFolder: subspace.getSubspaceTempFolderPath(),
linesToPrepend: extraNpmrcLines,
createIfMissing: this.rushConfiguration.subspacesFeatureEnabled,
supportEnvVarFallbackSyntax: this.rushConfiguration.isPnpm
supportEnvVarFallbackSyntax: this.rushConfiguration.isPnpm,
moveSensitiveSettingsToEnvironment: InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(
Comment thread
iclanton marked this conversation as resolved.
this.rushConfiguration
),
environmentVariableSettingNames
});
this._syncNpmrcAlreadyCalled = true;

if (environmentVariableSettingNames?.size) {
terminal.writeWarningLine(
`The "provideNpmrcCredentialsViaEnvironment" experiment does not translate project ` +
`.npmrc settings for PNPM ${packageManagerToolVersion}. PNPM will ignore environment ` +
`variables in these settings: ${Array.from(environmentVariableSettingNames).join(', ')}. ` +
`Supply credentials using URL-scoped "pnpm_config_//..." environment variables, or move ` +
`the settings to trusted user, global, CLI, or environment configuration. See the PNPM ` +
`11.6.0 section in the Rush upgrade notes.`
);
}

const npmrcHash: string | undefined = npmrcText
? crypto.createHash('sha1').update(npmrcText).digest('hex')
: undefined;
Expand Down
55 changes: 53 additions & 2 deletions libraries/rush-lib/src/logic/installManager/InstallHelpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import type { IConfigurationEnvironment } from '../base/BasePackageManagerOption
import type { PnpmOptionsConfiguration } from '../pnpm/PnpmOptionsConfiguration';
import { PnpmWorkspaceFile } from '../pnpm/PnpmWorkspaceFile';
import { merge } from '../../utilities/objectUtilities';
import { getNpmrcEnvironmentVariables } from '../../utilities/npmrcUtilities';
import type { Subspace } from '../../api/Subspace';
import { RushConstants } from '../RushConstants';

Expand Down Expand Up @@ -377,10 +378,41 @@ export class InstallHelpers {
};
}

/**
* Returns true if Rush (rather than PNPM) should expand the `${VAR}` tokens that appear in
* credentials and registry URLs in the `.npmrc` file. See the
* `provideNpmrcCredentialsViaEnvironment` experiment.
*/
public static shouldProvideNpmrcCredentialsViaEnvironment(rushConfiguration: RushConfiguration): boolean {
const {
isPnpm,
packageManagerToolVersion,
experimentsConfiguration: {
configuration: { provideNpmrcCredentialsViaEnvironment = false }
}
} = rushConfiguration;
if (!isPnpm || !provideNpmrcCredentialsViaEnvironment) {
return false;
}

// PNPM 11.6.0 added URL-scoped `pnpm_config_//...` credentials, which let CI bind a token to
// a registry without deriving that trusted binding from repository-controlled configuration.
// Keep this compatibility workaround only for patched versions that lack that native path.
return (
(semver.gte(packageManagerToolVersion, '10.34.2') && semver.lt(packageManagerToolVersion, '11.0.0')) ||
(semver.gte(packageManagerToolVersion, '11.5.3') && semver.lt(packageManagerToolVersion, '11.6.0'))
);
}

/**
* Returns the environment that the package manager should be invoked with, including any
* credentials that were moved out of the generated `.npmrc` file in `npmrcFolder`.
*/
public static getPackageManagerEnvironment(
rushConfiguration: RushConfiguration,
options: {
debug?: boolean;
npmrcFolder?: string;
Comment thread
iclanton marked this conversation as resolved.
} = {}
): NodeJS.ProcessEnv {
let configurationEnvironment: IConfigurationEnvironment | undefined = undefined;
Expand All @@ -393,7 +425,26 @@ export class InstallHelpers {
configurationEnvironment = rushConfiguration.yarnOptions?.environmentVariables;
}

return _mergeEnvironmentVariables(process.env, configurationEnvironment, options);
const packageManagerEnvironment: NodeJS.ProcessEnv = _mergeEnvironmentVariables(
process.env,
configurationEnvironment,
options
);

const { npmrcFolder } = options;
const shouldProvideCredentials: boolean =
InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(rushConfiguration);
if (npmrcFolder !== undefined && shouldProvideCredentials) {
Object.assign(
packageManagerEnvironment,
getNpmrcEnvironmentVariables({
npmrcFolder,
supportEnvVarFallbackSyntax: rushConfiguration.isPnpm
})
);
}

return packageManagerEnvironment;
}

/**
Expand Down Expand Up @@ -514,7 +565,7 @@ function _mergeEnvironmentVariables(
debug?: boolean;
} = {}
): NodeJS.ProcessEnv {
const packageManagerEnv: NodeJS.ProcessEnv = baseEnv;
const packageManagerEnv: NodeJS.ProcessEnv = { ...baseEnv };

if (environmentVariables) {
// eslint-disable-next-line guard-for-in
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -502,8 +502,10 @@ export class RushInstallManager extends BaseInstallManager {

const packageManagerEnv: NodeJS.ProcessEnv = InstallHelpers.getPackageManagerEnvironment(
this.rushConfiguration,
this.options
{ ...this.options, npmrcFolder: subspace.getSubspaceTempFolderPath() }
);
const keepEnvironment: boolean =
InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(this.rushConfiguration);

const commonNodeModulesFolder: string = path.join(
this.rushConfiguration.commonTempFolder,
Expand Down Expand Up @@ -622,6 +624,7 @@ export class RushInstallManager extends BaseInstallManager {
args: installArgs,
workingDirectory: this.rushConfiguration.commonTempFolder,
environment: packageManagerEnv,
keepEnvironment,
suppressOutput: false
},
this.options.maxInstallAttempts,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -494,8 +494,10 @@ export class WorkspaceInstallManager extends BaseInstallManager {

const packageManagerEnv: NodeJS.ProcessEnv = InstallHelpers.getPackageManagerEnvironment(
this.rushConfiguration,
this.options
{ ...this.options, npmrcFolder: subspace.getSubspaceTempFolderPath() }
);
Comment thread
iclanton marked this conversation as resolved.
const keepEnvironment: boolean =
InstallHelpers.shouldProvideNpmrcCredentialsViaEnvironment(this.rushConfiguration);
if (ConsoleTerminalProvider.supportsColor) {
packageManagerEnv.FORCE_COLOR = '1';
}
Expand Down Expand Up @@ -596,6 +598,7 @@ export class WorkspaceInstallManager extends BaseInstallManager {
args: installArgs,
workingDirectory: subspace.getSubspaceTempFolderPath(),
environment: packageManagerEnv,
keepEnvironment,
suppressOutput: false,
onStdoutStreamChunk: onPnpmStdoutChunk
},
Expand Down
Loading