GitHub - zeroecco/nvidia-converge: Clean up Nvidia drivers for Linux

3 min read Original article ↗

CI

nvidia-converge is a node-level NVIDIA driver stack reconciler. It reads a desired state, audits the host, explains breakage, previews package/kernel/module changes, optionally converges the machine, validates the GPU stack, pins compatibility-sensitive packages, records rollback metadata, and emits a machine-readable compliance report.

Default desired state:

desired:
  role: compute
  driver: 580-open
  cuda_compat: 13.0
  secure_boot: signed
  container_runtime: docker
  fabric_manager: true
  mig: disabled
  kernel_policy: pin-compatible

Usage

Commands print a compact human summary by default. Use --out report.json to write the full machine-readable report, or --json to print that report to stdout.

From a source checkout, run the CLI as a module:

python3 -m nvidia_converge --version
python3 -m nvidia_converge validate --desired config.yaml --out validation.json
python3 -m nvidia_converge doctor
python3 -m nvidia_converge plan --out report.json
sudo python3 -m nvidia_converge install --apply --out report.json
sudo python3 -m nvidia_converge verify --apply --out verify.json
sudo python3 -m nvidia_converge lock --apply --out lock.json
python3 -m nvidia_converge support
python3 -m nvidia_converge schema report
sudo python3 -m nvidia_converge rollback --snapshot /var/lib/nvidia-converge/snapshots/latest.json --apply

Or use the checkout-local launcher:

./nvidia-converge doctor
./nvidia-converge validate --desired config.yaml --out validation.json
./nvidia-converge plan --out report.json
sudo ./nvidia-converge install --apply --out report.json
sudo ./nvidia-converge verify --apply --out verify.json

On Ubuntu and other PEP 668 distributions, do not install into the system Python with python3 -m pip install -e .. Use a virtual environment:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -e .
nvidia-converge doctor

After venv installation, use the packaged command while the venv is active:

nvidia-converge doctor
nvidia-converge validate --desired config.yaml --out validation.json
nvidia-converge plan --out report.json
sudo nvidia-converge install --apply --out report.json
sudo nvidia-converge verify --apply --out verify.json
sudo nvidia-converge lock --apply --out lock.json
sudo nvidia-converge snapshot --apply --out snapshot.json
sudo nvidia-converge rollback --snapshot /var/lib/nvidia-converge/snapshots/latest.json --apply

Commands that execute host-mutating actions, checks, or rollback snapshot writes (install, verify, lock, snapshot, rollback) are dry-run unless --apply is supplied. Without --apply, verify still performs non-mutating checks but skips module-load and container GPU execution checks.

Support Matrix

The supported package managers, runtime assumptions, and known limits are available through:

nvidia-converge support
nvidia-converge support --json

The host-mutation integration test contract is documented in docs/integration-testing.md. Release promotion evidence should be recorded as integrations/results.<tag>.json and validated against the integration-results schema.

Development

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -e ".[test]"
python3 -m compileall -q nvidia_converge tests
python3 tests/run_tests.py
python3 -m ruff check .
python3 -m mypy nvidia_converge
python3 -m pytest -q
python3 -m build

Releases

Tagged releases build and smoke-test wheel/sdist artifacts in GitHub Actions, publish SHA256 checksums, and create GitHub artifact attestations:

git tag v0.1.0
git push origin v0.1.0

Report

Every command can write a JSON report with audit findings, diagnostics, proposed or applied actions, verification results, rollback snapshot metadata, and an SBOM-style package/module inventory.

JSON Schemas are available in schemas/ and through the CLI:

nvidia-converge schema desired
nvidia-converge schema integration-results
nvidia-converge schema report
nvidia-converge schema validation