Windows ML CLI

Windows ML CLI is a command-line tool for preparing AI models for Windows ML. Start with a supported pretrained model, convert it to ONNX if needed, and prepare it for your target hardware. Then measure how quickly it runs and evaluate its accuracy before integrating it into your Windows app.

This guide is for Windows app developers who want to use an existing model rather than train one. Familiarity with PowerShell and installing Python packages is helpful; you don't need to understand PyTorch export internals to complete the CPU quickstart.

The CLI provides a common workflow for CPUs, GPUs, and neural processing units (NPUs), including hardware from AMD, Intel, NVIDIA, and Qualcomm. It uses execution providers (EPs), software components that run model operations on a particular hardware target. You can run individual steps interactively or save a build configuration for use in scripts and CI/CD pipelines.

Windows ML CLI prepares and tests models; your app uses Windows ML to run them. For an introduction to the runtime and app integration, see Get started with Windows ML.

Key concepts

What you can do

Models and tasks

Windows ML CLI supports workflows across vision, language, audio, and multimodal models. Examples include image classification, object detection, segmentation, depth estimation, text classification, embeddings, question answering, speech recognition, and text generation.

Support is specific to the model architecture, task, precision, and target EP. A recognized task doesn't mean every model for that task has a validated export or evaluation path. Use winml inspect to check a model and winml catalog to browse curated starting points. See Supported models for validated configurations and compatibility results.

Some models contain cooperating components rather than a single graph. For example, CLIP uses separate text and vision encoders, and supported decoder language models can use a GenAI bundle. See the CLIP composite-model sample and Qwen3 bundle sample.

Command reference

Run the pipeline with winml build, or use individual commands when you need to inspect or customize a stage.

Prerequisites

• A Windows PC. For NPU workflows, use Windows 11, version 24H2 or later, with a compatible NPU, driver, and execution provider.
• Python 3.11, x64 (AMD64). Native Arm64 Python isn't supported by the CLI's current dependency set.
• The uv package manager.
• An internet connection to install packages and download models, execution providers, or datasets when required. Some Hugging Face resources require authentication or acceptance of their license terms.

You don't need an NPU to get started. The quickstart below uses CPU inference so you can establish a baseline before targeting other hardware.

Install Windows ML CLI

Run these commands in PowerShell from a directory where you want to create a Python environment:

uv python install cpython-3.11-windows-x86_64-none
uv venv --python cpython-3.11-windows-x86_64-none
uv pip install winml-cli

Use uv run to invoke the CLI without activating the environment:

uv run winml --version
uv run winml sys --list-device --list-ep

The sys command lists the compute devices and execution providers available on your machine. If an expected device or EP is missing, check your driver and provider requirements before building for that target. See the sys command reference.

Quickstart: Build and benchmark a model

This example uses ResNet-50, a pretrained model for classifying images. It exports and optimizes the model, then measures how quickly the resulting ONNX model runs on the CPU. It keeps 32-bit floating-point (FP32) values and skips quantization and compilation, giving you an initial performance measurement to compare with later changes.

1. Inspect model support

uv run winml inspect -m microsoft/resnet-50

Review the detected task, model class, and support status before starting the build. To browse other starting points, use uv run winml catalog or see Supported models.

2. Build the model

uv run winml build -m microsoft/resnet-50 -o .\resnet_out --device cpu --ep cpu --precision fp32 --no-quant --no-compile

The CLI generates a build configuration, exports the model to ONNX, and runs the enabled analysis and optimization steps. In this example, --no-quant skips quantization and --no-compile skips EP compilation.

For this single-model ONNX workflow, the output directory contains:

• model.onnx: The final model from the enabled build stages.
• winml_build_config.json: The build configuration, which you can inspect, customize, and reuse.
• Intermediate model files: Outputs from completed stages, such as export and optimization.
• Analysis output: Compatibility findings and optimization recommendations when analysis runs.

Keep any external weight files with the ONNX files that reference them. For details about intermediate files and deployment artifacts, see Output layout.

3. Measure performance

uv run winml perf -m .\resnet_out\model.onnx --device cpu --ep cpu --iterations 50 --monitor -o .\resnet_out\perf_cpu.json

The benchmark runs warm-up iterations before collecting timing samples. The JSON report includes latency statistics and throughput, plus available hardware metrics when monitoring is enabled.

By default, this ONNX benchmark uses generated inputs. It measures model execution, not the complete application pipeline or task accuracy. Use representative input tensors with --input-data when appropriate, and evaluate accuracy separately.

Target a GPU or NPU

Use --device to select a hardware category and --ep to select an execution provider. For example, on a machine with a compatible DirectX 12 GPU, benchmark the floating-point model with DirectML:

uv run winml perf -m .\resnet_out\model.onnx --device gpu --ep dml --iterations 50 -o .\resnet_out\perf_gpu.json

Common provider aliases include:

Provider availability depends on the device, driver, and installed runtime packages. Use winml sys to check your machine and see Execution providers and devices for the current compatibility details.

For commands that support automatic device selection, --device auto prioritizes an available NPU, then GPU, then CPU. This is a selection policy, not a benchmark-based guarantee that the selected device is fastest for your model.

NPU workflows can require a different precision or graph layout from a CPU baseline. Follow the Hugging Face model to NPU tutorial for a target-specific workflow.

Bring an existing ONNX model

You can use Windows ML CLI without exporting from Hugging Face. Start by analyzing a local model for your intended EP. The following example uses Qualcomm QNN as the analysis target:

uv run winml analyze -m .\my_model.onnx --device npu --ep qnn --optim-config .\optim_config.json

The analyzer reports supported, partially supported, unsupported, and unknown operators or patterns. It also writes recommended graph optimization settings to the specified JSON file. Treat unknown results as a need for further investigation, not as confirmation of support.

Apply the recommendations, then analyze the optimized result:

uv run winml optimize -m .\my_model.onnx -c .\optim_config.json --device npu --ep qnn -o .\my_model_optimized.onnx
uv run winml analyze -m .\my_model_optimized.onnx --device npu --ep qnn

You can also pass an existing ONNX file to winml build; the pipeline skips export. For the complete workflow, including quantization and compilation, see Bring your own ONNX model.

Customize model export

Use winml export to export a supported Hugging Face model or a compatible local checkpoint directory independently of the build pipeline:

uv run winml export -m microsoft/resnet-50 -o .\resnet_export.onnx --batch-size 1 --with-report

The exporter can preserve PyTorch module hierarchy as ONNX node metadata, helping you relate graph operators to their originating modules. Here, a module is a component of the original PyTorch model. With --with-report, the exporter also generates export reports for inspection. See Load and export for an introduction to the PyTorch export mechanisms and their options.

For models with specific input requirements, you can configure:

• Input shapes with --shape-config, such as sequence length or image dimensions.
• Input tensor specifications with --input-specs.
• Dynamic dimensions with --dynamic-axes, when the model and target provider support them.
• ONNX export settings with --export-config, including the requested opset.
• Exporter selection with --dynamo to opt into the TorchDynamo exporter rather than the default TorchScript path.

These options aren't interchangeable across every workflow. Provider requirements, static-shape constraints, and exporter operator decompositions can affect whether the result compiles or runs. See Load and export and the export command reference.

Quantize and calibrate a model

Quantization reduces the precision used for weights and activations. Depending on the model and target EP, it can reduce memory requirements or enable more efficient execution, but it can also change accuracy.

The build workflow supports precision selection such as fp32, fp16, int8, int16, and mixed weight/activation specifications, subject to the selected recipe and provider. The standalone winml quantize command provides four types of conversion:

Repeat --precision to chain supported conversions, such as --precision int4 --precision fp16. Build precision defaults and standalone conversion modes aren't interchangeable.

For example, quantize the optimized ResNet-50 model with task-based calibration:

uv run winml quantize -m .\resnet_out\model.onnx -o .\resnet_out\resnet_int8.onnx --precision int8 --task image-classification --model-id microsoft/resnet-50 --samples 32 --method minmax

For a supported task, --task and --model-id select the default calibration dataset and matching model preprocessing. Access to that dataset might require authentication or acceptance of its terms.

Quantization controls include:

• Calibration range estimation with minmax, entropy, or percentile.
• Separate weight and activation types, including mixed precision such as w8a16.
• Per-channel weight quantization and symmetric quantization.
• Calibration sample count.

When task-based calibration isn't available or no task is supplied, the quantizer can use generated calibration inputs. These are useful for exercising the pipeline, but shouldn't be treated as evidence of production accuracy. Check that calibration data represents your workload and evaluate the quantized model against its floating-point baseline.

See the quantize command reference for supported types, option precedence, and calibration behavior.

Compile and package for deployment

Use winml compile when your target EP supports preparing a model ahead of inference. For EPs with ahead-of-time compilation, the result can include an EP context containing the compiled execution plan, reducing work during model initialization.

For example, on a compatible Qualcomm NPU system, compile the quantized model from the preceding section:

uv run winml compile -m .\resnet_out\resnet_int8.onnx --device npu --ep qnn -o .\resnet_out\resnet_qnn.onnx

Compilation consumes the model you supply; it doesn't perform a separate quantization pass. Ensure the graph and precision meet the provider's requirements first.

For QNN, the default layout stores the compiled EP context in an external sidecar file. Use --embed to embed the context in the ONNX file instead. Embedding the context doesn't remove any separate external-weight requirements. Package all files referenced by the model and preserve their relative locations.

Compilation validates the result with a trial inference by default. That check requires compatible target hardware and doesn't replace task-level accuracy evaluation. You can also forward provider-specific settings with --ep-options.

Treat compiled artifacts as target-specific outputs. Their compatibility can depend on the provider, hardware, runtime, and driver versions; don't assume that one compiled artifact covers every deployment device.

See Compile and EPContext, the compile command reference, and Distribute models for Windows ML.

Diagnose performance and resource use

Beyond the quickstart benchmark, winml perf provides controls for investigating model performance:

Report fields and monitoring support vary by runtime, provider, and driver. An unavailable memory counter isn't equivalent to zero memory use. Keep the device, input shapes, runtime, precision, cache state, and monitoring settings consistent when comparing results.

For supported ONNX Runtime GenAI bundles, --runtime ort-genai measures generation-specific metrics, including time to first token and decode tokens per second. These aren't interchangeable with single-inference ONNX latency and throughput. Record the prompt, generation length, and runtime settings when comparing language-model results.

See the perf command reference for report fields and feature limitations.

Evaluate accuracy before deployment

Performance improvements must be considered alongside model quality. Use winml eval to:

• Evaluate a supported task on a dataset, with the appropriate model preprocessing and task metrics.
• Compare an ONNX candidate with a reference model using the same inputs.
• Evaluate a supported Hugging Face checkpoint directly with --runtime pytorch to establish a source-model baseline.

For example, compare the optimized ResNet-50 output with its exported ONNX baseline:

uv run winml eval -m .\resnet_out\model.onnx --mode compare --reference .\resnet_out\export.onnx --device cpu --ep cpu --reference-device cpu --reference-ep cpu -o .\resnet_out\compare.json

Compare mode reports output similarity and uses generated inputs unless you provide input data. It isn't a substitute for task-level accuracy evaluation on representative data. For quantized models, use representative calibration data and measure accuracy against your floating-point baseline.

See the eval command reference for dataset configuration, input requirements, and reference-model options.

Save and automate a build configuration

Use winml config when you want to review or customize the pipeline before running it:

uv run winml config -m microsoft/resnet-50 --device cpu --ep cpu --precision fp32 --no-quant --no-compile -o .\resnet_cpu.json
uv run winml build -c .\resnet_cpu.json -m microsoft/resnet-50 -o .\resnet_ci --no-quant --no-compile

Check the configuration into source control and reuse it in your build pipeline. The configuration stores loader and export settings, not the source checkpoint path, so pass the model identifier with -m when running the build. For repeatable comparisons, also control package versions, model and dataset revisions, calibration inputs, and the target runtime and hardware environment. A saved configuration alone doesn't guarantee identical results across different environments.

For the configuration format and stage controls, see Config and build.

You can also invoke supported model preparation and evaluation workflows from Python. See the Python API reference for configuration types and programmatic entry points.

More workflows

• Build language-model bundles. For supported model families and targets, winml build --export-type optimized creates an ONNX Runtime GenAI bundle instead of a single ONNX model. See the Qwen3 bundle sample. Supported targets are recipe-specific.
• Work with an AI agent. The repository provides skills for model preparation, adding model support, and experimental optimization investigations. Load the appropriate skill into an agent host that supports skills or custom instructions. See Use Windows ML CLI with AI agents.
• Use a graphical interface. The Foundry Toolkit extension for Visual Studio Code provides a Model Conversion workflow that invokes Windows ML CLI. See the UI quickstart for setup and the integration's current release status.

Preview and experimental workflows

Some repository workflows have additional release and dependency constraints:

• CGIR/CGC workflows use preview runtime or execution-provider packages. CGC adapter and pattern inspection also requires a compatible D3D12 runtime and driver for the relevant queries. Don't assume these prerequisites are included with the standard CLI installation.
• GGUF benchmarking through the Windows ML Tasks llama.cpp path is experimental. See the GGUF sample for requirements and report differences.

These workflows have their own status and requirements; CLI availability doesn't imply that every backend or integration is generally available.

Privacy and feedback

Windows ML CLI collects optional usage telemetry with consent. On the first interactive run, the CLI prompts for your choice; pressing Enter accepts and entering n declines. You can change the choice later through telemetry.consent in %USERPROFILE%\.winml\config.json. Telemetry is disabled in non-interactive and CI/CD contexts. See the Windows ML CLI privacy statement for details.

For CLI issues and feature requests, use the Windows ML CLI GitHub repository.

Related content

• Windows ML overview
• Get started with Windows ML
• Install Windows ML CLI
• Windows ML CLI command reference
• Windows ML CLI troubleshooting