Movie Encoder
Windows 10 & 11 Buy license · $89.95
GPU encoding

NVENC vs x264: GPU vs CPU H.264 Encoding Compared on Quality, Speed and Settings

NVENC is far faster, x264 gives better quality per bit at slow presets. Compare quality, speed and features, plus the best settings for each.

Updated 5 minute read The Movie Encoder team

NVENC is NVIDIA's dedicated hardware video encoder, built into GeForce, RTX and professional GPUs. x264 is the leading open-source software H.264 encoder and runs on your CPU. NVENC is much faster and barely loads the rest of the system. x264 at slower presets usually gives better quality for the same file size. For streaming, screen recording and quick exports, NVENC is often the better choice. For the smallest high-quality files, x264 (or x265 and AV1 in software) still wins.

The gap is smaller than it used to be, and it depends on the GPU generation, the settings and the content. This guide compares the two on quality, speed, features and practical use, and explains how to get the best from each.

How they differ

x264 is a software encoder. It can use every CPU core and spend as much time as you let it searching for efficient ways to compress each frame. Its presets, from ultrafast to placebo, trade speed for compression efficiency, and its psychovisual tuning has been refined over many years.

NVENC is a fixed-function block on the GPU, separate from the CUDA and graphics cores. Because the encoding happens in dedicated silicon, it is fast and power-efficient, and it leaves the CPU and most of the GPU free. That's why it is so popular for game streaming. The trade-off is that its algorithms are fixed in hardware and limited by real-time design goals, so it can't search as exhaustively as a slow software preset.

Quality per bit

At the same bitrate, quality comparisons broadly show this pattern:

  • Older NVENC (Kepler, Maxwell and Pascal-era GPUs such as the GTX 900 and 10 series) was clearly behind x264's medium preset, especially at low bitrates.
  • Turing-generation and later NVENC (RTX 20 series onwards, plus most GTX 16 cards) brought a significant quality improvement. In many tests it lands roughly in the region of x264's fast to medium presets for H.264, which is good enough that many streamers switched to it.
  • x264 slow and slower still generally beat NVENC H.264 at the same bitrate, with the difference most visible at low bitrates and in complex, grainy or high-motion content.

Results vary with content, bitrate, NVENC settings (preset, lookahead, adaptive quantisation, B-frames) and how the comparison is scored. At generous bitrates both can look excellent and the difference may not be visible. If you want to test your own footage, measure with VMAF. Our guide to VMAF, PSNR and SSIM explains how.

The GTX 1650 is a special case. The original version uses an older NVENC generation, while some later variants use the Turing encoder. Check your exact model if quality matters.

Speed and system load

NVENC is typically many times faster than x264 at slow presets. 1080p encodes often run at several hundred frames per second on modern cards, and 4K can run well above real time. Just as important, it leaves the CPU almost free. You can keep working, gaming or running other encodes in parallel.

x264's speed depends on your CPU and preset. A modern multi-core CPU can encode 1080p at medium faster than real time, but at slow or veryslow long videos can take a long time, and the CPU is at full load throughout.

Side-by-side comparison

NVENC (H.264)x264
Runs onDedicated block on NVIDIA GPUsCPU, any brand
SpeedVery fastFrom very fast to very slow, depending on preset
CPU loadLowHigh
Quality per bitGood on Turing and laterBetter at slow presets
Quality controlConstant quality (-cq), VBR, CBR, constant QPCRF, two-pass, ABR, CBR, constant QP
Two-passInternal multipass option within one encodeFull two-pass with stats file
Other codecs on the same hardwareHEVC (and AV1 on RTX 40 series and later)H.264 only (x265 is a separate encoder)
Best forStreaming, recording, fast exports, big batchesSmallest files, archives, final masters

Getting the most from NVENC

NVENC's defaults are tuned for speed. For file encodes where quality matters, you can do better:

ffmpeg -i input.mp4 -c:v h264_nvenc -preset p7 -tune hq -rc vbr -cq 21 -b:v 0 -rc-lookahead 32 -spatial-aq 1 -c:a copy output.mp4

  • -preset p7 is the slowest, highest-quality preset (p1 is fastest). Even p7 is fast compared with software.
  • -rc vbr -cq 21 -b:v 0 gives constant-quality behaviour, NVENC's closest equivalent to CRF. The CQ number isn't on the same scale as x264's CRF, so calibrate it separately.
  • -rc-lookahead and -spatial-aq improve how bits are distributed, especially in complex scenes. -temporal-aq can also help on suitable content.

Consider HEVC NVENC (hevc_nvenc) as well. On Turing and later, it typically gives clearly better compression than H.264 NVENC at similar speed. On RTX 40 series and later, AV1 NVENC (av1_nvenc) is more efficient still, for players that support AV1. Our guide to GPU video encoding on NVIDIA, Intel and AMD compares each vendor's support.

Getting the most from x264

For x264, the main controls are the preset and CRF. -preset slow is a common sweet spot, and slower presets bring diminishing returns. Use -crf between roughly 18 and 23 for most work. See what CRF is. -tune film, animation or grain can help with specific content.

ffmpeg -i input.mp4 -c:v libx264 -preset slow -crf 20 -c:a copy output.mp4

Practical limits worth knowing

  • Concurrent sessions. NVIDIA's documentation limits consumer GeForce cards to a set number of simultaneous NVENC sessions per system. The limit has been raised several times over the years and is currently 8. Professional cards are not limited in the same way.
  • Chroma formats. NVENC encodes 4:2:0 and 4:4:4 (generation-dependent). 4:2:2 encoding arrived only with the RTX 50 series (Blackwell). x264 supports 4:2:0, 4:2:2 and 4:4:4.
  • Drivers. NVENC features depend on the driver version as well as the hardware, and FFmpeg builds need a recent enough driver.

Which should you use?

  • Live streaming or recording while gaming: NVENC. Freeing up the CPU matters more than the last few per cent of efficiency.
  • Uploading to YouTube or other platforms: either works well at a generous bitrate, because the platform re-encodes anyway. NVENC saves a lot of time. See the best export settings for YouTube.
  • Large batch conversions: NVENC (ideally HEVC or AV1) if speed matters; software if storage efficiency matters more.
  • Archiving, or hitting a tight size limit: x264 slow (or x265 and AV1 in software) for the best quality per byte.

Doing it with Movie Encoder

Movie Encoder supports NVIDIA NVENC alongside x264, x265 and SVT-AV1. It detects what your card and drivers can really encode, and lets you choose CPU or GPU automatically or manually, with a configurable fallback if a hardware encode isn't possible. The built-in encoder benchmark helps you compare speeds on your own PC.

To compare quality as well, the Quality Lab runs real test encodes of a sample and measures size, time, PSNR, SSIM and VMAF (where the FFmpeg build includes libvmaf), with a side-by-side A/B view. That way you can decide whether NVENC is good enough for your footage before you commit a large batch.

Frequently asked questions

Is NVENC as good as x264?

On Turing-generation and later GPUs, NVENC H.264 often lands roughly around x264's fast to medium presets. x264's slower presets still usually give better quality at the same bitrate.

Does NVENC use the GPU's gaming performance?

NVENC is a separate dedicated block, so the impact on gaming is usually small, although some GPU memory and bandwidth are still used.

Should I use NVENC H.264 or HEVC?

If your playback devices support HEVC, hevc_nvenc typically compresses noticeably better at similar speed. On RTX 40 series and newer, AV1 NVENC is more efficient again.

Get three guides like this every month, plus software update notices.Subscribe →