NVENC vs. X264 — Does CPU or RTX GPU encoding work best for Twitch?

If you want to livestream to Twitch, Mixer, or YouTube Live, you’ve had two options when it comes to video encoding. You can set your CPU to do software encoding. You could alternatively select your Nvidia GPU to handle that task. Each of these have their benefits, but your best bet was to use your CPU. But with Nvidia’s new RTX video cards, that may have changed.
CPU versus GPU encoding was traditionally about quality versus performance. Livestreaming requires you to compress a video broadcast into a small amount of bandwidth. Twitch has a maximum upload speed of 6 Mbps, which isn’t a lot when you need to encode 60 frames of 1080p video every second. But X264 is efficient at smushing visual details into that tiny space. The problem is that if you’re running a game on the same system, CPU encoding is going to hurt performance.
If you have an older, slower CPU, however, you could use NVENC on the GPU instead. This option rarely hurts performance. But the problem in the past was that NVENC’s quality wasn’t comparable to X264 at 6 Mbps or lower. It wasn’t unusable, for sure, but NVENC used to eat a lot of details.
But that’s old NVENC. On the RTX cards, like the RTX 2080 Ti, Nvidia has improved its encoder. And it is now potentially better than X264 in a lot of ways.

RTX NVENC vs. X264 — which is better now?

So how do these two encoding standards stack up against one another today? Overall, they’re pretty even. X264 still does some things better, but I think NVENC may have the edge when it comes to what you want for livestreaming video.
For this text, I set NVENC and X264 to record at a bitrate of 6 Mbps. Both are also use the “High” profile and a keyframe interval of 2. Everything else is default for OBS Studio. I also included some HVEC H.265 4K footage recorded at 160Mbps.
To see the best possible comparison, you should watch the video at the top in 4K. But I’ll include some screenshots below that show both encoders in action.

X264 does better with bright flashes

NVENC still seems to struggle with shifting from black to white. You can see in the image below that the orange-yellow light on the bottom right of the image has some blocking for the GPU encoding. The CPU option, meanwhile, doesn’t look too different from the 160 Mbps H.265 4K encoder in the middle. At least when it comes to the smoothness of the light source.
 
You can also see that the white spotlight from the ceiling has some blocking around it for NVENC. Again, the CPU maintains an image that is more in-line with the high-bitrate footage.

NVENC is better at handling rapid changes

But X264 falls behind when it comes to fast-moving gameplay. I picked Rocket League because it suffers from a lot of compression due to its intense pace. And NVENC is definitely better at dealing with a lot of on-screen changes at once.
Take a look at this comparison. This screenshot is from a quick turn where the camera is panning with a lot of speed. The GPU footage looks sharper overall. The CPU footage, however, has a soft and fuzzy look to it. That’s close to the camera and at a distance. Look at the teal coloring of the car. It’s smooth and consistent with NVENC. On the CPU, you can notice some ugly blotches. Likewise, the basketball court looks blurrier on X264, and so does the dark crowd.

NVENC is just a speedier coded. That’s especially noticeable when you get a lot of thin strips of detail moving quickly left or right or up and down. In Rocket League, the playfield has a hexagonal containment grid that you can drive on. And X264 just doesn’t like that one bit.
Here’s a good example of what I mean:

Now, NVENC can also struggle with these kinds of rapid changes. It has some frames that look like the one on the right. Check that out by pausing around this point in the video. But the point is that it happens far more frequently on X264.

NVENC makes text more legible

Finally, and most importantly for livestreamers, NVENC is way better with text. At a distance, X264 smudges words together to the point that you cannot read them. NVENC, meanwhile, maintains their legibility.

This is crucial for Twitch or Mixer because you don’t want your viewers struggling to read words in the game environment. That’s really frustrating. It’s hard to maintain your bearings as a viewer if you cannot tell who is shooting your favorite broadcaster. Or if you can’t read signs in a town.
Now, X264’s text is really only a problem when it’s tiny, far away, or moving quickly, but that’s the case in a lot of shooters. Or in something like Rocket League.

So which one should you choose?

If all things are equal, I would now choose NVENC. Having clear text and fewer visual artifacts during fast motion is a big improvement. X264 isn’t awful, but at 6 Mbps, NVENC has the advantages that are more important for a livestream.
But all things aren’t equal. This new NVENC is only on RTX cards at the moment, and those are expensive. The RTX 2070 is coming soon for $600, but that’s twice as much as an AMD Ryzen 7 2700X. And a $300 CPU like the 2700X has 8 core with hyperthreading, which means it’s great for playing games and livestreaming at the same time. You likely won’t get much of a performance hit in most games while broadcasting on a chip like that.
Of course, exceptions to that will grow as games learn to take more advantage of hyperthreading. For example, the recent Assassin’s Creed games will use every CPU core you give it through at least 8 cores. But if we’re doing a cost-benefit analysis, I would never suggest you get an RTX card just for NVENC. Instead, save your money and get a 6-core-plus recent-generation CPU.
But all of that said, if you are using an RTX card already, go with NVENC. It is the better option.
Source: VentureBeat
To Read Our Daily News Updates, Please Visit Inventiva Or Subscribe Our Newsletter & Push.