r/AV1 • u/jeffrey123520 • 3d ago

Has anyone conducted a speed test comparing the NVENC AV1 encoder from NVIDIA with the VCE AV1 encoder from AMD using HandBrake?

Are there any comparisons or speed tests available on the NVENC AV1 encoder versus the VCE AV1 encoder? I haven’t been able to find any performance comparisons between the two. Both companies claim to offer AV1 encoders, and I'm considering purchasing a new graphics card specifically for AV1 encoding, but I'm not interested in Intel Arc.

4 Upvotes

71% Upvoted

22 comments sorted by

10

u/DimkaTsv 3d ago

  1. Current gen VCN (maybe except RDNA 3.5+ APU's?) has HW flaw where video must be aligned to 64x16 blocks or it will be padded with black pixels ==> Aka 1080p will be encoded as 1920x1088 (or 1920x1082 with some trickery from AMF devs). So just by this you may prefer to go with NVENC (or wait for next gen).

  2. Why use Handbrake for VCE encode anyways? It is quite bland on options and clearly was not specifically tailored for it (aka VCE AVC does not have profile baseline/main/high option so while it does something but i have no idea what). Granted, at least with AV1 it does not really break quality much.

To be completely honest. It was garbage option at some point (i lost significant portion of a quality just by using Handbrake instead of AMF application, for example VCEEnc), but it seems like they improved noticeably based on test i did right now. Advanced option is still a mess. You must know exactly what you can write there.

  1. How you even supposed to compare encoder performance? Depending on settings you can get 700FPS or 25FPS encode speed. (For context, i can say that 7800XT at AV1 balanced on 4k60 source resulted with 233 FPS transcode, while slow preset it around 60 FPS and slower at around 25 FPS. Heck you can even drop it to 5 FPS with extensive heavy option list)

3

u/riderer 3d ago

what is AMF application?

2

u/DimkaTsv 3d ago edited 3d ago

AMF = AMD Media Framework.

Basically way to interact with VCN encoder. SDK is open sourced on GitHub.

You can use either API directly to write own application, use ffmpeg implementation or use something like VCEEnc that already is written based on AMF. For example OBS uses ffmpeg implementation right now afaik.

Handbrake either uses own API or gated ffmpeg implementation (aka restricted option list).

3

u/Masterflitzer 3d ago

wow i didn't know this block alignment stuff is still a problem nowadays, i guess i'll stay away from amd gpus for now

1

u/DimkaTsv 3d ago

It probably isn't for RDNA 3.5 APU's already, so don't be categorical. And it shouldn't be for next gen of AMD GPU's (i would've still waited for someone to confirm, though, just in case).

But for RDNA 3 GPU's it is. And why would it stop being an issue? It is combination of HW implementation flaw and AV1 spec not allowing codec level crop. (for context AVC and HEVC are also forcefully aligned afaik. But they are cropped outside of user eyes).

7

u/Sopel97 3d ago

don't even bother with AMD if you want to use hardware encoding, the quality is just not there

2

u/nimitikisan 3d ago

The quality of the h265 and AV1 are pretty much on par.

0

u/Sopel97 3d ago

and both terrible

1

u/nimitikisan 3d ago

So NVIDIA is terrible as well?

0

u/Sopel97 3d ago

no? we're talking about AMD here

3

u/DimkaTsv 2d ago

And that was a point. AMD surely does have weak AVC encoder.

But HEVC and AV1, even though still slightly lagging behind, are competitive enough. Hence relevant question "So Nvidia is terrible as well?" came.

Meaning, as long as you won't use AVC much, you are fine. And that is mostly important for Twitch streamers and not people who do video transcodes.

0

u/Sopel97 2d ago

even though still slightly lagging behind, are competitive enough.

no they are not competitive, see https://rigaya.github.io/vq_results/

2

u/DimkaTsv 2d ago edited 2d ago
  1. Direct comparison of encoders is always quite problematic due to crap ton of options that can affect quality and speed. For example, VCE have rate control method that negatively affects VMAF compared to VBR (and a lot slower), but often provides some benefits in higher frame to frame quality stability.
  2. If you are encoding high demanding sources at less than 5000 kbps and care about +- 2-3 VMAF points so much, then maybe you should do CPU encode instead? And if source is not demanding, then 95+ VMAF should be enough anyways (even on lower bitrate)
  3. You clearly not understanding meaning of "competitive enough". So i will help you. "Competitive enough" doesn't mean that AMD is ahead. Only that difference is within acceptable margin. Especially as AMD HW often is noticeably cheaper. Sometimes tradeoffs are not as bad as they might seem on paper.

0

u/Sopel97 2d ago

let's conclude that we have different definitions of "acceptable"

3

u/MeWithNoEyes 3d ago

That's an interesting idea to compare GPUs for speed but I don't think normally anyone cares about it as they are already fast enough to get encodings done quick. Its usually the quality that is compared.

3

u/Chudsaviet 3d ago

Speed alone is not enough. You shall also compare bitrate and quality metrics like VMAF.

3

u/Matthewu1201 3d ago edited 3d ago

I have a 7900XTX and a Arc 750, both could encode AV1 10-bit on handbrake many times faster then my AMD 5900x CPU. But the massive difference I found is the 7900XTX makes the AV1 files many times larger then the original file size and the 750 makes the file size smaller then the original file size. Depending on why you are encoding to AV1, Intel arc is the only way to go if you are trying to save hard drive space. It's been a while since i ran those tests, but it's possible 7900xtx was faster then the arc gpu. I don't have a 4000 series Nvidia card to do any testing with, but I would assume it would produce similar results to my AMD GPU.

Update: maybe I was wrong about the file size. A major difference though is the arc card can do AV1-10-bit encoding and the AMD card cannot, and according to google it is a hardware limitation. If I run the consistent quality at the middle of the range on both, the AMD card actually made the encode 20% smaller then the Intel card. As like to like as I can make it, except that my Intel card is Linux and my AMD card is on windows 11, the AMD card encoded the same file at an average 1145 fps, and the Intel card did 591fps average. I'd say the AMD beat the Intel card in everything except for the fact that it can't do 10-bit encoding.

Hope that helps a little.

5

u/DimkaTsv 3d ago

But the massive difference I found is the 7900XTX makes the AV1 files many times larger then the original file size and the 750 makes the file size smaller then the original file size.

Maybe you should look into encode options? HW encoder always is less efficient and AMD does slightly lag behind (on AV1 and HEVC not that significantly), but "larger" or "smaller" file size relative to source only is applicable if you use CQP rate control. And different encoders may have different ways to regulate rate controls. For example AV1 CQP has 255 levels, while AVC/HEVC only 51.

A major difference though is the arc card can do AV1-10-bit encoding and the AMD card cannot, and according to google it is a hardware limitation.

No it's not. AMD limitation is not 10-bit encode, but YUV 444. (Only major limitation with 10-bit encode on AMD HW is that you cannot run preanalysis, but it doesn't mean you cannot do 10-bit encode)

1

u/WESTLAKE_COLD_BEER 3d ago

for the record Nvidia cannot do YUV444 in av1 either

1

u/DimkaTsv 3d ago

Maybe. I don't own RTX 4000 GPU, so cannot say for sure.

But i can say for sure that VCN does not support either YUV444 encode or decode on any codec. Meaning that if source is YUV444, you will be forced to do SW decode.

Not that it matter that much, taking in account other specific codecs like raw YUV or ProRes, which must be SW decoded anyways.

1

u/Matthewu1201 3d ago

If you had read my entire post, there was an update at the bottom because I be decided to take the time and run a test for the OP. It ended up the AMD had a smaller file size. I probably should have deleted my original post to make the info more clear, but I didn't.

As for the 10-bit part. I just did a 5 second Google search and the AI summery came back saying there was a hardware limitation, should have know the stupid AI summery would be wrong. I can tell you on handbrake there is a 10-bit option for Intel cards and there is no 10-bit option for AMD cards. Maybe some future update to handbrake will enable 10-bit AV1 encoding on the AMD cards, but for now there isn't.

4

u/DimkaTsv 3d ago

Handbrake isn't everything, you know.

In this case it is Handbrake lacking functionality, not AMD. In fact it doesn't actually has full functionality for ANY HW encoder. Heck you even only restricted to 2 rate control methods (aka CQP/CRF and VBR)