thats the issue, downscaling. when I downscale 4K to HD it takes about 45min for a 2h film, when I keep the resolution its like 5min.

We need more detailed information, what codec you want do encode in, what’s the settings to the encoder, what type of input file etc. And since you use windows it would be nice to check if you use the 3D accelerator or the Encoder. This is easy shown in the taskmanager, when you encode in hardware the hardware encoder should show usage. For me I use both intels quicksync aswell as nvidias nvec for encoding and never had any issue so it just might be an AMD issue or your specific GPU may be faulty-or driver issue.