My hopefully last challenge was to get the brightness control working with the hardware buttons and even with software. The proprietary Nvidia driver needs an additional config entry.
Add under “Device section”:
Option "RegistryDwords" "EnableBrightnessControl=1"
Now you should control your brightness via software and via hardware on most Laptops