And just to clarify, I'm still using GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset" as I assume that removing nomodeset would make the black screen return.
I ended up using this tutorial for purging everything Nvidia and reinstalled the nvidia-driver-535. Got a black screen while installing but after reboot it seems to work without any issues. I can now open the Nvidia settings and I see a lot more things in the settings plus it says "Nvidia Corporation" under Settings > About.
My pleasure. Note that every time you get a kernel update you need to re-install/build the driver when chose to manually install the nvidia driver by the guide you posted.