Just curious, but have you tried modifying grub (/etc/default/grub) so that the following entry:
And see if it boots to grub (without your script)? These newer kernels include the nvme drivers and this entry would tell grub to find and use them prior to the os search.
If you do this in the live usb, connect another usb (persistence hasn't worked yet due to a problem in the Ubuntu base image) and copy the following two files over:
Perform the install. Then close the installer when it asks to reboot by the x in the upper right. Mount the drive you installed to (on /mnt) and copy those files to the respective directories. Mount the following three also:
sudo mount --bind /dev /mnt/dev
sudo mount --bind /sys /mnt/sys
sudo mount --bind /proc /mnt/proc
Then mount the boot partition:
sudo mount /dev/nvmeXnXpX /mnt/boot
Then you can chroot into the /mnt to perform the update:
sudo chroot /mnt
Make sure you're in the right drive:
And make sure there are a few .mod files and the grub.cfg file. If not do not continue, go back and try your mounts again. Otherwise:
Exit and reboot.
That may solve your issue without needing the nvme boot script.
I'm sorry, but that would mean you'd have to copy off your custom efi and erase it to try... but it wouldn't be much to put it back using a live image if it doesn't work.