EDIT: After some testing, it introduces audio/video stutter. I'm going to see if i can find a way to fix that.
EDIT 2: After clearing all the files from ~/$USER/.config/pulse and rebooting, the audio seems to be playing properly. I'm going to test it for a while and see what happens.
EDIT 3: After testing for several hours, it seems while this solution does indeed get it working, there's still occasional audio/video stutters. Looks like 6.8.0-40 is a no go if you're having this issue, but YMMV. Good luck.
Ok, here's a method i found to fix it.
sudo apt install wireplumber pipewire-pulse
systemctl --user --now disable pulseaudio.service pulseaudio.socket
systemctl --user mask pulseaudio
systemctl --user --now enable pipewire pipewire-pulse wireplumber
Now reboot, and audio/video should play.
Source: How to Install and Configure PipeWire in Linux - Make Tech Easier
There is a section that says to remove pulseaudio, don't do this because it will try to remove half the damn system with it.