I have managed to resolve this issue, but not convinced it is the "right way".
I had tried just about everything, and through a combination of changes now have bluetooth audio now working through my Sony BT headphones.
- I disabled the onboard BT module in my HP Elitedesk BIOS.
- I plugged in a generic USB BT receiver.
- I re-installed all packages refering to bluetooth
- I used bluetoothctl and entered the command for "set default"
I did run across other issues which have now resolved themselves. These included what must have been conflisst with my BT keyboard + mouses's unifying receiver.
The headset now connects and works as expected, without jitter. It is still slow to be "found / connected" but I'm not going to pursue this further in case I break it.
So my problem / solution seems to revolve around incompatibility between the onboard BT and my headset, which I have resolved by using a 3rd party USB BT dongle.