How can totally remove packages of youtube-dl with every single file on the system?
If you installed it using the curl method described in their documentation page, you should be able to uninstall using:
sudo apt remove --purge youtube-dl
If you prefer you can also use the synaptic package manager. When you choose a program to be install it will also show you other dependencies that are required and will be installed, or in this uninstalled, it works both ways.