The thumbnails are generated on the fly as you open the image or something else triggers this process. Unfortunately the file picker on its own doesn't trigger the thumbnail generation.
Manually, you can click on the file so that it generates the preview on the right side; the next time you open the file picker that image would load the thumbnail normally.
Or you can run a script to do this for you, either at given intervals or on demand. There are already a few that I found online:
There's also a file size limit to avoid generating previews for files that are too large. You can follow these steps to change it: