Automatic Files Organisation (Intermediate Users)

My first tutorial!

Please forgive me if this is not the right section of the Forum for this one, more than happy to change it or remove it if it's off topic.

I've created a quick Python script that automatically organise the download folder (or whatever folder needs to be organised) based off the file type.

A downloaded .jpg file will be moved automatically in a folder called "images", for example, but the script is open to be personalised as much as you want!

Here's the code:

# afm.py - this is the name i gave to my script -

import os
from shutil import move

# directory paths

user = os.getenv('USER')
root_dir = '/home/USER/Downloads/'.format(user)
image_dir = '/home/USER/Downloads/images/'.format(user)
documents_dir = '/home/USER/Downloads/documents/'.format(user)
software_dir = '/home/USER/Downloads/software/'.format(user)
video_dir = '/home/USER/Downloads/video/'.format(user)
audio_dir = '/home/USER/Downloads/audio/'.format(user)
others_dir = '/home/USER/Downloads/others/'.format(user)


# category wise file types

doc_types = ('.doc', '.docx', '.txt', '.pdf', '.xls', '.ppt', '.xlsx', '.pptx')
img_types = ('.jpg', '.jpeg', '.png', '.svg', '.gif', '.tif', '.tiff', '.psd')
software_types = ('.exe', '.pkg', '.dmg', '.deb', '.flatpak', '.snap')
video_types = ('.mov', '.mkv', '.mp4')
audio_types = ('.mp3', '.wav')

def get_non_hidden_files_except_current_file(root_dir):
  return [f for f in os.listdir(root_dir) if os.path.isfile(f) and not f.startswith('.') and not f.__eq__(__file__)]
def move_files(files):
  for file in files:

    # file moved and overwritten if already exists

    if file.endswith(doc_types):
      move(file, '{}/{}'.format(documents_dir, file))
      print('file {} moved to {}'.format(file, documents_dir))
    elif file.endswith(img_types):
      move(file, '{}/{}'.format(image_dir, file))
      print('file {} moved to {}'.format(file, image_dir))
    elif file.endswith(software_types):
      move(file, '{}/{}'.format(software_dir, file))
      print('file {} moved to {}'.format(file, software_dir))
    elif file.endswith(video_types):
      move(file, '{}/{}'.format(video_dir, file))
      print('file {} moved to {}'.format(file, video_dir))
    elif file.endswith(audio_types):
      move(file, '{}/{}'.format(audio_dir, file))
      print('file {} moved to {}'.format(file, audio_dir))  
    else:
      move(file, '{}/{}'.format(others_dir, file))
      print('file {} moved to {}'.format(file, others_dir))

if __name__ == "__main__":
  files = get_non_hidden_files_except_current_file(root_dir)
move_files(files)

You can add as much as file type you want in the script, and as said before, modify the folders path.
For simplicity, just save the .py file in the folder you want to organise, and create also a log folder (to keep track of what the script does, the script will create a log.txt file).

To run it, you have two options:

  1. Launch the script from the terminal with the following:

python3 NAMEOFTHESCRIPT.py >>~/YOURFOLDER/LOGFOLDER/log.txt

  1. Automate the script with Crontab adding this line in the end of the file (i did a 2 minutes script):
  • Open your terminal

  • Run crontab -l (this will tell you if a cron file s already present)

  • Run crontab -e (this will let you open and edit the cron file - i selected nano editor for edit)

  • Add the following line in the end of the file:

*/2 * * * * cd ~/Downloads/ && python3 NAMEOFTHESCRIPT.py >>~/YOURFOLDER/LOGFOLDER/log.txt

  • Press CTRL + O to write the file and save it

  • Press CTRL + X to exit

  • Run this command to verify the file:

$ sudo ls -l /var/spool/cron/crontabs

I hope this can be helpful and feel free to reach out if needed!

5 Likes

OMG .... if I understand you correctly when you download something from the web and it goes to your download folder from there it will be placed in the correct folder for the correct type of file .... kinda like Internet Download Manager for win 10 ....

If this is the case I have been trying to find a program like this ever since I installed Zorin 16 Pro .... Linux work -a-rounds do not work and just screw everything up for me and as I don't use Snap I have just had to go to my download folder and transfer all my downloads to their proper folders ....

Now if I could just have someone take me by the hand and tell me what ... where and how to install the above code I will be most great full ... I know your instructions are pretty clear but I'm taking about someone who knows diddly squat about coding ....

I almost understand like open Python (have to see if it is already downloaded)

Copy ALL the contents listed above in the light gray box and paste into Python and place that into my downloads folder (where I want it) ......

I assume I need to put it into a folder or something .... after that I'm a bit fuzzy .... well lets face it a lot fuzzy ....

Interesting I typed ......... sudo apt update && sudo apt install python ...... into my terminal and it said

Preparing to unpack .../python2_2.7.17-2ubuntu4_amd64.deb ...
Unpacking python2 (2.7.17-2ubuntu4) ...
Selecting previously unselected package python-is-python2.
Preparing to unpack .../python-is-python2_2.7.17-4_all.deb ...
Unpacking python-is-python2 (2.7.17-4) ...
Setting up libpython2.7-stdlib:amd64 (2.7.18-1~20.04.1) ...
Setting up python2.7 (2.7.18-1~20.04.1) ...
Setting up libpython2-stdlib:amd64 (2.7.17-2ubuntu4) ...
Setting up python2 (2.7.17-2ubuntu4) ...
Setting up python-is-python2 (2.7.17-4) ...
Processing triggers for mime-support (3.64ubuntu1) ...
Processing triggers for gnome-menus (3.36.0-1ubuntu1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for desktop-file-utils (0.24-1ubuntu3) ...
mike@mike-ROG-Strix-G731GT-G731GT:~$

But I can't find it in my main menu under python .....

I can totally help you out, I’m not on my laptop anymore now, but we can do tomorrow eve if you like?

1 Like

The script use python 3 so you should use the following:

sudo apt update && sudo apt install python3

That should work :+1:t2:

1 Like

No problem not sure just what time zone you are in but where I live it is 8:30am in Southeast Asia .... I'll play around with it for a little bi but first I have to get python downloaded to my laptop ... thanks

I’m on UK timezone - we can work something out :+1:t2:

1 Like

OK sounds good I just ran

See ya tomorrow

@Frog, your system already has python 3 installed (python 3.8). To use it for running scripts shouldn't be problematic. If you want to get into development install a different version so as not to interfere with the systems version.

@dv1sual nice script. You may want to add .deb, .snap and .flatpak to the software list. Thank you for sharing this.

1 Like

File types added to the software list, thank you for the help!

I've been meaning to write a script like this for a long time! I wrote a script that renames all files in a given directory according to some specified parameters. For example you can rename this file 'Learn Linux in 5 days.pdf' into Learn_Linux_In_5_Days.pdf. Pretty useful if you get torrents and download files from several sources with inconsistent naming conventions, and wouldn't it be nice if in addition to renaming them they could also be places exactly where you need them? :smiley:

So when I saw this post of yours I finally got the inspiration I needed to give it a try. I wrote it as a command line script so that you can provide options including file associations so that you could specify right away whatever extension is missing, without having to modify the script at all.

I have only tested it briefly but added a --dry-run option to avoid making changes. I'd be extremely grateful if people could give it a try and let me know how it works. I will be integrating other options to rename files as well soon if anyone is interestedd (you could also just run both scripts from cron, but I need to re-write it anyway, so).

In any case, here's the link to the repo and some examples:

examples:

# Move only files with extension mp4 and wav, ignore the rest
./file_organizer.py -o mp4 wav

# Do not move files with extensions mp4 wav
./file_organizer.py -i mp4 wav

# Move epub and pdf files to ~/Sync/Books (default for those two is ~/Documents)
./file_organizer.py -m ~/Sync/Books=epub,pdf

# Move only epub files to ~/Sync/Books (default for those two is ~/Documents)
./file_organizer.py -o epub -m ~/Sync/Books=epub,pdf

# Move files from ~/Desktop (default is to move from ~/Downloads)
./file_organizer.py -s ~/Desktop

# Only move files that start with "distr", case sensitive e.g.: Distributed_Systems_With_Node_js.epub
./file_organizer.py -e '^Distr'

# Move files with multiple extensions to the Desktop e.g.: Shade of Z - Black.tar.xz, files_backup.tar.gz
./file_organizer.py -e '(\.tar\.[gx]z)$' -m ~/Desktop=.tar.gz,.tar.xz

# Run in test mode to verify changes before actually writing to disk
./file_organizer.py --dry-run

# Check script's version
./file_organizer.py --version
2 Likes

WOW i'm going off topic here, but i love your password checker code!
I mean, you're very good at python - amazing repo.

I added one issue i found in the file_organiser code - nothing major at all - i hope you don't mind!

Yes, my first issue!

As you may have noticed I'm not very creative when it comes to naming these kind of scripts :sweat_smile: I'll have a look at it later, thanks for the compliments!

1 Like

Yup I'm here had some stuff to do this morning .... sorry ..... just to let you know I have been using Zorn 16 Pro for about 4 months ..... before that I used Windows since Windows first came out I believe back in the 70' .... I know absolutely 0 about coding the bravest thing I have ever done in Win is tackle the registry on occasion .... so yes I am a complete newbie when it comes to everything else .... I can however use the terminal to some extent thanks to the good folks here on the board ....

Now that that is over .... the only thing I see we will need to add to your excellent script is in the doc_types .... WPS for WPS Office

I assume that all that code needs to be placed somewhere .... I have updated Python to 3.8.10 but no idea how to use it .....

@Frog type python3 in the terminal and then whatever the script is called.

Example: python3 file_organizer.py

Like any windows terminal command, or Linux terminal command, there are flags to change options on most scripts. --help will bring up usage details for most of them.

Enjoy

1 Like

OK I typed python3 # afm.py (the mane he gave his script) in the terminal and I got this ....

mike@mike-ROG-Strix-G731GT-G731GT:~$ python3 # afm.py
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

help
Type help() for interactive help, or help(object) for help about object.

help()

Welcome to Python 3.8's help utility!

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at The Python Tutorial — Python 3.8.12 documentation.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics". Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".

help>

Yes I know it told me to .....

check out the tutorial on the Internet at The Python Tutorial — Python 3.8.12 documentation. ..... so I did

OK I tried this but now I don't know what to do .....

Oh well my head is aching so going to leave this until tomorrow at which time we'll start again .... just deleted the terminal ....

I’m so sorry I felt asleep yesterday night, apologies :disappointed:
I’ve messaged you privately so we can figure out how can I help you better :+1:t2:

Hey @Frog, let me try to explain bit more in details, hope this make everything more clear!

So, this:

Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux

means that you have the right Python3 version installed, so we are all good on that side of things.

Now, what you need is an editor like Atom for example, that you can use to edit the script i posted.
You don't need to do much, just change the user name and the folder path accordingly to your machine.

Steps:

  • Open the text editor
  • Copy-paste the script i posted in the thread
  • Give it a name followed by .py and save it (eg. script.py) and place it in the download folder.
  • Go back to the text editor, and at this point:
user = os.getenv('USER')
root_dir = '/home/USER/Downloads/'.format(user)
image_dir = '/home/USER/Downloads/images/'.format(user)
documents_dir = '/home/USER/Downloads/documents/'.format(user)
software_dir = '/home/USER/Downloads/software/'.format(user)
video_dir = '/home/USER/Downloads/video/'.format(user)
audio_dir = '/home/USER/Downloads/audio/'.format(user)
others_dir = '/home/USER/Downloads/others/'.format(user)

Change everywhere you read "USER" with your username (your should be mike)
Save the file again. You finished!

Now, working on the folder side of things.
Create all the sub-folders in the download folder. You'll want:

  • images
  • documents
  • software
  • video
  • audio
  • others
  • log

This is where the script will move all the files it will find in the downloads folder.
The saved script must be placed into the downloads folder like this:

downloads

Almost there!

No open your terminal and digit:

cd Downloads

(this will let you enter the Downloads folder where the script is) followed by:

python3 script.py >>~/Downloads/log/log.txt

This piece here:

>>~/Downloads/log/log.txt

Will create a small text file that will register every file the script will move in the right folder. I use it cause i like to have control on what happens but it's not really necessary :slight_smile:
Keep it if you don't mind.

This is it, keep me posted on how it's going! Once it's working manually, we can proceed to automate with crontab :v:

2 Likes

Thanks I haven't forgotten you .... just working through some other problems right now ..... this stupid printer won't work again and I'm trying to make a hard copy of your instructions .... so as soon as I get that fixed I'll get back to you ..... thanks for being so patience .....

1 Like

OK here is what I've done so far ..... please check it for mistakes .... I did change one item in the doc_type I added '.wps' , ..... for WPS Office

afm.py - this is the name i gave to my script -

import osfrom shutil import move

directory paths

user = os.getenv('mike')root_dir =

root_dir = '/home/mike/Downloads/'.format(user)

image_dir = '/home/mike/Downloads/images/'.format(user)

documents_dir = '/home/mike/Downloads/documents/'.format(user)

software_dir = '/home/mike/Downloads/software/'.format(user)

video_dir = '/home/mike/Downloads/video/'.format(user)

audio_dir = '/home/mike/Downloads/audio/'.format(user)

others_dir = '/home/USER/Downloads/others/'.format(user)

category wise file types

doc_types = ('.doc', '.docx', '.txt', '.pdf', '.xls', '.ppt', '.xlsx', '.wps')

img_types = ('.jpg', '.jpeg', '.png', '.svg', '.gif', '.tif', '.tiff', '.psd')

software_types = ('.exe', '.pkg', '.dmg', '.deb', '.flatpak', '.snap')

video_types = ('.mov', '.mkv', '.mp4')

audio_types = ('.mp3', '.wav')

def get_non_hidden_files_except_current_file(root_dir):

return [f for f in os.listdir(root_dir) if os.path.isfile(f) and not f.startswith('.') and not f.eq(file)]def move_files(files):

for file in files:

# file moved and overwritten if already exists



if file.endswith(doc_types):

  move(file, '{}/{}'.format(documents_dir, file))

  print('file {} moved to {}'.format(file, documents_dir))

elif file.endswith(img_types):

  move(file, '{}/{}'.format(image_dir, file))

  print('file {} moved to {}'.format(file, image_dir))

elif file.endswith(software_types):

  move(file, '{}/{}'.format(software_dir, file))

  print('file {} moved to {}'.format(file, software_dir))

elif file.endswith(video_types):

  move(file, '{}/{}'.format(video_dir, file))

  print('file {} moved to {}'.format(file, video_dir))

elif file.endswith(audio_types):

  move(file, '{}/{}'.format(audio_dir, file))

  print('file {} moved to {}'.format(file, audio_dir))  

else:

  move(file, '{}/{}'.format(others_dir, file))

  print('file {} moved to {}'.format(file, others_dir))

if name == "main":

files = get_non_hidden_files_except_current_file(root_dir)

move_files(files)

1 Like