Linux Folder / Folders Backup on Yandex
Ok so a little while ago I made a Linux Yandex python script for backing MySQL databases and sync them on Yandex Disk, so if I already did that I thought I should use a part of that script and make a backup script for folders.
Most of the script is self-explanatory, all you have to do is to change the variables before the starting of the class keyword for your own needs. I put some comments that will better explain what you should define in each variable. Most of them are either of type string or number, just one is a bit different is a list of x lists of a 3 element list. It might sound confusing, but with this approach setting the script is actually easier.
I'll give you an example of how to set that variable, first for each folder that you want to sync on Yandex you have 3 settings in that variable:
- 1 path to the directory you want to backup (ex /home/andrei/backup-folder [do not include an ending slash])
- 2 path to the Yandex target dir where to save the archive of the backup folder ( this is a subfolder of Yandex backup-dir that you had set in Yandex config file ex /home/yandex_backup/files/my_backup )
- 3 and lastly the name of the archive that you want ( I will mention that the date in format year-month-day will be appended to this name)
So that is all, here is the code from the script, also I will put a download link below after the code:
#!/usr/bin/env python3
import sys
import os
import os.path
import subprocess
import time
from threading import Thread
from queue import Queue, Empty
from datetime import datetime
"""
-------------- -------- ------------------ --------------- ------------- ---------
A script to sync Yandex cloud backups for a specified folder in the system.
Program: Yandex Sync Script
Author: Andrei O. (andrei0x309)
Date: Nov 11, 2016
Revision: 0.1
Revision | Author | Comment
-------------- -------- ------------------ --------------- ------------- ---------
20161111-0.1 Andrei O. Initial creation of the script.
-------------- -------- ------------------ --------------- ------------- ---------
"""
#Configure These Variables
YFbkconfigList= [[ "/usr/share/nginx/azrael-sub7.ro", "/home/andrei/yandex_backup/01.140/FL/blog.flashsoft.eu", "blog.flashsoft.eu"], ["/home/alina/blackellis.eu", "/home/andrei/yandex_backup/01.140/FL/blackellis.eu", "blackellis.eu" ]]
# [ [ folder-to-backup, path-where-to-backup(folder in yandex sync), archive-name ], [next set like before] ]
MaxNumberOfBackupsToKeep = 3 # Number of bk to keep
YBKSLogPath = os.path.dirname(os.path.realpath(__file__)) + "/" + "YBKFLLog.log" # Log FIle path
DebugEnable = False # Print to logfile
YandexStopDaemonAfterSync = True # try to stop Yandex daemon if is started
YandexUserName = "andrei0x309" # Yandex Username
YandexUserPass = "**********" # Yandex Password
YandexAuthTimeOutSec = 600
#end Configure Variables
class YandexFolderBk:
def __init__(self):
for ent in YFbkconfigList:
fl=[]
currTimeLog = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
if DebugEnable: sys.stdout=open(YBKSLogPath,"a")
for file in os.listdir(ent[1]):
if str(file.name).find(ent[2]) != -1:
fl.append(file.name)
fl.sort()
fld = []
if len(fl) > MaxNumberOfBackupsToKeep:
fld = fl[MaxNumberOfBackupsToKeep:]
fld= [ent[0]+ "/" + x for x in fld]
subprocess.call("rm -rf " + " ".join(fld), shell=True)
currTime = datetime.now().strftime("%Y-%m-%d")
subprocess.call("tar -czf "+ent[1]+"/"+ent[2]+"-"+currTime+".tar.gz "+ ent[0] , shell=True)
try:
p = subprocess.Popen(['yandex-disk', 'sync'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
pline = p.stdout.read()
while True:
if str(pline).find("Syncing") == -1 :
print("{0}: Not Syncing probably auth error, request new token".format(currTimeLog), file=sys.stdout)
ON_POSIX = 'posix' in sys.builtin_module_names
def enqueue_output(out, queue):
for line in iter(out.readline, b''):
queue.put(line)
out.close()
pt = subprocess.Popen(['yandex-disk', 'token', YandexUserName, '-p', YandexUserPass], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=ON_POSIX)
q = Queue()
t = Thread(target=enqueue_output, args=(pt.stdout, q))
t.daemon = True # thread dies with the program
t.start()
def getOutTx():
try:
return q.get_nowait()
except Empty:
return None
while True:
lineTk = getOutTx()
if not t.is_alive():
subprocess.call("yandex-disk sync", shell=True)
break
time.sleep(0.7)
if not p.poll() is None:
break
if YandexStopDaemonAfterSync:
try:
subprocess.call("yandex-disk stop", shell=True)
except OSError as e:
print("{0}: Stoping Yandex Daemon failed".format(currTimeLog), e, file=sys.stdout)
except OSError as e:
print("{0}: Execution Of Yandex Sync failed:".format(currTimeLog), e, file=sys.stdout)
sys.stdout.close()
if __name__ == '__main__':
YandexFolderBk()
Also, you should use crontab for setting the repetition of execution, I use:
@monthly root /home/andrei/yandex_backup/bkscripts/YandexFolderBk.py
Download link:
[download id="3"]