[TheHijacker]

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensical defaults.

http://rdiff-backup.nongnu.org/

This is a first version of this package. I expect more to follow with fine tuning and tweaking according to usage by users.

1. Backing up from eTRAYz to USB External HDD (default setting in package):

Look at settings file in /home/sysadmin/.rdiff-backup/rdiff-backup.conf

# for how long to keep backups
KEEP=30D

# source and destination folder
SOURCE="/home/Public"
DESTINATION="/External/HDD/part1/backup/Public"

# which folders to exclude
EXCLUDE="--exclude /home/Public/eTorrent --exclude /home/Public/eDownloader"

# rdiff binary file
RDIFF="/usr/bin/rdiff-backup"

# some common options
COMMON_OPTIONS="-v5 --force --create-full-path --ssh-no-compression --exclude-fifos --exclude-device-files --exclude-sockets --exclude-other-filesystems --print-statistics"

This default settings are as follows.

Keep files for up to 30 days. Backup folder /home/Public but exclude folders /home/Public/eTorrent and /home/Public/eDownloader. Make backup on external USB hard drive into folder backup/Public which gets mounted on /External/HDD/part1.

You can edit this file according to your needs. To test it localy disk to disk, I changed the destination folder to /home/sysadmin/backup/Public.

There is also an option to backup to external location via SSH, but this is a bit more advanced. Lets keep it simple for now.

To start backup run this type in SSH:

rdiff-backup.sh

2. Backing up from Windows to eTRAYz:

You need to do a couple of steps.

First download native Windows build (rdiff-backup-1.2.8-win32.zip) and unzip it to some folder.

Next we need SSH client for windows. Best and easiest to use is our beloved putty. Download following files: putty.exe, puttygen.exe and plink.exe and to make things easier copy there files in the same folder as you unpacked rdiff-backup for Windows.

Now we need to make a private key in order to connect to eTRAYz over SSH without authentication. Run puttygen.exe. He will open a Window where you click on Generate button (1.). Now move your mouse under progress bar to create random key.



Then save it by clicking on Save private key (2.). He will nag if you want to save it without passphrase. Confirm with YES. Name the file privatekey.ppk and save it again in same folder where we have unpacked rdiff-backup files. DO NOT CLOSE PUTTYGEN WINDOW YET.

Login to your eTRAYz via SSH with root username. Now lets copy paste the private key into .ssh folder. Type this first to make the folder if its not existing:

mkdir -p /root/.ssh/

Then copy the string in Public key for pasting into OpenSSH authorized_keys file from puttygen window(3.). Then paste this line into black putty window (right mouse click):

echo "HERE_YOUR_KEY_ALL_IN_ONE_LINE" >> /root/.ssh/authorized_keys

This is example from my puttygen key:

echo "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEArzBORj2Xpu9ltwOvwv7vPY9L2MeD/1pz1bBAgwJA4bN90HWEi1bofbMUm57EZ9qu8GC/GV4vU/WgfN947byoCEH1YJmKyrKNHzElUavPXSf1NJJzZ95Wt25ynsCH4pUDM8BYtRKF4vXlzYqT6GUJyYBuKpJjGHMDGw7rMUfC+88= rsa-key-20100816" >> /root/.ssh/authorized_keys

Now make a new bat file in the same folder as we unpacked rdiff-backup. For example rdiff-backup.bat and put following text inside:

rdiff-backup.exe -v5 --no-hard-links --print-statistics --remote-schema "plink.exe -i privatekey.ppk %%s rdiff-backup --server" "c:\folder on\your hard drive\\." root@etrayzip::/home/folder/on/your/et

Replace the etrayzip with your etrayz hostname or IP number and replace source and target folder with the one you want to use.

To start making backup from windows to eTRAYz just run this bat file:

rdiff-backup.bat

3. Using crontab to schedule regular backup:

Type in console:

if [ ! -e /usr/bin/vi ]; then ln -s /bin/vi /usr/bin/vi; fi
crontab -eu root

First press the i key to put vi editor in edit mode.

And if you want the job to start daily at 00:05 (five minutes after midnight) then paste this into new line:

5 0 * * * /usr/bin/rdiff-backup.sh >/dev/null 2>&1

To close and save vi editor type:

:wq

Give it a try and please post your comments and possible improvements.

[sebek72]

Great stuff...old school bash scripts

[TheHijacker]

Hehe. All this newage kids that are growing up with Windows know nothing about this. DOS FTW!

Have you tried to install this package? Any comments on how to improve it are welcomed.

[sebek72]

TheHijacker, on 18 August 2010 - 10:25, said:

Hehe. All this newage kids that are growing up with Windows know nothing about this. DOS FTW!

Have you tried to install this package? Any comments on how to improve it are welcomed.

Installed it, but didn't have the time to test...

Maybe over weekend.

[epox]

got the following error???????




ogin as: root
Using keyboard-interactive authentication.
Password:
Last login: Thu Aug 19 20:05:28 IST 2010 from 192.168.1.3 on pts/1
[1:~]$ rdiff-backup.sh
Executing /usr/bin/rdiff-backup.sh
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 20, in <module>
import rdiff_backup.Main
File "/usr/lib/python2.6/site-packages/rdiff_backup/Main.py", line 23, in <module>
import getopt, sys, re, os, cStringIO, tempfile, errno
ImportError: No module named cStringIO
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 20, in <module>
import rdiff_backup.Main
File "/usr/lib/python2.6/site-packages/rdiff_backup/Main.py", line 23, in <module>
import getopt, sys, re, os, cStringIO, tempfile, errno
ImportError: No module named cStringIO
[2:~]$ login as: root
Password:
Last login: Thu Aug 19 20:05:28 IST 2010 from 192.168.1.3 on pts/1
[1:~]$ rdiff-backup.sh
Executing /usr/bin/rdiff-backup.sh
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 20, in <module>
import rdiff_backup.Main
File "/usr/lib/python2.6/site-packages/rdiff_backup/Main.py", line 23, in <module>
import getopt, sys, re, os, cStringIO, tempfile, errno
ImportError: No module named cStringIO
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 20, in <module>
import rdiff_backup.Main
File "/usr/lib/python2.6/site-packages/rdiff_backup/Main.py", line 23, in <module>
import getopt, sys, re, os, cStringIO, tempfile, errno
ImportError: No module named cStringIO
[2:~]$


cheers mark

[TheHijacker]

Try this:

export PYTHONHOME=/usr
rdiff-backup.sh

Python is broken in firmware v1.0.5.

Let me know if it worked.

[epox]

got the error below

login as: root
Using keyboard-interactive authentication.
Password:
Last login: Thu Aug 19 20:13:43 IST 2010 from 192.168.1.3 on pts/1
[1:~]$ export PYTHONHOME=/usr
[2:~]$ rdiff-backup.sh
Executing /usr/bin/rdiff-backup.sh
Fatal Error: Wrong number of arguments given.
See the rdiff-backup manual page for more information.
Using rdiff-backup version 1.2.8
Fatal Error: Switches missing or wrong number of arguments
See the rdiff-backup manual page for more information.
[3:~]


mark

[TheHijacker]

Better. Did you edit /home/sysadmin/.rdiff-backup/rdiff-backup.conf?

From where to where are you backing up?

[epox]

trying this

# for how long to keep backups
KEEP=30D

# source and destination folder
SOURCE="/home/public/Music(iTunes)"
DESTINATION="/External Disk/USB1/part1/backup/Public"


mark

[TheHijacker]

You need external USB HDD then inside. And make sure that path is correct. You DO NOT want to copy files from /home into OS partition. It’s only 2GB. Filling that up will quite possible brick it and you will need to reinstall firmware.

Try something simpler to start with. This is what I used to test:

# for how long to keep backups
KEEP=30D

# source and destination folder
SOURCE="/home/Public"
DESTINATION="/home/sysadmin/backup/Public"

# which folders to exclude
EXCLUDE="--exclude /home/Public/eTorrent --exclude /home/Public/eDownloader"

# rdiff binary file
RDIFF="/usr/bin/rdiff-backup"

# some common options
COMMON_OPTIONS="-v5 --force --create-full-path --ssh-no-compression --exclude-fifos --exclude-device-files --exclude-sockets --exclude-other-filesystems --print-statistics"

I copy from HDD to HDD.

[epox]

Hi changed

/External disk/USB1/part1/backup/Public.
to
/External/USB1/part1/backup/Public.

and got the following error

login as: root
Using keyboard-interactive authentication.
Password:
Last login: Thu Aug 19 21:06:50 IST 2010 from 192.168.1.3 on pts/1
[1:~]$ export PYTHONHOME=/usr
[2:~]$ rdiff-backup.sh
Executing /usr/bin/rdiff-backup.sh
Fatal Error: Bad directory /External/USB1/part1/backup/Public.
It doesn't appear to be an rdiff-backup destination dir
Using rdiff-backup version 1.2.8
Unable to import module xattr.
Extended attributes not supported on filesystem at /home/Public/Music(iTunes)
Unable to import module posix1e from pylibacl package.
POSIX ACLs not supported on filesystem at /home/Public/Music(iTunes)
Unable to import win32security module. Windows ACLs
not supported by filesystem at /home/Public/Music(iTunes)
escape_dos_devices not required by filesystem at /home/Public/Music(iTunes)
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import module xattr.
Extended attributes not supported on filesystem at /External/USB1/part1/backup/Public/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
POSIX ACLs not supported on filesystem at /External/USB1/part1/backup/Public/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import win32security module. Windows ACLs
not supported by filesystem at /External/USB1/part1/backup/Public/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices not required by filesystem at /External/USB1/part1/backup/Public/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions Off
High-bit permissions On
Symlink permissions Off
Extended filenames On
Windows reserved filenames Off
Access control lists Off
Extended attributes Off
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Fatal Error: Fatal Error: The file specification
'/home/Public/eTorrent'
cannot match any files in the base directory
'/home/Public/Music(iTunes)'
Useful file specifications begin with the base directory or some
pattern (such as '**') which matches the base directory.
[3:~]$ ^C


ps using 2 1tb drives in liner and using 500gb external hd to back up to

will keep trying!

mark
[3:~]$

[TheHijacker]

Please check where exactly your external HDD is mounted (paste output of command mount and command df -T)! And escape the folders with strange characters like:

/home/Public/Music\(iTunes\)/

[hawkeye]

Thanks Hijacker for the instructions it worked like a charm backing up to external HD but can it be kept running after closing down putty or scheduled to run on the etrayz without being prompted via putty or other external software.
I'm still learning what the etrayz can and cannot do by itself.

Thanks

[TheHijacker]

You can put it in crontab job if you want? That would automaticly start it at specific time in background.

[Viking69]

Tried it, but only gets:

Using rdiff-backup version 1.2.8
Fatal Error: Invalid remote schema:

plink.exe -i privatekey.ppk %%s rdiff-backup --server

??

[TheHijacker]

Post here the content of your edited config file /home/sysadmin/.rdiff-backup/rdiff-backup.conf.

[epox]

Hi
got it working using your posted settings
some info on how to set up a crontab job would be great

thanks Mark

[TheHijacker]

Extra instructions about using cron to schedule task added to first post in this topic.

[epox]

etrayz software v 1.05

rdiff-backup settings works by running

export PYTHONHOME=/usr
rdiff-backup.sh

in putty


# for how long to keep backups
KEEP=30D

# source and destination folder
SOURCE="/home/Public/"
DESTINATION="/External/USB1/part1/backup/Public"

# which folders to exclude
EXCLUDE="--exclude /home/Public/eTorrent --exclude /home/Public/eDownloader"

# rdiff binary file
RDIFF="/usr/bin/rdiff-backup"

# some common options
COMMON_OPTIONS="-v5 --force --create-full-path --ssh-no-compression --exclude-fifos --exclude-device-files --exclude-sockets --exclude-other-filesystems --print-statistics"





however trying to set auto back up wont work

test /usr/bin/vi || ln -s /bin/vi /usr/bin/vi
crontab -eu root
6 18 * * * /usr/bin/rdiff-backup.sh >/dev/null 2>&1

in putty rdiff wont back up

6 18 * * * = 18.06pm?

any ideas

thanks Mark

[TheHijacker]

You of course need your extenal HDD inserted and make sure its mounted as /External/USB1/part1/.

Crontab job looks OK to me. You will not see it execute in putty. Its all done in background. Check by looking at destination folder.