Sunday, August 26, 2012

Backup/Restore software - Amanda

It is tiresome work to implement a backup system with even Open Source Software  or Enterprise Backup Software, while it is also absolutely necessary to restore data after its loss or corruption.  

"Backups have two distinct purposes. The primary purpose is to recover data after its loss, be it by data deletion or corruption. Data loss can be a common experience of computer users. The secondary purpose of backups is to recover data from an earlier time, according to a user-defined data retention policy, typically configured within a backup application for how long copies of data are required.", says Wikipedia about bakcup.

I believe that Amanda Open Source Backup(Community Edition) is relatively easy and rapidly to set up and well documented on their wiki about how to install, tune the parameters or troubleshoot. It is enable to back up multiple hosts over network to tape changers, disks, optical media or AWS S3.

I would like to introduce how to set up Amanda server/client and try backup/restore process.
  • The relation between amanda server and client 
  • Prepare Amanda server and client in common
・/etc/hosts
As Amanda uses /etc/hosts to resolve their hostname, both Amanda server and client have their hostname on /etc/hosts.
# cat > /etc/hosts <<EOF
192.168.0.192 amanda_client
192.168.0.193 amanda_server
EOF

・install xinetd and the related libraries
# yum -y install xinetd.x86_64 gnupg.x86_64 sharutils.x86_64

・start xinetd
# /etc/init.d/xinetd start

・activate xinetd
# chkconfig xinetd on
 
・allow amanda backup services with iptables if necessary
# vi /etc/sysconfig/iptables
...
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -s 192.168.0.0/24 -p tcp --dport 10080 -j ACCEPT
...
COMMIT
# /etc/init.d/iptables restart

  • Set up Amanda Client
・install amanda client
# wget http://www.zmanda.com/downloads/community/Amanda/3.3.2/Redhat_Enterprise_6.0/amanda-backup_client-3.3.2-1.rhel6.x86_64.rpm
# rpm -ivh amanda-backup_client-3.3.2-1.rhel6.x86_64.rpm
# rpm -qa | egrep 'amanda-backup'
amanda-backup_client-3.3.2-1.rhel6.x86_64

・create amandahosts file
# cat > ~amandabackup/.amandahosts << EOF
amanda_server amandabackup amdump
EOF
# chmod 700 ~amandabackup/.amandahosts
・create amanda-client.conf(Amanda client configuration file)
# mkdir /etc/amanda/bk01
# cat > /etc/amanda/bk01 <<EOF
conf "default"               
index_server "amanda_server"
tape_server  "amanda_server"
tapedev      "file://var/lib/amanda/vtapes"
auth "bsdtcp"
ssh_keys "/var/lib/amanda/.ssh/id_rsa_amrecover"
EOF
・create amandates file
※It has to be there whether it is currently used or not, as it was used to calculate the file size.
http://wiki.zmanda.com/index.php/FAQ:What_is_the_'amandates'_file_for%3F
# touch /var/amanda/amandates


・setup the directories' owner
# chown -fR amandabackup:disk /var/*/amanda /etc/amanda

  • Set up Amanda server
・install amanda server
# wget http://www.zmanda.com/downloads/community/Amanda/3.3.2/Redhat_Enterprise_6.0/amanda-backup_server-3.3.2-1.rhel6.x86_64.rpm
# rpm -ivh amanda-backup_server-3.3.2-1.rhel6.x86_64.rpm
# rpm -qa | egrep 'amanda-backup'
amanda-backup_server-3.3.2-1.rhel6.x86_64


・create amanda.conf(Amanda server configuration file)
※There's sample and template files under /var/lib/amanda/example and /var/lib/amanda/template.d.
$ cat > /etc/amanda/bk01/amanda.conf <<EOF
org      "bk01"
send-amreport-on all
dumpuser "amandabackup"
inparallel 4            
dumporder "sssS"        
taperalgo first         
displayunit "k"         
netusage  100000 Kbps   
dumpcycle 1 weeks       
runspercycle 7        
tapecycle 16 tapes      
bumpsize 20 Mb          
bumppercent 20          
bumpdays 1              
bumpmult 4              
ctimeout 120
etimeout 1800
dtimeout 300
connect-tries 3
req-tries 5
device_output_buffer_size 1280k
usetimestamps yes
flush-threshold-dumped 0
flush-threshold-scheduled 0
taperflush 0
autoflush no
runtapes 1                      
maxdumpsize -1          
labelstr "^default-[0-9][0-9]*$"
amrecover_changer "changer"     
holdingdisk hd1 {
    comment "main holding disk"
    directory "/var/lib/amanda/holding"
    use 3 Gb                    
    chunksize 1Gb       
    }
infofile "/etc/amanda/default/state/curinfo"     
logdir   "/etc/amanda/default/state/log"         
indexdir "/etc/amanda/default/state/index"               
tpchanger "chg-disk"
tapedev "file:/var/lib/amanda/vtapes"
tapetype HARDDISK
define tapetype global {
    part_size 1G
    part_cache_type none
}
define tapetype HARDDISK {
 length 3072 mbytes
}
define dumptype global {
    comment "Global definitions"
    index yes
    auth "bsdtcp"
}
define dumptype root-tar {
    global
    program "GNUTAR"
    comment "root partitions dumped with tar"
    compress none
    index
    priority low
}
define dumptype user-tar {
    root-tar
    comment "user partitions dumped with tar"
    priority medium
}
define dumptype comp-user-tar {
    user-tar
    compress client fast
    estimate calcsize
}
define taperscan taper_lexical {
    comment "lexical"
    plugin "lexical"
}
taperscan "taper_lexical"
 
・create the directories
# mkdir -p /var/lib/amanda/holding /etc/amanda/default/state/{curinfo,log,index}
 
・setup the directories' owner
# chown -fR amandabackup:disk /var/*/amanda /etc/amanda
 
・create the virtual tape drive
# su - amandabackup
$ for slot_num in `seq 1 25` ; do
  mkdir -p /var/lib/amanda/vtapes/slot${slot_num}
done
 
・set up the virtual tape drive
$ ln -s /var/lib/amanda/vtapes/slot1 /var/lib/amanda/vtapes/data
 
・label the volume in the slot
$ for i in `seq 1 9`; do
  amlabel default default-0${i} slot ${i}
done

Reading label...
Found an empty tape.
Writing label 'default-01'...
Checking label...
Success!
...
Reading label...
Found an empty tape.
Writing label 'default-09'...
Checking label...
Success!
 
・show the contents of all slot
$ amtape default show
slot   9: date X              label default-09
slot  10: unlabeled volume
slot  11: unlabeled volume
slot  12: unlabeled volume
slot  13: unlabeled volume
slot  14: unlabeled volume
slot  15: unlabeled volume
slot  16: unlabeled volume
slot   1: date X              label default-01
slot   2: date X              label default-02
slot   3: date X              label default-03
slot   4: date X              label default-04
slot   5: date X              label default-05
slot   6: date X              label default-06
slot   7: date X              label default-07
slot   8: date X              label default-08
 
・reset the tape changer
$ amtape default reset
changer is reset
 
・create disklist(the directories to be archived)
$ cat > /etc/amanda/default/disklist <<EOF
amanda_client /var/www comp-user-tar
EOF
 
・run the self-check on both the amanda tape server and amanda client hosts
$ amcheck default
Amanda Tape Server Host Check
-----------------------------
WARNING: holding disk /var/lib/amanda/holding: only 3076096 KB available (3145728 KB requested)
found in slot 1: volume 'default-01'
slot 1: volume 'default-01' is still active and cannot be overwritten
found in slot 2: volume 'default-02'
slot 2: volume 'default-02'
Will write to volume 'default-02' in slot 2.
NOTE: skipping tape-writable test
NOTE: info dir /etc/amanda/default/state/curinfo/amanda_client/_var_www does not exist
NOTE: it will be created on the next run.
NOTE: index dir /etc/amanda/default/state/index/amanda_client/_var_www does not exist
NOTE: it will be created on the next run.
Server check took 1.683 seconds

Amanda Backup Client Hosts Check
--------------------------------
Client check: 2 hosts checked in 2.160 seconds.  0 problems found.

(brought to you by Amanda 3.3.2)
  • verify backups
[Amanda server]
・backup the disk
$ amdump defaut

・show the archived data
$ amadmin defaut find
date                host          disk     lv tape or file      file part status
2012-08-15 18:21:59 amanda_client /var/www  0 defaut-02    1  1/1 OK

[Amanda client]
Amanda offers 2 ways to restore the archived data with amrecover and amrestore.
I am using amrecover in an interactive manner.
・connect to the amanda server
# amrecover -s amanda_server -t amanda_server -C default
AMRECOVER Version 3.3.2. Contacting server on amanda_server ...
220 magento AMANDA index server (3.3.2) ready.
Setting restore date to today (2012-08-15)
200 Working date set to 2012-08-15.
200 Config set to default.
200 Dump host set to amanda_client.
Use the setdisk command to choose dump disk to recover
 
・list the all diskname on the amanda client hosts
amrecover> listdisk
200- List of disk for host amanda_client
201- /var/www
200 List of disk for host amanda_client
 
・specify which disk to restore
amrecover> setdisk /var/www
200 Disk set to /var/www.
 
・specify the working directory which the archived data is restored
amrecover> lcd /tmp/var
amrecover> lpwd
/tmp/var
 
・add the specified files to be restored(all of the data specified here with the wild card)
amrecover> add *
Added dir /icons/ at date 2012-08-15-18-21-59
Added dir /html/ at date 2012-08-15-18-21-59
Added dir /error/ at date 2012-08-15-18-21-59
Added dir /cgi-bin/ at date 2012-08-15-18-21-59


・restore
amrecover> extract
...
./icons/small/unknown.png
./icons/small/uu.gif
./icons/small/uu.png
amrecover> exit
 
・verify the difference between the data to be archived and the one restored
# diff -r /var/www/ /tmp/var/


verify the size between the data to be archived and the one restored
# du -cks /var/www/ /tmp/var/
1176    /var/www/
1176    /tmp/var/
2352    total

That's all!

12 comments:

  1. Great blog, very informative. The diagram was very helpful, thank you for sharing.

    Call virus removal Lexington or removal of all malicious software.

    ReplyDelete
  2. Great blog. By the way, I am curious about taperscan. Could you please tell the reason to use lexical taperscan instead of traditional? Does one have to use lexical while using virtual tapes in Amanda?

    ReplyDelete
  3. I wanna appreciate for your hard working ............ this is really superb....
    Download Full Version Software7zip | Free Download

    ReplyDelete
  4. Amanda Open Source Backup is a very simple and easy to use. thankful for great sharing!
    download free software 2014

    ReplyDelete
  5. Amanda Open Source Backup is very easy and useful.
    Portable Software Download

    ReplyDelete
  6. Thanks a for informing! Amanda Open Source Backup is very easy and useful.
    Portable Software

    ReplyDelete
  7. I must say this blog indicates the sincerety with other bloggers, It's an amazing blog.I always love to read these kind of blogs.Amazing discussion on software technology. Go ahead and update us through these type of posts.
    Full crack Software Download

    ReplyDelete
  8. The foundations and organizations offering trainings in PHP will convey the applicable information about the dialect to the applicants and give them a few openings for work by chipping away at live tasks.visit site

    ReplyDelete
  9. Every business has certain requirements like IT resource, infrastructure, labour, finance and data security and management. Though large businesses don't face any issues in getting them all but small businesses still struggle to expand and spend in the key business areas.
    most affordable backup solution

    ReplyDelete