I've used UFS2 snapshots on FreeBSD by means of the dump command to make stateful filesystem backups in the past, so I recently wanted to take a similar approach to backup some Linux servers. Most GNU/Linux distributions use ext3 as their root filesystem by default these days which does not natively support snapshots, so I needed to find another way to do it. As it turns out, I used LVM on my main Linux servers to add a layer of virtualization to the storage subsystem. LVM does natively support snapshots, and here's how to do it.

1. Make room for the snapshot

I ran into a major problem right off the bat - the server I was testing LVM snapshots on had one hard drive with 2 partitions, 1 for /boot and one for LVM. The LVM partition contained one Volume Group called "TERA-VIRTUAL" and this VG had 2 Logical Volumes - "root" (mounted on /) and swap_1 (swap). The root LV had all the PE allocated to it and the filesystem beneath it was using all of the available space. The problem is that since there were no available Physical Extents in the VG (or in my PVs), I couldn't create a snapshot volume. This is important: when you create a snapshot on LVM you are really creating another Logical Volume that is used to hold the changes to the volume. This volume needs to me in the same Volume Group as the volume you are taking a snapshot of - so you need some extra space in that VG! In my case, I needed to resize the root fs, which was on a Logical Volume - this to me an hour plus to achieve since most LiveCDs either have no support for LVM (gparted-live) or the support is broken (backtrack 3 final usb). I booted ubuntu server 7.10 64bit into rescue mode and started a shell in the installer environment and once at a prompt typed vgmknodes to make the /dev/ entries for my LVs, then used resize2fs to shrink my FS and lvreduce to shrink my LV, leaving me with 23G of available Physical Extents in my VG (seriously, if you don't know what I'm talking about, don't try it - you will trash your hard drive!).

2. Create the snapshot and run backup

To create a snapshot you first need to figure out how much space you need - it is critical that you have enough space to hold all the changes to your volume until the snapshot is removed!. For me 10% of my 200G is fine - 20G.
The process is to create a snapshot LV, mount it, do your backup, unmount it, and remove the snapshot:


root@TERA-VIRTUAL:~# lvcreate --size 20G --snapshot --name snap /dev/TERA-VIRTUAL/root
Logical volume "snap" created
root@TERA-VIRTUAL:~# lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
root TERA-VIRTUAL owi-ao 200.00G
snap TERA-VIRTUAL swi-a- 20.00G root 0.00
swap_1 TERA-VIRTUAL -wi-ao 9.46G
root@TERA-VIRTUAL:~# lvdisplay /dev/TERA-VIRTUAL/snap
--- Logical volume ---
LV Name /dev/TERA-VIRTUAL/snap
VG Name TERA-VIRTUAL
LV UUID fFbaH4-22Hq-s7a2-e1mo-DPov-8wff-dvuAiI
LV Write Access read/write
LV snapshot status active destination for /dev/TERA-VIRTUAL/root
LV Status available
# open 0
LV Size 200.00 GB
Current LE 51200
COW-table size 20.00 GB
COW-table LE 5120
Allocated to snapshot 0.01%
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 254:2

root@TERA-VIRTUAL:~# mount /dev/TERA-VIRTUAL/snap /backup_snapshot/
root@TERA-VIRTUAL:~# cp -Rf /backup_snapshot/VirtualMachines/TERA-WEBSERVER /backup/
root@TERA-VIRTUAL:~# lvdisplay /dev/TERA-VIRTUAL/snap
--- Logical volume ---
LV Name /dev/TERA-VIRTUAL/snap
VG Name TERA-VIRTUAL
LV UUID fFbaH4-22Hq-s7a2-e1mo-DPov-8wff-dvuAiI
LV Write Access read/write
LV snapshot status active destination for /dev/TERA-VIRTUAL/root
LV Status available
# open 1
LV Size 200.00 GB
Current LE 51200
COW-table size 20.00 GB
COW-table LE 5120
Allocated to snapshot 1.96%
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 254:2

root@TERA-VIRTUAL:~# umount /backup_snapshot/
root@TERA-VIRTUAL:~# lvremove /dev/TERA-VIRTUAL/snap
Do you really want to remove active logical volume "snap"? [y/n]: y
Logical volume "snap" successfully removed


As you can see from the transcript above, you can keep an eye on your snapshot size by using lvdisplay and looking for "Allocated to snapshot". As seen above, before the backup the snapshot volume was 0.01% full, but by the time my backup finished, it was up to 1.96%.

Now I've got a sane copy of my data without having to dismount my filesystem!

Having done Linux administration for over 10 years now, I use the watch command almost every day. On Linux, watch will run a command every couple seconds. It's particularly nice for watching files change:


# watch ls -lah


This example will keep showing the output of ls -lah every 2 seconds.

About a year ago I switched a couple servers over to FreeBSD 6.1 because of it's security features and more stable development model. For the last year I've been occasionally using the watch command on FreeBSD and getting an annoying error like this:


[root@devel /backup]# watch ls -lah
watch: fatal: bad device name


Today it finally occured to me to look at the manpage:


WATCH(8) FreeBSD System Manager's Manual WATCH(8)

NAME
watch -- snoop on another tty line

SYNOPSIS
watch -cinotW -f snpdev tty

DESCRIPTION
The watch utility allows the user to examine all data coming through a
specified tty using the snp(4) device. If the snp(4) device is not
available, watch will attempt to load the module (snp). The watch util-
ity writes to standard output.


It hardly looks like the watch I know! It turns out that FreeBSD has a different use for watch - snooping on other consoles. On FreeBSD, the GNU watch (like on Linux) is called gnu-watch. If you don't want to install it, you can easily whip up a poor-man's replacement:


#!/usr/local/bin/bash
while [ 1 -lt 2 ]
do
clear
date
echo ------------------------------------
eval $@
sleep 2
done


TIP: To replace the default 'watch' with this script, save the above script as /root/watch, then make it executable, chmod +x /root/watch and alias it in your shell, alias watch='/root/watch'. You can save this alias permenantly with the bash shell by adding it to ~/.bash_profile .

Here's a sample output from the poor-man's watch:

Thu Jul 10 22:09:27 EDT 2008
------------------------------------
total 17113832
drwxr-xr-x 3 root wheel 512B Jul 10 21:03 .
drwxr-xr-x 21 root wheel 512B Jul 10 18:25 ..
-rw-r--r-- 1 root wheel 97M Jul 10 15:41 archive1.tar
-rw-r--r-- 1 root wheel 40M Jul 10 15:41 archive1.tar.bz2
-rw-r--r-- 1 root wheel 308M Jul 10 17:19 archive2.tar
-rw-r--r-- 1 root wheel 93M Jul 10 17:19 archive2.tar.bz2
-rw-r--r-- 1 root wheel 600M Jul 10 17:13 archive2a.tar
-rw-r--r-- 1 root wheel 192M Jul 10 17:13 archive2a.tar.bz2
-rw-r--r-- 1 root wheel 3.3G Jul 10 19:09 archive2b.tar
-rw-r--r-- 1 root wheel 1.4G Jul 10 19:09 archive2b.tar.bz2
-rw-r--r-- 1 root wheel 5.4G Jul 10 19:33 archive2c.tar
-rw------- 1 root wheel 1.9G Jul 10 22:09 archive2c.tar.bz2
-rw-r--r-- 1 root wheel 2.1G Jul 10 17:44 archive2d.tar
-rw-r--r-- 1 root wheel 886M Jul 10 16:08 archive3.tar
drwxr-xr-x 2 root wheel 512B Jul 10 19:44 scripts


I use Adobe Creative Suite 3 (CS3) Web Premium every day, Flash, Fireworks, Photoshop, but most of all Dreamweaver. I consider Dreamweaver CS3 to be the best HTML/CSS visual editor available and a it's a very handy FTP client as well. It also serves as a nice text editor with syntax highlighting for PHP (although PDT for Eclipse is MUCH better). My problem is that it crashes on me almost every day with this really annoying error: Runtime Error! This application has requested the Runtime to terminate it in an unusual way. Here's what it looks like:



This error makes my blood boil! I've done everything to try it including, but not limited to:

Delete the configuration Files

This seems to give me the most success. Dreamweaver stores a file cache and some non-site related settings in C:\Users\[your username]\AppData\Roaming\Adobe\Dreamweaver 9\Configuration. If you delete or rename this directory then Dreamweaver will recreate it when it starts up.

Delete the Dreamweaver Windows Prefetch file

When a program starts up, Windows trys to optimize the startup process by creating a "cache" type file in C:\windows\Prefetch. Next time the program opens it uses this file to speed up the process. You can safely delete everything in the Prefetch folder. Doing this helped me get Dreamweaver working yesterday.

Get rid of Logitech SetPoint

Someone on an Adobe forum said this worked for them - I haven't had much success

Restart computer with only essential processes enabled

Also from Adobe, this seems to have no impact on my computer - it still crashes.

Maybe it's the 8KB file bug???

According to many people on Adobe's forum, if you a working on a file that is exactly 8,192 bytes (8KB) Dreamweaver will crash. Well, my files are taking up 8,192K because the block size on my disk pushes 7+K files up to 8,192 (look at "size on disk"), but I put a huge HTML comment at the end of the file to push it past 10K and it didn't help.

Doing the exact same thing every time will make Dreamweaver crash every time

Here's my most effective solution as of this morning: once Dreamweaver crashes, you can't do the exact same thing when you reopen it or it will just keep crashing. I was trying to add a 4x3 table to a document and everytime I did Insert->Table->OK it immediately crashed (I did this 10+ times while changing other Windows stuff). Then I went to show my business partner and instead of adding a table, I went to a different Site and changed some code around, then when back to the original site and BAM! it works fine now! What the HECK is going on here?

Final thoughts

You know, I've been working with Linux and open source software for over 10 years, and I finally break down and by a $2,000+ software package - I expect it to work! If it doesn't work I expect that I can get support to make it work (and not that "please restart your computer then uninstall/reinstall" BS)! Come on Adobe!!! If I charge my clients >$100/hr and I have to fight with Dreamweaver for 1 hr per day, where's my $36,500 from Adobe for the last year?

P.S. I think the problem is related to MS Visual Studio 2005's runtime update and Vista or something.

Although I use AMFPHP RemoteObjects with the Cairngorm Framework everyday, I never had a need for a simple LocalConnection. LocalConnections let you communicate between running SWFs, the only problem is that they are unidirectional. SWF A can make a new LocalConnection to SWF B and invoke it's methods, but SWF B can't contact SWF A. The way to get around this is to make another LocalConnection back from SWF B to SWF A. Trying to wrap my head around receiving and sending connections was starting to make me angry! They are annoyingly misleading - the receiving SWF needs to .connect() to a named connection, whereas the sending SWF doesn't - it just calls .send() with the same named connection. I figured I could do the world a favor and abstract this confusion for you.

Here is the result (demonstrated with a Flex SWF and a Flash SWF):

Flex SWF

Flash SWF

In both the Flex MXML and the Flash FLA, I am including my BiDirLocalConnection class and it does all the hard work for me.

To make this bidirectional concept easier to handle, I introduced something called Roles. There are two roles available to your SWFs: Master and Slave. You must pick one of these roles when you instantiate the BiDirLocalConnection Object. If you have more than one running SWF in a given Role you will get an error on the newest one.

Here's an example of how to use the class in Flex:


import net.teratechnologies.common.BiDirLocalConnection;

private var connection:BiDirLocalConnection;

private function init():void{
connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_MASTER,this);
connection.connect();
}


The constructor takes three arguments (the third is optional):

BiDirLocalConnection(role:String,callbackScope:Object,connectionBaseName:String="BiDirConnection")


- role: The role of this SWF (ROLE_MASTER or ROLE_SLAVE)
- callbackScope: When a SWF connects to this one and tries to invoke a method (call a function), where should it look to find the method? Normally you specify this so the remote SWF has access to the functions in your current scope.
- connectionBaseName: This optional argument is only required if you have more than one BiDirLocalConnection at the same time. You can use any string here and it will serve as the prefix for the two LocalConnection connectionNames that are used.

Here is the complete code from the Slave in Flash CS3 as seen above. There are three things on the stage: TextArea (receiveText), TextInput (sendText) and a Button (sendButton):

import net.teratechnologies.common.BiDirLocalConnection;

var connection:BiDirLocalConnection;

function init():void{
sendButton.addEventListener(MouseEvent.CLICK,send);
connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_SLAVE,this);
connection.connect();
}
function send(e:Event):void{
connection.send('showText',sendText.text);
}
function showText(t:String):void{
receiveText.text = "Received: "+t+"\n"+receiveText.text;
}

init();
stop();


Now here is the complete code for the Flex MXML file:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" creationComplete="init()">
<mx:Script>
<![CDATA[
import net.teratechnologies.common.BiDirLocalConnection;

private var connection:BiDirLocalConnection;

private function init():void{
sendButton.addEventListener(MouseEvent.CLICK,send);
connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_MASTER,this);
connection.connect();
}
private function send(e:Event):void{
connection.send('showText',sendText.text);
}
public function showText(t:String):void{
receiveText.text = "Received: "+t+"\n"+receiveText.text;
}

]]>
</mx:Script>
<mx:Panel width="100%" height="100%" layout="absolute" title="Flex 2/3">
<mx:TextArea id="receiveText" right="10" bottom="39" left="10" top="10."/>
<mx:TextInput id="sendText" left="10" right="72" bottom="10"/>
<mx:Button label="Send" id="sendButton" right="10" bottom="10"/>
</mx:Panel>

</mx:Application>


It really is quite simple, the SWFs are calling each other's "showText" functions with the text of the TextInput box as an argument. You can pass as many arguments as you want to connection.send(), and they don't need to be simple Strings.

Finally, here is the ActionScript Code for the BiDirLocalConnection class itself:


package net.teratechnologies.common {
import flash.net.LocalConnection;

public class BiDirLocalConnection{

public static const ROLE_MASTER:String = "master";
public static const ROLE_SLAVE:String = "slave";

private var txConnName:String;
private var rxConnName:String;
private var txLC:LocalConnection = new LocalConnection();
private var rxLC:LocalConnection = new LocalConnection();
private var callbackScope:Object;

public function BiDirLocalConnection(role:String,callbackScope:Object,connectionBaseName:String="BiDirConnection"){
if(role == ROLE_MASTER){
txConnName = connectionBaseName + "_TX";
rxConnName = connectionBaseName + "_RX";
}else{
rxConnName = connectionBaseName + "_TX";
txConnName = connectionBaseName + "_RX";
}
this.callbackScope = callbackScope;
}
public function connect():void{
trace(rxConnName);
rxLC.connect(rxConnName);
rxLC.client = callbackScope;
}
public function send(methodName:String,...rest):void{
txLC.send(txConnName,methodName,rest);
}
}
}


You can also download the Flex 2/3 file, Flash FLA and Class file in a ZIP file.

Quick Linux Tip:

If you're trying to delete a very large number of files at one time (I deleted a directory with 485,000+ today), you will probably run into this error:


/bin/rm: Argument list too long.


The problem is that when you type something like "rm -rf *", the "*" is replaced with a list of every matching file, like "rm -rf file1 file2 file3 file4" and so on. There is a reletively small buffer of memory allocated to storing this list of arguments and if it is filled up, the shell will not execute the program.

To get around this problem, a lot of people will use the find command to find every file and pass them one-by-one to the "rm" command like this:


find . -type f -exec rm -v {} \;


My problem is that I needed to delete 500,000 files and it was taking way too long.

I stumbled upon a much faster way of deleting files - the "find" command has a "-delete" flag built right in! Here's what I ended up using:


find . -type f -delete


Using this method, I was deleting files at a rate of about 2000 files/second - much faster!

You can also show the filenames as you're deleting them:


find . -type d -print -delete


...or even show how many files will be deleted, then time how long it takes to delete them:


root@devel# ls -1 | wc -l && time find . -type f -delete
8532

real 0m3.660s
user 0m0.036s
sys 0m0.552s