SSH for Web Developers: Transfer Sites Fast

SSH for Web Developers: Transfer Sites Fast
http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/digg_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/reddit_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/dzone_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/stumbleupon_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/delicious_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/blogmarks_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/newsvine_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/google_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/myspace_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/facebook_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/yahoobuzz_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/sphinn_16.png http://www.nurelm.com/themanual/wp-content/plugins/sociofluid/images/twitter_16.png

Imagine this typical scenario: You have a client who needed work to be done in a development environment, so you setup a new cPanel account for them, did the work, tested it out, and now you need to move your work to the live server.  This involves a lot of things that are fairly slow to do using an FTP (or SFTP) client, phpMyAdmin, and transfers to your local machine followed by transfers to the live server.

A little command line magic will help this process go much faster, and will increase your Web Geek superpowers immensely. Unlike using GUI clients for every step of the process, understanding a few command line basics gives you nearly infinite flexibility, and will boost your speed 10x for scenarios such as that described above.  Here’s a quick walk through of how to handle that scenario on a typical Linux server setup (in this case using the paths you often see when your server has WHM / cPanel managing hosting accounts).

First, get your files and database ready on the development server.  SSH into the server (if you don’t know how to SSH into a server, visit this site for more information: www.liquidweb.com/kb/new-user-tutorial-basic-ssh) , and navigate to the directory where your files are located:

[localmachine] ~ > ssh devserver Password: 
Last login: Wed Nov  7 09:38:56 2012 from 151.201.108.169

[devserver] ~ > cd /home/mysite/www
[devserver] /home/mysite/www >

Now tar/gzip all of your files, then grab a dump of your database, gzip it as well, which results in a grand total of 2 files that you’ll have to transfer:

[devserver] /home/mysite/www > tar zcvf files.tar.gz . 
... lots of output as tar shows you files ...
[devserver] /home/mysite/www > mysqldump -hdbserver -udbuser -pdbpass dbname > db.sql
[devserver] /home/mysite/www > gzip db.sql

Here’s a quick review of what those commands actually did.  The first combined all the files in the current directory (and directories and files in those directories, etc) into one big archive, then compressed that archive, and saved it all in a file called files.tar.gz.  The 2nd line dumped your database (you’ll need to fill in dbserver, dbuser, dbpass, dbname), and the 3rd just compressed that db.sql file into a new file called db.sql.gz.

Now that you’ve done everything you need to do on the development server, open a new SSH window and log into the live server:

[localmachine] ~ > ssh liveserver Password:

Last login: Wed Nov  7 09:38:56 2012 from 151.201.108.169

[liveserver] ~ > cd /home/mysite/www
[liveserver] /home/mysite/www >

It’s a good idea to first move your old files out of the way, and to backup your database before you overwrite everything. The following will create a backup directory, move all your files into it, and backup your database:

[liveserver] /home/mysite/www > mkdir backup [liveserver] /home/mysite/www > mv ./* backup 
[liveserver] /home/mysite/www > mv ./.* backup 
[liveserver] /home/mysite/www > mysqldump -hdbserver -udbuser -pdbpass dbname > backup/db.sql 
[liveserver] /home/mysite/www > gzip backup/db.sql

Note those 2 separate mv commands … one moves regular files, and the other moves hidden files.  If you want to make sure you got everything, do an “ls -la” to view everything still in your directory.  When you’re satisfied that your backup is complete (and please make totally completely for sure that it is), then grab those files you just created on the development server with wget …

[liveserver] /home/mysite/www > wget http://devserver/files.tar.gz 
... wget output as it retrieves the file ...

[liveserver] /home/mysite/www > wget http://devserver/db.sql.gz 
... more wget output ...

We’re getting close now.  Next, uncompress those files and install your new database …

[liveserver] /home/mysite/www > tar zxvf ./files.tar.gz 
... lots of tar output ...
[liveserver] /home/mysite/www > gunzip db.sql.gz 
[liveserver] /home/mysite/www > mysql -hdbserver -udbuser -pdbpass dbname < db.sql

At this point, you are done with the essential stuff.  A few cleanup issues remain, though.  To save on disk space and avoid having a bunch of junk setting in your directories, you’ll want to delete those files you created on your development server and the live server.  Just use the “rm” command.  And, you might need to deal with the issue of how some software packages store your URL in their database, making the new site at the new location misbehave.  In order to do this, just open “db.sql” in the text editor of your choice (I prefer VIM because I’ve been using it for decades, it’s everywhere, and it’s insanely powerful for this sort of thing), and replace things like “devserver” with “liveserver”, then reimport your database as shown in the last line above.

About the Author

Sam is one of NuRelm's founders. After a few years designing computer chips for organizations the size of small countries, he was tempted by the irresistible lures of starting a business: more work, less salary, and the ability to be as creative as he pleased provided the bills still got paid. This has worked out surprisingly well over the past 13 years, resulting in a successful company and a lot of happy clients. Sam's electrical engineering background and years of Web application development experience, combined with the wonders of an MBA, allow him to perform feats such as installing wiring while expounding on the viability of selling Web-based software to simplify the process.

Author Profile: