scp foo user@host:~/bar
# To copy a folder use the prefix -R foo/, like this
scp -R foo/ user@host:~/

ssh -l user host -D 8080

ssh -l user host -L 8080:localhost:80

Host foo
        User root
        HostName host
        IdentityFile ~/.ssh/otherkey
        Compression yes
        CompressionLevel 9
        KeepAlive Yes
Host bar
        User foo
        HostName host

First we need to enable some modules:

a2enmod rewrite vhost_alias
/etc/init.d/apache2 restart

Next up we do some rewrite magic, you need to change some paths and domains to reflect your setup, personally I just use some search and replace in nano to reflect whatever domain and site I’m setting up.

<VirtualHost *:80>
ServerName eksempel.no
ServerAlias *.eksempel.no #wildcard catch all
VirtualDocumentRoot /var/www/%1
UseCanonicalName Off
IndexOptions FancyIndexing
### Use mod_rewrite to direct eksempel.no to www.eksempel.no
RewriteEngine On
RewriteCond %{HTTP_HOST} ^eksempel.no
RewriteRule (.*) http://www.%{HTTP_HOST}$1 [R=301,L]
### Logging
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog /var/log/apache2/access_log_eksempel combined
<Directory /var/www>
AllowOverride None
</Directory>
</VirtualHost>

You must also restart Apache2 for your changes to apply.

/etc/init.d/apache2 restart

You might have to do small adjustments to stuff like users and parent directory. In this guide i use /root/ as parent directory for most actions, and all commands should work then.

Upgrading and Installing

It’s important that you do this on your target server and the server you are backuping, or else rdiff-backup will not be able to use ssh as it starts an rdiff-server on your target machine.

apt-get update
apt-get install rdiff-backup

Setting up SSH with keys

After installing rdiff we need to generate ssh-keys so that you can have automated rdiff-backups. I’ll only cover this briefly, but if you are unsure try to consult another guide i made about setting up key-based ssh authentication.

ssh-keygen
scp .ssh/id_rsa user@host:.ssh/id_rsa

This generates the keys you need and send them to the appropriate user on your remote machine for storing backups. If you are setting up more servers to use the same backup location copy id_rsa.pub over to them.

Using rdiff-backup

rdiff-backup -v5 –print-statistics /local-dir user@host::/remote-dir
rdiff-backup –remove-older-than 2W user@host::/remote-dir

Now we have created the first and initial backup, this might take some time, depending on your connection speed, disks and CPU. You should be able to watch what’s stores since we issued the -v5 comand (verbos 5).

Automating it

echo “#!/bin/sh” >> rdiff-backup.sh
echo “rdiff-backup -v5 –print-statistics /local-dir user@host::/remote-dir” >> rdiff-backup.sh
echo “rdiff-backup –remove-older-than 2W user@host::/remote-dir” >> rdiff-backup.sh

Running it

crontab -e

And insert the following line

* 0 * * * /root/rdiff-backup.sh

Save it! Now you can just sit back and relax and let ‘em backups flow, each night at twelve o’ clock.

Restoring

Maybe the most boring part but also the most important part, restoring. When you restore with rdiff-backup you can restore to any date you want, as long as it’s still being kept. In this guide we used 2W, or two weeks, so we should be able to restore a file 14 days back. Nifty!

rdiff-backup -r 14D user@host::/remote-dir/file /local-dir/file

This will restore the 14 days old file (-r 14D) of “file” to “/local-dir/file”. Pretty nifty, ey? If you are doing disaster recovery you can also recover an entire directory.

Final words

Do some simple tests, like backing up and restoring. This way you don’t get caught with your pants down when you actually need to restore something, and you ensure that backups actually are working – as this guide is not fully tested nor foolproof.

All the benchmarks in this article is based on Apache Benchmarking (ab) tool, so the numbers will not translate directly to a production environment. This was all done locally on a virtually hosted Debian Lenny (5.0.4) on VMWare Fusion. You should see much higher numbers than this in a real world scenario. If not you should highly consider buying a new server. The virtual server was running on a single 2,4Ghz shared core, with 1024MB virtually allocated memory and 8GB virtual drive.

For the actual tests i make ab run a series of 1000 requests with 10 concurrent requests at a time. In order for it to be a fair fight I run all tests twice. The page that is requested is a Pressflow-installation with a story consisting of three paragraphs of random bits and bytes, in total the document is 6755 bytes (very small). Apache is set up to respond to rewrites too, as most servers are.

My main configuration consists of Apache2, php5, phpmyadmin and mysql-server. In a production environment you might not need packages such as phpmyadmin and mysql-server, this is just installed for testing purposes. For bleeding edge packages i suggest taking a look at dotdeb, but i cannot vouch for the stability of said repository.

apt-get install apache2 php5 phpmyadmin mysql-server

With this fairly plain and simple setup up and running it’s time to crack some numbers. What we are focusing on is average request time, requests per second and time per request. On a basic Apache2 setup with default configuration and MPN Worker we get these numbers. You find the whole output at http://static.vvvegard.net/files/benchmarking/.

Apache2 Default Configuration
Time taken for tests:   97.409 seconds
Requests per second:    10.27 [#/sec] (mean)
Time per request:       974.094 [ms] (mean)

From this we can read that we get around 10 req/s. This is not bad – considering the setup – but you’d want to see a lot more if you are to survive a sudden traffic rush, keep in mind that most of a sites traffic hits within a short time. Also, it took a whopping 974ms (on average, 10 concurrent!) to load each page. That almost 1 second, and that’s not counting in other factors such as network latency and the rendering speed of your visitors browser. This is just to actually get the damn data!

Moving on we’re going to try out some of the most popular opcode cachers for PHP. First of is my personal favorite, as before of writing this article, APC – Alternative PHP Cache.

apt-get intall php-apc
apt-cache show php-apc

As the last command shows us we’ve installed “Version: 3.0.19-2″ of php-apc. This is the latest version of APC in the repository as of writing this. APC is very minimalistic in it’s design and does not require a lot of configuration to get it working. But we add the following line in order to beef it up a bit, you might want to experiment with this, depending on how many sites you host or how heavy the scripts are. Numbers are in MB, duh.

echo "apc.shm_size = 128" >> /etc/php5/conf.d/apc.conf
apache2ctl restart

Now that we’ve got that settled we move on to the actual benchmarking. As with Apache2 default configuration you can access the full benchmark athttp://static.vvvegard.net/files/benchmarking/. You will also find a sample configuration here.

APC Default Configuration
Time taken for tests:   26.041 seconds
Requests per second:    38.40 [#/sec] (mean)
Time per request:       260.414 [ms] (mean)

We see a great improvement in time taken, requests served per second and the time it took per request. As we can see it’s almost 4 times faster with little to none actual work done.

Moving on we’re going to test a bit more advanced opcode cacher, XCache. It’s actually a side project from the super snappy and lightweight web server lighttpd but since is’s made for PHP it works with Apache2 to.

apt-get install php5-xcache
apt-cache show php5-xcache

The latest stable release in the repository is “Version: 1.2.2-3″. Like with APC I  did little modifications to the configuration shipped with Debian. Please note that XCache is a lot more flexible and secure because of it’s design; but also more complex and easier to break.

nano /etc/php5/conf.d/xcache.ini
Edit xcache.size to 128M

Just as with the others you find the complete benchmark at http://static.vvvegard.net/files/benchmarking/ and configurations in http://static.vvvegard.net/files/benchmarking/config.

XCache Default Configuration
Time taken for tests:   25.523 seconds
Requests per second:    39.18 [#/sec] (mean)
Time per request:       255.233 [ms] (mean)

As we can see it’s only slightly faster than APC with a pretty basic configuration, but you should be able to squeeze a little more juice out of it by tuning the configuration a little more. Not groundbreakingly much faster than APC, but I guess the added flexibility might make XCache a better choice than APC.

Moving further along we hit the first obstacle, eAccelerator. It’s not in Debian Lenny repository, so we have to compile it from source. This will probably make eAccelerator much more up to date and some of the difference shown might be because of this. There are also people who have made their own packages of this, use at own risk!

First we need to install some essentials for building a package form source. I will not get into detail in how to install and configure eAccelerator but if you run into any problems be sure to drop a line. These commands should do the magic! Please note that due to restrictions, (that you can remove) the maximum allowed shared memory under Linux Kernel 2.6 is 32MB so we run it on 32MB. This should not show any effects during this test as it’s a small site and not that much to actually cache, in bytes.

apt-get install build-essential autoconf automake libtool m4 php5-dev
wget http://bart.eaccelerator.net/source/0.9.6/eaccelerator-0.9.6.tar.bz2
tar -xjvf eaccelerator-0.9.6.tar.bz2
phpize5
./configure
make; make install
cd /etc/php5/conf.d/
wget http://static.vvvegard.net/files/benchmark/config/eaccalerator.ini
mkdir /var/cache/eaccelerator
chmod 777 /var/cache/eaccelerator
apache2ctl restart

Like with XCache eAccelerator has a lot more flexible features than APC. There is a lot of tuning that can be done if you take your time. As with the rest you’ll find the complete benchmark at http://static.vvvegard.net/files/benchmarking/ and configurations at http://static.vvvegard.net/files/benchmarking/config.

eAccelerator Default Configuration
Time taken for tests:   24.341 seconds
RRequests per second:    41.08 [#/sec] (mean)
Time per request:       243.408 [ms] (mean)

Here we see a marginal speed increase from both XCache and APC, but this might just be because if runs the latest available version of eAccelerator as of writing. You might also tweak eAccelerator a lot for more juice by visiting their wiki to get some good hint’s and tips. When it comes to security I guess eAccelerator is in for a beating, from my point of perspective you should not run this in a multi-user environment as users potentially can access and read each others PHP-files.

Moving yet again along we’re going to put php5-memcache up to the tests. Php5-memcache is a lot like APC with fewer possible configurations than XCache and eAccelerator. And like the others you find the complete benchmark at http://static.vvvegard.net/files/benchmarking/ and configurations in http://static.vvvegard.net/files/benchmarking/config. We’re using the default configuration shipped with Debian, no modifications.

apt-get install php5-memcache
apt-cache show php5-memcache

We’re getting “Version: 3.0.1-1″ of php5-memcache. This seems to be quite new as far as I could see. And that makes what comes next even more depressing.

php5-memcache Default Configuration
Time taken for tests:   98.439 seconds
Requests per second:    10.16 [#/sec] (mean)
Time per request:       984.391 [ms] (mean)

Woha! It’s actually worse than native PHP. This might be because of the configuration, but my gut feeling tells me there is something more in the grass, the server load went trough the roof with php5-memcache.

To sum it all up, most OPCode cachers do a really good job in both speeding up you requests and makes your server scale a little more than it does without any caching capability. Running a cacher is highly recommended as it reduces hits on your disks and makes your server scale a lot better. The tests also showed that most cachers make the server peak 4x as much on requests per second, witch should do the trick if you experience slowdown during peak hour on your server. In a combination with Varnish and a fine tuned setup you should be able to get around 150-200 req/s on backend servers while serving 1500+ req/s from Varnish, with a 90% cache-hit rate.

Next time I’ll make some nice graphs with rrdtool too

Please note that this is my first encounter with Nginx, so bare with me if I do some weird stuff. I try to document whatever I am doing, mostly to educate my self. You should not use any configuration found in this article in a production environment if you are not one hundred percent clear what it does and why it does it.

In this article we’ll install nginx from source as the latest stable release in Debian Lenny is very old. Please note that this might not be the case with Squeeze/SID, but in my articles I focus on using the latest stable release of Debian.

apt-get install build-essential libpcre3-dev libpcre3 libssl-dev zlib1g-dev

This will install all essential build utilities that you will need to build a program from source. In addition to this we install some libraries to support SSL and PECR to support rewrites.

cd /root
wget http://nginx.org/download/nginx-0.7.65.tar.gz
tar -zxvf nginx-0.7.65.tar.gz
cd nginx-0.7.65

Please go to nginx.org and fetch the latest stable version of Nginx. As of writing this 0.7.65 is the latest stable release. We download it, untar it and move in to the folder. You probably knew this (4)

./configure --prefix=/usr/local /
--sbin-path=/usr/local/sbin /
--conf-path=/etc/nginx/nginx.conf /
--pid-path=/var/run/nginx.pid /
--error-log-path=/var/log/nginx/error.log /
--http-log-path=/var/log/nginx/access.log /
--with-http_ssl_module /
--with-debug

This tells the compiler where to put stuff and what modules we want to configure it with or without. Please refer to nginx wiki if you want more prefixes. If you encounter any errors, read them and install whatever it is missing. Use some good old fashion Google-Fu! And please let me know in a comment if I made a Boo-Boo. (3)

make; make install

This uses the information given to it when we configured it and installs nginx. Normally this should go by without any errors as the step above should have found any short comings.

cd /etc/init.d/
wget http://static.vvvegard.net/files/nginx/nginx
chmod +x  nginx

This is a modified version of the startup script to comply with the special configuration we use. The original script can be found at articles.slicehost.com. With this startup script you should be able to start, stop, restart or just reload it. I recommend you to visit the site that i snagged this startup script from, they have a lot of good articles on nginx and hosting in general! (2)

/etc/init.d/nginx start

This starts nginx and you should now be able to access it via your favorite browser.

Server: nginx/0.7.65

This is the header that nginx puts out, and this verifies that nginx is actually serving us pages. I prefer to use Live HTTP Headers add-on for Firefox to inspect HTTP Headers.

cd /etc/nginx
mv nginx.conf old_nginx.conf
wget http://static.vvvegard.net/files/nginx/nginx.conf

This replaces the standard configuration to a modified version that comply with the standards of Debian. This is fairly basic and should work in most cases. (1)

mkdir sites-enabled/
cd sites-enabled/
wget http://static.vvvegard.net/files/nginx/000-default

Again, to comply with Debian standards we store all of our virtual hosts in it’s own folder and instead of including everyone manually we include everything in that folder with wildcard. (1)

/etc/init.d/nginx restart

Now this won’t do much, it will basically just serve static files located in “/var/www”. If we’d want PHP you’d have to either do it via fastcgi or use nginx as a frontend for Apache2, primarily for caching and compressing and let Apache2 handle all dynamic data.

Resources:
(1) http://www.ubuntugeek.com/using-nginx-as-a-reverse-proxy-to-get-the-most-out-of-your-vps.html
(2) http://articles.slicehost.com/2007/10/19/debian-etch-installing-nginx
(3) http://wiki.nginx.org/Main
(4) http://nginx.org/

Due to the strict rules of Debian many of the packages in stable releases are very old. In some cases though we want to run the latest version of a program, mainly due to new features and increased speed. The latest version of Varnish under Debian Lenny’s repositories is 1.1.2 (apt-cache show varnish), but in most cases we’d want to run the latest, especially with rapidly developed programs like Varnish. Varnish 2.0.6 is alos available in the Debian repository for SID/Squeeze or via backports.

apt-get install build-essential

Now we got most of the essential stuff for building the packages under Debian Lenny.

cd /root
wget http://sourceforge.net/projects/varnish/files/varnish/2.0.6/varnish-2.0.6.tar.gz/download
tar -zxvf varnish-2.0.6.tar.gz
cd varnish-2.0.6/

Now we have fetched the (as of now) latest stable version of Varnish from the developers them selves. Moving on we configure it and install it.

./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var/lib --mandir=/usr/share/man
make
make install

This tells the compiler where to put stuff and then installs it. We’ve chosen most of the common places on Debian Lenny. This may vary on different operating systems. If you get any errors read them and install whatever it is missing. This should not be necessary if you remembered the first step.

 ldconfig

This tells Debian to update the location of the libs. This is vital to get Varnish to work on Debian Linux. Failing to do this step will result in a “file not found” when trying to start Varnish. (2)

cp redhat/varnish.sysconfig /etc/default/varnish

This is the main varnish configuration file. This has nothing to do with how Varnish handles request but how the daemon is run. If you are unsure how to configure this just run with “Alternative 2″.

DAEMON_OPTS="-a <out-side IP>:80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -u varnish -g varnish \
             -s file,/var/lib/varnish/varnish_storage.bin,512M"

This is a pretty basic setup that listens to port 80. The control daemon runs on localhost:6082. This you can telnet into and get various stats, load/unload configuration files when Varnish is running. Very useful tool when tuning your Varnish configuration. Also, this setup uses a 512MB cache-file, this might be too high or to low for your site, depending on how much memory you have avaliable and how much content needs/can be cached.

nano /etc/varnish/default.vcl

This is where the main configuration is done. You should set it up with atleast one backend server. Personally i run Apache2 with appropriate tools as backends, but you may run any web server of your choice. Try running your backend servers locally bound to port 80, this way you can trick them into not rewriting url when Varnish forwards the requests to them. If you do this remember to bind Varnish to your out-side IP-address, or else you will not be able to bind two applications to the same port.

/usr/local/sbin/varnishd -f /etc/varnish/default.vcl

This will start Varnish, telling it to use the configuration file located in /etc/varnish/. If you did all your steps right you should be seeing something like this in your HTTP headers. You should make a startup-script for this if you are running it on a production server. (1)

X-Varnish: 1861896702
Age: 0
Via: 1.1 varnish

Now we’ve got a fully functioning setup of Varnish. Before you run this on a production site you may want to consult with the developers of Varnish to tune the installation to your needs. You should also make varnish run under it’s own user and group. Varnish does this by default but you have to actually add the user first.

adduser varnish --disabled-login --no-create-home

This should add the user.

Resources:
(1) http://www.all2e.com/Ressourcen/Artikel-und-Fallstudien/Varnish-installation-and-setup-on-ez-publish-based-systems
(2) http://varnish-cache.org/changeset/4420
(3) http://varnish-cache.org/
(4) http://kristianlyng.wordpress.com/