Introduction
Red Hat is one of the largest Linux distributions in the enterprise market and there are a multitude of other RPM based distributions such as CentOS, Fedora, Mandriva, SuSE. Many people are deploying their web projects into RPM based environments and it makes a lot of sense to try to do things the “Red Hat Way”. This post deals with introducing the Red Hat filesystem layout and automatically deploying a web application into it with the RPM package management tool and a YUM repository.
Filesystem Hierarchy Standard
Most Linux distributions adhere to the Filesystem Hierarchy Standard, which attempts to standardise the location of files within the filesystem. This tells us that the bootloader should live in /boot or that systemwide application config files should live in /etc for example. Our web project probably has a configuration file and probably produces log files as well as temporary files or maybe it has supporting shell scripts. It would be best practice for us to deploy these assets of our application into the relevant areas of the Linux filesystem so that logs can be rotated, temporary files cleaned out or configs backed up by other system daemons for example. Your operations team will thank you for not cluttering up the /var/www or /home file systems with inappropriate files. Imagine for example a runaway logfile filling a partition on /var/www because your web app hit a problem or your temporary files filling /home because they were not being managed by system housekeeping scripts.
Application Configuration Files
These live under /etc in a directory named after the application. Our example project is called ‘ubertool’ so the config files would live in /etc/ubertool.
Application Log Files
These should be put in the /var/log area of the filesystem again in a subdirectory named after the application, so in our case this would be /var/log/ubertool
Application Temporary Files
Files which are volatile and only have a temporary life-span should be written to /tmp or /var/tmp. The difference being that files stored in /tmp are usually cleared on a system reboot, whereas /var/tmp files should survive a system reboot. Once again we use a directory named after the application to separate our files from other system software. We’ll use /var/tmp in this instance so our temporary files will be written to /var/tmp/ubertool
Web Application Code
Source code which is interpreted by Apache lives in /var/www in a subdirectoy as before. So our PHP files will be deployed to /var/www/ubertool . In this example, our project is a Zend Framework application which contains a subdir called ‘public’. It is this public directory which will be assigned the DocumentRoot in Apache for our application. This enables us to hide all but the bootstrap code from the web root. This is useful incase of a mis-configuration of the server causes source files to be made visible as plain text for example.
Apache Configuration Files
Apache application configuration files are usually stored in /etc/httpd/conf.d/ in a file named ubertool.conf for example. The main Apache config file looks in the conf.d directory for any files named *.conf and loads them in turn. This allows us to keep our configurations away from other software which you have got deployed on the server. It is usually where you would declare a virtual host or a Directory which houses your application.
Building the Source
Here we use Phing to build our application. I’m not going to go into the basics of Phing here as it is convered elsewhere in depth. I will show you how we use config files and template files to build a package specification though.
Build Properties
We have 4 possible build environments. Development, Testing, Staging, and Production. Each environment has its own properties file which contains build specific configurations like the following (production.properties):
build.summary = "Uber Tool - Our Fantastic Tool" build.version = 1.0.3 build.release = 4 build.target = production apache.documentroot = /var/www apache.configroot = /etc/httpd/conf.d application.configroot = /etc apache.name.en = www.ubertool.com apache.style = vhost build.rpm.buildroot = build/package/
We call phing with an attribute to set which environment we want to build for:
phing -Dbuild.target=production
Writing an Apache Configuration
Using the above properties, we use a build target in phing to write out a ubertool.conf file for Apache:
<target name="apache-config" depends="check-environment"> <echo msg="Creating Apache Config File" /> <if> <equals arg1="${apache.style}" arg2="vhost" /> <then> <echo msg="configuring Apache as VHost Stye site" /> <copy file="build/apache-vhost.conf" tofile="config/${phing.project.name}.conf" overwrite="true"> <filterchain> <replacetokens begintoken="##" endtoken="##"> <token key="TARGET" value="${build.target}" /> <token key="APPCONFIGROOT" value="${application.configroot}" /> <token key="DOCUMENTROOT" value="${apache.documentroot}" /> <token key="PROJECT" value="${phing.project.name}" /> <token key="VHOST-EN" value="${apache.name.en}" /> </replacetokens> </filterchain> </copy> </then> <elseif> <equals arg1="${apache.style}" arg2="directory" /> <then> <echo msg="configuring Apache as Directory Stye site" /> <copy file="build/apache-directory.conf" tofile="config/${phing.project.name}.conf" overwrite="true"> <filterchain> <replacetokens begintoken="##" endtoken="##"> <token key="TARGET" value="${build.target}" /> <token key="APPCONFIGROOT" value="${application.configroot}" /> <token key="DOCUMENTROOT" value="${apache.documentroot}" /> <token key="PROJECT" value="${phing.project.name}" /> <token key="VHOST" value="${apache.name.en}" /> </replacetokens> </filterchain> </copy> </then> </elseif> </if> </target>
And a template file for the config file (apache-vhost.conf):
<VirtualHost *:80> <Directory ##DOCUMENTROOT##/##PROJECT##/public> SetEnv APPLICATION_ENV ##TARGET## SetEnv APPLICATION_CONFIG ##APPCONFIGROOT##/##PROJECT## SetEnv APPLICATION_NAME ##PROJECT## AllowOverride All RewriteEngine On RewriteCond %{REQUEST_FILENAME} -s [OR] RewriteCond %{REQUEST_FILENAME} -l [OR] RewriteCond %{REQUEST_FILENAME} -d RewriteRule ^.*$ - [NC,L] RewriteRule ^.*$ index.php [NC,L] </Directory> ServerAdmin dof@llgc.org.uk DocumentRoot ##DOCUMENTROOT##/##PROJECT##/public ServerName ##VHOST-EN## ErrorLog /var/log/httpd/##PROJECT##-error_log </VirtualHost>
You can see that we replace a few tokens with the properties brought into phing from the production.properties file and we write out a new apache config file into config/ubertool.conf
Building a Tarball
Packaging up all of the files into a single tarball (tar.gz) file makes handling the package easier and it a pre-requisite of the RPM build process. You may also wish to deploy this tar file somewhere useful. Thie build target looks like this:
<target name="build-tarball"> <echo msg="Creating Tarball of deployable source" /> <mkdir dir="build/package/${phing.project.name}-${build.version}-${build.release}" /> <copy todir="build/package/${phing.project.name}-${build.version}-${build.release}"> <fileset dir="."> <include name="application/**" /> <include name="config/**" /> <include name="lib/**" /> <include name="public/**" /> </fileset> </copy> <delete file="build/package/${phing.project.name}-${build.version}-${build.release}.tar.gz" /> <tar destfile="build/package/${phing.project.name}-${build.version}-${build.release}.tar.gz" basedir="build/package/${phing.project.name}-${build.version}-${build.release}" prefix="${phing.project.name}-${build.version}" compression="gzip"> </tar> <echo msg="Deleting temporary directory" /> <delete dir="build/package/${phing.project.name}-${build.version}" /> </target>
Building the RPM
Now we have a source tarball, we can look at generating an RPM file. We have a template spec file which contains the basics for a web application being deployed on a RedHat Enterprise server.
This again takes strings from the properties file and writes them into the spec file and looks like this (rpm.spec.in):
%define version ##VERSION## %define release ##RELEASE## %define name ##PROJECT## %define contentdir /var/www %define apacheconfdir %{_sysconfdir}/httpd/conf.d %define applicationconfdir %{_sysconfdir}/%{name} Summary: ##SUMMARY## Name: %{name} Version: %{version} Release: %{release} License: GPL URL: http://developer.com/projects/%{name} Group: Applications/Internet Source: %{name}-%{version}-%{release}.tar.gz BuildArch: noarch Requires: httpd, php >= 5.1.0, php-soap, php-mysql, php-pdo, php-pear, php-xml, php-gd Prereq: httpd, php >= 5.1.0 BuildRoot: ##BUILDROOT## %description ##SUMMARY## %prep %setup -q %build %install rm -rf $RPM_BUILD_ROOT mkdir -p -m0755 $RPM_BUILD_ROOT%{contentdir}/%{name} mkdir -p -m0755 $RPM_BUILD_ROOT%{apacheconfdir} mkdir -p -m0755 $RPM_BUILD_ROOT%{applicationconfdir} install -m 755 config/%{name}.conf $RPM_BUILD_ROOT%{apacheconfdir} install -m 755 application/config/application.ini $RPM_BUILD_ROOT%{applicationconfdir} cp -rp application lib public $RPM_BUILD_ROOT%{contentdir}/%{name} %clean rm -rf $RPM_BUILD_ROOT %files %defattr(-,root,root) %attr(640,root,apache) %config(noreplace) %{_sysconfdir}/%{name}/application.ini %config %dir %{_sysconfdir}/%{name} %config(noreplace) %{_sysconfdir}/httpd/conf.d/*.conf %dir %{contentdir}/%{name} %{contentdir}/%{name}/application %{contentdir}/%{name}/public %{contentdir}/%{name}/lib %post /usr/sbin/setsebool -P httpd_can_network_connect 1 RET=`/etc/init.d/httpd status` if [ "$?" -ne "0" ]; then /etc/init.d/httpd start 1>/dev/null else /etc/init.d/httpd graceful 1>/dev/null fi %changelog * Mon Mar 8 2010 Dan Field <dof@llgc.org.uk> - Local Build fixes for OS X * Fri Dec 4 2009 Dan Field <dof@llgc.org.uk> - First RPM build release
This file is our input to the RPM build process. We use a phing target to build the RPM like so:
<target name="build-rpm" depends="check-environment,apache-config,application-config,build-tarball" > <echo msg="Creating RPM Build Environment" /> <mkdir dir="build/package/SPECS" /> <mkdir dir="build/package/SOURCES" /> <mkdir dir="build/package/BUILD" /> <mkdir dir="build/package/RPMS" /> <mkdir dir="build/package/RPMS/noarch" /> <mkdir dir="build/package/SRPMS" /> <echo msg="Creating RPM Spec File" /> <!-- Apply some string substitution on the spec file and copy it to a new location --> <copy file="build/rpm.spec.in" tofile="build/package/SPECS/${phing.project.name}.spec" overwrite="true"> <filterchain> <replacetokens begintoken="##" endtoken="##"> <token key="SUMMARY" value="${build.summary}" /> <token key="PROJECT" value="${phing.project.name}" /> <token key="VERSION" value="${build.version}" /> <token key="RELEASE" value="${build.release}" /> <token key="BUILDROOT" value="${build.rpm.buildroot}" /> </replacetokens> </filterchain> </copy> <!-- copy source into the users RPM build environment --> <copy file="build/package/${phing.project.name}-${build.version}-${build.release}.tar.gz" todir="build/package/SOURCES/" overwrite="true" /> <!-- Build the RPM in the users RPM build environment --> <exec command="/usr/local/bin/rpmbuild -bb --target=noarch-redhat-linux --clean -D='%_topdir ${project.basedir}/build/package' build/package/SPECS/${phing.project.name}.spec" escape="true" passthru="true" /> </target>
This should leave you with an RPM file built into the build/package/RPMS/noarch directory within your build dir.
Deploying the RPM
These days, most of us use package tools to search, browse and update from a package repository. In the RPM world, this is usually a YUM repo, and it makes a lot of sense to run your own local repo for locally developed applications. We simply SCP (with SSH keys) the RPM to our repo, where cron looks for new additions and regenerates the package index every few minutes. This tutorial goes into more detail on the setup of a local repo. Naturally, we use a phing target to to this too:
<target name="deploy-rpm" depends="build-rpm"> <exec command="/usr/bin/scp build/package/RPMS/noarch/${phing.project.name}-${build.version}-${build.release}.noarch.rpm repo.develper.com:/var/yum-repo-developer/" /> </target>
And that’s it! When you have made changes to your code which you want to deply, simply open the properties file in a text editor, and modify the version and release numbers to your desired values, and run:
phing -Dbuild.target=production deploy-rpm
RaiyaRaj
June 15th, 2010
This is really beautiful way of deployment. I like it.
Jason Gordon
July 20th, 2010
Nice tutorial. Really appreciate your effort on this.