Bundling Javascript

Javascript is becoming increasingly popular. Frameworks like jQuery, MooTools and Prototype/Scriptaculous have opened up a wide array of possibilities to webdevelopers. As a result, many webpages depend on a series of Javascript source files to be included. All these Javascript files are retrieved through separate HTTP requests, which are - by concept - very slow and will increase the loading time of your website. This article will describe my approach to solving this.

Bundling your Javascript

First of all, I started by combining all Javascript files into 1 request. As you can see in the source of this page, I depend on several script but request all of them through one script tag:

<script type='text/javascript'
src='core/jsmin/bundle.js?js=js/mootools-core.js,js/mootools-more.js,js/monkey.js,js/tooltipinside.js&amp;m=4948599067'>
</script>

It is not hard to understand what's going on here: the bundle.js request will simply concatenate all the scripts passed in the js GET parameter. As such, the above script tag will include the files:

• js/mootools-core.js
• js/mootools-more.js
• js/monkey.js
• js/tooltipinside.js

This bundling is not done by a physical bundle.js file, but through PHP. This is achieved by rewriting the request from bundle.js to bundle.php through .htaccess:

RewriteEngine on
RewriteRule ^bundle.js$ bundle.php [QSA,L]

The actual PHP code to bundle the scripts should not be a challenge for you. It is basically just looping through the $_GET['js'] files and outputting the referenced files, which would be accomplished by:

$files = explode(',', $_GET['js']);
foreach ($files as $file) {
	readfile($file);
}

However - because PHP is used to manage the output, we can add a few additional optimizations that will help the process even more.

Compressing (minifying) the Javascript

Instead of outputting the source Javascript we can save some bandwidth by simply minifying the Javascript before sending it to a client. I used the excellent jsmin-php open-source project. Minifying will (among other things) strip whitespace and comments, and might additionally shorten small scoped variable names without affecting any of your code flow. While most Javascript frameworks will be already compressed, your own code may easily be reduced in size by around 30%.

include "JSMin.php";

$files = explode(',', $_GET['js']);
foreach ($files as $file) {
	echo JSMin::minify(file_get_contents($file));
}

I would also strongly recommend you to verify if the requested $file has a .js extension to prevent users from requesting other files.

Server-side caching

At this point, your bundle.php script will have a (possibly) heavy and boring job. Most of the time it will simply concatenate and minify the same Javascript source files over and over again. To reduce the load at this point, I've implemented a caching system that will simply dump the ouput into a cached .js file, which will be served the next time an identical request is received. The name of the cache file is a hash of all files requested.

The bundle.php script will thus first calculate a cache filename based on the input parameters. If this cache file exists, it may simply serve the file from cache - if not, it will have to generate the output, and create the cache itself. To avoid caching of files that have been changed between requests, I also use the "file modified time" (filemtime() in PHP) in my cache filename generation:

$cache = '';
foreach ($files as $file) {
	$cache .= $file;
	$cache .= filemtime($file);
}
$cache = 'cache/'.md5($cache);

Client-side caching

However, this will still generate bandwidth every time the Javascript is requested. Because of all the parameters and missing response headers (we have implemented none yet!), most browsers will not risk caching the response, so they request the Javascript over and over again. This is why we need one more additional piece of code: support for client-side caching.

First of all, since the Javascript source files may have changed between requests, we need a way to force the browser to re-request the file if the signatures of any of the files has changed. This is where the $_GET['m'] parameter is for (refer to <script> tag example above). The $_GET['m'] is the sum of all file modification times (available through the PHP function filemtime()). When this parameter changes, any browser will simply request the file again, since it has a different "name".

Secondly, we will start responding with corrent MIME-type and Status headers, which allows us to respond with a "Status 304 (not modified)" when 2 identical requests are received from the same client. Instead of clouding you with words, I'll just show my code and talk you through:

// calculate last-modified & etag caching headers
$last_modified_time = filemtime($cache);
$etag = md5_file($cache);

header("Content-type: text/javascript");
header("Pragma: public");
header("Cache-Control: public");
header("Last-Modified: ".gmdate("D, d M Y H:i:s", $last_modified_time)." GMT");
header("Etag: ".$etag); 

if (@strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) == $last_modified_time || trim($_SERVER['HTTP_IF_NONE_MATCH']) == $etag) { 
	header("HTTP/1.1 304 Not Modified");
	exit;
}

This code is pretty straightforward. In the end, it will simply send a 304 header and stop processing the script (which would otherwise output all Javascript source) when a request is received from a browser that supports caching (and as such sends the HTTP_IF_MODIFIED_SINCE and HTTP_IF_NONE_MATCH request headers is response to our response headers which include the Last-Modified and Etag headers). I hope you understand that last sentence. If not, read it again, it makes sense!

Complete code

This is the complete code that I finally came up with for bundle.php:

<?php
	
	include "JSMin.php";
	$path    = "../../";
	$files   = explode(",", $_GET['js']);
	$missing = array();
	
	$cache = '';
	foreach ($files as $index => $file) {
		if (strtolower(substr($file, -2)) != 'js') {
			unset($files[$index]);
		} else if (file_exists($path.$file)) {
			$cache .= $file;
			$cache .= filemtime($path.$file);
		} else {
			$missing[] = $file;
			unset($files[$index]);
		}
	}
	
	$cache = 'cache/'.md5($cache);
	
	if (count($missing)) {
		header("Content-type: text/javascript");
		header("Pragma: no-cache");
		header("Cache-Control: no-cache");
		echo "alert('Could not load the following javascript source files:\\n\\n- ".implode("\\n- ", $missing)."\\n\\nJavascript not loaded / running!');";
		exit;
	}
	
	if (count($files)) {
		// create cached version if not present
		if (!file_exists($cache)) {
			$js = '';
			foreach ($files as $file) {
				$js .= JSMin::minify(file_get_contents($path.$file));
			}
			
			file_put_contents($cache, $js);
		}
		
		// calculate last-modified & etag send caching headers
		$last_modified_time = filemtime($cache);
		$etag = md5_file($cache);
		
		header("Content-type: text/javascript");
		header("Pragma: public");
		header("Cache-Control: public");
		header("Last-Modified: ".gmdate("D, d M Y H:i:s", $last_modified_time)." GMT");
		header("Etag: ".$etag); 
		
		if (@strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) == $last_modified_time || trim($_SERVER['HTTP_IF_NONE_MATCH']) == $etag) { 
			header("HTTP/1.1 304 Not Modified");
			exit;
		}
		
		readfile($cache);
	}

Tips, feedback and questions are welcome.