View Full Version : How do you compress (and operating) JS code on release?

1 May 2008, 5:27 AM

When developing large application, you have big amount of .js files and index.jsp(php, asp) looks like this:

<script src="mymodule1.js"...></script>
<script src="mymodule2.js"...></script>
<script src="mymodule3.js"...></script>

I'm developing application that have at least 150 such Javascript files (each is from 100 to 2000 lines of code). It takes additional time for browser to load all this files.
It no problem on developer machine, but on production server it will be good to compose all this files to one big file, and compress it using utility like Yahoo compressor.

Utility to compress Javascript files must scan my index.jsp, find all js files, put their in one big file and compress it. Then replace all <script> tags in index.jsp with one <script> tag with composed and compressed file.

1. Do you faced (or solved) this problem?
2. Is there are some tools to automate it, or you wrote your own?
3. Or you maybe using other approach to compose and compress a lot of js files?

I'm using Netbeans as IDE.

1 May 2008, 5:33 AM
I suggest nightly builds (minify) of your CSS and JS instead of adhoc. Which speeds up the webserver,etc.

YUI has a great compression utility.

1 May 2008, 5:58 AM
Thanks for fast reply!

Sure, I mean that it (composing and compression) will be on build stage. Only once, not for every browser request.

How to achieve this on build process?

1 May 2008, 5:59 AM
it depends on your platform. Windows, task scheduler. On unix, you can use cron.


1 May 2008, 6:15 AM
It is not problem to adjust sheduler.

The problem is how to compose Javascript files on build process. I found Animal post using ANT in build process:

I need to learn how using ANT on build Ext Application and provide step-by-step guide for others.

1 May 2008, 9:45 AM
I'll be interested to see what you are able to put together (especially if you're developing on windows box). This might help:


Make sure to read the "see also" link

1 May 2008, 11:58 AM
I have a similar method, I have this php file, that searches between <!--JS Compile Start--> and <!--JS Compile Stop--> comments in HTML, and concenates the files. (just call the js_compile function)

function js_include_parse($js_includes){

$xml_inc = '<?xml version="1.0"?><xmlroot>'.$js_includes."</xmlroot>";

if (! ($xmlparser = xml_parser_create())){
die ("Cannot create XML parser");

xml_set_element_handler($xmlparser, "js_start_tag", "js_end_tag");

xml_parse($xmlparser,$xml_inc, true);


global $scripts;

return $scripts;

//Using Tutorial from http://www.devpapers.com/article/61/5

function js_start_tag($parser, $name, $attribs) {
if (is_array($attribs)) {
if (isset($attribs["SRC"])){
global $scripts;
// echo $attribs["SRC"]."<br>";
array_push($scripts, $attribs["SRC"]);

function js_end_tag(){}

function concatenate_js($array,$dir){
$merged_js = "";
foreach($array as $file){

$merged_js .= "\n //JS File: $file \n ".file_get_contents("$dir/$file");
return $merged_js;

function js_compile($file){
$filedata = file_get_contents($file); //load file
$js_includes = get_string_between($filedata,"<!--JS Compile Start-->","<!--JS Compile Stop-->");

$script_array = js_include_parse($js_includes);
return concatenate_js($script_array,dirname($file));


Then you can just make an ajax bookmarklet that compiles (I have a "dev" toolbar menu on my app in development.). It may work better than saki's regexp method (since it runs the files through a full XML parser to extract src tags... but i donno)

1 May 2008, 1:14 PM
Im using packtag.


It unifies and minify all js files in a servlet (with cache). No need to nightly build or doing thing manually. Just a couple of tags in jsp and it does all the work.

Currently im having some trouble with encoding, but the author tried it out and said he had no problems so it might be a bug in my project

1 May 2008, 2:15 PM

Does JS and CSS, and will do it on the fly, and also cache the results.

PS: Combine with CSSTidy: http://csstidy.sourceforge.net/

6 May 2008, 6:42 PM
I've written things for exactly this reason though unfortuantely current company is very IP restrictive.

One of the problems I found even with compressing and minifying is that you are still delivering a crapload of data to the client on their first load (and each subsequent uncached request) - even though they probably aren't actually using the entire gamut of scripts at the same time. The best solution Ive found is a hybrid of both the on-demand (cached) concatenation, minification and gz compression, as well as an on demand client side loader which simply loads "groups" of scripts as it is requested - and only if it hasn't previously be loaded.

6 May 2008, 6:46 PM
That is ideal, though you need to watch out for duplicate dependency issues. In addition, the nice part of that type of plan is that after you do the upfront download in order to paint the page, you can start preloading things into the users browser cache, even if they have yet to request something that requires it. Or into Google Gears, or whatever, for later downloads.

6 May 2008, 7:05 PM
my method:

1)define all js files and loading order in a package.txt file like this:


2)use java code to find all the package.txt files, open and read js file defined, then concatenate them into one js file.

3)use ant target to compress the js file like this:

<target name="gen_app.js" description="" depends="js.concatenate">
<apply executable="java" parallel="false">
<fileset dir="public/js" includes="app-debug.js"/>
<arg line="-jar"/>
<arg path="WEB-INF/lib/yuicompressor-2.3.5.jar"/>
<arg line="-o"/>
<mapper type="glob" from="*.js" to="*-min.js"/>
<copy file="app-debug-min.js" tofile="public/js/app.js" />
<delete file="app-debug-min.js"/>

sorry for my poor English.

6 May 2008, 8:35 PM
Okay I've spent a bit of time stripping out some of the more proprietry stuff so I can provide an example of the preloading. I can't actually provide my class that handles the compression, concatenation and minification as I'm sure bossman would flip, but this should at least provide a few shortcuts. This uses EXT's core stylesheet to provide ajax loading throbber feedback and masking to the user; though you could easily adjust to taste.


* Javascript ResourceManager definition inc preloaders
* @author Adam J Benson
* @todo Add in the CSS on demand loaders
* @version 1.3
* EXAMPLE PACKAGE DEFINITION: (All packages should be defined prior to trying to load them)
* var pack={
* name: "ExamplePack",
* description:"This is an example 'package object' which is used by the resource managers",
* scripts:["url1","url2","url3","url4"],
* callback: function(){},
* requires:["packageName"] //Dependancies for this package to be loaded before it...
* }
* EXAMPLE USAGE (with callback)
* ScriptManager('Common', function(){
* var simple = new Ext.FormPanel();
* });
* EXAMPLE USAGE (without callback)
* ScriptManager('Common');

var ScriptManager = window.ScriptManager = function( packageName, callback ) {
* Constructor

ScriptManager.prototype = {
registeredPackages:[], //List of packages that have been registered
loadQueue:[], //Queue of packages that have been requested to load (we load sequentially to avoid dependancy issues)
processing: false,

register: function(packageConfig){
//Register the package for loading

load: function(packageName, callback) {
* Load the specified package, with an optional callback (in addition to the callback configured by the packages config)
//Package Not Found
console.log("SPCore Notice: The package "+packageName+" has not been registered but was requested by a script.");
//Already loaded... Trigger the scripted callback
//Required Scripts
var required=this.registeredPackages[packageName].requires
for(var i=0, m=required.length; i < m;i++){
//There is a dependant not loaded, so add self to the queue and load it instead...
//Already proccessing a script (or document body hasn't finished loading)! Add it to the queue...
ScriptManager.srcScript(this.registeredPackages[packageName], callback);
genScriptNode : function() {
var scriptNode = document.createElement("script");
scriptNode.setAttribute("type", "text/javascript");
return scriptNode;
srcScript : function(packageConfig, callback) {
var scriptNode = ScriptManager.prototype.genScriptNode();
scriptNode.setAttribute("src", "<?php echo ROOT_URL; ?>fetchResource.php?scripts[]=" + packageConfig.scripts.join("&scripts[]="));
scriptNode.onload = scriptNode.onreadystatechange = function() {
if (!scriptNode.readyState || scriptNode.readyState == "loaded" ||
scriptNode.readyState == 4 && scriptNode.status == 200) {
}, 200);
var headNode = document.getElementsByTagName("head")[0];
nextQueueItem : function(){
if(this.loadQueue.length > 0){
var currentItem=this.loadQueue.shift();

scriptLoaded : function(){
* Callback function for whenever a script finishes loading

setMask: function(maskText){
var mask = document.createElement("div");
var maskMessage=document.createElement("div");
maskMessage.className="ext-el-mask-msg x-mask-loading";
maskMessage.innerHTML="<div>Loading... "+maskText+"</div>";
maskMessage.style.left=((window.innerWidth/2)-(maskMessage.offsetWidth/2)) + "px";
maskMessage.style.top=((window.innerHeight/2)-(maskMessage.offsetHeight/2)) + "px";
clearMask: function(){
var mask=document.getElementById("ScriptManagerMask");
var maskMessage=document.getElementById("ScriptManagerMaskMessage");
ScriptManager.register = ScriptManager.prototype.register;
ScriptManager.load = ScriptManager.prototype.load;
ScriptManager.loadQueue = ScriptManager.prototype.loadQueue;
ScriptManager.registeredPackages = ScriptManager.prototype.registeredPackages;
ScriptManager.srcScript = ScriptManager.prototype.srcScript;

* Dynamically load, compress and concatenate the javascript resources
* @author Adam J Benson
* @package Spellrus
* @todo CSS Fetching
require_once (ROOT_PATH . "classes/ResourceCompressor.php");

$compressor=new ResourceCompressor();

foreach($_GET['scripts'] as $script){

header("location:" . $compressor->fetchJavascript());

You should be able to create your own resource compressors easily using scripts such as jsminify, a simple loop to concatenate, gzip if the server supports it (remember to check if the client supports it too!), and most importantly caching.

Try to construct your packages into very logical units, i.e some of my packages are:
name: "Common",
description:"Common Libraries",
name: "EXTCore",
description:"GUI Framework",
name: "EXTPlugins",
description:"Shiney squiggles!",
name: "WYSIWYG",
description:"What You See Is What You Get Editor

6 May 2008, 8:45 PM
Our JS Builder tool includes a command line version that can be automated via Ant or anything else that can execute a script. It's currently Windows-only, but if that's not a problem, it's a useful tool:


8 May 2008, 11:22 AM
I use ant for cutting the release. I learned it 2 days ago, it's really easy. Here's the script (similar to @fangzhouxing, but all in ant):

<project name="myProject" default="compile" basedir=".">
<property name="jsSource" value="../foo/"/>
<property name="outputDir" value="../bar"/>
<property name="buildNumber" value="5000"/>
<property name="yuicompressor" value="yuicompressor-2.3.5.jar"/>

<!-- delete the old compile directory -->
<target name="clean">
<!-- delete the old directory -->
<delete dir="${outputDir}"/>

<!-- create a concatenated and YUICompressed application -->
<target name="compile" depends="clean">
<!-- create a new compile directory -->
<mkdir dir="${outputDir}"/>

<!-- create a temp file for use in the next several steps -->
<tempfile property="temp.file" destDir="."/>

<!-- concatenate together the files that make up [email protected]@.js -->
<concat destfile="${temp.file}" fixlastline="yes">
<fileset file="${jsSource}/license.js"/>
<fileset file="${jsSource}/dashboard.js"/>
<fileset file="${jsSource}/welcome.js"/>
<fileset dir="${jsSource}/applicationUtil"/>
<fileset dir="${jsSource}/userAdmin"/>

<!-- use YUI Compressor to compress the js file -->
<java jar="${yuicompressor}" fork="true">
<arg value="-o"/>
<arg value="${outputDir}/application${buildNumber}.js"/>
<arg value="--type"/>
<arg value="js"/>
<arg file="${temp.file}"/>

<!-- delete the temp file -->
<delete file="${temp.file}"/>

This takes less than 3 seconds for 10k LOC (2.6ghz MBP). Could almost be used for development.

For development, instead of including 100's of js files, I include application.js.php which is a PHP script that concatenates together the files on the fly. This takes under 300ms, so it's fine for development. This makes the development environment more like the deployment environment. Try using this function:

function output($filename) {
echo "\n/* $filename */\n";
echo "\n";

5 Jul 2008, 4:55 AM
Here's something I came across that is fairly related to this thread so I thought I'd post it here.


7 Jul 2008, 1:32 PM
I use Ant to concatenate and minify my JavaScript/CSS. See this great blog entry from the creator of YUI Compressor: http://www.julienlecomte.net/blog/2007/09/16/

Also, JAWR (https://jawr.dev.java.net/) looks really enticing as it has excellent tie in with DWR as well.

27 Aug 2008, 5:43 AM
I also use JAWR with Ext. As useful as ext-all-debug is, it really does bring my firebug debugging to its knees, so instead I configure JAWR to use the uncompressed sources for debug and the provided release for non-debug ...

{other ext code and settings}

# ext-all - either use the minified version, or the source.

# ext-all: provided compressed version

# ext-all-source: uncompressed source files (see extjs/sources/ext.jsb for the correct list for your version. This is ext-2.2)
The source list is derived from the 'ext.jsb' file, so you will have to compare your version's ext.jsb with the mappings list above, but other than that I've seen no ill effects of debugging this way, and firebug screams now by comparison.

The only gocha I had was trying to get dynamic theme swapping working. Here's my workaround for setting my themes using JAWR's path mangling..

<script type="text/javascript" src="../jawr_loader.js" ></script>
<script type="text/javascript">
setThemeCSS: function(path,media) {
replaceTheme: function(path) {
insertCondComment : function(condition,func,path,media){


// Set the theme
JAWR.loader.setThemeCSS('/extjs/resources/css/xtheme-'+ theme + '.css')
Naturally this isn't the only way to do this, but I thought I'd share it anyway in the spirit of open-source.

BTW, great job ext-developers!

27 Aug 2008, 7:32 AM
www.scriptalizer.com (http://www.scriptalizer.com/)
this lets you upload multiple js files. it then minifies and combines them into one

19 Dec 2008, 7:30 AM
heres details on another one.


22 Dec 2008, 12:40 AM
I selected packtag.

It is simple to use:

// detect should we compress JS and CSS or not
String compressed = "true";
String reqNonCompressed = request.getParameter("noncompressed");

if (reqNonCompressed != null) {
compressed = "false";

pageContext.setAttribute("compressed", compressed);

and in html:

<pack:script enabled="${compressed}">
<!-- LIB: ExtJs Framework -->


While developing, I use GET parameter "noncompressed" and scripts are not compressed.

PS. But have troubles with cyrillic characters in scripts on Tomcat with packtag.
On Glassfish all is fine.

14 Sep 2009, 7:36 AM
We also use Jawr on our project. We are in the process of converting from ExtJS 2 to ExtJS 3. Do you have the ExtJS 3 JavaScript listing you could post? Thanks!

17 Sep 2009, 12:44 PM
If I find any additional errors in the order, I will update this post, but this seems to be the correct order.


19 Sep 2009, 8:25 AM
The first, i will merge all the js files into one file, and then use this online tool (http://javascriptcompressor.com/) to compress. I think it very useful.

21 Sep 2009, 5:23 AM
We use JAWR to compress our js files. It works great. It also has a debug mode so that you can load them individually if you are developing and need to debug something. The list I posted is the ExtJS list for debugging purposes. If running in release mode you should use the compressed versions ExtJS provides, ext-all.js an ext-base-all.js.

20 May 2010, 4:03 PM
Sorry for resurrecting an old thread, but it seems to be still relevant.

I'm wanting to be able to be able to automatically produce a number of cut-down versions of Ext for deployment to different parts of my web app so I'm setting up an Ant task to control YUI Compressor along the lines of the example shown by chazmatazz.

Is there an official (i.e. Ext developer) approved ordered list for the Ext JS files to be concatenated or does somebody have a way of extracting the list from the ext-all-debug.js files?

I'm also hoping that the order of the files won't change much between Ext versions otherwise it will be a chore to maintain the file list in Ant. Currently I'm on Ext 3.2.1

20 May 2010, 11:18 PM
@Wedgie: is there a reason why you can't use JSBuilder? http://www.extjs.com/products/jsbuilder/

20 May 2010, 11:44 PM
I'm wanting to be able to be able to automatically produce a number of cut-down versions of Ext for deployment to different parts of my web app so I'm setting up an Ant task to control YUI Compressor along the lines of the example shown by chazmatazz.

ext-all.js gezipped is 186kb (3.2.0). It is isn't exactly small, but it isn't huge either. And if many of your users are returning users it will be in their cache. So I would be hesitant to make "a number of cut-down versions", as it might introduce errors.

Regarding the orginal question of this thread. I use a publish/release script which cat's all my js files and then uses yui (I guess similar to JSBuilder). Also it gives the js directory a revision tag, to force download of the latest code (the ext lib files are in a different directory).

21 May 2010, 12:03 AM
rockstarapps - in all projects one click to concatenate and compress all, there is no better solution.

21 May 2010, 12:06 AM
why would you get an application to do this, if a few lines of script does the same and give you full control?

21 May 2010, 12:14 AM
because rockstarapps do a lot more like on-the-fly-jslint while typing a.s.o. And why should i write a script for each project if all is done and can be configured visually? Check it out and you never will miss it.

Mike Robinson
21 May 2010, 7:19 AM
I personally think that the only technique that you should use (and, I think, you should use it uniformly for all content) is: gzip. That's it ... that's all.

Here's why:

(1) "Obfuscating" code, in a vain attempt to make it unintelligible, really doesn't work anyhow. You can instantly "prettify" it.

(2) Memory sizes at the client end are now a lot bigger. Transmission time, also, is much faster. Servers are a lot smarter; so are browsers. Parsing an entire "big" file is an almost-instantaneous process, and it forevermore will be.

(3) A compression algorithm like gzip is designed to compress things and it doesn't have to worry that the compressed file is "directly readable by the parser." So, gzip can routinely deliver compression ratios that are better than anything a "minifier" could do. And it does it... on the fly, and for all content. For all of your AJAX exchanges, too. (And that's very significant.)

(4) If anyone's going to "split up" ExtJS, the party that does so must be ExtJS, Inc. And, they have to take a very hard look at the cost/benefit picture: not only with regard to themselves but also with regard to the deployed code as it is operating in the field. (What if, for instance, a user accesses two different ExtJS apps back-to-back, each of which were split in different ways, with different content for identically-named files? "What Would The (Client's) Cache Do?" Think about it ... a lot ...

It would be remarkably easy to open up a huge bucket of expensive worms, in quest for a time/cost savings that you don't get! While incurring a high cost that you don't need! Happens all the time. It's just the nature of the beast. Any beast.

21 May 2010, 7:36 AM
I agree that minimizing JavaScript code has not a lot of benefit in combination with gzip (and the latter you should use). And, I also think it is not a good idea to break up the Ext library in smaller versions.

However, one other technique you should use is concatenation. You don't want 150 browser requests, where 1 is sufficient.

21 May 2010, 8:35 AM
i don't agree :) Minifying / compressing is also something protecting the original source. If only gzipped, you can extract and have the original code. But i don't want other people to "steal" my code.

And after minify you can gzip the compressed one. In my latest project i have eg
143KB concatenated js
92KB yui-compressed of the concatenated
17KB gzipped of the yui compressed

(this all is application only)

And this all is done by rocketstarapps automatically.

You should / must do what ever you think is the best way, this is the way i go.

21 May 2010, 8:42 AM
Minifying / compressing is also something protecting the original source

It doesn't protect, it only makes it slightly harder to read..

How large is the file if you skip the yui-compression? Probably around 20-25KB.

See my table here (http://www.extjs.com/forum/showthread.php?97359-Minify-Javascript-and-Client-Performance-Impact/page2) (I already had this discussion with Mike).

21 May 2010, 8:58 AM
sure it doesn't protect really, but with vars A,B,C, ... it makes it hard to use. Even better than original ;)

21 May 2010, 12:27 PM
Those of you who argue against minifying code, must not be running on legacy browsers, with minimal memory or slow connections :)

Minifying beyond gzip is a major improvement for sites with large js files. Yeah, you can run the minified code back thru a beautifier and get munged var names with no comments - that's not the point. It's the further reduction you get beyond simple white space removal - e.g. single char var names.

25 May 2010, 3:41 PM
Lots of interesting posts there!

FWIW here are my thoughts:

GZIP is good, but only if you have control over the server settings, and it's often not enabled by default. I prefer to not rely on it always being there.
Effective caching depends on server and browser settings beyond the control of the developer.
Minifying for optimisation (as opposed to obfuscation) should not break the code.
Ext JS LLC recommend producing cut-down versions of the Ext library when required. For example why download the whole library when all you want to do is pop up an Ext window?
The Ext library should be allowed to grow in size at each release without having to worry about the negative effect on existing apps, or put conversely you should be able to upgrade your apps to a later version of Ext without having to worry about its size.
Using additional tools carries non-obvious but significant costs for a software shop in terms of training/learning, managing dependencies, upgrading and migrating. The question should always be "why should I use this tool" rather than "why shouldn't I?"

In the absence of a compelling case for using something else I just want to use the standard YUI compressor (which we already use for our own JS library) automated with Apache Ant, along the lines as has been suggested by other posters in this thread.

So I'd really like for there to be an official dependency tree, or at least an ordered list that I can use to help me do my builds.

15 Apr 2011, 3:24 PM
I recommend to use Granule Tag lib the http://code.google.com/p/granule/

It gzip and combine javascripts wrapped by g:compress tag

code sample is:
<script type="text/javascript" src="base"/>
<script type="text/javascript" src="myui.js"/>