PDA

View Full Version : Working off of the SVN tree



davidascher
6 Apr 2007, 4:27 PM
The SVN tree (which premium customers get access to) doesn't have the built files (e.g. ext-all.js). While there are plans to provide tools to premium customers to do those builds on demand, that's not there yet. To make my own life easier in the interim, I wrote a script that I run passing it the root of the svn tree and it creates the packaged files. If you want to work off of the head, it's handy.

UPDATED:

Also, people should know that trbs and I have created a google hosted project for it, so that people's suggestions can be incorporated, at http://code.google.com/p/extjs-py-builder/
(http://code.google.com/p/extjs-py-builder/)
Notes:

If your Python installation doesn't have them already, you'll need to install elementtree (http://effbot.org/zone/element-index.htm) or cElementTree (http://effbot.org/zone/celementtree.htm).

It uses a less aggressive minimization strategy than Jack does. I don't care about that
when working off of SVN at this stage. (Minimization needs java & shrinksafe)

If you have issues, find bugs, feature requests, please use the issue tracker (http://code.google.com/p/extjs-py-builder/issues/list).

trbs
7 Apr 2007, 12:09 PM
There's also a Python version of jsmin:
http://www.crockford.com/javascript/jsmin.py.txt

For if u want to use ShrinkSafe and JsMin to safe some extra bytes :)

Jul
7 Apr 2007, 12:18 PM
David,

I just had the same issue. Jack pointed out in another thread that you can build the *-all.js files yourself pretty easily. Just download and install jsbuilder found here: http://code.google.com/p/js-builder/

There are three .jsb files. To build one, just doubleclick on a file, set the output directory and generate. The three .jsb files are found here:

\Ext\src\ext.jsb
\Ext\resources\resources.jsb
\Ext\examples\examples.jsb

- Jul

davidascher
9 Apr 2007, 9:20 PM
Thanks, I know I could run JSBuilder myself, except that I don't have a Windows machine handy right now =). Besides, it feels like overkill =).

--david

papasi
10 Apr 2007, 1:55 PM
I tried to run jsbuilder on 1.0beta1 and the generated ext-all-debug.js is broken (size is smaller than the one in the zip file and a lot of js error on load)

I wonder if the jsb file is out of sync with the include javascript files or a bug in js builder.

mystix
10 Apr 2007, 5:53 PM
this post might help: http://extjs.com/forum/showthread.php?t=4382

trbs
13 Apr 2007, 3:26 PM
Hey David

Love your script, for us people working under Unix it's just wonderful :)
Hope u don't mind me making some small adjustments/improvements.

Changes:

Added jsmin (included in the source for ease of use)
Added options (enable/disable ShrinkSafe and jsmin)
Recover from failure if ShrinkSafe or Java does not exist.

fuzzymonk
26 Apr 2007, 9:11 AM
While I am using windows, I would like to keep all my build scripts in python, so this python jsb processor is awesome. I reworked it a little to get it to work under windows, and to be generic for for any jsb file. I would like to incorporate David's changes as well, but thought I could enlighten other windows users to what I did:

Install ElementTree package (cElementTree does not exists for windows, as far as I can tell)

Change:

from cElementTree import ElementTree as ET
to

from elementtree import ElementTree

Change:

jsb = ET(file=fname)
to

jsb = ET.parse(fname)

trbs
26 Apr 2007, 4:23 PM
i've updated my version of the build script with your changes and assumed that u meant: "from elementtree import ElementTree as ET".

could u please check the attached version works in windows ? (thanks in advance!)

the codebase is now getting pretty ugly, but if works fine. when i have some time i want to clean the code up and port as many features as possible from the java-builder and js builder to the python-builder. (maybe even throw a small cross platform gui in there :)

if people like this idea and seriously use the python script then please post a message here or talk to me on irc about it. that will help move the rewrite plans up in my todo list a bit ;)

p.s. davidascher havn't seen u around much lately, does this sound good to you ?

changes:

windows fixes
smaller filesize (ShrinkSafe every source file)
added continue option (skip non-existing files)
use elementtree if cElementTree is not available

thejoker101
3 May 2007, 8:43 PM
I've never used python before, so this may be an obvious answer, but how do I use this script?

After browsing through the source, I think I am doing it correctly, but I get this error:


$ python build_ext_packages.py ../svn/ext/src/
Traceback (most recent call last):
File "build_ext_packages.py", line 25, in ?
from elementtree import ElementTree as ET
ImportError: No module named elementtree

I tried adding -S and -J as options and get the same thing. I downloaded the last listed python script.

Can someone help me out here? TIA

davidascher
4 May 2007, 6:49 AM
ElementTree and cElementTree are available for vesions of Python which don' already include it from http://effbot.org/zone/element-index.htm and http://effbot.org/zone/celementtree.htm

Also, people should know that trbs and I have created a google hosted project for it, so that people's suggestions can be incorporated, at http://code.google.com/p/extjs-py-builder/.

I haven't had time to maintain the script, but trbs might be able to update the version on there soon. Patches, but reports, etc., should probably go there.

Cheers,

--david

trbs
4 May 2007, 9:48 AM
You need ElementTree or cElementTree for this script.
This is needed to parse the .jsb files in the ExtJS source.

I added some extra print statements around the Imports so it should be more clear now that Element Tree is a dependency.

keithpitt
13 May 2007, 7:42 PM
Hello all,

Great script by the way. I gave it a go, works like a charm. I did however try and specify my output dir, but failed. I looked through the code and didnt see any evidence of such an option. Ive created a path to enable the functionality. Ive only tested it on Windows, seems to be working fine for me.

P.S I dont know python, so I hope what Ive done is ok. =P~
P.S.S It wouldnt let me upload a .patch file, so I changed its extension to .txt

keithpitt
13 May 2007, 8:37 PM
I also noticed that when the file processes the resources.jsb file, it uses jsmin on the ext-all CSS file at the end. This removes the spaces from within the file and consequently breaks the style sheet rules. I've got another patch here with a fix.

trbs
14 May 2007, 12:20 PM
sorry guys, still havn't got around to fix the latest version in code.google.com.

if u have trouble please use davidascher's original script in the mean time.

keithpitt Thanks for your patches ! I will put them in svn as soon as i can.

trbs
23 May 2007, 3:56 PM
okey i uploaded a new version into svn
this version is an intermediate version between David's original script and my additions to it. because my version still has issues that need to be worked out and my time to work on it is limited. :(

I plan to port my additional features and keithpitt's patches back into the main version as soon as possible. But at least now the version in svn is working and correctly builds the Ext source tree :)

It lacks the command line options; but don't worry that's first on my list to add the next time i code on it B)

The most stable version is now:
http://extjs-py-builder.googlecode.com/svn/trunk/build_ext_packages.py

trbs
2 Jun 2007, 4:01 PM
forgot to post an update announcement here.

the newest version 0.6 which is in svn now contains:
* command line options
* 3 different compressors/packers to choice from
* support for Ext2.0
* search CLASSPATH for ShrinkSafe
* continues building after missing files (but with option to stop auto continue)

upcoming features are:
* jsdoc support (for building api documentation)
* shrinking of css files
* optional target directory (for building outside of the ext svn source)

thejoker101
23 Aug 2007, 12:47 PM
What version of python does this require?

It doesn't seem to work on this version:

===>$ python
Python 2.3.5 (#2, Oct 16 2006, 19:19:48)
[GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2


I know it's an older version of python, but I don't have control of it. Can it work with this version, or am I missing something?

trbs
23 Aug 2007, 2:25 PM
Thanks for reporting this thejoker101, it was indeed incompatible with python <2.4 i've fixt this problem. The build script should now run on any major python version.

For good meassure this is the error i got when running with python2.3:


File "../extjs-py-builder/build_ext_packages.py", line 742
all = '\n'.join(open(p(file)).read() for file in files)
^
SyntaxError: invalid syntax

which is now fixed B)

please note that the buildscript will not work on python 2.2 or below because of some missing base libraries. if somebody has a real need for python 2.2 i'm willing to create a patch for that.

thejoker101
23 Aug 2007, 6:15 PM
Yep, that was the error I was getting. Is the google code repository updated then? Going to check now...

thejoker101
23 Aug 2007, 6:28 PM
K, the 0.7 version is there, great. I no longer get the error I had before.

However, the server I'm on apparently doesn't have elementtree, now to see if I get that installed on a shared host... Doesn't usually work out.

thejoker101
23 Aug 2007, 6:43 PM
I've downloaded elementtree from the link provided in this tread, however I'm not sure where I should put the files such that the script will look for them. I don't have permissions such that I can install down to /usr/lib/ directories.

I keep getting this error:
ElementTree not found; u need ElementTree of cElementTree to run this script.

The instructions on the download page say this:


Note that if you only need the core functionality, you can include the ElementTree.py file in your own project. To get path (http://effbot.org/zone/element-xpath.htm) support, you also need ElementPath.py. All other modules are optional.

I have those two files in the same directory as build_ext_packages.py, but no go.

trbs
24 Aug 2007, 8:57 AM
If your familiar with the likes of Java or other languages which use dependencies the setup is similar.

You can install elementree where ever you want, as long as python knows where to find it while starting up the builder.

Think the easiest will be too download elementtree, extract the tarball and copy the folder named elementtree to some path which is writable to you. For the example i'll take /home/user/pydeps/



$ cd /tmp
$ wget http://effbot.org/media/downloads/elementtree-1.2-20040618.tar.gz
$ tar xzf elementtree-1.2-20040618.tar.gz
$ mkdir -p ~/pydeps
$ mv /tmp/elementtree-1.2-20040618/elementtree ~/pydeps


* please note that this doesn't work as easily with cElementTree because of the extra c modules that will need to be compiled, but the overall idea is similar

Now because you cannot write into any systemwide folders you'll have to let python know about this ~/pydeps directory another way. This is done by settings the PYTHONPATH environment variable like so:



$ PYTHONPATH=~/pydebs/:$PYTHONPATH; export PYTHONPATH


You can check if everything worked okey by doing:



$ python -c 'import elementtree'


If this raises an error like: ImportError: No module named elementtree
Then there's still something wrong and you'll need to check your path's and files since elementtree is a pure-python module this should work without any additional steps like running setup.py in the installation directory, if it just returns silently your all set to run the build script :)

You can add the new PYTHONPATH to your ~/.profile and/or ~/.bashrc to make it sticky the next time you ssh into the server.

* note that you don't have to use ~/pydebs, you can also just extract the elementtree tarball in your homedirectory and use PYTHONPATH=~/elementtree-1.2-20040618:$PYTHONPATH; export PYTHONPATH but i like reasonable clean directory structures :)

hope this helps.

trbs
24 Aug 2007, 9:04 AM
I've downloaded elementtree from the link provided in this tread, however I'm not sure where I should put the files such that the script will look for them. I don't have permissions such that I can install down to /usr/lib/ directories.

I keep getting this error:
ElementTree not found; u need ElementTree of cElementTree to run this script.

The instructions on the download page say this:



I have those two files in the same directory as build_ext_packages.py, but no go.

you could indeed install elementtree in the same directory as build_ext_packages.py, but it will need to be the Directory elementtree like described in my previous post. Also when you install it in the same directory you cannot startup build_ext_packages.py from another directory anymore since it will only look in the current directory.

What will work however is if you copy the directory elementtree to your ext/ checkout and download build_ext_packages.py in the same directory. Then you can run the buildscript as:



$ cd ~/ext
$ svn up
....
$ test -d elementtree && echo "elementree exists here :)"
$ test -f build_ext_packages.py && echo "build_ext_packages.py also exists"
$ python ./build_ext_packages.py .

trbs
30 Aug 2007, 8:11 AM
FYI: the lastest version (0.7.1) which is available in the SVN repository but not yet as download in google code has now support for psyco. If you have psyco installed the build will be likely to be 10 times faster. Specially jsmin get's a dramatic speed up from using the JIT compiler!