View Full Version : Ext.data.Store with persistence

18 Oct 2007, 4:18 PM
In line with the other thread, I wrote a basic extension to Ext.data.Store to allow persistence. Additionally, instead of using the modified property to track modified records, it uses a basic journal to record all transactions.

It works as follows:

The reader is assumed to have a parallel method to read(), specifically write(), that takes object data to convert to its appropriate format (Json, XML, etc.).
A separate config option, updateProxy, is used to configure the proxy used to write. This could, of course, be the same proxy used to read, if the url, etc. are the same. This proxy needs to have an update(params) function. The params are set by the Store, and include any passed by the caller, as well as one 'data', that includes the encoded information to be sent.
When the store receives a commitChanges() call, it flushes the journal out to the server using the pre-configured updateProxy, if one is available.
When the store receives a rejectChanges() call, it undoes all the journal changes.

This is really a first cut, but I hope it is useful. Included are:

Added and updated methods and properties to Ext.data.Store to allow for appropriate journalling and writes.
Added method for JsonReader to support the conversion from objects to JSON (that one was real easy).
Added methods to HttpProxy to support the updating.

Feedback and improvements are appreciated. I will post the actual code in a separate posting in this thread.


18 Oct 2007, 4:19 PM
* @author adeitcher
// to keep track of changes
journal : [],
maxId : 0,

// fixed types
types : {change: 'change', add: 'add', remove: 'remove'},

// the added config is automatically there - these are defaults
updateProxy : null,

// if you want to replace all, set options.replace = true
write : function(options) {
if (this.fireEvent('beforewrite',this,options) != false) {
// get the appropriate records
var records = this.writeRecords(options);
// structure the params we need for the update to the server
var params = {
data: this.reader.write(records)
// add any options
Ext.applyIf(params, options);

writeRecords : function(options) {
var data = [];
var tmp;
// we take only the journal, unless we have explicitly asked to replace all

// sure, we could use a trinary operator, but this may get more complex in the future,
// and if-then-else is cleaner to understand

// to be supported later
if (options != null && options.replace == true) {
tmp = this.journal;
//data = this.data.getRange(0);
} else {
tmp = this.journal;

// get the actual data in the record
for (var i=0; i<tmp.length; i++) {
if (tmp[i] != null) {
data[i] = {
type: tmp[i].type,
data: tmp[i].record.data


// these all need to be modified to keep track of real changes
add : function(records){
records = [].concat(records);
for(var i = 0, len = records.length; i < len; i++){
var index = this.data.length;
for (var i=0; i<records.length; i++) {
this.journal.push({type: this.types.add, index: index+i, record: records[i]});
this.fireEvent("add", this, records, index);

remove : function(record){
var index = this.data.indexOf(record);
this.journal.push({type: this.types.remove, index: index, record: record});
this.fireEvent("remove", this, record, index);

removeAll : function(){
// record that all objects have been removed
for (var i=0,len = this.data.getCount(); i<len; i++) {
this.journal.push({type: this.types.remove, index: i, record: this.data[i]});
this.fireEvent("clear", this);

insert : function(index, records){
records = [].concat(records);
for(var i = 0, len = records.length; i < len; i++){
this.data.insert(index, records[i]);
for (var i=0; i<records.length; i++) {
this.journal.push({type: this.types.add, index: index+i, record: records[i]});
this.fireEvent("add", this, records, index);

commitChanges : function(){
// commit the changes and clean out
var m = this.journal.slice(0);
this.journal = [];
// only changes need commitment
for (var i=0, len=m.length; i<len; i++) {
if (m[i].type == this.types.change) {

// now write the changes to persistent storage
if (this.updateProxy != null) {

rejectChanges : function(){
// back out the changes in reverse order
var m = this.journal.slice(0).reverse();
this.journal = [];
for(var i = 0, len = m.length; i < len; i++){
var jType = m[i].type;
if (jType == this.types.change) {
// reject the change
} else if (jType == this.types.add) {
// undo the add
} else if (jType == this.types.remove) {
// put it back

* The next three are for changes affected directly to a record
* Ideally, this should never happen: all changes go through the Store,
* and are passed through after being recorded. However, in an object-oriented paradigm,
* it is accepted that you can gain access to the direct object, unlike a SQL paradigm.
* Thus, we need to put events on the Record directly.

// if we edited a record directly, we need to update the journal
afterEdit : function(record){
this.journal.push({type: this.types.change, index: this.data.indexOf(record), record: record});
this.fireEvent("update", this, record, Ext.data.Record.EDIT);

// if we rejected a change to a record directly, we need to remove it from the journal
afterReject : function(record){
// find the last edit we had, and remove it
for (var i=this.journal.length-1; i>=0; i++) {
if (this.journal[i].type == this.types.change && this.journal[i].record == record) {
this.journal = this.journal.splice(i,1);
this.fireEvent("update", this, record, Ext.data.Record.REJECT);

// if we committed a change to a record directly, we still keep it in the journal
afterCommit : function(record){
this.fireEvent("update", this, record, Ext.data.Record.COMMIT);

getNextId : function() {
// send back the maxId + 1

getMaxId : function() {
var maxId = 1000;
if (this.data != null) {
var records = this.data.getRange(0);
for (var i=0; i<records.length; i++) {
if (records[i].id > maxId)
maxId = records[i].id;

// basic data writer
Ext.data.DataWriter = function(meta, recordType) {
this.meta = meta;
this.recordType = recordType instanceof Array ?
Ext.data.Record.create(recordType) : recordType;

// Json data writer extension to Reader
Ext.apply(Ext.data.JsonReader.prototype, {

// write - input objects, write out JSON
write : function(records) {
// hold our new structure
var j = Ext.util.JSON.encode(records);
if (!j) {
throw{message: "JsonWriter.write: unable to encode records into Json"};


// extend the HttpProxy to write
update : function(params){
if(this.fireEvent("beforeupdate", this, params) !== false){
var o = {
params : params || {},
callback : this.updateResponse,
method: 'POST',
scope: this
Ext.applyIf(o, this.conn);
this.activeRequest = Ext.Ajax.request(o);
callback.call(scope||this, null, arg, false);

updateResponse : function(o, success, response){
delete this.activeRequest;
this.fireEvent("updateexception", this, o, response);
this.fireEvent("update", this, o, o.request.arg);


19 Oct 2007, 3:54 AM
This looks perfect for my next Ext application, thank you.

One question which probably illustrates my limited understanding of OO JavaScript. Within an application scope, are you changing the behavior of the base Ext DataStore class?

If so did you consider deriving a new class called PersistentDataStore to host the persistent overrides?

19 Oct 2007, 4:12 AM
Glad it helped. If you make any adjustments to improve it, please post them.

I went back and forth between extension of the class and application to the base prototype. In the end I chose the latter, since it seems a fairly basic element to have. The original code even has update and updateResponse methods for the proxy, but unimplemented.

Check out the docs and examples for Ext.apply as opposed to Ext.extend.

22 Oct 2007, 8:37 PM
Well, I found some shortcomings and weaknesses.
The minor shortcoming is that write and writeexception events were not being handled properly. I will post code in the next few days that fixes it.

The major weakness is that the data stored in the Ext.data.Store is actually only a subset of the JSON or XML data, i.e. only those that make up Records below the root (not even including the root itself). So, when you go to write, you end up with a JSON without a root, let alone anything else that was in there. This weakness only applies if you use replace mode, i.e. each write is a full overwrite of the data server-side. This is not an issue if you use journal mode.

I have been working on SecureReader which handles encryption/decryption, and uses the Decorator pattern to wrap the native reader (Json, XML, etc.). I should be able to just Decorate another intermediate layer to ensure we get the whole dataset, not just what the store wants.

11 Dec 2007, 4:13 PM
hmm... interesting...

ok deitch... i tryed your code but noting... can you give me code sample or files ?

12 Dec 2007, 5:39 AM
Sure. I will post it in a few days. I wrote a significant extension, which includes StoreProxy and some major patches. I need a few days to clean it up and versionize it properly.

The short form:
You use it like any other store, with one added config parameter: updateProxy with another proxy (e.g. HttpProxy). If this proxy is non-null, then when you call commitChanges() on the store, it will push the changes out.

I will get this uploaded in a few days.

12 Dec 2007, 11:32 AM
very cool

ok... i will wait your post... please hurry up :-)

13 Dec 2007, 7:50 AM
Hi Deitch,

We are waiting for your code bunch..............

13 Dec 2007, 12:16 PM
OK, I have posted it under Ext 2.0 user extensions and will close down this thread.
See http://extjs.com/forum/showthread.php?t=20783