yathit/ydn-db

Javascript database module for Indexeddb, Web SQL and localStorage storage mechanisms supporting version migration, advanced query, SQL and transaction.

Setup

To use this , include one of the ydn-db minified js files from download page in your HTML page.

• User Guide
• API Reference
• Demo applications
• Example applications
• Release notes
• Download

Supported browsers

This library can be used in almost any browser.

• Chrome 4+
• Firefox 3+
• IE 6 (userdata), IE7+ (localStorage), IE10+ desktop/mobile (indexeddb)
• Safari 3.1+ desktop/mobile (websql)
• Android browser 2.1+ (websql), 4+ (indexeddb)
• Android web client, iOS web client (websql)
• Opera 10+ (websql), Opera 15+ (indexeddb)

Features

• Unified data access layer on IndexedDB, WebDatabase and WebStorage storage mechanisms.
• Support all features of asynchronous IndexedDB API.
• The implementation of WebSQL supports schema reflection, untyped column key, composite index, multiEntry and IndexedDB-like aborting and implicit commit transaction. localStorage use on-memory AVL tree index for key range query and its performance is in-par with database.
• Support for on-the-fly database schema generation, IndexedDB-style versioned schema migration and advance schema-centric (auto-version) migration by reflecting on the existing schema.
• Well tested closure library module including 234 unit test functions in addition to qunit end-to-end test to validate library API specification.
• Advance transaction workflow and managed request (meaning you will never ever see an InvalidStateError).
• Designed for high performance index query (only).
• Customized log messages, improper usage protection and guided error messages on dev distribution.
• Basic support for high level query using SQL.
• Full text search (via ydn-db-text module).
• Client-server Synchronization (via ydn-db-sync module).
• We adopt strict javascript coding patterns for performance and robustness: no global; no eval; no error globbing; parameterized query; all public methods and constructors are strongly typed; this is this; and coding errors throw errors.

Examples

Schema definition

var schema = {
  stores: [{
    name: 'people',
    indexes: [{
       name: 'age'
    }, {
       name: 'age, name',
       keyPath: ['age', 'name']
    }]
  ]
}
db = new ydn.db.Storage('db-name', schema);

If the database exists, it will be opened and updated with the given schema if necessary. In doing so, object stores and indexes will be created or deleted.

Simple usage

Simple usage for opening, storing and retrieving by a primary key id1.

db = new ydn.db.Storage('db-name', schema);
db.put('people', {name: 'John', age: 10, sex: 'Male'}, 'id1');
db.get('people', 'id1').done(function(record) {
  console.log(record);
});

Query

The following snippet shows querying from the people object store using index age by a key range bounded by 25. The result will be sorted by age.

var q = db.from('people').where('age', '>=', 25);
var limit = 10;
q.list(limit).done(function(objs) {
  console.log(objs);
});

Sorting using an index with filtering on another index.

var q = db.from('people').where('age', '=', 25);
q.order('name').list().done(function(objs) {
  console.log(objs);
});

Note that the above sort query requires a compound index ['age', 'name'].

Transaction

By default, database requests are executed in separate transactions and executed in order. The following code snippet shows running all database requests in a single transaction.

var req = db.run(function update_prop (run_db) {
    run_db.get('player', 1).done(function(data) {
        data.health += 10;
        run_db.put('player', data).done(function(key) {
          if (data.health > 100) {
            req.abort();
          }
        });
      }
    }, ['player'], 'readwrite');
    req.then(function() {
      console.log('updated.');
    }, function(e) {
      console.log('transaction aborted');
});

Events

ydn.db.Storage dispatch events for connection and error. Additionally modification of records events can be installed by defining in schema.

Data heavy query should be execute after database connection is established by listening ready event.

db.onReady(function (err) {
  if (err) {
    console.error(err);
    return;
  }
  // heavy database operations should start from this.
);

Library developer guide

If you haven't try Closure Tools before, setup can be time consuming and painful. I recommend to read Michael Bolin book's Closure: The Definitive Guide. A good understanding of closure coding pattern is necessary to understand and follow this library codes.

Apache ant is used to build javascript compiler. ydn-base repo build.xml defines compiler and others tools setting. You must change according to your local machine setting. Specifically check property values of closure-library.dir and closure-compiler.dir, which point to respective directries.

Downloads the following three repos a directory.

svn checkout http://closure-library.googlecode.com/svn/trunk/
git clone [email protected]:ytkyaw/ydn-db.git
git clone https://bitbucket.org/ytkyaw/ydn-base.git

that should create three directories for closure-library, ydn-base and ydn-db.

Run local apache (recommended) or a static server on that directory.

Go to ydn-db folder and run ant deps and ant ydn-base.deps to generate closure dependency tree.

Use HTML files in the /test folder for getting started. These files are also used for debug development.

Note: we use master track version of closure tools. Compiling with pre-build jar may encounter compile error.

Note: precompile files are built by using custom compiler to strip debug messages. See detail on ydn-base/tools/strip_debug.txt.

Additional features requires the following optional repos.

1. Full text search https://github.com/yathit/ydn-db-fulltext.git
2. Dependency for ydn-db-fulltext https://github.com/yathit/fullproof
3. Dependency for ydn-db-fulltext https://github.com/yathit/natural
4. Synchronization https://bitbucket.org/ytkyaw/ydn-db-sync (private)

Testing

You should be able to run /ydn-db/test/all-test.html or run tests individually. Since all tests are async, disable run the 'in parallel' check box. These test files are for basic testing and debugging.

The coverage test is performed by JsTestDriver test. Notice that ant gen-alltest-js generates jsTestDriver.conf to prepare the testing configuration.

java -jar JsTestDriver.jar --tests all

End-to-end testing for distribution can be found in the test/qunit folder as well as online [qunit test kits] (http://dev.yathit.com/index/demos.html).

Deployment

For update bower, create a tag

git tag -a v1.3.3 -m "bug fixes"

and push the tag to github

git push origin master v1.3.3

Contributing

Sending pull requests is easiest. For large or architectual changes, please email one of the authors in the source code.

We follow the Google JavaScript Style Guide. All commits on the master branch must pass the most stringent compilation settings and pass all unit tests.

A few coding dialects we follow:

• Preferred variable naming is like_this, notLikeThis. Function names areLikeThis.
• Assume native types (boolean, number, string) are not nullable. If a nullable type is used, it is different from undefined. Using undefined for missing values in native type is encourage over null.

Library design

• Library API should be similar to IndexedDB API and use exact terminology and concept in the IndexedDB specification. So that, people who already familiar with it can pick up immediately as well as go forward with native API.
• Simple operations should be easy to use as well as optimized for it. Also impose user to use efficient methods while making inefficient ways very difficult or impossible.
• For complex query, helper utility functions and classes will be provided. Storage class has deep understanding about these helper classes and do optimization behind the sense.
• Memory efficient and must not use buffer memory. If buffer is used, it must be explicit. Memory leak is unacceptable.
• Provide extensive error and log message in debug mode, spare no expense since we will strip them in production binary. Error and exception should be thrown as soon as possible, preferable before async callback.
• Since this API is very simple, fallback to WebSQL and WebStorage should be straight forward. This library design have no consideration for these storage mechanisms.

Bug reports

Please file an issue for bug reports describing how we could reproduce your problem. We will try address any subtle problems, memory and speed performance issues, and even extending the features of the IndexedDB API.

You may also ask questions in Stackoverflow #ydn-db with ydb-db hash, or follow us on Twitter @yathit.

License

Licensed under the Apache License, Version 2.0

ydn-db's Issues

404 on examples

Hi, I tried to run your examples but it return a 404. Im wondering if you can fix it. Since I wanted to test your tool really quick on ie9 and see how it works.

But for the mean time I wanted to try building it on my own. And this is really painful to set it up.

Async/Transaction/Something problem

I'm trying to cache data in a simple cacheService module I built for my angular app.

The data gets returned from the WebApi. I "put" the data into a store and return a promise so my calling client can continue the process.

From here I want to access the store where I just saved data and apply various filters.

I think the data is not ready for me to query because I can't seem to get to it.

I can't seem to find much documentation on "put" to understand how to know when it's done saving so that I know when it's safe to fetch the data again. All the examples fetch the data immediately as if it were a synchronous call. Maybe it is. I'm new so I don't quite understand yet.

All I know is I spent all day on this and now all night.

What happens is when I press a button to retrieve data I get nothing back into the UI, when I have not previously requested the data. The second time I press my button I get data but it's the data from the original request and not the newest data.

Summary:

1. Fetch data via Web Api and place it into an IndexedDB data store with YDN-DB
2. Return control to my UI
3. UI executes a query against the IndexedDB data store.
4. No data is to be found.

Thoughts?

Here is my cacheService. It's probably a mess to a veteran but again, I'm new.

mashupApp.service('cacheService', function ($http, $q, $log) {

var isCacheStale = function (cacheName, minutes) {
    // dbCache: is 'mashCache'
    // store: No store is provided but might be added later to remove constaint 'mashCacheAge'
    // subject: is the name of the cache being evaluated.
    // minutes: is the number of minutes before the cache is considered stale.

    var deferred = $q.defer();
    var result = true;

    // get milliseconds version of minutes.
    var ageToleranceMilliseconds = (minutes * 60) * 1000;
    var currentDateMilliseconds = new Date().getTime();

    // checking dbCacheReady because, if a page loads to quickly, before the cache database is ready
    // then accessing it will cause an error.
    (function wait() {
        if (dbCacheReady) {

            // The database is ready so go ahead and begin checking for stale data.
            try {
                dbCache.executeSql('SELECT * FROM mashCacheAge WHERE id = \'' + cacheName + '\'').then(function (record) {
                    $log.log('mashCacheAge record for: [ ' + cacheName + ' ] cache.');
                    $log.log(record);

                    // if no record is returned then it is considered stale.
                    var recordCount = record.length;
                    if (recordCount > 0) {
                        var durationMilliseconds = currentDateMilliseconds - record[0].updatedDate;

                        // Check if the data is stale.
                        if (durationMilliseconds > ageToleranceMilliseconds) {
                            result = true;
                        }
                        else { result = false; }

                        deferred.resolve(result);
                    }
                    else {
                        // no records found so this cache is considered stale.
                        deferred.resolve(true);
                    }
                });
            }
            catch (e) {
                // no data store for the cache was found so it is considered stale.
                deferred.resolve(true);
            }

        } else {
            // Giving the cache database a moment to set up.
            setTimeout(wait, 500);
        }
    })();

    return deferred.promise;
};

// Updates the age of any cacheName.
var updateCacheAge = function (cacheName) {
    // updates the mashCacheAge.  This helps track how old/stale a cache is.

    var cacheJSON = {
        id: cacheName,
        updatedDate: new Date().getTime()
    };

    dbCache.put({ name: 'mashCacheAge', keyPath: 'id' }, cacheJSON);

}

var getCache = function (cacheName) {
    var deferred = $q.defer();

    // waiting for dbCacheReady because accessing the cache to early causes an error.
    (function wait() {
        if (dbCacheReady) {

            dbCache.executeSql('SELECT * FROM \'' + cacheName + '\'').then(function (record) {
                deferred.resolve(record);
            });

        } else { setTimeout(wait, 500); }
    })();

    return deferred.promise;
}

// --------------------------------------------------------------------------------
// If the machCache is too old then all the data is wiped out and the user starts again.
// --------------------------------------------------------------------------------
var minutesInOneWeek = 10080;  // 10080 = 1 week
isCacheStale('mashCacheStart', minutesInOneWeek).then(function (result) {
    //alert(result);
    if (result) {
        // TODO: Add this to mashup logging.
        // the mashCache is too old and needs cleared.
        //ydn.db.deleteDatabase('mashCache');
        //setTimeout(function () { alert('mashCache was stale so deleted.'); }, 1);
        //dbCache.clear();
        // The first time through there will be no objects and this clear will fail. 
        // TODO: update this to could objects before calling
        try { dbCache.clear(); } catch (e) { }
        $log.log('mashCache was stale so deleted.');
        updateCacheAge('mashCacheStart');
    };
});
// --------------------------------------------------------------------------------
// --------------------------------------------------------------------------------
// --------------------------------------------------------------------------------

return {
    // Retrieve cache
    getCache: function (cacheName) {
        var deferred = $q.defer();
        getCache(cacheName).then(function (data) {
            deferred.resolve(data);
        });
        return deferred.promise;
    },
    getData: function (cacheName, schema, webApiUrl, staleMinutes) {

        var deferred = $q.defer();
        // var cacheName = 'getItems2';

        // Check if the cache is stale.
        isCacheStale(cacheName, staleMinutes).then(function (cacheIsStale) {
            // If cache stale then get new data.
            if (cacheIsStale) {
                // cache has become stale so retrieving fresh data.
                $http.get(webApiUrl, { withCredentials: true })
                    .success(function (data) {

                        //#region

                        // -------------------------------------------------------------------------------
                        // Schema will be passed in so that it can be generated on the fly.
                        // When a schema for an object store changes it will break and cause
                        // an error.  We will look for that error and simply delete the
                        // indexedDB database so the next time this is caused the new schema
                        // can take hold.  This doesn't seem ideal but the impact is only
                        // to performance when schemas change for cached data.
                        // -------------------------------------------------------------------------------
                        // Normally the schema is defined up front and versioned.  For the cache we want
                        // developers to have one less thing to consider and allow the mashup core
                        // to be less coupled to the caching needs of other mashup applications.
                        // -------------------------------------------------------------------------------

                        //#endregion

                        try {
                            // add data to cache
                            dbCache.put(schema, data);
                            // updateCacheAge
                            updateCacheAge(cacheName);
                        }
                        catch (err) {
                            $log.log(err);
                            indexedDB.deleteDatabase('mashCache');
                            $log.log("IndexedDB error on updating a cache. Deleted database to allow new schema.");
                            alert("IndexedDB error on updating a cache. Deleted database to allow new schema.");

                        }

                        // return web api data to the client
                        // async alert() so the performance perception isn't affected.
                        //setTimeout(function () { alert('web api data'); }, 1);
                        deferred.resolve(data);
                    })
                .error(function () {
                    // if the call fails then return the current cache.
                    // TODO: make an async call to let someone know a service failed?
                    alert("Web Api Error");
                    getCache(cacheName).then(function (data) {
                        // async alert()
                        // setTimeout(function () { alert('cache data'); }, 1);
                        deferred.resolve(data);
                    });
                });
            } else {
                // cached data is still good so return it.
                getCache(cacheName).then(function (data) {
                    // async alert()
                    //setTimeout(function () { alert('cache data'); }, 1);
                    deferred.resolve(data);
                });
            }
        });
        return deferred.promise;
    }

};

});

Here is the client code. Or at least it's a client to the service. (portion)

},

    getExample4items: function (staleMinutes, id) {
        var deferred = $q.defer();

        var cacheName = 'MashupExamples_example4items';
        var schema = { name: cacheName, keyPath: 'id' };
        var webApiUrl = 'http://localhost:50000/api/MashupExamples/Items/';

        try {
            cacheService.getData(cacheName, schema, webApiUrl, staleMinutes).then(function (data) {

                try {

                    var record = _.where(data, { 'id': parseInt(id) });
                    deferred.resolve(record);

                } catch (e) { $log.log(e); }

            });
        } catch (e) { $log.log(e); }


        return deferred.promise;
    },

You'll notice that I'm passing in the schema with the call. That's because I don't want to define a schema for the entire application up front and deal with all the versioning. I seem to have been able to work around this.

Thanks,
Bob

Retrieving datatype(INTEGER) sorted list.

My Code:

var schema = {
                    version: 2,
                    stores: [{
                        name: 'Employee',
                        keyPath: "name",
                        indexes: [{
                            keyPath: 'phone',
                            type: 'INTEGER'
                        }]
                    }]
                };

I want to retrieve my data sorted my phone which is INTEGER. I have tried all Sorting method like ,

db.values("Employee", "phone", null, 100)).done(function(r) {
  console.log("list of objects sorted by Phone", r);
});

or

db.values("Employee", "phone")).done(function(r) {
  console.log("list of objects sorted by phone", r);
 });

Result :

1
11
2
3
33
4

Where my expected result should be:

1
2
3
4
11
33

Sorting its considering INTEGER as string. Is there way to sort data by datatype while retrieving from object store.?

SortedMerge and bound keyrange doesn't make it

Hi,
I would like to execute a query like that : "select id from storage where userId=toto and read=1 and date between date1 and date2"

Using SortedMerge on indexIterators doesnt give coherent result. I think the reason is that the bounded iterator isn't sorted by primary key. What would be the best way to solve it ? Imagine that i have other criterias (and possibly another date range)

Here is my code :

var schema = {
stores : [{
name : "storage",
keyPath : [
"id" ],
indexes : [
{ keyPath : [ "userId", "read"]},
{ keyPath : [ "userId", "date"]},
]
}]};
...
var i1 = new ydn.db.IndexIterator("storage", "userId, read", ydn.db.keyrange.only(["toto",1]));
var i2 = new ydn.db.IndexIterator("storage", "userId, date", ydn.db.keyrange.bound(["toto",new Date(0)], ["toto",new Date()]));

var out = [];
var join_algo = new ydn.db.algo.SortedMerge(out);
db.scan(join_algo, [i1, i2]).done( ... );

removing based on keyrange

I'm sure it's something I'm doing wrong but here is my code. I'm trying to remove all records with an index value of less than a value I give it. In this case I'm storing the index as a long date value so I want to remove all records older than a week.

There error I get says that it cannot find the index "logId" but it's definitely there.
"Uncaught ydn.error.ArgumentException: index: logId not found in log."

Here is the table in the Resource tab.

What do you think?

    var schema = {
        stores: [{
            name: 'log',
            keyPath: 'logId',
            autoIncrement: false,
        },
        {
            name: 'heartbeat',
            keyPath: 'logId',
            autoIncrement: false
        }]
    };

    var logDb = new ydn.db.Storage('logServiceDB', schema);

                   // delete all records older than 1 week.
                    var indexLongDate = new Date().getTime();// - 7;
                    // TODO: figure out how to add ydn to angular modules to avoid implicit declaration.
                    var keyRange = ydn.db.KeyRange.upperBound(indexLongDate, true);

                    var req = logDb.remove('log', logId, keyRange)
                    .fail(function (e) {
                        alert(e);
                        throw e;
                    })
                        .done(function (count) {
                            alert(count);
                        });

404 on todo example, missing ydn-db-min-0.3.js

Getting this error when trying to run the included todo example:
Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost/ydn/jsc/ydn-db-min-0.3.js

I couldn't find a compiled version of the ydn-db js file anywhere in the repository or anywhere for download, just the jquery version which has no documentation. It would really lower the bar and make this project much more popular if it worked out of the box and did not require that developers learn closure and ant just to get the js file. Hope this gets fixed.

Thanks

Feedback

Hello, I could be mistaken but "limit" is not used in SortedMerge solver (V 1.0.4). Any plan to implement it ? Actually i have to store all keys in memory to paginate results also, adding an "offset" feature will be great for large dataset :)

However, great library !

How do I build a single file that also includes fulltext module?

I got the example to work but I have no clue how to build it to a single file...

Internet Explorer 7 string indexes

Application return proper number for rows found for db.values, however they are all undefined, while in localStorage data is present.

//src/ydn/db/base/utils.js:503
  return this.current = this.index < this.lastIndex ? parseInt(this.string[++this.index] +
    this.string[++this.index], 16) : null;

Using indexes in string is unsupported in ES3 (IE7+) rendering library useless for IE7-, however this easy fix restore the support for IE7, while not affecting modern browsers:

//src/ydn/db/base/utils.js:503
  return this.current = this.index < this.lastIndex ? parseInt(this.string.charAt(++this.index) +
    this.string.charAt(++this.index), 16) : null;

Use of window and console objects in the library while in web worker process

When I was trying to use the ydn-db in web-worker process, I found that there are many places where window object is being used.

However worker process do not allow usage of window in it, so we cannot use the library in worker.

Secondly, it is not recommended to use console objects in worker for logging, making it difficult to use the ydn in worker.However console works in chrome webworker but not in safari webworker.

I request to make one library where dom objects(window) and console is not used.

how to implement promise-based CRUD events?

from what i understand of the docs, the following adds a single event listener for all update events:

db.addEventListener(['updated'], function(event) {

how do i get a callback for the last update event, or a specific one? i'm trying to wrap YDN with something like this:

mystore.save('users', { name: 'joe' }, 24).then(function(result) {
   // handle something here
});

two, ... indexs multiEntry

hello, how you can implement a request?

shema = {
stores: [{
names: 'table1',
keyPath: 'id',
indexes[
{
keyPath: 'tag1',
multiEntry: true
},{
keyPath: 'tag2',
multiEntry: true
}
]
}]
}

WHERE tag1 = 1 and tag2 = 2

Tried and failed: where().where() and Nested-loop join

Deleting database

Hi,

I am using ydn library from the link below :- http://dev.yathit.com/ydn-db/getting-started.html

It has created a database in indexed db . Now I want to know how to remove that database as did not get any code to delete a database code is there only to create database and clear records of database.

Intertab communication support

I recommend to add support for intertab communication, similar to this solution based on the storage event of localstorage:
https://github.com/diy/intercom.js

I mean, that you can build something like this on top of your facade. (I am not sure whether your current lib supports this event based technology.)

multiple where conditions

Is it possible to make multiple where conditions?

var q = db.from('people').where('age', '=', 25);

i want to get people who are age 25 and who live in state = ny

different result using polyfill websql (see event.getVersion result), data lost on apps re-open

thanks for the great wrapper,

my default setting options for creating db was :

var options = {mechanisms: ['websql','indexeddb']};

using indexeddb, the data still remain on device even if i closed and re-open apps again, but when i forced it to use websql (for ios and older samsung's android), the data was lost when i closed the application and reopen it.

i'm using latest ydn-db library (1.0.3).

var is_updated = event.getVersion() != event.getOldVersion();
        if (is_updated) {
            console.log('database connected with new schema');
        } else if (isNaN(event.getOldVersion()))  {
            console.log('new database created');
       } else {
            console.log('existing database connected');
        }

results (websql) :
database connected with new schema

results (indexeddb) :
existing database connected

can you explain me the different ?

Querying with multiple conditions on the same field

Hi.

How do I write a query similar to SQL IN (or OR WHERE) ?

Something like SELECT * FROM table WHERE id IN (1,5,10)
or
SELECT * FROM table WHERE id = 1 OR id = 5 OR id = 10

Thanks

Securing data

I recommend to add support encryption technologies, for example

• http://www.jcryption.org/
• https://code.google.com/p/crypto-js/
• https://github.com/bitwiseshiftleft/sjcl
  etc...

Ofc. I mean that an encryption interface is required...

How to convert ts file of ydn to js

I am no able to convert ts file to js to use full fledged ydn

please help me to convert

Any example of integrating ydn-db with backbonejs

Wondering whether this library could be integrated with backbonejs. Have anyone done this? Any examples would be appreciated.

data fetching in query problem

No comment

Full text search - broken in IE11?

Hi there,

I'm having trouble with the full text searching library in YDN-DB. It worked previously in v0.8.4 beta, but it seems to be broken in v1.1.1...

Fiddle here: http://jsfiddle.net/hmtphw6b/1/ (example taken from documentation page: http://dev.yathit.com/api/ydn/db/storage.html)

If you load in IE11, it throws a 'DataError'. If you load in Chrome it returns results as expected.

Any ideas? Thanks in advance!

u.openDatabase

Hello ,

Thanks for answering all my queries related to the database.

when ever I try to delete the database by the below code
ydn.db.deleteDatabase('dbname').done(function(){
}
it gives error in firebug ' u.openDatabase is not a function ' in ydn.db-dev.js however the database is deleted.

Thanks

Deletion of a range of items

I am writing an app using YDN db as database.

I will like a scenario where only x items are available in a particular store most times, i.e at intervals i delete old items such that only x items are left.
The problem is, this works well upto a point. Sometimes, i just find out the whole items have been deleted.
I have searched and traced to see what might be responsible for this. i think maybe there is a race issue or something.
This is my code below, pls can you help check it?

ydb.count(store).done(function(cnt) {
    var x=20;
    var DeleteEndsAt = cnt - x;

    if(DeleteEndsAt>0){
        // console.log('Ended');
        ydb.from(store).order('utcdate').list(DeleteEndsAt).done(function(list){

           for (var i = 0; i < list.length; i++) {
            var r=list[i];    
            ydb.remove(store,r.id);
           }
        });
    }else{
        // console.log('Elsed');
    }
}

Native promises

Hi,

I see you're using Goog's Deferred. Could it be possible to use native promises instead, when available (and fallback on Deferred when they're not, as in IE) ? The main issue is that their API are not identical (for example, Promises have a 'catch' method while Deferreds don't), so a Deferred wrapper would be required.

I think it could only improve the library and keep it up-to-date with the latest browser evolutions. I'm currently working on a Promise-heavy application, and it's a bit weird to have some places where I know that I'm working with something that look like a Promise, smell like a Promise, but is not a Promise and will crash the code if used wrong :)

ydn-db needs a website

When you have >20 questions about your project on StackOverflow, it's time to start thinking about a real website : )

Demos link is not working :/

http://dev.yathit.com/index/demos.html

iOS 8 not working at all.

Safari in iOS 8 is not recognizing the ydn plugin at all. Have you been able to identify the issues? And do you have any time frame in mind for releasing an update that will work with iOS 8?

Where clause insensitive query search.

Want to query database in insensitive manner. Is there anyway to achieve it?

where(field_name, op, value, op2, value2)

produce Case Sensitive query need to avoid case sensitiveness.

'webkitIDBRequest' is deprecated. Please use 'IDBRequest' instead

Hi,

Chrome version 38.0.2125.101 m (latest update to this date)

Modify schema in WebSQL: Data is lost

Hi,

I noticed that if I modify the schema of a certain store in WebSQL, all the data in that store is lost. In IndexedDB it works fine, the data is kept.

I created a simple jsfiddle to show it:

https://jsfiddle.net/otb1wbet/

If you look at the console you'll see that once the database is created with another schema I cannot retrieve the entries anymore.

If this data should be migrated manually by the developer, can you please give me some hints on how to do it?

Thank you

ydn.db.deleteDatabase not working in ipad

Hello , I am creating an application which run both in desktop and ipad. Below is the schema
var schema1 ={ name:'todo', keyPath:"id", autoIncrement: true, indexes: [{ keyPath: 'opt', name: 'opt', // usually omitted. generally same as keyPath. multiEntry: true }, { name: 'qid', keyPath:'qid' }, { name: 'sidn', keyPath:'sidn' }, { name: 'license', keyPath:'license' }, { name: 'publisher', keyPath:'publisher' }, { name: 'lp', keyPath: ['license', 'publisher'] } ] };

var schema2 = { name:'offline_sdn', keyPath:"id", autoIncrement: true, indexes: [{ keyPath: 'sdn', name: 'sdn' }] };

var schema ={stores:[schema1,schema2]};
var db = new ydn.db.Storage('todo_2',schema);

when I run the application in desktop it gives database type indexed db and in ipad it gives websql. when I use the below code

ydn.db.deleteDatabase(db.getName(),db.getType()).done(function(){ alert('deleted db'); db.close(); }).fail(function(e){ alert('failed to delete'+e); });

it worked in case of desktop as on page refreshing show new instance of database where as in case of ipad it does not delete the database . please help.

Where() on compount indexes does not work

I have tried several times to query on a compound index without success.

{
  keyPath: ['foo', 'bar']
}

.where(store, ['foo', 'bar'], ...) // Using compound index
.where(store, 'foo, bar', ...)  // Expected name by YDN?
.where(store, 'foo,bar', ...)  // Expected name by YDN?
.where(store, 'foo_bar', ...) // Custom name

With, or without name, lead to no success. Sometimes it appeared that it was working but in fact the where was not filtering anything. I might have done something wrong, but I couldn't debug because the source code was minified and I do not have time to start building the app.

Cheers,
Fred

q = new ydn.db.Query() undefined

Hello I am using code in the example folder from the link below

https://github.com/yathit/ydn-db

but when I tried to use q = new ydn.db.Query(); it give undefined error should include other js files too as it using only ydn.db-dev.js file

also I want to know how to get the out of line auto incremented primary key in for loop
since other indexes are fetched by their index name but this key has no name so how to get it.

iOS 8 error with multiple tables in one database (IndexedDB is used, not WebSQL)

Hei there,

i got a problem on iOS in Safari if I create on startup a Storage with more then one table. The error I get is the following:

"DOM IDBDatabase Expception 8: An operation failed because the requested database object could not be found."

This only appaers if you open the Website in Safari direct and not as an Homescreen-App. I think apps from the homescreen run in a different state and dont support indexddb already.

To recreate this error take your ToDo-List Example App and add a second table into your database and run the app.

var schema = {
  stores:[
  {
    name:'todo',
    keyPath:"timeStamp"
  },
  {
    name:'todo2',
    keyPath:"timeStamp"
  }
  ]
};

After some research i discoverd that on iOS 8 we use indexeddb, not like on iOS 7 WebSQL. Since your own code suggets to not use indexedDB on Safari (cause of their bugs), it looks like the following code returns true, and not false for iOS devices:

/**
 * @final
 * @return {boolean} return indexedDB support on run time.
 */
ydn.db.con.IndexedDb.isSupported = function() {
  if (!ydn.db.base.indexedDb) {
    return false;
  }
  if (goog.userAgent.product.SAFARI)
  {
    // IndexedDB in Safari is too buggy at this moment.
    return false;
  }
  return !!ydn.db.base.indexedDb;
};

To fix the problem, I only changed the above code to:

/**
 * @final
 * @return {boolean} return indexedDB support on run time.
 */
ydn.db.con.IndexedDb.isSupported = function() {
  if (!ydn.db.base.indexedDb) {
    return false;
  }
  if (goog.userAgent.product.SAFARI || goog.labs.userAgent.browser.matchIosWebview_)
  {
    // IndexedDB in Safari is too buggy at this moment.
    return false;
  }
  return !!ydn.db.base.indexedDb;
};

Now on iOS we use WebSQL again and all Works like before.
I hope this is detailed enought to describe the Problem and the ongoing fix.

Greets Sascha.

PS: Thanks for this nice Wrapper for Databases!

Error using RequireJS module

ReferenceError: Can't find variable: exports
global codeydn.db-npm-dev.js:333
dns.js:111

Which points to the last line of the main file:

= a.f, c.a || (c.a = []), c.a.push(b), a.b=!0), jc().b = Wb)
});
oa("ydn.db.Storage", wh);
})();

//# sourceMappingURL=ydn.db-dev.js.map

exports.ydn = ydn;

I'm getting it in both Firefox and Safari, here is my include code:

define('DNS', function(require) {
  'use strict';

  var jquery = require('jquery'),
    URI = require('URI'),
    SecondLevelDomains = require('SecondLevelDomains'),
    punycode = require('punycode'),
    Record = require('Record'),
    ydn = require('ydn-db');

  function DNS() {
    this.db = new ydn.db.Storage('speechjs');
    this.pubs = {};
    //TODO: Load providers from localStorage
    this.pubs.speechis = function(name) {
      $.ajax({
        url     : 'http://api.bits.name',
        type    : 'get',
        data    : { name: name },
        dataType: 'json',
        async   : false,
        fail    : function(jqXHR, status, error) {
          console.error(
            'Call failed :' + status,
            "Error: " + error,
            jqXHR);
        },
        done    : function(data) {
          return data;
        }
      });
    }
  }

And my function calls this.db.put('records', r, r.name); as well as:

    db.get('records', name).always(function(record){
      console.log(record);
    });

I'll get it live on a server ASAP.

Schema Versioning in IndexedDB not working

If I add schema = {stores : [some_schema], version : 1}; It works in websql but it does not work in indexeddb

Firefox 36 Beta bug

Auto updates just moved me to FF 36 Beta, which caused my application to stop working. I tried out the demo at http://yathit.github.io/ydndb-demo/todo/todo.html and it has the same error to the console as follows. I am searching FF bugs to see if there are any reports but thought I would give you a heads up.

log:open request to database "todo_2"  cause error of UnknownError

21:6:59.180 log:undefined
21:6:59.181 log:Storage:todo_2: opening fail
21:6:59.182 log:Storage:todo_2: database connection fail undefined
UnknownError    

...aultPrevented)){e.q.close();e.q=null;var b=Error();b.name=a.type;e.Fa(b)}},f.cal...

ydn.db-dev.js (line 257)
21:6:59.197 log:Purging 0 transactions request.

Checking local Storage database size

Hello,

Is there any way to get the size of local database created by browser on ipad and how much space is left .

Web Worker Access

I noticed in another thread that you planned on supporting Web Workers via a post-message hack. Have you had any progress on that? Do you still plan on implementing it?

compound index and key joining

Hello Yathit

Thanks you for your patience for answering my queries.

I checked the link below -
http://dev.yathit.com/ydn-db/nosql-query.html

for compound index and key joining but did not get it . I am attaching snap shot of scheme with data . Can you please provide the proper syntax for query or provide me the link of working example.
I am using the below code here 1817 is my qid and 2 is sidn

key_range = ydn.db.KeyRange.bound(['1817', '2']);
db.values(new ydn.db.IndexValueCursors('todo',key_range)).done(function(d) {
console.log(d);
}
);

It gives me error undefined is not a function in line ( new ydn.db.IndexValueCursors() )

can you please let me know is there I have to include another js file as I am using only ydn.db-dev.js and is my syntax is write

Problem on IE 11

Running on IE11 I get this error on StoreSchema:

connection failed with ydn.error.ConstraintError by different schema: store: "Catalog" index "category, name" keyPath, expect: category,name, but: category,name

Any suggestion?

bower.json main is wrong

Hey, they file defined in bower.json as main is not there. It should be "jsc/ydn.db-dev.js" not "jsc/ydn.db-npm-dev.js",

Provide non-minified version

It can be really handy at times to have the non-minified version.

Please, could you ship them?

warning: "'webkitIDBRequest' is deprecated. Please use 'IDBRequest' instead."

Browser: Chrome Version 41.0.2272.76 (64-bit)
"ydn.db": "~1.0.3"

encountered at load time

NOTE: i'm using browserify to concatenate ydn-db into a single minified vendor bundle

Deprecation warnings...

On recent versions of Chrome, I get warnings "'webkitIDBRequest' is deprecated. Please use 'IDBRequest' instead.". While it appears to be a simple fix, I leave it to someone who already knows this code.

safari 8

[Error] NotFoundError: DOM IDBDatabase Exception 8: An operation failed because the requested database object could not be found.
transaction
M
onsuccess

getLastVersion returns NAN

getLastVersion returns NAN
console.log(event.getOldVersion());

        db.addEventListener('ready', function(event) {
            var is_updated = event.getVersion() != event.getOldVersion();
            if (is_updated) {
                console.log('database connected with new schema');
                console.log(event.getOldVersion());
                migrations.run(event.getOldVersion());
            } else if (isNaN(event.getOldVersion()))  {
                console.log('new database created');
            } else {
                console.log('existing database connected');
            }
            app.db = db;
            app.DBconnected = true;
        });

FireFox not working

http://yathit.github.io/ydndb-demo/todo/todo.html

Demo not working in FireFox 33

Dev and production environments

I recommend you to add support for the main component managers:

• https://github.com/component/component
• https://github.com/bower/bower
• https://www.npmjs.org/
• http://jamjs.org/docs

and test the lib on

• http://browserify.org/
• http://requirejs.org/

too with

• https://github.com/karma-runner
  (since jstd is no longer supported).

I found an older lib, which contains adapters to other storage technologies:
http://pablotron.org/?cid=1557
Maybe it is possible to merge in for example a cookie fallback, etc...