Collection Data Store

A collection data store is simply a collection of items. Data stored in collection data stores is indexed to make it easy to work with.

API

The data storage API is available in WebApps and RESTApps.

// retrieve an instance of the storage API
const storage = require('storage');
// get a collection store
// if no collection store exists with the provided name, one will be created
// (note: the name must also be explicitly registered in the app's manifest.json)
const collectionDataStore = storage.getCollectionDataStore('myCollectionStore');

API variants [@since 5.2]

The data storage API is available in two variants. One is callback-based whereas the other is callback-free. Please note that both variants execute synchronously, which probably makes the callback-free approach easier to work with.

add(data) [@since 5.2]

Adds an item to a collection.
Returns the stored item.

Argument

Description

data

JSON data to store

const person = {
   name: 'Foo',
   age: 33
};

try {
   collectionDataStore.add(person);
} catch (e) {
   // [Error handling](https://developer.sitevision.se/docs/data-storage/error-handling)
}

addAll(data) [@since 5.2]

Adds an array of items to a collection

Argument

Description

data

An array of objects to store

const people = [
   {
      name: 'Foo',
      age: 33
   },
   {
      name: 'Bar',
      age: 34
   }
];


try {
   collectionDataStore.addAll(people);
} catch (e) {
   // [Error handling](https://developer.sitevision.se/docs/data-storage/error-handling)
}

get(dsid) [@since 5.2]

Gets a specific item in a collection

Argument

Description

dsid

The dsid to identify the item to retrieve data from

try {
   item = collectionDataStore.get(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

set(dsid, data) [@since 5.2]

Updates a specific item in a collection (partial update)
Returns the stored item.

Argument

Description

dsid

The dsid to identify the item to update

data

JSON data to update the item

try {
   collectionDataStore.set(dsid, data);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

If a property is set to null, the property will be removed

remove(dsid) [@since 5.2]

Removes a specific item in a collection.
Returns the removed item.

Argument

Description

dsid

The dsid to identify the item to remove

try {
   collectionDataStore.remove(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

removeAll() [@since 5.2]

Removes all items in a collection

try {
   collectionDataStore.removeAll();
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

First level data is asynchronously indexed in a new DataStorage Solr index whenever an entry is mutated (add/update/remove). Queries are performed using regular Solr-syntax.

Index fields (object keys) are prefixed with 'ds' and its type. Numerical values will also get a numerical field.

Values for keys that contains "sortable" (case-insensitive) somewhere in the name will also get a sortable field [@since 8.1].

Index field name

Technical description of index field

ds.analyzed.<key>

Analyzed, multi-valued and stored solr.TextField

ds.double.<key>

Numerical, single-valued and stored solr.TrieDoubleField

ds.sortable.<sortable-named key>

Single-valued sortable field [@since 8.1]


find(query, count, skip)

Returns a search result

ArgumentDescriptionDefault
queryThe search query (Solr query syntax)
countMaximum number of hits to return10
skipNumber of leading hits to be skipped0

find(query, options) [@since 8.1]

Returns a search result, potentially in a custom sort order

Options is a object with three optional properties:
nameDescriptionDefault
countMaximum number of hits to return10
skipNumber of leading hits to be skipped0


orderBy

Object (or array of objects) that describes the sort order

keyvalue
fieldName of the sortable index field
order"ASC" or "DESC" (ascending or descending)

dstimestamp

DESC


const result = store.find('cheese');
store.find('ds.analyzed.name:jane');
store.find('ds.double.age:[30 TO 40]', {count: 50});

store.find('*'); // Iterates the data store without querying the index

Search results are always sorted!

If no explicit orderBy is given, it will be sorted by the modified field (i.e. "dstimestamp descending").

Hence the search result is "last updated data first" if no explicit orderBy is given.

Custom sorting examples

// Find max 10 entries that has a sortableName value that starts with "mag"
// ...sorted by the sortableName field
find('ds.analyzed.sortableName:mag*', {
   orderBy: {
      field: 'ds.sortable.sortableName',
      order: 'ASC'
   }
});

// Find max 100 entries that has data in the employee field
// ...sorted by the sortableName field
// ...(and if equivalent sortableName, do secondary sort by the salary field)
find('ds.analyzed.employee:*', {
   count: 100,
   orderBy: [
   	{
      	field: 'ds.sortable.sortableName',
      	order: 'ASC'
   	},
   	{
      	field: 'ds.double.salary',
      	order: 'DESC'
   	}
   ]
});

Match-all queries ('*', '*:*') bypasses the index and iterates the data store directly if no explicit orderBy is specified. Typical use-case is when you just want to get the last updated entries.

instantIndex(dsid) [@since 6.1]

Performs instant indexing (blocking) of a collection data store post.

This function is typically useful for apps that uses query-based data that always needs to be up-to-date. Such apps would typically call this function after each "add entry" and "remove entry" operation.

Note! The blocking "index now" operation will only be performed on the index of the local cluster node! Indexes on other cluster nodes still relies on the asynchronous indexing triggered by data mutations!

Argument

Description

dsid

The dsid to identify the item to instantly index

try {
   store.instantIndex(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

Search result

The find method returns a search result wrapper which exposes helpful methods to operate on the data.

toArray() [@since 5.2]

Gets an array of items from the search result.

try {
   data = result.toArray();
   // data -->
   [
      {
         name: 'Bar',
         dstimestamp: 1554719693812,
         dsid: 'f2412b40-59e9-11e9-ae61-7a4f433f610f',
         age: 30
      },
      {
         name: 'Foo',
         dstimestamp: 1554719663367,
         dsid: 'e01ba170-59e9-11e9-ae61-7a4f433f610f',
         age: 33
      }
   ]
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

each(callback)

Calls the supplied callback for every item in the result

Argument

Description

callback

Called after execution. The first parameter of the callback will contain an error object if an error occurred, undefined otherwise. The second parameter will contain an item from the result

result.each((err, data) => {
   if (err) {
   // handle error  
   }
   handleItem(data);
});

hasNext()

Returns a boolean indicating whether there are available items in the search result iterator

 if (result.hasNext()) {
   // ...
}

next() [@since 5.2]

Reads one item at a time from the result iterator

try {
   item = result.next();
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

length(callback)

Get the number of items in the search result

Argument

Description

callback

Called after execution. The first parameter of the callback will contain an error object if an error occurred, undefined otherwise. The second parameter will contain the number of hits in the search result

result.length((err, length) => {
   if (err) {
   // handle error  
   }
   // ...
});