Collection Data Store

A collection data store is simply a collection of items. Data stored in collection data stores is indexed to make it easy to work with.

API

The data storage API is available in WebApps and RESTApps.

// retrieve an instance of the storage API
const storage = require('storage');
// get a collection store
// if no collection store exists with the provided name, one will be created
// (note: the name must also be explicitly registered in the app's manifest.json)
const collectionDataStore = storage.getCollectionDataStore('myCollectionStore');

API variants [@since 5.2]

The data storage API is available in two variants. One is callback-based whereas the other is callback-free. Please note that both variants execute synchronously, which probably makes the callback-free approach easier to work with.

Switch to callback-based API documentation (pre Sitevision 5.2)

add(data) [@since 5.2]

Adds an item to a collection.
Returns the stored item.

Argument	Description
`data`	JSON data to store

const person = {
   name: 'Foo',
   age: 33
};

try {
   collectionDataStore.add(person);
} catch (e) {
   // [Error handling](https://developer.sitevision.se/docs/data-storage/error-handling)
}

addAll(data) [@since 5.2]

Adds an array of items to a collection

Argument	Description
`data`	An array of objects to store

const people = [
   {
      name: 'Foo',
      age: 33
   },
   {
      name: 'Bar',
      age: 34
   }
];


try {
   collectionDataStore.addAll(people);
} catch (e) {
   // [Error handling](https://developer.sitevision.se/docs/data-storage/error-handling)
}

get(dsid) [@since 5.2]

Gets a specific item in a collection

Argument	Description
`dsid`	The `dsid` to identify the item to retrieve data from

try {
   item = collectionDataStore.get(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

set(dsid, data) [@since 5.2]

Updates a specific item in a collection (partial update)
Returns the stored item.

Argument	Description
`dsid`	The `dsid` to identify the item to update
`data`	JSON data to update the item

try {
   collectionDataStore.set(dsid, data);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

If a property value in the data is set to null, the property will be removed

remove(dsid) [@since 5.2]

Removes a specific item in a collection.
Returns the removed item.

Argument	Description
`dsid`	The `dsid` to identify the item to remove

try {
   collectionDataStore.remove(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

removeAll() [@since 5.2]

Removes all items in a collection

try {
   collectionDataStore.removeAll();
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

Search

First level data is asynchronously indexed in a new DataStorage Solr index whenever an entry is mutated (add/update/remove). Queries are performed using regular Solr-syntax.

Index fields (object keys) are prefixed with 'ds' and its type. Numerical values will also get a numerical field.

Values for keys that contains "sortable" (case-insensitive) somewhere in the name will also get a sortable field [@since 8.1].

Index field name	Technical description of index field
`ds.analyzed.<key>`	Analyzed, multi-valued and stored solr.TextField
`ds.double.<key>`	Numerical, single-valued and stored solr.TrieDoubleField
`ds.sortable.<sortable-named key>`	Single-valued sortable field [@since 8.1]

Value note! Only base-type values (string, boolean, number) and arrays of such values will be indexed. Hence, object values and array of object(-s) won't be indexed.

You can search multiple fields at once to create specific queries. E.g:
+ds.analyzed.name:Foo +ds.analyzed.surname:Bar

find(query, count, skip)

Returns a search result

Argument	Description	Default
`query`	The search query (Solr query syntax)
`count`	Maximum number of hits to return	`10`
`skip`	Number of leading hits to be skipped	`0`

find(query, options) [@since 8.1]

Returns a search result, potentially in a custom sort order

name Description Default

count Maximum number of hits to return 10

skip Number of leading hits to be skipped 0

orderBy

Object (or array of objects) that describes the sort order

key	value
`field`	Name of the sortable index field
`order`	"ASC" or "DESC" (ascending or descending)

modified

DESC

const cheeseResult = store.find('cheese');
const janeResult = store.find('ds.analyzed.name:jane');
const ageResult = store.find('ds.double.age:[30 TO 40]', {count: 50});

const allResult = store.find('*'); // Iterates the data store without querying the index

Search results are always sorted!

If no explicit orderBy is given, it will be sorted by the modified field.

Hence the search result is "last updated data first" if no explicit orderBy is given.

Custom sorting examples

// Find max 10 entries that has a sortableName value that starts with "mag"
// ...sorted by the sortableName field
const magResult = store.find('ds.analyzed.sortableName:mag*', {
   orderBy: {
      field: 'ds.sortable.sortableName',
      order: 'ASC'
   }
});

// Find max 100 entries that has data in the employee field
// ...sorted by the sortableName field
// ...(and if equivalent sortableName, do secondary sort by the salary field)
const employeeResult = store.find('ds.analyzed.employee:*', {
   count: 100,
   orderBy: [
   	{
      	field: 'ds.sortable.sortableName',
      	order: 'ASC'
   	},
   	{
      	field: 'ds.double.salary',
      	order: 'DESC'
   	}
   ]
});

Match-all queries ('*', '*:*') bypasses the index and iterates the data store directly if no explicit orderBy is specified. Typical use-case is when you just want to get the last updated entries.

instantIndex(dsid) [@since 6.1]

Performs instant indexing (blocking) of a collection data store post.

This function is typically useful for apps that uses query-based data that always needs to be up-to-date. Such apps would typically call this function after each "add entry" and "remove entry" operation.

Note! The blocking "index now" operation will only be performed on the index of the local cluster node! Indexes on other cluster nodes still relies on the asynchronous indexing triggered by data mutations!

Argument	Description
`dsid`	The `dsid` to identify the item to instantly index

try {
   store.instantIndex(dsid);
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

Search result

The find method returns a search result wrapper which exposes helpful methods to operate on the data.

toArray() [@since 5.2]

Gets an array of items from the search result.

try {
   data = result.toArray();
   // data -->
   [
      {
         name: 'Bar',
         dstimestamp: 1554719693812,
         dsid: 'f2412b40-59e9-11e9-ae61-7a4f433f610f',
         age: 30
      },
      {
         name: 'Foo',
         dstimestamp: 1554719663367,
         dsid: 'e01ba170-59e9-11e9-ae61-7a4f433f610f',
         age: 33
      }
   ]
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

each(callback)

Calls the supplied callback for every item in the result

Argument	Description
`callback`	Called after execution. The first parameter of the callback will contain an error object if an error occurred, `undefined` otherwise. The second parameter will contain an item from the result

result.each((err, data) => {
   if (err) {
   // handle error  
   }
   handleItem(data);
});

hasNext()

Returns a boolean indicating whether there are available items in the search result iterator

 if (result.hasNext()) {
   // ...
}

next() [@since 5.2]

Reads one item at a time from the result iterator

try {
   item = result.next();
} catch (e) {
   // [Error handling](/docs/data-storage/error-handling)
}

length(callback)

Get the number of items in the search result

Argument	Description
`callback`	Called after execution. The first parameter of the callback will contain an error object if an error occurred, `undefined` otherwise. The second parameter will contain the number of hits in the search result

result.length((err, length) => {
   if (err) {
   // handle error  
   }
   // ...
});

Did you find the content on this page useful?