Indexing

The process of extracting and providing data to an index is called Indexing. There are two indexing processes in Sitevision:

Eventual consistency

When a Sitevision object is mutated, the object will be added to an Indexing Queue. The Queue is processed asynchronously by the indexing process.

First step of the processing is to extract new indexable data. Next step is to use the extracted data, i.e. update all affected indexes with fresh data. The first step is executed on the Sitevision cluster node where the mutation occurred. The second step is propagated and executed on all Sitevision cluster nodes. In other words - it will take a moment before all indexes in the Sitevision cluster are updated. Typically within a minute but when the workload is high it might take longer.

The Indexer User

When the Node indexing process extracts new data (e.g. renders a page that was just published), it will do so as the Indexer User.

The Indexer User is of type sv:systemUser and there are some methods in SystemUserUtil that might be helpful if you want to provide custom data to indexes in your Script module or WebApp.

var systemUserUtil = require('SystemUserUtil');
 
if (systemUserUtil.isIndexer()) {
   /* Render custom data for indexing */
} else {
   /* Render data to other users */
}

Excluding blocks of content

A Script module or WebApp that explicitly wants to exclude data from being indexed by Sitevision can use the check-if-Indexer trick (see above) but a much simpler way is by adding specific sv-no-index HTML comments to the rendered output.

  • Start excluding comment: <!--sv-no-index-->
  • Stop excluding comment: <!--/sv-no-index-->

The exclusion must include both comments to be valid.

<p>
   Some text
</p>
   
<!--sv-no-index-->
<h3>
   Legal agreement
</h3>
<p>
   Some legal stuff...
</p>   
<form action="/agreements" method="post">
	<label for="name">Name: </label>
	<input type="text" name="name">
	<button type="submit">Agree</button>
</form>   
<!--/sv-no-index-->
   
<p>
   More text
</p>