BDB queues are said to support high concurrency since it only locks records not pages, and if you only have 64 byte (or so) records and your page size is 512, then locking a page could lock a large number of records. But I ran into a problem with it today...
We have multiple writes and multiple readers for the index stats table. Writes update the stats from multiple indexing threads, while consumers check the status of the index using IsIndexStale. We use snapshot isolation to prevent read locks during the IsIndexStale call, but here comes the rub.
BDB queues don't support snapshot isolation.
So that's that.
How did you get round this issue (i.e. the lack of snapshot isolation)?
ReplyDeleteI had to switch back to btrees.
ReplyDeleteThen the problem is that we need to make sure that the latest reduced, latest mapped and touches are on a different pages. The only way to do that (that I can figure out) is to put them in separate virtual databases in the physical file (index_stats.db).
So for each index, we create three virtual databases _stats, _reduce, _touches. Each one of these databases has a single key/value pair that contains the data.
This allows us to make sure that for every index, all of the stats are separately updatable.
https://github.com/mweber26/ravendb/blob/bdb-1.2/Raven.BdbProvider/IndexingStatsTable.cs