Mutate and append to value support

[cmumford]

Original issue 65 created by morten.haraldsen on 2012-01-10T09:33:14.000Z:

It would be nice if one could:

• Append to an existing value atomically
• Mutator operation on the value, like +/-n

Thanks!

[cmumford]

Comment #1 originally posted by sanjay@google.com on 2012-01-18T18:06:23.000Z:

Noted. This has come up before, but we haven't decided on whether
or not to do it. You have two options without direct leveldb support:

(1) Do a read-modify-write operation to update (can be expensive
and you will have to implement some concurrency control to lock
out other writers to the same key during this oepration).

(2) Arrange your schema in such a way that every logical key
maps to a set of neighboring leveldb keys (e.g.,
logical_key#seq_number). Then updates can be done by just
creating a new key and writing the delta under that key.
And reads can be done by adding a wrapper that merges
together values that fall under the same logical key.

[cmumford]

Comment #2 originally posted by morten.haraldsen on 2012-02-03T15:58:53.000Z:

Thanks, I already use your first suggestion, as I need to read the old value to check that the new is a valid state change anyway.

[cmumford]

Comment #3 originally posted by jyf1987 on 2012-09-11T08:09:56.000Z:

i am interesting of the secondary suggestion, maybe we need to introduce a chain like store unit in the DB

[cmumford]

Comment #4 originally posted by jtconnor on 2014-03-29T20:07:54.000Z:

For option (2), how would you recommend allocating sequence numbers?

• Could have sequence number in memory in process reading/writing to db but awkward to recover sequence number if process crashes (would have to iterate over all keys and use max_sequence_number + 1).
• Could store sequence number in db but would then have to use option (1) above in order to read-modify-write to allocate a new sequence number.
• Could use a time-based sequence number but seems dangerous to trust clock after crash/restart.
• Could use an external service to allocate sequence numbers but that seems like kicking the can down the road.

What do you think?

[rickbeeloo]

Is this not supported yet? To append a value to an existing key? We could of course manually check using the get first however this is very expensive and we would not be able to use batch inserts - specifically relevant in our case (terabytes of data)