so, i am progressing in my understanding of how to program key value stores
i had a scheme set up, where for a second level caching scheme, there was an access counter record with also the size of the event itself, and i was zeroing that size as well as flipping the event record for its ID
this turned out to be a problem somehow, i can't quite figure out how, but eventually the mark step of the garbage collection code was marking already deleted events and for several days i have been fighting with this and got one small victory yesterday when i slimmed it down a bit
i just was deleting the counter record, then it wasn't found in the mark step and voila, no more sweeps of already swept items
so, i've learned something here, that is a reflection of other learning about simplicity
every key in a key value store should be simple, one key, one value
it is a code smell when your value is composite, but divisible, ie, in my example, the access timestamp AND the storage size of the event
so, tomorrow i'm revising the code so the access counter is just a timestamp, and the event itself signifies having been pushed out to a second level store by flipping the whole event for the event id (hash)
just as previously, removing the counter event solved the problem, i expect this will solve the problem, as now the access counter only does one thing: update it every time it comes up in a query
and the event itself is the primary source of "is this now pushed out to the level 2 store?"
some time in the future i am going to write a go-native style key value store wrapper around badger that lets you abstract out all the codec parts of the code so you just store keys that are based on structs and define them with simple encoders that turn things like hashes into 8 byte prefix codes and such
don't have the time to build out the nice clean interface for now but i've learned something in the last week of pain, and i will definitely be using this hard won knowledge in the future
#devstr #golang #keyvalue #deardiary #programming #nostr
Showing page 1 of
1 pages
