super excited today
https://github.com/Hubmakerlabs/replicatr
my relay project, being built for a company working to make decentralised collaboration systems for business
officially has passed from WIP to late alpha, pending some cleanup and a few more parts on our first distributed database back end implementation, with the Internet Computer Protocol (yes, a smart contract blockchain thing)
the important facts are this:
- built off the base of fiatjaf's khatru
- using an extensively rewritten and reorganised fork of fiatjaf's work for zebedee, go-nostr
- a heavily modified version of fiatjaf's eventstore badger implementation, with a simple atomic mark-and-sweep garbage collection scheme that can run either
- unbounded, like relay.damus.io 🤣
- as a standalone store with a configurable static low/high water mark and maximum size and GC frequency
- as the previous, but with a second level data store that is intended to be a shared (replicated) distributed database store
there is also the beginnings of an access control system that will enable automated and manual control of access for subscription based payments and private use cases, which is configured via a DM based CLI, and of course it has full support for NIP-42 and restricts access to privileged events when in auth-required mode so that only parties to events can fetch such events
in the next week i will be polishing it and testing it extensively to make sure that it's happy and probably in the next week throw it up as a beta test for anyone to spam at it, but with auth required, so there's that also, to get more data on what hasn't been done properly yet
aside from that, the event store module and the nostr library will be separated from the main code base, and later on a khatru- style highly modular launch configuration scheme will be fully added, currently there is one main launcher with several options for the various components, ideally these become modules and can be mixed and matched and new modules easily added in the same way as khatru - it just wasn't done already because we are focusing on a specific back end target due to them being the source of funding to do the work
will be updating again when i get an instance up and running live to battle test the core and the GC enabled badger event store
#devstr #replicatr #deardiary #nostr
Remember SNL from the 80s and Deep Thoughts with Jack Handy?
I was thinking about Stephen Hawking and the woman who told him, "the earth is flat and it is turtles all the way down." That was one of the original memes. The word meme is courtesy of Richard Dawkins, a stanch atheist.
All of a sudden, I thought, "is it turtles all the way up?" What I mean by that is, how did existence come to be? I know about the Big Bang Theory and all. I also was educated in religious schools, Muslim (Sudan, Saudi Arabia) and Catholic (Williamsburg, VA). That is to say, I know about God creating the world in 7 days. Yes, it is the same story in Christianity and Islam. Surprised?
But just like a fish in a fish bowl can't "see" past its world (the bowl) and "feels" something is off because that isn't its natural habitat (the river, the sea, the ocean), we humans keep asking ourself what is the meaning of life? Where do come from? If God created us, who created God? What is the beginning of the beginning?
Maybe we are two lost souls living in a fish bowl year after year. Pink Floyd.
Maybe we, like the fish, are not in our "natural habitat" and we need help getting back home. Bitcoin fixes this.
#nostr #plebchain #grownostr #Bitcoin #asknostr
so, i am progressing in my understanding of how to program key value stores
i had a scheme set up, where for a second level caching scheme, there was an access counter record with also the size of the event itself, and i was zeroing that size as well as flipping the event record for its ID
this turned out to be a problem somehow, i can't quite figure out how, but eventually the mark step of the garbage collection code was marking already deleted events and for several days i have been fighting with this and got one small victory yesterday when i slimmed it down a bit
i just was deleting the counter record, then it wasn't found in the mark step and voila, no more sweeps of already swept items
so, i've learned something here, that is a reflection of other learning about simplicity
every key in a key value store should be simple, one key, one value
it is a code smell when your value is composite, but divisible, ie, in my example, the access timestamp AND the storage size of the event
so, tomorrow i'm revising the code so the access counter is just a timestamp, and the event itself signifies having been pushed out to a second level store by flipping the whole event for the event id (hash)
just as previously, removing the counter event solved the problem, i expect this will solve the problem, as now the access counter only does one thing: update it every time it comes up in a query
and the event itself is the primary source of "is this now pushed out to the level 2 store?"
some time in the future i am going to write a go-native style key value store wrapper around badger that lets you abstract out all the codec parts of the code so you just store keys that are based on structs and define them with simple encoders that turn things like hashes into 8 byte prefix codes and such
don't have the time to build out the nice clean interface for now but i've learned something in the last week of pain, and i will definitely be using this hard won knowledge in the future
#devstr #golang #keyvalue #deardiary #programming #nostr
Showing page 1 of
2081 pages
