Nostr View

GitCitadel @GitCitadel - 6mo

We at nostr:npub1s3ht77dq4zqnya8vjun5jp3p44pr794ru36d0ltxu65chljw8xjqd975wz are proud to announce the start of an "other stuff" relay at https://thecitadel.nostr1.com/ focused on documents and application data, not social media content. It is free-to-use for the general public and would be a great addition to any "other stuff" client default list. The only events we are currently blocking are Kind 01 notes and anything that breaks our terms of service. We are using the following relay icon: https://i.nostr.build/O4qGo.jpg Please refer to our associated wiki page for details on which events we are currently focused on. https://wikifreedia.xyz/gitcitadel-relay/

4
1
3

Laeserin @Laeserin - 6mo

nostr:npub10npj3gydmv40m70ehemmal6vsdyfl7tewgvz043g54p0x23y0s8qzztl5h nostr:npub1l2vyh47mk2p0qlsku7hg0vn29faehy9hy34ygaclpn66ukqp3afqutajft nostr:npub15qydau2hjma6ngxkl2cyar74wzyjshvl65za5k5rl69264ar2exs5cyejr nostr:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6 nostr:npub1wqfzz2p880wq0tumuae9lfwyhs8uz35xd0kr34zrvrwyh3kvrzuskcqsyn nostr:npub1ecdlntvjzexlyfale2egzvvncc8tgqsaxkl5hw7xlgjv2cxs705s9qs735

1
0
3

Laeserin @Laeserin - 6mo

Thank you to nostr:npub1356t6fpjysx9vdchfg7mryv83w4pcye6a3eeke9zvsje7s2tuv4s4k805u for the relay icon! ☺️

0
0
3

Laeserin @Laeserin - 6mo

This doesn't solve for the versioning of replaceable articles, but at least we have an additional free-to-use relay for our various clients.

0
0
3

Laeserin @Laeserin - 6mo

nostr:npub12262qa4uhw7u8gdwlgmntqtv7aye8vdcmvszkqwgs0zchel6mz7s6cgrkj

1
0
3

Laeserin @Laeserin - 6mo

I had the idea of a 20/20 rule. Keep newest 20 revisions of replaceable events or (if they have fewer) keep revisions for max 20 months.

1
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

yes, i removed the replaceable delete function, you can literally just write a filter that asks for an unbounded number of replaceable event kind associated with a pubkey and the newest version comes first and all the rest you'll find it's really easy to modify any relay to do this, just change the special cases for replaceable events as someone pointed out to me already, clients already sort and pick the newest if multiple come back from these queries, so it's just a matter of removing that racy delete also, deleting events other than when the author (or admin) sends a delete message is a silly idea, much better to have garbage collection that just scans from time to time and removes the most stale versions long after they are out of date if clients were more savvy with this, they could easily implement rollback when you make a wrong update

0
1
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

writing a GC is a solution that covers more cases with less special cases though, all it requires is a last accessed timestamp

1
1
3

Laeserin @Laeserin - 6mo

GC?

1
0
3

Laeserin @Laeserin - 6mo

Ah, garbage collection. 😅 Thanks!

0
1
3

Laeserin @Laeserin - 6mo

nostr:npub1gnwpctdec0aa00hfy4lvadftu08ccs9677mr73h9ddv2zvw8fu9smmerrq nostr:npub1wtuh24gpuxjyvnmjwlvxzg8k0elhasagfmmgz0x8vp4ltcy8ples54e7js

0
0
3

Laeserin @Laeserin - 6mo

nostr:npub18kzz4lkdtc5n729kvfunxuz287uvu9f64ywhjz43ra482t2y5sks0mx5sz

0
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

garbage collector you track each event's last access time and then when you do a GC run you grab all the event serials, along with their last access timestamp, collect the size of each of these events, sort them from lowest to highest, total it up, then count off as many events of the oldest last access that hits your "low water mark" target and then delete them in badger db i did this in a "batch" transaction which uses a divide and conquer to break the operation into a series of parallel operations (like, ideal is number of CPU threads) and it happens so fast by doing it this way, you solve multiple problems with one action, and events that haven't been accessed in a long time are the best candidates for removal... old replaceable events will naturally fall into that because clients only mainly want the newest version and most of the time accessing old versions would be an infrequent event and more often you would only want the next oldest or so anyway, so they will expire from the cache without any extra rules or logic

1
1
3

Laeserin @Laeserin - 6mo

Ah, very clever.

1
0
3

semisol @Semisol - 6mo

all relay implementations suck I’m tired of putting 2 strfry instances on different boxes and calling that a “fault tolerant” relay and others doing the same

0
1
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

storing full events you have to enable the GC size limit for it to have a high and low water mark, and you can additionally configure that if the defaults don't fit your case, and even further, you can create a second level data store, which would presumably be a shared data store that is accessed over the network, and the headroom above the high water mark will then store the indexes of events that have fallen out of the local cache but still allow fast filter searches https://mleku.net/replicatr is the core, which is a fork of khatru, and https://mleku.net/eventstore is the eventstore with the GC enabled for the "badger" and there is a "l2" event store that lets you plug in two event stores, one is usually badger, and the other can be anything else, and there is a "badgerbadger" which i wrote using two levels of badger event store, one with GC on and L2 enabled that tests the GC once your event and index storage size exceeds the size limit

1
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

it's how Go manages memory, pretty much, their algorithm is substantially more clever than mine and refined over 15 years, i would eventually want to make it dynamic so it reacts to bursts in traffic and spaces out and shortens the length of GC passes to minimise latency for client requests during heavy traffic, all of these are similar cases handled by the Go GC, which, BTW, was mostly engineered by the guy who built the V8 javascript engine, the basis of nodejs and chrome's javascript interpreter

0
1
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

btw, fiatjaf is wrong about badger, he just doesn't know how to use it or write good, bugproof binary encoding libraries... the batch processing functions are incredibly fast, like, 15gb of database can be measured in ~8 seconds and if a GC pass is needed that might take another 5-12 seconds deponding on how far over the limit it got

1
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

also, yes, that will scale, on a 20 core threadripper with 40Mb of cache and a 128gb of memory it would probably zip through that job in less than half that time

0
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

the eventstore is almost drop-in except for the definition of the (basically identical) eventstore interface most code written to work with khatru's arrays of closures can also be quickly adapted no, i haven't got to doing - full text search, right? it requires writing another index, though that may be easier to get happening sooner if you use a DB engine that already has that as a turn-key option the Internet Computer database engine has some kind of complex indexing scheme on it and likely would be easy to make it do this but the badger event store is bare bones, all it is built to do is fast filter searches and GC... it would not be hard to add more indexes but it would be a couple month's work i'd estimate

1
0
3

mleku ✝Ⓐ☯︎ @mleku - 6mo

well, i think i could get MVP in 1 month anyhow

0
0
3
Showing page 1 of 2 pages