i could be wrong, but i think that if #realy has a full text search capability it will be the first nostr relay that has this built into it, instead of being a ghey DVM. the nip has existed for a long time, https://github.com/nostr-protocol/nips/blob/master/50.md
i never implemented it before though it was always on my mind, because getting realy into a fully working nip-01 compliant state was quite a lot of effort because of the shitty code i based it on, which i basically completely rewrote, almost 90% of it i'd estimate.
this will also be good for my efforts to sell the idea to my boss of shifting our data storage systems for chat and forums over to a nostr base, and because i made a HTTP API already, i only have to explain the event format to them, not the whole retarded nip-01 "API" that dare not speak that name (because it's a shit API, in so many ways)
it could well be that in the end, i'm going to be the one out of all the nostr devs who actually successfully builds nostr into a for real commercial protocol, because building a replacement for slack and gsuite has been on his mind since the beginning, i think most of what my boss is doing is trying to get funding for something bigger, so if we have already integrated chat and forum tech with nostr events, it's only a small step to include documentation, i have already now also had experience with working with international time and schedules, we can have calendar, messaging, chat, documents, i mean what more do you need out of a remote collab or just plain business data, comms and scheduling system than those?
i've now drafted the full text indexer, it uses unicode to split up the words, eliminates the duplicates, and then feeds that to a background thread that updates the indexes for each word to add the database sequence record for the event to the list of them that contain each word
it's too late in the night for me to add the search function tho, but i will continue this work tomorrow, it's much more easy for me to do this than the bunker signer at this point, because i'm new to working with teh GUI i'm using for that, and i had to refresh my memory about how to encrypt keys and all the bits associated with that.
not gonna even test the indexer tonight, i'm poopered, but i will test it tomorrow afternoon after my fiat mine work, to make sure it isn't bombing out with a panic or anything, and then to work on the actual search function
the search function is a bit complicated... first you have to take the search term, break it down into words, and find all the records and get all of their lists of database sequence numbers, and then out of that, first, assemble a progressive list of the events that come up in all of them, and then one less, and one less and one less until there is no search terms left
then you go through those lists, and you scan the content field for the locations of the text matches, and find the ones that have the longest sequences of matches in the same order as the search request text, to sort them by, and then you return the events in descending order of exact matching, and all the events that have several of the terms but not in order after that
and then the task will be complete, and #realy will have a full text search capability with sort by relevance (relevance meaning how closely the result event matches the search terms).
making the index is the easy part. finding the matches and ranking them will be quite a long function i expect. the indexer is only 117 lines of code.
Showing page 1 of
1 pages
