#devstr #realy #progressreport
i have been in the process of building the new HTTP protocol but i wanted to first actually upgrade a production system to whatever new code i have got running to sorta make sure that it's really working, with a reasonable amount of actual users and spiders making connections to it
well, anyway, the purpose was pretty small, mainly
there is now a new index, which consists of an event ID, and the event created_at timestamp
for now, the production system has just made these for every event it has, and will generate them for every new event that comes in
but the reason for it was so that as soon as i update to the full finished MVP implementation of the protocols, that the necessary indexes are already in place
i have basically already implemented the fetch by ID endpoint and the event publish via http, the last piece is the http `/filter` endpoint, which basically provides for doing a search based on kinds, authors and tags.
the "search" field is a separate thing anyway, and is intended for full text indexes and ... well, DVMs, which are basically what i'm superseding btw
these return only the event IDs and to enable that, i needed to create a new index that stores the event ID and created_at timestamp into an index, so i can find the event by index, then use the index to find the FullIdIndex entry and then from that i can assemble a list and sort it either ascending or descending based on the timestamp in the index
without having to decode the event data, that's important, because that's an expensive operation when those two fields are all i need to get the result
and then the caller can then know, that at the moment the results were delivered, that is correct for the state, and it can then segment that list, if necessary, and then request the individual events that it actually needs, which is a big bandwidth saving as well as enabling simpler pagination by shifting the query state to the client
of course, clients can decide to update that state, and because they already have the same query's results, if they store it, they can even then also see if new event spopped up in between, as chaos tends to allow (clock skew and network latency) but the client doesn't have to have those events be thrown at it immediately as is the case with nostr standard nip-01 EVENT envelope responses on the websocket
now, they can literally just ask for a set of event IDs, and have them spewed back as line structured JSON (jsonl) and voila
far simpler to parse and understand for a humble web developer
oh yeah, the `/filter` endpoint does do time windows, and it can now also support sort orders
so it will be https://mleku.realy.lol/filter?since=123546&until=23455&sort=desc&limit=100
if you click on that link once it's implemented, it will fill your browser with JSON with one line break per result, assuming you put a set of parameters in the POST payload with the pubkeys, kinds and tags you want to search for (tag means letter and a keyword/prefix to find in the second field of the tag)
that was what this minor update was for, mainly, preparing the ground so once i have the filter endpoint, and tested it so i can get a query, it returns a list of events, then i can just send that list up on the `/events` endpoint and there you have it, same flow as before, yes, it is a two round trip but probably later this can be collapsed into a single endpoint, i made them separate so they are simpler and i want to encourage this method as it saves bandwidth and power for mobile devices, which only have to download the list of events first, and then only ask for the ones that it needs to be able to show on the display, which will be, obviously, only a few plus some extra to allow immediate scrolling
Showing page 1 of
1 pages 