so, couldn't sleep because allergy still
i forgot how long the egg allergy takes to fully clear up, and it also seems to onset most of all at bedtime
so... i have an espresso to calm it down, and sit to write this binary encode/decode test
i have found the bug, i think
it's the old empty tag that should be a nil tag trick
original:
> "tags":[]
re-encodes from runtime format:
> "tags":[]
decodes back from being encoded from runtime to binary and binary back to runtime back to json:
> "tags":[[]]
well, that's the first, but hundreds of examples of this... it's from empty tags... most clients don't actually produce events with no tags at all, mostly... they are kooky looking events too, with strings of decimal numbers comma separated like they are encoding binary with decimal for whatever reason, instead of UTF-8
but digging further I also see another error:
```
{"id":"6f38c8cce14f986c86b0ee9808643d2f6827ac29168f9f96a96b6a521ea25841","pubkey":"9303ab260a6c7653fc8ea3b041d533c2c678c1e3c4772a05e9f53b8be99869c7","created_at":1715054396,"kind":5,"tags":[["e","7aea510b0c26be894ef641f0478efae0089ba1efb2f3165ad6136279bbb30bbe"],["a",
"31234:9303ab260a6c7653fc8ea3b041d533c2c678c1e3c4772a05e9f53b8be99869c7:de9917f2-9211-4375-8765-39babb31c177"],["k","31234"],["alt","Deletion event"]],"content":"","sig":"1ff9965d3bd7b4d4bcdf23312de2779f2faec93a160ab57ba570268a9042083a0b6e07cedf5cb1df8ec470c28850d21efd998619b4f58641542571ad075548e3"}
{"id":"6f38c8cce14f986c86b0ee9808643d2f6827ac29168f9f96a96b6a521ea25841","pubkey":"9303ab260a6c7653fc8ea3b041d533c2c678c1e3c4772a05e9f53b8be99869c7","created_at":1715054396,"kind":5,"tags":[["e","7aea510b0c26be894ef641f0478efae0089ba1efb2f3165ad6136279bbb30bbe"],["a",
"0:39333033616232363061366337363533666338656133623034316435333363326336373863316533633437373261303565396635336238626539393836396337:de9917f2-9211-4375-8765-39babb31c177"],["k","31234"],["alt","Deletion event"]],"content":"",
"sig":"1ff9965d3bd7b4d4bcdf23312de2779f2faec93a160ab57ba570268a9042083a0b6e07cedf5cb1df8ec470c28850d21efd998619b4f58641542571ad075548e3"}
```
there is an `a` tag in there (it's a delete) where the re-decoded form has failed to encode correctly, i've written code in the binary encoder to correctly handle a tags, they are kind:pubkey:string and it looks a lot like this one did not format the value correctly, those numbers are clearly decimal form of hexadecimal, and the initial kind value was not encoded (or failed decoding back)
i'd say i'm going to find several bugs in there with this, this is two to chalk up for tomorrow morning's work
i'd say i'll have every last bug in the binary encode/decode ironed out with this in hand
almost there with this...
found one problem, the tag writer was not putting a zero byte in to indicate the tag field is empty
this was also causing the content field to not be decoded properly because it was decoding a length value of the content instead of the tag, and finding invalid bytes or even varints following it and giving up on the decode
next obvious problem has to do with a tags, that i mentioned also... a typical example:
original
```
["a",
"32123:7759fb24cec56fc57550754ca8f6d2c60183da2537c8f38108fdf283b20a0e58:a470b862-1522-4c91-bb11-ded892d8d1
61","wss://relay.wavlake.com"]
```
mangled
```
["a",
"0:3737353966623234636563353666633537353530373534636138663664326336303138336461323533376338663338313038666
4663238336232306130653538:a470b862-1522-4c91-bb11-ded892d8d161","wss://relay.wavlake.com"]
```
if you are familiar with ascii, you'd know that 37 is the number `7` and 35 is `5` so you can see that what has happened is it has failed to encode the kind prefix (which is a decimal) and it has instead decoded the hex value, which is decoded to binary in my binary encoder for a, p and e tags field 2 and for id, pubkey and signature
the a tag is the most complicated because it contains two numbers, which i have encoded as a 16 bit prefix value and a 32 byte for the npub that follows in an `a` tag.
likely again it's some mess to do with the `a` tag processing either it's writing, or not writing the varint prefix correctly... really, the a tag should not need a length prefix until the third arbitrary text field, so i've probably messed that up somewhere
note that all of the bugs i'm finding here were outliers... events with no tags are not common, and `a` tags are mostly only used with delete events. still a problem though, because for those cases it fails to decode properly, or encode properly, or both, and this is unexpected behaviour for clients
probably there will be one or two more little glitches to get through here
like, for example, maybe i should try to make a prefix code for these tag types so if it spots this prefix code it knows the field is noncompliant and stores it verbatim
nostr:nevent1qvzqqqqqqypzqnyqqft6tz9g9pyaqjvp0s4a4tvcfvj6gkke7mddvmj86w68uwe0qyghwumn8ghj7mn0wd68ytnvv9hxgtcpz4mhxue69uhhgetnwsh8yetpd3ujumr0dshsz9mhwden5te0d4kx26m49ehx7um5wgcjucm0d5hszythwden5te0dehhxarj9emkjmn99uqzpu0wdtqumzdlrwt2ctqaatylhwcuw8j6ewyw66wypth5lhhf0xfjvhkccv
Showing page 1 of
1 pages
