Federate emoji as anonymous objects #815

Merged
floatingghost merged 1 commit from Oneric/akkoma:emoji-id into develop 2024-10-16 14:58:47 +00:00
Member

Similarly small change as those in #814, but may warrant some more attention wrt possibly federation effects.

Federation with other *omas and with *keys (+IceShrimp) should work; not sure how Mastodon and its forks handle null ids. (e.g. fedibird and chuckya support emoji reactions)


Usually an id should point to another AP object
and the image file isn’t an AP object. We currently
do not provide standalone AP objects for emoji and
don't keep track of remote emoji at all.
Thus just federate them as anonymous objects,
i.e. objects only existing within a parent context
and using an explicit null id.

IceShrimp.NET previously adopted anonymous objects
for remote emoji without any apparent issues. See:
333611f65e

Fixes: #694

Similarly small change as those in #814, but may warrant some more attention wrt possibly federation effects. Federation with other \*omas and with \*keys (+IceShrimp) should work; not sure how Mastodon and its forks handle null ids. (e.g. fedibird and chuckya support emoji reactions) --- Usually an id should point to another AP object and the image file isn’t an AP object. We currently do not provide standalone AP objects for emoji and don't keep track of remote emoji at all. Thus just federate them as anonymous objects, i.e. objects only existing within a parent context and using an explicit null id. IceShrimp.NET previously adopted anonymous objects for remote emoji without any apparent issues. See: https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/commit/333611f65eb2a65b2779ece0435b5ba84bf60e99 Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/694
Oneric added 1 commit 2024-06-23 18:49:00 +00:00
Federate emoji as anonymous objects
All checks were successful
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-amd64 Pipeline was successful
ci/woodpecker/pr/build-arm64 Pipeline was successful
ci/woodpecker/pr/docs Pipeline was successful
4ff5293093
Usually an id should point to another AP object
and the image file isn’t an AP object. We currently
do not provide standalone AP objects for emoji and
don't keep track of remote emoji at all.
Thus just federate them as anonymous objects,
i.e. objects only existing within a parent context
and using an explicit null id.

IceShrimp.NET previously adopted anonymous objects
for remote emoji without any apparent issues. See:
333611f65e

Fixes: #694
Author
Member

I was now able to confirm this works fine in practice with:

  • current Akkoma (no surprise)
  • Mastodon 4.2.10
  • Sharkey 2024.5.1 (federation code is supposed to almost match upstream Misskey sans their extensions and as mentioned before i’ve seen explicit null-id emoj handling in upstream Misskey code, so this likely holds for all *keys)

However issues arose with GtS 0.16.0; emoji got stripped with only the shortcode text remaining. I brought it up with GtS upstream

I was now able to confirm this works fine in practice with: - current Akkoma *(no surprise)* - Mastodon 4.2.10 - Sharkey 2024.5.1 *(federation code is supposed to almost match upstream Misskey sans their extensions and as mentioned before i’ve seen explicit null-id emoj handling in upstream Misskey code, so this likely holds for all \*keys)* However issues arose with GtS 0.16.0; emoji got stripped with only the shortcode text remaining. I brought it up with GtS upstream
Author
Member

GtS intends to fix this, but it’s not high prio and thus might take a while

We could either:

  • wait until the GtS fix arrives, leaving others broken instead
  • already apply this now and accept GtS users only receiving text instead of images for emoji until they get a fix
  • provide AP objects for local, admin-managed emoji using nil for all else like IceShrimp. Considering atm remote emoji can only be used in emoji reactions which are not supported by GtS and we axed fully-custom user emoji as a causality of dropping C2S support, this should cover all emoji GtS will see from us (for now)

However as is now, this can only work by tying the AP id to the shortcode, which i’m not fond of. Distinct emoji (old gets deleted, (much) later a different one added) will share an ID which might confuse some caches (especially since we can’t provide real updated timestamps yet) and the conceptually same emoji might change ID after rename (e.g. to be more consistent with other emoji of the same pack, or because the name is too generic, etc).
It might also lock us into this ID scheme even if we get better emoji handling in the future. Though perhaps, since emoji aren't to important just changing all IDs in the future might actually not be too bad in practice?

GtS intends to fix this, but it’s not high prio and thus might take a while We could either: - wait until the GtS fix arrives, leaving others broken instead - already apply this now and accept GtS users only receiving text instead of images for emoji until they get a fix - provide AP objects for local, admin-managed emoji using `nil` for all else like IceShrimp. Considering atm remote emoji can only be used in emoji reactions which are not supported by GtS and we axed fully-custom user emoji as a causality of dropping C2S support, this should cover all emoji GtS will see from us (for now) However as is now, this can only work by tying the AP id to the shortcode, which i’m not fond of. Distinct emoji *(old gets deleted, (much) later a different one added)* will share an ID which might confuse some caches *(especially since we can’t provide real `updated` timestamps yet)* and the conceptually same emoji might change ID after rename *(e.g. to be more consistent with other emoji of the same pack, or because the name is too generic, etc)*. It might also lock us into this ID scheme even if we get better emoji handling in the future. Though perhaps, since emoji aren't to important just changing all IDs in the future might actually not be too bad in practice?

i am going to make the executive decision of "lol gts can fix it their side" hehe
maybe extra breakages will spur them to bump it up the prio list

i am going to make the executive decision of "lol gts can fix it their side" hehe maybe extra breakages will spur them to bump it up the prio list
floatingghost merged commit f101886709 into develop 2024-10-16 14:58:47 +00:00
floatingghost deleted branch emoji-id 2024-10-16 14:58:47 +00:00
First-time contributor

I think this change is making emoji attachments federate as "id": null, which causes JSON-LD expansion errors. 😬

I think this change is making emoji attachments federate as `"id": null`, which causes JSON-LD expansion errors. 😬
First-time contributor

Context: We've received reports of federation issues with akko.wtf, and I verified that the affected users both have this in their actor objects:

"tag": [
  {
    "id": null
  }
]
Context: We've received reports of federation issues with akko.wtf, and I verified that the affected users both have this in their actor objects: ``` "tag": [ { "id": null } ] ```
First-time contributor

The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded

The specific error is `Invalid JSON-LD syntax; "@id" value must a string.`, verifiable at https://transform.tools/jsonld-to-expanded
First-time contributor

The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.

The issue is that `null` and `undefined` (no `id`/`@id` key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.
Author
Member

The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded

The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.

This appears to directly contradict section 3.1 “Object identifiers” of AP spec. For reference I’m copying it in full here:

ActivityPub 3.1 Object identifiers

All Objects in [ActivityStreams] should have unique global identifiers. ActivityPub extends this requirement; all objects distributed by the ActivityPub protocol MUST have unique global identifiers, unless they are intentionally transient (short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications). These identifiers must fall into one of the following groups:

  1. Publicly dereferencable URIs, such as HTTPS URIs, with their authority belonging to that of their originating server. (Publicly facing content SHOULD use HTTPS URIs).
  2. An ID explicitly specified as the JSON null object, which implies an anonymous object (a part of its parent context)

Identifiers MUST be provided for activities posted in server to server communication, unless the activity is intentionally transient. However, for client to server communication, a server receiving an object posted to the outbox with no specified id SHOULD allocate an object ID in the actor's namespace and attach it to the posted object.

All objects have the following properties:

id
   The object's unique global identifier (unless the object is transient, in which case the id MAY be omitted).

type
   The type of the object.

Ignoring the note about C2S API it says transient objects omit id fully, while anonymous object explicitly specify id as null.
Based on this i believe this is an error in the JSON-LD implementation (or alternatively, APs requirements are incompatible with JSON-LD requirements and nobody noticed so far). Am I missing something?

> The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded > > The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion. This appears to directly contradict section [3.1 “Object identifiers” of AP spec](https://www.w3.org/TR/activitypub/#obj-id). For reference I’m copying it in full here: <details> <summary>ActivityPub 3.1 Object identifiers</summary> > All Objects in [[ActivityStreams](https://www.w3.org/TR/activitypub/#bib-ActivityStreams)] should have unique global identifiers. ActivityPub extends this requirement; all objects distributed by the ActivityPub protocol *MUST* have unique global identifiers, unless they are intentionally transient (short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications). These identifiers must fall into one of the following groups: > > 1. Publicly dereferencable URIs, such as HTTPS URIs, with their authority belonging to that of their originating server. (Publicly facing content SHOULD use HTTPS URIs). > 2. An ID explicitly specified as the JSON `null` object, which implies an anonymous object (a part of its parent context) > > Identifiers *MUST* be provided for activities posted in server to server communication, unless the activity is intentionally transient. However, for client to server communication, a server receiving an object posted to the outbox with no specified `id` SHOULD allocate an object ID in the actor's namespace and attach it to the posted object. > > All objects have the following properties: > > **id** > &nbsp;&nbsp; The object's unique global identifier (unless the object is transient, in which case the id MAY be omitted). > > **type** > &nbsp;&nbsp; The type of the object. </details> Ignoring the note about C2S API it says **transient** objects *omit* `id` fully, while **anonymous** object explicitly specify `id` as `null`. Based on this i believe this is an error in the JSON-LD implementation *(or alternatively, APs requirements are incompatible with JSON-LD requirements and nobody noticed so far)*. Am I missing something?
First-time contributor

I'm not sure, but all JSON-LD implementations I'm aware of can't parse it, including the official JSON-LD playground

I'm not sure, but all JSON-LD implementations I'm aware of can't parse it, including the official [JSON-LD playground](https://json-ld.org/playground/)
First-time contributor

This would make AP explicitly incompatible with JSON-LD 🥴

This would make AP explicitly incompatible with JSON-LD 🥴
First-time contributor
https://github.com/digitalbazaar/jsonld.js/blob/5367858d28b6200aaf832d93eb666d4b819d5d4f/lib/expand.js#L527-L534
First-time contributor

From the JSON-LD spec:

If expanded property is @id:
If value is not a string, an invalid @id value error has been detected and processing is aborted. When the frameExpansion flag is set, value MAY be an empty map, or an array of one or more strings.

From the JSON-LD spec: > If expanded property is @id: > If value is not a string, an invalid @id value error has been detected and processing is aborted. When the frameExpansion flag is set, value MAY be an empty map, or an array of one or more strings.
First-time contributor

This is definitely not compliant.

This is definitely not compliant.
First-time contributor

Just verified, this is also mentioned in JSON-LD 1.0:

7.4.3) If expanded property is @id and value is not a string, an invalid @id value error has been detected and processing is aborted. Otherwise, set expanded value to the result of using the IRI Expansion algorithm, passing active context, value, and true for document relative.

(https://www.w3.org/TR/2014/REC-json-ld-api-20140116/)

Just verified, this is also mentioned in JSON-LD 1.0: > 7.4.3) If expanded property is @id and value is not a string, an invalid @id value error has been detected and processing is aborted. Otherwise, set expanded value to the result of using the IRI Expansion algorithm, passing active context, value, and true for document relative. (https://www.w3.org/TR/2014/REC-json-ld-api-20140116/)
First-time contributor

I wonder if this is an error in the AP spec

I wonder if this is an error in the AP spec
Author
Member

welp °~°

(I was trying to figure out if the allowed "empty value" for XML’s xsd:anyURI JSON-LDs anyURI range appears to be modeled after allows explicit nulls, but if the spec includes further wording specific to @id this has no relevance either way)

Tbqh I’m not sure what to do here; AP spec explicitly instructs us to define this to null but at the same time JSON-LD, via ActivityVocabulary (the latter defining id as @id), apparently tells us to never ever use null here. This should probably be brought up to the relevant standard bodies. Having anonymous objects is useful, so i guess either

  • JSON-LD is changed to allow null for @id
  • ActivityVocaulary is changed to use something other than @id for id
  • ActivityPub is changed to completely drop id for both transient and anonymous objects, instead distinguishing the two solely by whether it’s a top-level or child object
    (note: AP spec is not explicit about transient objects only being allowed at the top-level, but I’m interpreting the description of their purpose such that they only make sense when used at the top level)

Either change might break some existing implementation though...

Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null

welp °~° *(I was trying to figure out if the allowed "empty value" for XML’s `xsd:anyURI` JSON-LDs `anyURI` range appears to be modeled after allows explicit nulls, but if the spec includes further wording specific to `@id` this has no relevance either way)* Tbqh I’m not sure what to do here; AP spec explicitly instructs us to define this to `null` but at the same time JSON-LD, via ActivityVocabulary *(the latter defining `id` as `@id`)*, apparently tells us to never ever use `null` here. This should probably be brought up to the relevant standard bodies. Having anonymous objects is useful, so i guess either - JSON-LD is changed to allow `null` for `@id` - ActivityVocaulary is changed to use something other than `@id` for `id` - ActivityPub is changed to completely drop `id` for both transient and anonymous objects, instead distinguishing the two solely by whether it’s a top-level or child object *(note: AP spec is not explicit about transient objects only being allowed at the top-level, but I’m interpreting the description of their purpose such that they only make sense when used at the top level)* Either change might break some existing implementation though... Btw, how did you deal with this in IceShrimp.NET? Iinm https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/commit/333611f65eb2a65b2779ece0435b5ba84bf60e99 also sets some `id` fields to `null`
First-time contributor

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not, but I think you have more experience than me in wording these things.

Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null

C# doesn't differentiate between null and undefined, and our JSON-LD library interprets null as undefined (read: not including the key in the rendered JSON).

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not, but I think you have more experience than me in wording these things. > Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null C# doesn't differentiate between `null` and `undefined`, and our JSON-LD library interprets `null` as `undefined` (read: not including the key in the rendered JSON).
Author
Member

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not

It would be great if you could lead this; I’m lacking time atm. But feel free to ping me if anything needs clarification

> Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not It would be great if you could lead this; I’m lacking time atm. But feel free to ping me if anything needs clarification
First-time contributor

I'll open an issue, thank you!

I'll open an issue, thank you!
First-time contributor
Done. https://github.com/w3c/activitypub/issues/476
Sign in to join this conversation.
No description provided.