Federate emoji as anonymous objects #815

Merged

floatingghost merged 1 commit from Oneric/akkoma:emoji-id into develop 2024-10-16 14:58:47 +00:00

Conversation 20 Commits 1 Files changed 4 +5 -3

Oneric commented 2024-06-23 18:49:00 +00:00

Member

Copy link

Similarly small change as those in #814, but may warrant some more attention wrt possibly federation effects.

Federation with other *omas and with *keys (+IceShrimp) should work; not sure how Mastodon and its forks handle null ids. (e.g. fedibird and chuckya support emoji reactions)

---

Usually an id should point to another AP object
and the image file isn’t an AP object. We currently
do not provide standalone AP objects for emoji and
don't keep track of remote emoji at all.
Thus just federate them as anonymous objects,
i.e. objects only existing within a parent context
and using an explicit null id.

IceShrimp.NET previously adopted anonymous objects
for remote emoji without any apparent issues. See:
333611f65e

Fixes: #694

Similarly small change as those in #814, but may warrant some more attention wrt possibly federation effects. Federation with other \*omas and with \*keys (+IceShrimp) should work; not sure how Mastodon and its forks handle null ids. (e.g. fedibird and chuckya support emoji reactions) --- Usually an id should point to another AP object and the image file isn’t an AP object. We currently do not provide standalone AP objects for emoji and don't keep track of remote emoji at all. Thus just federate them as anonymous objects, i.e. objects only existing within a parent context and using an explicit null id. IceShrimp.NET previously adopted anonymous objects for remote emoji without any apparent issues. See: https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/commit/333611f65eb2a65b2779ece0435b5ba84bf60e99 Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/694

Oneric added 1 commit 2024-06-23 18:49:00 +00:00

Federate emoji as anonymous objects ... 4ff5293093

Usually an id should point to another AP object
and the image file isn’t an AP object. We currently
do not provide standalone AP objects for emoji and
don't keep track of remote emoji at all.
Thus just federate them as anonymous objects,
i.e. objects only existing within a parent context
and using an explicit null id.

IceShrimp.NET previously adopted anonymous objects
for remote emoji without any apparent issues. See:
333611f65e

Fixes: #694

Oneric referenced this pull request 2024-07-13 18:04:42 +00:00

[bug] Invalid Emoji ID #820

silverpill referenced this pull request 2024-07-13 19:49:37 +00:00

[bug] Invalid Emoji ID #820

Oneric commented 2024-10-04 02:28:16 +00:00

Author

Member

Copy link

I was now able to confirm this works fine in practice with:

• current Akkoma (no surprise)
• Mastodon 4.2.10
• Sharkey 2024.5.1 (federation code is supposed to almost match upstream Misskey sans their extensions and as mentioned before i’ve seen explicit null-id emoj handling in upstream Misskey code, so this likely holds for all *keys)

However issues arose with GtS 0.16.0; emoji got stripped with only the shortcode text remaining. I brought it up with GtS upstream

I was now able to confirm this works fine in practice with: - current Akkoma *(no surprise)* - Mastodon 4.2.10 - Sharkey 2024.5.1 *(federation code is supposed to almost match upstream Misskey sans their extensions and as mentioned before i’ve seen explicit null-id emoj handling in upstream Misskey code, so this likely holds for all \*keys)* However issues arose with GtS 0.16.0; emoji got stripped with only the shortcode text remaining. I brought it up with GtS upstream

Oneric commented 2024-10-14 00:17:30 +00:00

Author

Member

Copy link

GtS intends to fix this, but it’s not high prio and thus might take a while

We could either:

• wait until the GtS fix arrives, leaving others broken instead
• already apply this now and accept GtS users only receiving text instead of images for emoji until they get a fix
• provide AP objects for local, admin-managed emoji using nil for all else like IceShrimp. Considering atm remote emoji can only be used in emoji reactions which are not supported by GtS and we axed fully-custom user emoji as a causality of dropping C2S support, this should cover all emoji GtS will see from us (for now)

However as is now, this can only work by tying the AP id to the shortcode, which i’m not fond of. Distinct emoji (old gets deleted, (much) later a different one added) will share an ID which might confuse some caches (especially since we can’t provide real updated timestamps yet) and the conceptually same emoji might change ID after rename (e.g. to be more consistent with other emoji of the same pack, or because the name is too generic, etc).
It might also lock us into this ID scheme even if we get better emoji handling in the future. Though perhaps, since emoji aren't to important just changing all IDs in the future might actually not be too bad in practice?

GtS intends to fix this, but it’s not high prio and thus might take a while We could either: - wait until the GtS fix arrives, leaving others broken instead - already apply this now and accept GtS users only receiving text instead of images for emoji until they get a fix - provide AP objects for local, admin-managed emoji using `nil` for all else like IceShrimp. Considering atm remote emoji can only be used in emoji reactions which are not supported by GtS and we axed fully-custom user emoji as a causality of dropping C2S support, this should cover all emoji GtS will see from us (for now) However as is now, this can only work by tying the AP id to the shortcode, which i’m not fond of. Distinct emoji *(old gets deleted, (much) later a different one added)* will share an ID which might confuse some caches *(especially since we can’t provide real `updated` timestamps yet)* and the conceptually same emoji might change ID after rename *(e.g. to be more consistent with other emoji of the same pack, or because the name is too generic, etc)*. It might also lock us into this ID scheme even if we get better emoji handling in the future. Though perhaps, since emoji aren't to important just changing all IDs in the future might actually not be too bad in practice?

floatingghost commented 2024-10-16 14:58:41 +00:00

Owner

Copy link

i am going to make the executive decision of "lol gts can fix it their side" hehe
maybe extra breakages will spur them to bump it up the prio list

i am going to make the executive decision of "lol gts can fix it their side" hehe maybe extra breakages will spur them to bump it up the prio list

😆 1

floatingghost merged commit f101886709 into develop 2024-10-16 14:58:47 +00:00

floatingghost deleted branch emoji-id 2024-10-16 14:58:47 +00:00

floatingghost referenced this pull request from a commit 2024-10-16 14:58:49 +00:00

Merge pull request 'Federate emoji as anonymous objects' (#815) from Oneric/akkoma:emoji-id into develop

zotan commented 2024-11-01 12:48:15 +00:00

First-time contributor

Copy link

I think this change is making emoji attachments federate as "id": null, which causes JSON-LD expansion errors. 😬

I think this change is making emoji attachments federate as `"id": null`, which causes JSON-LD expansion errors. 😬

zotan commented 2024-11-01 12:50:09 +00:00

First-time contributor

Copy link

Context: We've received reports of federation issues with akko.wtf, and I verified that the affected users both have this in their actor objects:

"tag": [
  {
    "id": null
  }
]

Context: We've received reports of federation issues with akko.wtf, and I verified that the affected users both have this in their actor objects: ``` "tag": [ { "id": null } ] ```

zotan commented 2024-11-01 12:50:23 +00:00

First-time contributor

Copy link

The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded

The specific error is `Invalid JSON-LD syntax; "@id" value must a string.`, verifiable at https://transform.tools/jsonld-to-expanded

zotan commented 2024-11-01 12:57:46 +00:00

First-time contributor

Copy link

The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.

The issue is that `null` and `undefined` (no `id`/`@id` key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.

Oneric commented 2024-11-01 15:38:42 +00:00

Author

Member

Copy link

The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded

The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion.

This appears to directly contradict section 3.1 “Object identifiers” of AP spec. For reference I’m copying it in full here:

ActivityPub 3.1 Object identifiers

All Objects in [ActivityStreams] should have unique global identifiers. ActivityPub extends this requirement; all objects distributed by the ActivityPub protocol MUST have unique global identifiers, unless they are intentionally transient (short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications). These identifiers must fall into one of the following groups:

1. Publicly dereferencable URIs, such as HTTPS URIs, with their authority belonging to that of their originating server. (Publicly facing content SHOULD use HTTPS URIs).
2. An ID explicitly specified as the JSON null object, which implies an anonymous object (a part of its parent context)

Identifiers MUST be provided for activities posted in server to server communication, unless the activity is intentionally transient. However, for client to server communication, a server receiving an object posted to the outbox with no specified id SHOULD allocate an object ID in the actor's namespace and attach it to the posted object.

All objects have the following properties:

id
   The object's unique global identifier (unless the object is transient, in which case the id MAY be omitted).

type
   The type of the object.

Ignoring the note about C2S API it says transient objects omit id fully, while anonymous object explicitly specify id as null.
Based on this i believe this is an error in the JSON-LD implementation (or alternatively, APs requirements are incompatible with JSON-LD requirements and nobody noticed so far). Am I missing something?

> The specific error is Invalid JSON-LD syntax; "@id" value must a string., verifiable at https://transform.tools/jsonld-to-expanded > > The issue is that null and undefined (no id/@id key) are handled differently in JSON-LD. The latter is valid (and how anonymous objects are supposed to work), the former is not, and throws on deserialization/expansion. This appears to directly contradict section [3.1 “Object identifiers” of AP spec](https://www.w3.org/TR/activitypub/#obj-id). For reference I’m copying it in full here: <details> <summary>ActivityPub 3.1 Object identifiers</summary> > All Objects in [[ActivityStreams](https://www.w3.org/TR/activitypub/#bib-ActivityStreams)] should have unique global identifiers. ActivityPub extends this requirement; all objects distributed by the ActivityPub protocol *MUST* have unique global identifiers, unless they are intentionally transient (short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications). These identifiers must fall into one of the following groups: > > 1. Publicly dereferencable URIs, such as HTTPS URIs, with their authority belonging to that of their originating server. (Publicly facing content SHOULD use HTTPS URIs). > 2. An ID explicitly specified as the JSON `null` object, which implies an anonymous object (a part of its parent context) > > Identifiers *MUST* be provided for activities posted in server to server communication, unless the activity is intentionally transient. However, for client to server communication, a server receiving an object posted to the outbox with no specified `id` SHOULD allocate an object ID in the actor's namespace and attach it to the posted object. > > All objects have the following properties: > > **id** > &nbsp;&nbsp; The object's unique global identifier (unless the object is transient, in which case the id MAY be omitted). > > **type** > &nbsp;&nbsp; The type of the object. </details> Ignoring the note about C2S API it says **transient** objects *omit* `id` fully, while **anonymous** object explicitly specify `id` as `null`. Based on this i believe this is an error in the JSON-LD implementation *(or alternatively, APs requirements are incompatible with JSON-LD requirements and nobody noticed so far)*. Am I missing something?

zotan commented 2024-11-01 16:08:11 +00:00

First-time contributor

Copy link

I'm not sure, but all JSON-LD implementations I'm aware of can't parse it, including the official JSON-LD playground

I'm not sure, but all JSON-LD implementations I'm aware of can't parse it, including the official [JSON-LD playground](https://json-ld.org/playground/)

zotan commented 2024-11-01 16:08:53 +00:00

First-time contributor

Copy link

This would make AP explicitly incompatible with JSON-LD 🥴

This would make AP explicitly incompatible with JSON-LD 🥴

zotan commented 2024-11-01 16:13:14 +00:00

First-time contributor

Copy link

5367858d28/lib/expand.js (L527-L534)

https://github.com/digitalbazaar/jsonld.js/blob/5367858d28b6200aaf832d93eb666d4b819d5d4f/lib/expand.js#L527-L534

zotan commented 2024-11-01 16:15:54 +00:00

First-time contributor

Copy link

From the JSON-LD spec:

If expanded property is @id:
If value is not a string, an invalid @id value error has been detected and processing is aborted. When the frameExpansion flag is set, value MAY be an empty map, or an array of one or more strings.

From the JSON-LD spec: > If expanded property is @id: > If value is not a string, an invalid @id value error has been detected and processing is aborted. When the frameExpansion flag is set, value MAY be an empty map, or an array of one or more strings.

zotan commented 2024-11-01 16:16:03 +00:00

First-time contributor

Copy link

This is definitely not compliant.

This is definitely not compliant.

zotan commented 2024-11-01 16:19:05 +00:00

First-time contributor

Copy link

Just verified, this is also mentioned in JSON-LD 1.0:

7.4.3) If expanded property is @id and value is not a string, an invalid @id value error has been detected and processing is aborted. Otherwise, set expanded value to the result of using the IRI Expansion algorithm, passing active context, value, and true for document relative.

(https://www.w3.org/TR/2014/REC-json-ld-api-20140116/)

Just verified, this is also mentioned in JSON-LD 1.0: > 7.4.3) If expanded property is @id and value is not a string, an invalid @id value error has been detected and processing is aborted. Otherwise, set expanded value to the result of using the IRI Expansion algorithm, passing active context, value, and true for document relative. (https://www.w3.org/TR/2014/REC-json-ld-api-20140116/)

zotan commented 2024-11-01 17:23:47 +00:00

First-time contributor

Copy link

I wonder if this is an error in the AP spec

I wonder if this is an error in the AP spec

Oneric commented 2024-11-01 17:37:03 +00:00

Author

Member

Copy link

welp °~°

(I was trying to figure out if the allowed "empty value" for XML’s xsd:anyURI JSON-LDs anyURI range appears to be modeled after allows explicit nulls, but if the spec includes further wording specific to @id this has no relevance either way)

Tbqh I’m not sure what to do here; AP spec explicitly instructs us to define this to null but at the same time JSON-LD, via ActivityVocabulary (the latter defining id as @id), apparently tells us to never ever use null here. This should probably be brought up to the relevant standard bodies. Having anonymous objects is useful, so i guess either

• JSON-LD is changed to allow null for @id
• ActivityVocaulary is changed to use something other than @id for id
• ActivityPub is changed to completely drop id for both transient and anonymous objects, instead distinguishing the two solely by whether it’s a top-level or child object
  (note: AP spec is not explicit about transient objects only being allowed at the top-level, but I’m interpreting the description of their purpose such that they only make sense when used at the top level)

Either change might break some existing implementation though...

Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null

welp °~° *(I was trying to figure out if the allowed "empty value" for XML’s `xsd:anyURI` JSON-LDs `anyURI` range appears to be modeled after allows explicit nulls, but if the spec includes further wording specific to `@id` this has no relevance either way)* Tbqh I’m not sure what to do here; AP spec explicitly instructs us to define this to `null` but at the same time JSON-LD, via ActivityVocabulary *(the latter defining `id` as `@id`)*, apparently tells us to never ever use `null` here. This should probably be brought up to the relevant standard bodies. Having anonymous objects is useful, so i guess either - JSON-LD is changed to allow `null` for `@id` - ActivityVocaulary is changed to use something other than `@id` for `id` - ActivityPub is changed to completely drop `id` for both transient and anonymous objects, instead distinguishing the two solely by whether it’s a top-level or child object *(note: AP spec is not explicit about transient objects only being allowed at the top-level, but I’m interpreting the description of their purpose such that they only make sense when used at the top level)* Either change might break some existing implementation though... Btw, how did you deal with this in IceShrimp.NET? Iinm https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/commit/333611f65eb2a65b2779ece0435b5ba84bf60e99 also sets some `id` fields to `null`

zotan commented 2024-11-01 18:00:06 +00:00

First-time contributor

Copy link

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not, but I think you have more experience than me in wording these things.

Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null

C# doesn't differentiate between null and undefined, and our JSON-LD library interprets null as undefined (read: not including the key in the rendered JSON).

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not, but I think you have more experience than me in wording these things. > Btw, how did you deal with this in IceShrimp.NET? Iinm 333611f65e also sets some id fields to null C# doesn't differentiate between `null` and `undefined`, and our JSON-LD library interprets `null` as `undefined` (read: not including the key in the rendered JSON).

zotan referenced this pull request 2024-11-01 18:27:08 +00:00

[bug] Custom emoji objects aren't valid JSON-LD #848

Oneric commented 2024-11-01 19:37:04 +00:00

Author

Member

Copy link

Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not

It would be great if you could lead this; I’m lacking time atm. But feel free to ping me if anything needs clarification

> Bringing this up with the W3C sounds like the right move. Do you feel like doing this? I can do it if not It would be great if you could lead this; I’m lacking time atm. But feel free to ping me if anything needs clarification

zotan commented 2024-11-01 20:21:24 +00:00

First-time contributor

Copy link

I'll open an issue, thank you!

I'll open an issue, thank you!

zotan commented 2024-11-01 20:24:44 +00:00

First-time contributor

Copy link

Done. https://github.com/w3c/activitypub/issues/476

Done. https://github.com/w3c/activitypub/issues/476

👍 1 ❤️ 1

Oneric referenced this pull request 2024-11-24 20:56:18 +00:00

meta: closeable issues #758

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

approved, awaiting change

  

bug

Something is not working

  

configuration

This requires config changes

  

documentation

This is about human-readable docs

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

extremely low priority

  

feature request

Something new?

  

Fix it yourself

Support has been attempted

  

help wanted

Need some help

  

invalid

Something is wrong

  

mastodon_api

  

needs docs

  

needs tests

  

not a bug

  

planned

  

pleroma_api

  

privacy

  

question

More information is needed

  

static_fe

  

triage

  

wontfix

This won't be fixed

No labels

approved, awaiting change

bug

configuration

documentation

duplicate

enhancement

extremely low priority

feature request

Fix it yourself

help wanted

invalid

mastodon_api

needs docs

needs tests

not a bug

planned

pleroma_api

privacy

question

static_fe

triage

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#815

No description provided.