[rfc] pre-parse mfm_content as plaintext

Ghost commented

2022-07-08 19:05:43 +00:00

this is a problem I encountered while writing marked-mfm:

the mfm_content attribute is completely plaintext as opposed to raw_html. this is fine for things like markdown or HTML, since Marked is going to parse them on the frontend, with two exceptions: mentions and hashtags.

I could parse them on the frontend through Marked, but I think that's not a good idea for a few reasons:

mentions and tags aren't a "misskey markdown" feature, specifically
pleroma-fe already has components for HashtagLink and MentionsLine given the proper attributes
implementing those twice, once in the backend and once in the frontend (but only sometimes), sounds... bad
the parser can't know the URL of the mentioned account, requiring extra post-processing in the frontend anyway

what if instead of being completely plaintext, mfm_content came with the mentions and hashtags already linked? then all we'd have to do is parse it using marked-mfm, shove it in a RichContent (in the case of pleroma-fe) and it's done.

this is a problem I encountered while writing marked-mfm: the `mfm_content` attribute is completely plaintext as opposed to `raw_html`. this is fine for things like markdown or HTML, since Marked is going to parse them on the frontend, with two exceptions: mentions and hashtags. I *could* parse them on the frontend through Marked, but I think that's not a good idea for a few reasons: - mentions and tags aren't a "misskey markdown" feature, specifically - pleroma-fe already has components for `HashtagLink` and `MentionsLine` given the proper attributes - implementing those twice, once in the backend and once in the frontend (but only sometimes), sounds... bad - the parser can't know the URL of the mentioned account, requiring extra post-processing in the frontend anyway what if instead of being completely plaintext, `mfm_content` came with the mentions and hashtags already linked? then all we'd have to do is parse it using `marked-mfm`, shove it in a RichContent (in the case of pleroma-fe) and it's done.

Ghost commented

2022-07-08 19:19:15 +00:00

Author

my rationale for this is that, while mfm rendering is done entirely by the client, even plaintext posts have mentions and hashtags

my rationale for this is that, while mfm rendering is done entirely by the client, even `plaintext` posts have mentions and hashtags

Ghost referenced this issue from AkkomaGang/akkoma-fe

2022-07-09 11:40:36 +00:00

Use Marked and marked-mfm for MFM rendering #29

Ghost commented

2022-07-09 19:32:32 +00:00

Author

ALSO!!! this would be very useful for people who don't wanna see mfm formatting
if we show them mfm_content regardless, but just don't parse it, they're gonna see the $[tada plain mfm] instead of italic text

killing two birds with one stone

ALSO!!! this would be very useful for people who don't wanna see mfm formatting if we show them `mfm_content` regardless, but just don't parse it, they're gonna see the `$[tada plain mfm]` instead of *italic text* killing two birds with one stone

Ghost commented

2022-07-09 19:47:17 +00:00

Author

psuedocode (i don't know elixir hhh):

if mfm_content exists {
	raw_html = parse(mfm_content)
	is_mfm = true
}

and on the frontend:

if is_mfm && render_mfm {
	content = marked.parse(raw_html) // fully formatted mfm
} else {
	content = raw_html // either plaintext mfm OR not mfm at all
}

basically, completely ignoring what HTML Misskey chose to send us (it probably wasn't anything good anyway)

psuedocode (i don't know elixir hhh): ```go if mfm_content exists { raw_html = parse(mfm_content) is_mfm = true } ``` and on the frontend: ```go if is_mfm && render_mfm { content = marked.parse(raw_html) // fully formatted mfm } else { content = raw_html // either plaintext mfm OR not mfm at all } ``` basically, completely ignoring what HTML Misskey chose to send us (it probably wasn't anything good anyway)

floatingghost commented

2022-07-09 20:18:03 +00:00

Owner

yeah, we probably should be able to pre-link it

but i'm not a fan of the per-content-type flag, it feels prone to being extended in the future and we won't want is_someothertype checks

my proposal would be to pre-link in the same way as plaintext, but then have the frontend rely on a status.content_type field

yeah, we probably should be able to pre-link it but i'm not a fan of the per-content-type flag, it feels prone to being extended in the future and we won't want `is_someothertype` checks my proposal would be to pre-link in the same way as plaintext, but then have the frontend rely on a `status.content_type` field

Ghost commented

2022-07-09 20:23:28 +00:00

Author

that's a better solution, yes
currently the frontend relies on status.mfm_content existing to figure out if it's mfm or not but that would work with this solution so i added a flag
but using Content-Types is better

that's a better solution, yes currently the frontend relies on `status.mfm_content` existing to figure out if it's mfm or not but that would work with this solution so i added a flag but using Content-Types is better

floatingghost commented

2022-07-10 15:38:32 +00:00

Owner

ok, i've got an example format for you

{
  "account": "...",
  "akkoma": {
    "source": {
      "content": "<span class=\"h-card\"><a class=\"u-url mention\" data-user=\"AL121PGGiXpmpNN6TA\" href=\"https://akkoma.local.live/users/akkoma_user\" rel=\"ugc\">@<span>akkoma_user</span></a></span> linkifylink <a class=\"hashtag\" data-tag=\"dancedance\" href=\"https://akkoma.local.live/tag/dancedance\" rel=\"tag ugc\">#dancedance</a> $[jelly mfm goes here]",
      "mediaType": "text/x.misskeymarkdown"
    }
  }
}

returned as the status json

that feeling better to you?

ok, i've got an example format for you ```json { "account": "...", "akkoma": { "source": { "content": "<a class=\"u-url mention\" data-user=\"AL121PGGiXpmpNN6TA\" href=\"https://akkoma.local.live/users/akkoma_user\" rel=\"ugc\">@akkoma_user</a> linkifylink <a class=\"hashtag\" data-tag=\"dancedance\" href=\"https://akkoma.local.live/tag/dancedance\" rel=\"tag ugc\">#dancedance</a> $[jelly mfm goes here]", "mediaType": "text/x.misskeymarkdown" } } } ``` returned as the status json that feeling better to you?

Ghost commented

2022-07-10 15:46:19 +00:00

Author

yes
one other thing i noticed is how the backend rewrites \n to   on all types other than mfm
so currently i replace it in the frontend but if it can be replaced here that would be better i think, for consistency

yes one other thing i noticed is how the backend rewrites `\n` to ` ` on all types other than mfm so currently i replace it in the frontend but if it can be replaced here that would be better i think, for consistency

floatingghost commented

2022-07-10 16:01:21 +00:00

Owner

yea luckily a side-effect of using the linker is that it does that - so it'll be   post change

source: %{"content" => "@akkoma_user@akkoma.local.live linkifylink <a class=\"hashtag\" data-tag=\"dancedance\" href=\"http://localhost:4001/tag/dancedance\" rel=\"tag ugc\">#dancedance</a> $[jelly  mfm goes here] <br><br>## aaa", "mediaType" => "text/x.misskeymarkdown"}

(obv ignore the unlinked @ i have a buggo)

yea luckily a side-effect of using the linker is that it does that - so it'll be ` ` post change ```elixir source: %{"content" => "@akkoma_user@akkoma.local.live linkifylink <a class=\"hashtag\" data-tag=\"dancedance\" href=\"http://localhost:4001/tag/dancedance\" rel=\"tag ugc\">#dancedance</a> $[jelly mfm goes here] ## aaa", "mediaType" => "text/x.misskeymarkdown"} ``` (obv ignore the unlinked @ i have a buggo)