Use FEP-c16b: Formatting MFM functions #410

ilja · 2024-08-04T09:33:26Z

ilja commented

2024-08-04 09:33:26 +00:00

After discussing in https://akkoma.dev/FoundKeyGang/FoundKey/issues/343 a scrubber-friendly way was discovered for representing the MFM functions in HTML. These are the front-end changes needed for that. This should fix the MFM in Akkoma and provide compatibility for the same MFM functions as Foundkey has.

In this PR we:

first we need to add the Foundkey SCSS
- I added this in a separate commit so I could add Johan as author. If there's something I should do different, do let me know.
then adapt where needed, including backwards compatibility for posts in the "old" format
make sure the setting for disabling MFM still works
then we need to add JS for the data attr variables
make sure all supported tags are added to ./src/components/emoji_input/suggestor.js
does it work as expected?
- Do the functions display properly, see "Extra notes" below
  - For custom emojis in x2, x3, x4, I took the sizes that we already used in Akkoma
- Don't allow going outside of the borders of the status (we use overflow: hidden; on the StatusBody now)
- other things someone can think of?

Broader scope where this PR is part of:

FEP-c16b is a work in progress. It can currently be seen at https://codeberg.org/fediverse/fep/pulls/382
The MfmParser needs big adaptions. The work in progress can be seen at https://codeberg.org/ilja/mfm_parser/pulls/4
Akkoma backend needs changes, see https://akkoma.dev/AkkomaGang/akkoma/pulls/823

Extra notes:

Sparkle has not been implemented here. It didn't work before and it requires some extra JS. This is already a strict improvement, and also a very fundamental change over multiple repositories. So it seems better to me, to do sparkle in a separate PR.
We can check MFM on https://genau.qwertqwefsday.eu/ > three dots menu > About Foundkey > click the "I ❤️ Foundkey" button > type MFM and click the "Show preview" <> button

List of MFM functions I tested

These are all the supported functions and different combinations of attributes, and whether they have been confirmed to work. We test text, emoji, and custom emoji.

$[center tofu 🐈 :hehe:]
$[flip tofu 🐈 :hehe:]
$[flip.v tofu 🐈 :hehe:]
$[flip.v,h tofu 🐈 :hehe:]
$[font.serif tofu 🐈 :hehe:]
$[font.monospace tofu 🐈 :hehe:]
$[font.cursive tofu 🐈 :hehe:]
$[font.fantasy tofu 🐈 :hehe:]
$[font.emoji tofu 🐈 :hehe:]
$[font.math tofu 🐈 :hehe:]
$[blur tofu 🐈 :hehe:]
$[rotate tofu 🐈 :hehe:]
$[rotate.deg=30 tofu 🐈 :hehe:]
$[x2 tofu 🐈 :hehe:]
$[x3 tofu 🐈 :hehe:]
$[x4 tofu 🐈 :hehe:]
$[x2 $[x2 tofu 🐈 :hehe:]]
$[x2 $[x2 $[x2 tofu 🐈 :hehe:]]]
$[position.x=3 tofu 🐈 :hehe:]
$[position.y=5 tofu 🐈 :hehe:]
$[position.x=3,y=5 tofu 🐈 :hehe:]
$[scale.x=3 tofu 🐈 :hehe:]
$[scale.y=5 tofu 🐈 :hehe:]
$[scale.x=3,y=5 tofu 🐈 :hehe:]
$[fg tofu 🐈 :hehe:]
$[fg.color=#fff tofu 🐈 :hehe:]
$[bg tofu 🐈 :hehe:]
$[bg.color=#fff tofu 🐈 :hehe:]
$[jelly tofu 🐈 :hehe:]
$[jelly.speed=5s tofu 🐈 :hehe:]
$[twitch tofu 🐈 :hehe:]
$[twitch.speed=5s tofu 🐈 :hehe:]
$[shake tofu 🐈 :hehe:]
$[shake.speed=5s tofu 🐈 :hehe:]
$[spin tofu 🐈 :hehe:]
$[spin.speed=5s tofu 🐈 :hehe:]
$[spin.y tofu 🐈 :hehe:]
$[spin.y,speed=0.3s tofu 🐈 :hehe:]
$[spin.x tofu 🐈 :hehe:]
$[spin.x,speed=0.3s tofu 🐈 :hehe:]
$[spin.alternate tofu 🐈 :hehe:]
$[spin.alternate,speed=0.3s tofu 🐈 :hehe:]
$[spin.left tofu 🐈 :hehe:]
$[spin.left,speed=0.3s tofu 🐈 :hehe:]
$[jump tofu 🐈 :hehe:]
$[jump.speed=5s tofu 🐈 :hehe:]
$[bounce tofu 🐈 :hehe:]
$[bounce.speed=5s tofu 🐈 :hehe:]
$[rainbow tofu 🐈 :hehe:]
$[rainbow.speed=5s tofu 🐈 :hehe:]
$[tada tofu 🐈 :hehe:]
$[tada.speed=5s tofu 🐈 :hehe:]

Related https://akkoma.dev/AkkomaGang/akkoma/issues/381 After discussing in <https://akkoma.dev/FoundKeyGang/FoundKey/issues/343> a scrubber-friendly way was discovered for representing the MFM functions in HTML. These are the front-end changes needed for that. This should fix the MFM in Akkoma and provide compatibility for the same MFM functions as Foundkey has. In this PR we: - [x] first we need to add the Foundkey SCSS - I added this in a separate commit so I could add Johan as author. If there's something I should do different, do let me know. - [x] then adapt where needed, including backwards compatibility for posts in the "old" format - [x] make sure the setting for disabling MFM still works - [x] then we need to add JS for the data attr variables - [x] make sure all supported tags are added to `./src/components/emoji_input/suggestor.js` - [x] does it work as expected? - [x] Do the functions display properly, see "Extra notes" below - For custom emojis in `x2`, `x3`, `x4`, I took the sizes that we already used in Akkoma - [x] Don't allow going outside of the borders of the status (we use `overflow: hidden;` on the StatusBody now) - other things someone can think of? Broader scope where this PR is part of: - FEP-c16b is a work in progress. It can currently be seen at <https://codeberg.org/fediverse/fep/pulls/382> - The MfmParser needs big adaptions. The work in progress can be seen at <https://codeberg.org/ilja/mfm_parser/pulls/4> - Akkoma backend needs changes, see <https://akkoma.dev/AkkomaGang/akkoma/pulls/823> Extra notes: - Sparkle has not been implemented here. It didn't work before and it requires some extra JS. This is already a strict improvement, and also a very fundamental change over multiple repositories. So it seems better to me, to do sparkle in a separate PR. - We can check MFM on https://genau.qwertqwefsday.eu/ > three dots menu > About Foundkey > click the "I ❤️ Foundkey" button > type MFM and click the "Show preview" `<>` button <details> <summary>List of MFM functions I tested</summary> These are all the supported functions and different combinations of attributes, and whether they have been confirmed to work. We test text, emoji, and custom emoji. - [x] $[center tofu 🐈 :hehe:] - [x] $[flip tofu 🐈 :hehe:] - [x] $[flip.v tofu 🐈 :hehe:] - [x] $[flip.v,h tofu 🐈 :hehe:] - [x] $[font.serif tofu 🐈 :hehe:] - [x] $[font.monospace tofu 🐈 :hehe:] - [x] $[font.cursive tofu 🐈 :hehe:] - [x] $[font.fantasy tofu 🐈 :hehe:] - [x] $[font.emoji tofu 🐈 :hehe:] - [x] $[font.math tofu 🐈 :hehe:] - [x] $[blur tofu 🐈 :hehe:] - [x] $[rotate tofu 🐈 :hehe:] - [x] $[rotate.deg=30 tofu 🐈 :hehe:] - [x] $[x2 tofu 🐈 :hehe:] - [x] $[x3 tofu 🐈 :hehe:] - [x] $[x4 tofu 🐈 :hehe:] - [x] $[x2 $[x2 tofu 🐈 :hehe:]] - [x] $[x2 $[x2 $[x2 tofu 🐈 :hehe:]]] - [x] $[position.x=3 tofu 🐈 :hehe:] - [x] $[position.y=5 tofu 🐈 :hehe:] - [x] $[position.x=3,y=5 tofu 🐈 :hehe:] - [x] $[scale.x=3 tofu 🐈 :hehe:] - [x] $[scale.y=5 tofu 🐈 :hehe:] - [x] $[scale.x=3,y=5 tofu 🐈 :hehe:] - [x] $[fg tofu 🐈 :hehe:] - [x] $[fg.color=#fff tofu 🐈 :hehe:] - [x] $[bg tofu 🐈 :hehe:] - [x] $[bg.color=#fff tofu 🐈 :hehe:] - [x] $[jelly tofu 🐈 :hehe:] - [x] $[jelly.speed=5s tofu 🐈 :hehe:] - [x] $[twitch tofu 🐈 :hehe:] - [x] $[twitch.speed=5s tofu 🐈 :hehe:] - [x] $[shake tofu 🐈 :hehe:] - [x] $[shake.speed=5s tofu 🐈 :hehe:] - [x] $[spin tofu 🐈 :hehe:] - [x] $[spin.speed=5s tofu 🐈 :hehe:] - [x] $[spin.y tofu 🐈 :hehe:] - [x] $[spin.y,speed=0.3s tofu 🐈 :hehe:] - [x] $[spin.x tofu 🐈 :hehe:] - [x] $[spin.x,speed=0.3s tofu 🐈 :hehe:] - [x] $[spin.alternate tofu 🐈 :hehe:] - [x] $[spin.alternate,speed=0.3s tofu 🐈 :hehe:] - [x] $[spin.left tofu 🐈 :hehe:] - [x] $[spin.left,speed=0.3s tofu 🐈 :hehe:] - [x] $[jump tofu 🐈 :hehe:] - [x] $[jump.speed=5s tofu 🐈 :hehe:] - [x] $[bounce tofu 🐈 :hehe:] - [x] $[bounce.speed=5s tofu 🐈 :hehe:] - [x] $[rainbow tofu 🐈 :hehe:] - [x] $[rainbow.speed=5s tofu 🐈 :hehe:] - [x] $[tada tofu 🐈 :hehe:] - [x] $[tada.speed=5s tofu 🐈 :hehe:] </details>

ilja added 2 commits 2024-08-04 09:33:27 +00:00

Message from commiter: Add Foundkey MFM stylesheet 1f778ba38b

This is part of a bigger work to fix MFM in Akkoma
See <AkkomaGang/akkoma#381>

Here we add the MFM stylesheet as it is used by Foundkey
See <b22e627089>

Foundkey uses MFM and both the Founkey and Akkoma projects and communities, have historically been closely related
As such it makes sense to start with feature-parity with Foundkey

This commit only adds the stylesheet so that correct attribution is given
Further work will happen in later commits

Addapt MFM stylesheet for Akkoma

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

d1b2e7b51e

In the previous commit we added the Foundkey stylesheet,
here we adapt it to be used in Akkoma

The original mfm.css is removed and all needed for backwards compatibility has been added to the new stylesheet
Akkoma doesn't currently have support for disabling the animated mfm, so the stylesheet has been adapted for this as well

These are only the SCSS adaptions. Attributes with variables will not work, nor will sparkle
In both cases we need Javascript aditions. Those are for later commits.

ilja referenced this pull request from AkkomaGang/akkoma

2024-08-04 09:50:45 +00:00

Use FEP-c16b: Formatting MFM functions #823

ilja force-pushed use_fep-c16b_formatting_mfm_functions from d1b2e7b51e to aa173b8642

2024-08-04 11:27:40 +00:00

Compare

ilja force-pushed use_fep-c16b_formatting_mfm_functions from aa173b8642 to 7a7fdae4ae

2024-08-04 17:05:26 +00:00

Compare

ilja force-pushed use_fep-c16b_formatting_mfm_functions from 7a7fdae4ae to 3065416c93

2024-08-04 17:10:34 +00:00

Compare

ilja added 1 commit 2024-08-05 10:00:40 +00:00

fill in data-mfm- variables

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

f12068d6c2

Things like `speed=0.1s` now work

I also noticed a class was set on StatusBody, but we don't use it, we use StatusContent.
Therefor I removed it now.

We do still pass the setting through StatusBody to RichContent bc it's used there to decide to not show greentext for arrows when MFM was used.
Note that while this setting still works
* You have to refresh the page to see it working (was already like this, so I didn't touch it here)
* It explicitly checks for content type. If womeone provides MFM-like HTML, then it will still show as greentext if that option is enabled
  I think it's a bit inconsistent, but otoh, the inconsistency to me seems more that we ignore the greentext option for one input type specifically

I do still notice generall bugs with MFM.
* Position doesn't seem to work, neither does scale.
* There also seems to be a regression where custom emojis don't become larger any more with e.g. `$[x2 :hehe: ]`

I don't assume the regression is made in this commit, so I add this already. The rest needs to be fixed before merging.

ilja force-pushed use_fep-c16b_formatting_mfm_functions from f12068d6c2 to 56a59e1b55

2024-08-05 15:23:18 +00:00

Compare

ilja added 1 commit 2024-08-10 10:47:06 +00:00

Fix MFM functions x2, x3, and x4

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

ba4ae5badb

These now work for the new, FEP-c16b compliant, representation
Nesting also works

It already worked for text and "normal" emoji, but now it also works for custom emoji

ilja added 1 commit 2024-08-10 11:16:21 +00:00

Fix mfm-position and mfm-scale

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

f5f9949253

The `span`'s needed an inline-block for the transform to wrok
I also added an `overflow: hidden;` because these functions can make the text go beyond the borders of the StatusBody
With `overflow: hidden;`, it won't show outside of the borders

ilja added 1 commit 2024-08-10 11:57:19 +00:00

MFM make all supported tags suggested

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

3210873d7f

When typing MFM, a sugestor drop-down appears so you can see and/or choose what MFM function to use
The new MFM functions we support have now also been added

ilja changed title from ~~WIP: Use FEP-c16b: Formatting MFM functions~~ to Use FEP-c16b: Formatting MFM functions

2024-08-10 18:49:22 +00:00

ilja changed title from ~~Use FEP-c16b: Formatting MFM functions~~ to WIP: Use FEP-c16b: Formatting MFM functions

2024-08-11 12:38:51 +00:00

ilja changed title from ~~WIP: Use FEP-c16b: Formatting MFM functions~~ to Use FEP-c16b: Formatting MFM functions

2024-08-11 12:50:19 +00:00

ilja added 1 commit 2024-08-11 15:49:11 +00:00

MFM only use sanitised data-* attribute values

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

dcd09d38e8

We take the value from a data-* attribute and then add this to the style attribute.
This will probably be OK in most cases, but just to be sure, we check for "weird" characters first.
For now we only allow letters, numbers, dot and minus sign, because those are the ones I currently know of who are used in MFM.
The data-* attribute remains because it was already considered proper HTML as-is.

ilja force-pushed use_fep-c16b_formatting_mfm_functions from dcd09d38e8 to 6666a273a4

2024-08-11 16:11:50 +00:00

Compare

ilja commented

2024-08-17 20:36:40 +00:00

~~found a bug ☹️~~ fixed what we can, upstream issue created where we can't

https://mk.absturztau.be/notes/9vdb32gycp0700sc

"_misskey_content": "<center>$[fg.color=ffaa77 <b>MISSKEY FLAVORED</b>]\n<b>$[fg.color=ffaaaa $[x2 FEDI ⭐️ VERSE]]</b></center>"

The newline doesn't show on my instance for some reason.
- Is a bug in EarmarkParser, so I don't think it should be fixed here, see https://github.com/RobertDober/earmark_parser/issues/158. I wasn't able to work-around it by trying different versions of the dependency.
- I also made an issue on the Akkoma issue tracker so that we know about it, AkkomaGang/akkoma#829
~~The colours don't work. Apparently it uses ffaa77, but I think I expect the colour values to start with #. Need to check what's the correct way(s) to do.~~ Only without # is valid for value for color, this is now fixed

~~found a bug ☹️~~ fixed what we can, upstream issue created where we can't https://mk.absturztau.be/notes/9vdb32gycp0700sc ``` "_misskey_content": "<center>$[fg.color=ffaa77 <b>MISSKEY FLAVORED</b>]\n<b>$[fg.color=ffaaaa $[x2 FEDI ⭐️ VERSE]]</b></center>" ``` 1. [x] The newline doesn't show on my instance for some reason. * Is a bug in EarmarkParser, so I don't think it should be fixed here, see <https://github.com/RobertDober/earmark_parser/issues/158>. I wasn't able to work-around it by trying different versions of the dependency. * I also made an issue on the Akkoma issue tracker so that we know about it, https://akkoma.dev/AkkomaGang/akkoma/issues/829 2. [x] ~~The colours don't work. Apparently it uses `ffaa77`, but I think I expect the colour values to start with `#`. Need to check what's the correct way(s) to do.~~ Only without `#` is valid for value for `color`, this is now fixed

ilja added 1 commit 2024-08-18 13:54:22 +00:00

Don't require # in the data-mfm-color attribute

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

25681cf5f6

For colour in MFM attributes, we expected a `#`, but that's apparently wrong. The BE
translates the `color` attribute in `$[fg.color=000 text]` into `data-mfm-color=000`.
But for the SCSS to work, we need to put it in the style attribute as `--mfm-color: #000`.

Generally we just add the attribute value as-is in the `style` attribute, but now we
have a special exception for color so we add a `#` before the value.

Oneric reviewed 2024-08-25 01:21:11 +00:00

Oneric left a comment

I continue to think it’d be best for API users to perform the “data → style variables” transformation in the backend, so any frontend/app can easily support MFM rendering by just including a static CSS file. CSS rendering is a lower bar to clear than running scripts to parse and transform each post (and then still apply CSS rendering)

src/components/status_content/mfm.scss

					
				@ -0,0 +69,4 @@

				    */

				    .emoji {

				      --mfm-emoji-zoom-size: 2.63;

				    }

Oneric commented

2024-08-25 01:15:18 +00:00

I think the whole emoji size scaling setup is overly complicated. Just setting --mfm-emoji-zoom-size: 2.63; etc as here in all relevant cases and using this with a default of 1.0 directly in the base emoji height definition would achieve the same effect as now but avoid manually overriding height directly and --nested-base-emoji-size for some MFM effects.

However, while i appreciate the effort which went into preserving Akkoma’s old emoji scaling exactly (except for nested scaling which previously was a noop), this scaling behaviour is not compatible with Foundkey, iceshrimp and likely all other keys and imho leads to a jarring discrepancy as emoji get scaled noticeable more than text. Foundkey and IceShrimp set the emoji height to 2em instead of a fixed pixel value which means they’ll get automatically be scaled together with and by the same amount as text without any emoji-specific scaling logic. Imho we should adopt this too; improving portability and simplifying our logic at the same time.

I think the whole emoji size scaling setup is overly complicated. Just setting `--mfm-emoji-zoom-size: 2.63;` etc as here in all relevant cases and using this with a default of `1.0` directly in the base emoji `height` definition would achieve the same effect as now but avoid manually overriding `height` directly and `--nested-base-emoji-size` for some MFM effects. However, while i appreciate the effort which went into preserving Akkoma’s old emoji scaling exactly *(except for nested scaling which previously was a noop)*, this scaling behaviour is not compatible with Foundkey, iceshrimp and likely all other keys and imho leads to a jarring discrepancy as emoji get scaled noticeable more than text. Foundkey and IceShrimp set the emoji height to `2em` instead of a fixed pixel value which means they’ll get automatically be scaled together with and by the same amount as text without any emoji-specific scaling logic. Imho we should adopt this too; improving portability and simplifying our logic at the same time.