Use FEP-c16b: Formatting MFM functions #410
No reviewers
Labels
No labels
a11y
Bug
Bug fix
Critical Priority
Documentation
Feature
Feature request
Held for next release cycle
High Priority
Low Priority
Medium Priority
Minor change
Translation/Locale
WIP
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: AkkomaGang/akkoma-fe#410
Loading…
Reference in a new issue
No description provided.
Delete branch "ilja/akkoma-fe:use_fep-c16b_formatting_mfm_functions"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Related AkkomaGang/akkoma#381
After discussing in https://akkoma.dev/FoundKeyGang/FoundKey/issues/343 a scrubber-friendly way was discovered for representing the MFM functions in HTML. These are the front-end changes needed for that. This should fix the MFM in Akkoma and provide compatibility for the same MFM functions as Foundkey has.
In this PR we:
./src/components/emoji_input/suggestor.jsx2,x3,x4,tada, the sizes that Foundkey uses are usedoverflow: hidden;on the StatusBody now)Broader scope where this PR is part of:
Extra notes:
Sparkle has not been implemented here. It didn't work before and it requires some extra JS. This is already a strict improvement, and also a very fundamental change over multiple repositories. So it seems better to me, to do sparkle in a separate PR.
We can check MFM on https://genau.qwertqwefsday.eu/ > three dots menu > About Foundkey > click the "I ❤️ Foundkey" button > type MFM and click the "Show preview"
<>buttonList of MFM functions I tested
These are all the supported functions and different combinations of attributes, and whether they have been confirmed to work. We test text, emoji, and custom emoji.
$[center tofu 🐈 :hehe:]
$[flip tofu 🐈 :hehe:]
$[flip.v tofu 🐈 :hehe:]
$[flip.v,h tofu 🐈 :hehe:]
$[font.serif tofu 🐈 :hehe:]
$[font.monospace tofu 🐈 :hehe:]
$[font.cursive tofu 🐈 :hehe:]
$[font.fantasy tofu 🐈 :hehe:]
$[font.emoji tofu 🐈 :hehe:]
$[font.math tofu 🐈 :hehe:]
$[blur tofu 🐈 :hehe:]
$[rotate tofu 🐈 :hehe:]
$[rotate.deg=30 tofu 🐈 :hehe:]
$[x2 tofu 🐈 :hehe:]
$[x3 tofu 🐈 :hehe:]
$[x4 tofu 🐈 :hehe:]
$[x2 $[x2 tofu 🐈 :hehe:]]
$[x2 $[x2 $[x2 tofu 🐈 :hehe:]]]
$[position.x=3 tofu 🐈 :hehe:]
$[position.y=5 tofu 🐈 :hehe:]
$[position.x=3,y=5 tofu 🐈 :hehe:]
$[scale.x=3 tofu 🐈 :hehe:]
$[scale.y=5 tofu 🐈 :hehe:]
$[scale.x=3,y=5 tofu 🐈 :hehe:]
$[fg tofu 🐈 :hehe:]
$[fg.color=#fff tofu 🐈 :hehe:]
$[bg tofu 🐈 :hehe:]
$[bg.color=#fff tofu 🐈 :hehe:]
$[jelly tofu 🐈 :hehe:]
$[jelly.speed=5s tofu 🐈 :hehe:]
$[twitch tofu 🐈 :hehe:]
$[twitch.speed=5s tofu 🐈 :hehe:]
$[shake tofu 🐈 :hehe:]
$[shake.speed=5s tofu 🐈 :hehe:]
$[spin tofu 🐈 :hehe:]
$[spin.speed=5s tofu 🐈 :hehe:]
$[spin.y tofu 🐈 :hehe:]
$[spin.y,speed=0.3s tofu 🐈 :hehe:]
$[spin.x tofu 🐈 :hehe:]
$[spin.x,speed=0.3s tofu 🐈 :hehe:]
$[spin.alternate tofu 🐈 :hehe:]
$[spin.alternate,speed=0.3s tofu 🐈 :hehe:]
$[spin.left tofu 🐈 :hehe:]
$[spin.left,speed=0.3s tofu 🐈 :hehe:]
$[jump tofu 🐈 :hehe:]
$[jump.speed=5s tofu 🐈 :hehe:]
$[bounce tofu 🐈 :hehe:]
$[bounce.speed=5s tofu 🐈 :hehe:]
$[rainbow tofu 🐈 :hehe:]
$[rainbow.speed=5s tofu 🐈 :hehe:]
$[tada tofu 🐈 :hehe:]
$[tada.speed=5s tofu 🐈 :hehe:]
d1b2e7b51etoaa173b8642aa173b8642to7a7fdae4ae7a7fdae4aeto3065416c93f12068d6c2to56a59e1b55WIP: Use FEP-c16b: Formatting MFM functionsto Use FEP-c16b: Formatting MFM functionsUse FEP-c16b: Formatting MFM functionsto WIP: Use FEP-c16b: Formatting MFM functionsWIP: Use FEP-c16b: Formatting MFM functionsto Use FEP-c16b: Formatting MFM functionsdcd09d38e8to6666a273a4found a bug ☹️fixed what we can, upstream issue created where we can'thttps://mk.absturztau.be/notes/9vdb32gycp0700sc
The colours don't work. Apparently it usesOnly withoutffaa77, but I think I expect the colour values to start with#. Need to check what's the correct way(s) to do.#is valid for value forcolor, this is now fixed#in the data-mfm-color attributeI continue to think it’d be best for API users to perform the “data → style variables” transformation in the backend, so any frontend/app can easily support MFM rendering by just including a static CSS file. CSS rendering is a lower bar to clear than running scripts to parse and transform each post (and then still apply CSS rendering)
@ -0,0 +69,4 @@*/.emoji {--mfm-emoji-zoom-size: 2.63;}I think the whole emoji size scaling setup is overly complicated. Just setting
--mfm-emoji-zoom-size: 2.63;etc as here in all relevant cases and using this with a default of1.0directly in the base emojiheightdefinition would achieve the same effect as now but avoid manually overridingheightdirectly and--nested-base-emoji-sizefor some MFM effects.However, while i appreciate the effort which went into preserving Akkoma’s old emoji scaling exactly (except for nested scaling which previously was a noop), this scaling behaviour is not compatible with Foundkey, iceshrimp and likely all other keys and imho leads to a jarring discrepancy as emoji get scaled noticeable more than text. Foundkey and IceShrimp set the emoji height to
2eminstead of a fixed pixel value which means they’ll get automatically be scaled together with and by the same amount as text without any emoji-specific scaling logic. Imho we should adopt this too; improving portability and simplifying our logic at the same time.Yes! I didn't understand why I needed extra CSS for emoji while Foundey didn't ^^' Using
emworks perfectly, thx!One problem I saw with this, as you say, *key uses 2em. But Akkoma uses 38px, which corresponds to ~2.9px. So that means that we'd have smaller emoji than what people are used to on Akkoma-fe. I see two options
--emoji-size: 2em;. In normal situations, emoji will be 38px as they are now. When a zoom is used, the emoji will be e.g.2em * 200%. For tada, it will start at2em, so smaller than "normal", but then it grows bigger bc that's what tada does.When I try, this looks OK for me. It may cause things to be slightly off for things like mfmArt, but if we want to be completely compatible with the most advanced mfmArt, we have to change a lot more I'm afraid. So the question then becomes how far we want to go with that.
I now pushed a new commit with the second option, because it looks best to me, but it can still be changed if needed.
i was thinking of continuing to retain the current base size for non-MFM posts, but use
2emfor MFM posts only.This way scaling remains striaghtforward and we’re more compatible while the majority of posts (not-MFM) doesn't change at all
That one has crossed my mind too, but also has its problems (and worse imo).
From a code-complexity perspective; We now need a way for css to know what posts are mfm and not. It's doable, we can add a class somewhere with what content they were written in (or at least that it's MFM). But that's more complexity than what I did now (just one extra line in the css that already handles the zoom), so not an advantage in that regard, on the contrary.
It also means that input method now has an effect on the end result. But input method should purely be a client thing imo, transforming input into a proper representation. With MFM we couldn't do that, hence the work that this PR is part of. Bringing back the requirenment for knowing the input method and doing special handling goes against the "raison d'être" of this work for me.
There's also the fact that it may not be obvious to people why a custom emoji is smaller in some cases and bigger in others. Not all posts who are marked as having MFM are always MfmArt. So in those cases you'd have a perfectly normal-looking post, but suddenly the emoji have a different size than what people expect. This is internally inconsistent and I fear this will bring more confusion.
If it wasn’t dropped as part of this change we already have that via the
.mfmclass. In fact this change is part of the client-side overrides I’m applying atm to make MFM post behave better.current MFM CSS overrides for reference
Input methods already have an effect on the end result, e.g. MFM is rendered as an inline block, line spacing differs between MFM,Markdown and BBCode,plain text, plain text quotes get coloured while in Markdown and MFM it gets shifted, set in faint text /Or italics atm) and a bar is added, etc
Each input method is a different typeset language and to get the same result across different input methods, different input is required (if possible at all).
MFM in particular comes with stricter expectations on what constitutes a correct display and is defined by *key instances. One of those requirements being
2em-sized emoji.I should maybe rephrase myself. We get an object who has a
contentfield. It also has asourcefield. Currently we use thesourcefield, and that causes problems. This whole excercise here is about making it so that thecontentcan be used without requiring thesourcefield.Afaik, the differences you mention don't have anything to do with the
source(i.e. stated input method) field at all, they are purely derived from the content. The only exception is MFM, and that's what this is trying to fix.I certainly do not have the expectation of emoji to suddenly look different, and I very much believe it will only cause more confusion. Emoji are federated just like any one else federates them, so there's no special expectation there. There's is also no specific mention of
2emon https://misskey-hub.net/en/docs/for-users/features/mfm/Besides, if that's really the case to make, that the content should look like it does on Misskey; how far should this be taken? Because there's more than just "emoji are
2emsize". Should the text font be different? Should the spacings be different? Should we render the spacing differend? Heck, should the width of the content be changed? And so on... If yes, then we dissagree on a very fundamental level which I do not see how to resolve.Maybe a more important question; Do you consider this a blocking issue? If not, I'd like to keep it for now, as I do consider it a strict improvement regardless, and then see what people think.
This feels wrong to me. The back-end provides what to show, but how to exactly implement this, is then up to the front-end. In this case, it may by using
var(), but another front-end may want to do something different. Maybe they use a different technology, or maybe some day in the future data-* attributes become easier to work with and front-ends don't need this any more. Other problems I see are where to do this. Are we gonna store the style variables in the database? That means we're also federating it. What about scrubbers, won't they give problems? Or is this purely when a post is fetched. That means we have to run over each post in the backend when it's fetched, and implement a parser there. It seems like moving the problem from one side to another, while breaking responsabilities, for very few gain imo. Especially since sparkle (which isn't implemented yet, but I'd like to keep that for a future PR) needs something more advanced regardles.The backend provides safe HTML content to show and frontends just display the received HTML.
dataattributes don't actually tell the frontend what display is expected and require MFM-specific logic to be implement in every frontend and app.Effectively this means MFM won't ever display correctly outside akkoma-fe.
Transforming MFM data attributes into CSS variables in the backend on the other hand is simple and we already process HTML anyway to make it safe. Frontends/Apps then only need to include a simple static CSS file. If they also contain their own safety stripping logic they need to make sure CSS variables are retained, but CSS variables are safe anyway and this variable mechanism could be shared with other more powerful source formats.
Chances for correct MFM display in multiple frontends/apps seem higher this way imho.
Note:
sparklecan also be implemented as a static effect, it’s just not a straightforward copy from current Misskey since Miskkey adds and removes each sparkle node dynamically atm. (Instead of adding and removing nodes, just use a fixed amount and hide/unhide them during animation)Taking “how to exactly implement this, is then up to the front-end” further would mean all frontends completely ignore HTML content and create their own display directly from the provided post source.
Yes it does, that's what FEP-c16b is for. I linked to it in the description.
Providing the variables in
styledoesn't change that.It's simple on the front-end, and HTML is already processed there too for e.g. custom emoji, or names of people who are tagged. For backend I'm not so sure that it's simple. Currently (on main) the MFM attributes don't work. We change the content of the
sourceproperty to change the HTML, and at one time, this worked, also for attributes. But then a scrubber was changed, and suddenly it didn't get through any more. I'm not sure how it all flows in the back-end, but knowing that, I'm not so sure that it really is easier on the back-end.That's false. The
contentis a "textual representation of the Object". It describes how something should look (using HTML by default, extended with FEP-c16b for those who choose to implement FEP-c16b, still allowing a certain degree of freedom for the things who aren't specified). We can disagree on practical considerations for implementers, but this is what it is.Don't get me wrong, I do agree that adding the variables in
stylecan make things slightly easier on other front-ends. But I still also feel it's breaking responsabilities between front-end and back-end, and that simply doesn't sit right with me. Especially for a first implementation. It comes at a cost, and I'm not convinced yet that it's worth it. The way I see it; We have something that works now, if it turns out that it's indeed better to have this done on the back-end (as well?) then it can still be done later by whoever wants to implement that. But if we do it now, and it turns out to give unexpected problems, then going back is way harder because that will be a breaking change. Therefor I would like to keep this as it is now.ok so fun fact part of the reason mfm support in current stable is so patchy is that *oma supports pure html posting, which means our scrubber needs to be very resilient to not allow shenanigans through
which means the amount of processing we can actually do on the backend safely is limited - I actually removed some functionality after it was shown to allow bad remote html through because the style attribute is mostly unbound in flexibility
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.