FEP-dc88: Formatting Mathematics #642

Open
pounce wants to merge 1 commit from pounce/akkoma:formatting-mathematics into develop
First-time contributor

See #641

We're so back

this is how it looks in akko-fe
image
and how it is supposed to look, in mastodon-fe:
image

See #641 # We're so back this is how it looks in akko-fe ![image](/attachments/f5230c17-8d95-4c68-aeec-b09a7d156d01) and how it is supposed to look, in mastodon-fe: ![image](/attachments/5ac79f28-38cc-487d-aa9a-cf6c249fcdc6)
pounce added 1 commit 2023-09-16 21:46:42 +00:00
Allow MathML core tags in sanitized content
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
1b838627df
Member

Good news, the Vue bug appears to have been fixed a few week s ago: vuejs/core@d42b6ba3 and this fix is part of release 3.4.0+

Good news, the Vue bug appears to have been fixed a few week s ago: [vuejs/core@d42b6ba3](https://github.com/vuejs/core/commit/d42b6ba3f530746eb1221eb7a4be0f44eb56f7d3) and this fix is part of release 3.4.0+
Contributor

I wonder, is the front-end the only thing blocking this? As I understand, it does work with other front-ends like masto-fe, so I wonder if we should really wait until front-end is fixed. People generally install Akkoma-fe, but as I understand the idea is that Akkoma doesn't force/push one specific front-end.

I wonder, is the front-end the only thing blocking this? As I understand, it does work with other front-ends like masto-fe, so I wonder if we should really wait until front-end is fixed. People generally install Akkoma-fe, but as I understand the idea is that Akkoma doesn't force/push one specific front-end.
Member

Btw, re

As it's all comprised of macros around partial function definitions, it's not possible to use very much abstraction (such as for comprehensions or variable definitions) to help define a sanitizer, which leads to very ugly code.

I’m not too familiar with Elixir’s macro stuff, but some examples I’ve seen seem to suggest using regular for is at least sometimes capable of looping over macros and the below at least compiles (with math enabled) fine for me. Is there going something wrong during runtime later? (might want to add some basic tests)

  if Pleroma.Config.get([:markup, :allow_math]) do
    Meta.allow_tag_with_these_attributes("annotation", ["encoding"])
    Meta.allow_tag_with_these_attributes(:"annotation-xml", ["encoding"])

    Meta.allow_tag_with_these_attributes(:math, [
      "display",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])

    basic_math_tags = [
      "maction",
      "merror",
      :mi,
      "mmultiscripts",
      :mn,
      "mphantom",
      "mprescripts",
      "mroot",
      "mrow",
      "ms",
      "msqrt",
      "mstyle",
      "msub",
      "msubsup",
      "msup",
      "mtable",
      "mtext",
      "mtr",
      "semantics"
    ]

    for tag <- basic_math_tags do
      Meta.allow_tag_with_these_attributes(tag, ["mathvariant", "displaystyle", "scriptlevel"])
    end

    for tag <- ["mover", "munder", "munderover"] do
      Meta.allow_tag_with_these_attributes(tag, [
        "accent",
        "accentunder",
        "mathvariant",
        "displaystyle",
        "scriptlevel"
      ])
    end

    Meta.allow_tag_with_these_attributes("mfrac", [
      "linethickness",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])

    Meta.allow_tag_with_these_attributes(:mo, [
      "form",
      "stretchy",
      "symmetric",
      "largeop",
      "movablelimits",
      "lspace",
      "rspace",
      "minsize",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])

    Meta.allow_tag_with_these_attributes("mpadded", [
      "width",
      "height",
      "depth",
      "lspace",
      "voffset",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])

    Meta.allow_tag_with_these_attributes("mspace", [
      "width",
      "height",
      "depth",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])

    Meta.allow_tag_with_these_attributes("mtd", [
      "columnspan",
      "rowspan",
      "mathvariant",
      "displaystyle",
      "scriptlevel"
    ])
  end
Btw, re > As it's all comprised of macros around partial function definitions, it's not possible to use very much abstraction (such as for comprehensions or variable definitions) to help define a sanitizer, which leads to very ugly code. I’m not too familiar with Elixir’s macro stuff, but some examples I’ve seen seem to suggest using regular `for` is at least _sometimes_ capable of looping over macros and the below at least compiles (with math enabled) fine for me. Is there going something wrong during runtime later? *(might want to add some basic tests)* ```elixir if Pleroma.Config.get([:markup, :allow_math]) do Meta.allow_tag_with_these_attributes("annotation", ["encoding"]) Meta.allow_tag_with_these_attributes(:"annotation-xml", ["encoding"]) Meta.allow_tag_with_these_attributes(:math, [ "display", "mathvariant", "displaystyle", "scriptlevel" ]) basic_math_tags = [ "maction", "merror", :mi, "mmultiscripts", :mn, "mphantom", "mprescripts", "mroot", "mrow", "ms", "msqrt", "mstyle", "msub", "msubsup", "msup", "mtable", "mtext", "mtr", "semantics" ] for tag <- basic_math_tags do Meta.allow_tag_with_these_attributes(tag, ["mathvariant", "displaystyle", "scriptlevel"]) end for tag <- ["mover", "munder", "munderover"] do Meta.allow_tag_with_these_attributes(tag, [ "accent", "accentunder", "mathvariant", "displaystyle", "scriptlevel" ]) end Meta.allow_tag_with_these_attributes("mfrac", [ "linethickness", "mathvariant", "displaystyle", "scriptlevel" ]) Meta.allow_tag_with_these_attributes(:mo, [ "form", "stretchy", "symmetric", "largeop", "movablelimits", "lspace", "rspace", "minsize", "mathvariant", "displaystyle", "scriptlevel" ]) Meta.allow_tag_with_these_attributes("mpadded", [ "width", "height", "depth", "lspace", "voffset", "mathvariant", "displaystyle", "scriptlevel" ]) Meta.allow_tag_with_these_attributes("mspace", [ "width", "height", "depth", "mathvariant", "displaystyle", "scriptlevel" ]) Meta.allow_tag_with_these_attributes("mtd", [ "columnspan", "rowspan", "mathvariant", "displaystyle", "scriptlevel" ]) end ```
Author
First-time contributor

I wonder, is the front-end the only thing blocking this? As I understand, it does work with other front-ends like masto-fe, so I wonder if we should really wait until front-end is fixed. People generally install Akkoma-fe, but as I understand the idea is that Akkoma doesn't force/push one specific front-end.

Yes agree. I'll rebase this soon to get it ready to merge

> I wonder, is the front-end the only thing blocking this? As I understand, it does work with other front-ends like masto-fe, so I wonder if we should really wait until front-end is fixed. People generally install Akkoma-fe, but as I understand the idea is that Akkoma doesn't force/push one specific front-end. Yes agree. I'll rebase this soon to get it ready to merge
Author
First-time contributor

As it's all comprised of macros around partial function definitions, it's not possible to use very much abstraction (such as for comprehensions or variable definitions) to help define a sanitizer, which leads to very ugly code.

I’m not too familiar with Elixir’s macro stuff, but some examples I’ve seen seem to suggest using regular for is at least sometimes capable of looping over macros and the below at least compiles (with math enabled) fine for me. Is there going something wrong during runtime later? (might want to add some basic tests)

so this doesn't work, but it turns out if you unquote(tag) then it does. thanks for pointing me in the right direction!

> > As it's all comprised of macros around partial function definitions, it's not possible to use very much abstraction (such as for comprehensions or variable definitions) to help define a sanitizer, which leads to very ugly code. > > I’m not too familiar with Elixir’s macro stuff, but some examples I’ve seen seem to suggest using regular `for` is at least _sometimes_ capable of looping over macros and the below at least compiles (with math enabled) fine for me. Is there going something wrong during runtime later? *(might want to add some basic tests)* so this doesn't work, but it turns out if you `unquote(tag)` then it does. thanks for pointing me in the right direction!
pounce force-pushed formatting-mathematics from 1b838627df to bfde898225 2024-08-16 15:17:13 +00:00 Compare
pounce force-pushed formatting-mathematics from bfde898225 to 1cf33a318d 2024-08-16 15:20:40 +00:00 Compare
pounce changed title from WIP: FEP-dc88: Formatting Mathematics to FEP-dc88: Formatting Mathematics 2024-08-16 15:20:51 +00:00
Author
First-time contributor

ok my bad i forgor about this pr
but now it's ready to merge

ok my bad i forgor about this pr but now it's ready to merge
Member

Thanks!

Is there a post to test with which neither requires auth-fetch (like https://types.pl/@pounce/111105464618076520) nor is MFM (like https://genau.qwertqwefsday.eu/notes/9k1517t772; since MFM posts get reparsed from source atm and we don't parse LaTeX input in MFM (yet))?

Creating a local HTML post with content copied from a FoundKey example confirms the backend part works fine, but there are still some catches wrt to frontend.
(side note: i’m surprised HTML posts are still supported; i thought they were intended to be dropped over a year ago)

Example HTML
<p>
Testing MathML:
</p>

<math xmlns="http://www.w3.org/1998/Math/MathML">
<semantics>
<annotation encoding="application/x-tex">
x= \frac{-b' \pm \sqrt{(b')^2-ac}}{a}
</annotation>

<mrow><mi>x</mi><mo>=</mo><mfrac><mrow><mo></mo><msup><mi>b</mi><mo lspace="0em" rspace="0em" mathvariant="normal"></mo></msup><mo>±</mo><msqrt><mrow><mo stretchy="false">(</mo><msup><mi>b</mi><mo lspace="0em" rspace="0em" mathvariant="normal"></mo></msup><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mo></mo><mi>a</mi><mi>c</mi></mrow></msqrt></mrow><mi>a</mi></mfrac></mrow>

</semantics>
</math>

<br />

<span class="mfm-flip" data-mfm-h data-mfm-v>
FoundKey expands the world of the Fediverse
</span>
me being confused while things actually work fine

Taken as is only the annotation shows up in Firefox and no math at all in Chromium:
h.webp

Moving </semantics> to only enclose the annotation leads to the intended displa in chromium, but both annotations and the intended MathML showing up in Firefox, but FoundKey really seems to enclose everything in <semantics>, check e.g. the post above. Is this a FoundKey bug?

i.webp

EDIT: but also the above example post linked above seems to work fine. It turns out Firefox or MathML in general is sensitive to whether annotation comes before or after math. If it’s placed at the end of but still inside <semantics>, after all the actual MathML everything works as expected
Real FoundKey posts also place it at the end so there’s nothing wrong here

I upgraded all vue related packages in the frontend and assume we’ll wan’t to add CSS to explicitly hide annotations. Are there any other frontend changes you’d suggest?

Thanks! Is there a post to test with which neither requires auth-fetch *(like https://types.pl/@pounce/111105464618076520)* nor is MFM *(like https://genau.qwertqwefsday.eu/notes/9k1517t772; since MFM posts get reparsed from source atm and we don't parse LaTeX input in MFM (yet))*? Creating a local HTML post with content copied from a FoundKey example confirms the backend part works fine, but there are still some catches wrt to frontend. *(side note: i’m surprised HTML posts are still supported; i thought they were intended to be dropped over a year ago)* <details> <summary>Example HTML</summary> ```html <p> Testing MathML: </p> <math xmlns="http://www.w3.org/1998/Math/MathML"> <semantics> <annotation encoding="application/x-tex"> x= \frac{-b' \pm \sqrt{(b')^2-ac}}{a} </annotation> <mrow><mi>x</mi><mo>=</mo><mfrac><mrow><mo>−</mo><msup><mi>b</mi><mo lspace="0em" rspace="0em" mathvariant="normal">′</mo></msup><mo>±</mo><msqrt><mrow><mo stretchy="false">(</mo><msup><mi>b</mi><mo lspace="0em" rspace="0em" mathvariant="normal">′</mo></msup><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mo>−</mo><mi>a</mi><mi>c</mi></mrow></msqrt></mrow><mi>a</mi></mfrac></mrow> </semantics> </math> <br /> <span class="mfm-flip" data-mfm-h data-mfm-v> FoundKey expands the world of the Fediverse </span> ``` </details> <details> <summary>me being confused while things actually work fine</summary> Taken as is only the annotation shows up in Firefox and no math at all in Chromium: ![h.webp](/attachments/410f9267-ecb1-421d-bbd4-09e33c2cd526) Moving `</semantics>` to only enclose the annotation leads to the intended displa in chromium, but both annotations and the intended MathML showing up in Firefox, but FoundKey really seems to enclose everything in `<semantics>`, check e.g. the post above. Is this a FoundKey bug? ![i.webp](/attachments/31030285-2467-4f7c-b1b1-918b20062269) **EDIT**: but also the above example post linked above seems to work fine. It turns out Firefox or MathML in general is sensitive to whether annotation comes before or after math. If it’s placed at the end of but still inside `<semantics>`, after all the actual MathML everything works as expected Real FoundKey posts also place it at the end so there’s nothing wrong here </details> I upgraded all vue related packages in the frontend ~~and assume we’ll wan’t to add CSS to explicitly hide annotations.~~ Are there any other frontend changes you’d suggest?
7.3 KiB
9.3 KiB
Oneric approved these changes 2024-08-17 00:54:21 +00:00
Author
First-time contributor

EDIT: but also the above example post linked above seems to work fine. It turns out Firefox or MathML in general is sensitive to whether annotation comes before or after math. If it’s placed at the end of but still inside <semantics>, after all the actual MathML everything works as expected
Real FoundKey posts also place it at the end so there’s nothing wrong here

indeed, this stuff is usually supposed to be autogenerated. I use something like https://temml.org/ most of the time with 'annotate' checked on. In this case the annotation(s) should go at the end.

I upgraded all vue related packages in the frontend and assume we’ll wan’t to add CSS to explicitly hide annotations. Are there any other frontend changes you’d suggest?

The current vue package ref in the frontend already works for this (we just need 3.4). but cutting a new release would probably be good. I think the frontend is mostly equipped to handle these changes, but everything will look like crap if an incomming post with math is scrubbed (e.g. if :allow_math is false) either way. I was thinking about implementing the other half of the fep (correct scrubbing) in another PR, but i can also do it here if need-be.

> **EDIT**: but also the above example post linked above seems to work fine. It turns out Firefox or MathML in general is sensitive to whether annotation comes before or after math. If it’s placed at the end of but still inside `<semantics>`, after all the actual MathML everything works as expected > Real FoundKey posts also place it at the end so there’s nothing wrong here indeed, this stuff is usually supposed to be autogenerated. I use something like https://temml.org/ most of the time with 'annotate' checked on. In this case the annotation(s) should go at the end. > I upgraded all vue related packages in the frontend ~~and assume we’ll wan’t to add CSS to explicitly hide annotations.~~ Are there any other frontend changes you’d suggest? The current vue package ref in the frontend already works for this (we just need 3.4). but cutting a new release would probably be good. I think the frontend is mostly equipped to handle these changes, but everything will look like crap if an incomming post with math is scrubbed (e.g. if `:allow_math` is false) either way. I was thinking about implementing the other half of the fep (correct scrubbing) in another PR, but i can also do it here if need-be.
Oneric referenced this pull request from a commit 2024-08-17 16:04:25 +00:00
Author
First-time contributor

not sure if :allow_math should be default or not tbh
we need better scrubbing if it's off

not sure if `:allow_math` should be default or not tbh we need better scrubbing if it's off
Member

Personally i’d even prefer math to be enabled by default (after all we want to improve the reliability of math federation), but it’s ofc up to floati

re better scrubbing: this prob needs changes in the scrubber library. It currently has the option to handle scrubbing of style fields with a custom function (which we don't use atm). Something similar would need to be added for math nodes, so we can e.g. replace the whole math env with its annotation content if present and fallback to something else (current behaviour?) for annotation-less nodes

Personally i’d even prefer math to be enabled by default *(after all we want to improve the reliability of math federation)*, but it’s ofc up to floati re better scrubbing: this prob needs changes in the scrubber library. It currently has the option to handle scrubbing of `style` fields with a custom function *(which we don't use atm)*. Something similar would need to be added for math nodes, so we can e.g. replace the whole math env with its annotation content if present and fallback to something else (current behaviour?) for annotation-less nodes
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u formatting-mathematics:pounce-formatting-mathematics
git checkout pounce-formatting-mathematics
Sign in to join this conversation.
No description provided.