AkkomaGang/akkoma

Fork 94

[feat] compute (and federate) resolution and blurhash for instance's own media uploads #446

New issue

Open

opened 2023-01-25 14:42:50 +00:00 by lnl · 4 comments

lnl commented

2023-01-25 14:42:50 +00:00

The idea

if someone uploads an image or video on an Akkoma instance, the width, height, and blurhash should be calculated for them, and get federated with other instances (as it already happens with incoming federation from mastodon)

The reasoning

better API compatibility with Mastodon clients (e.g. https://github.com/mastodon/mastodon-android/issues/69)
better support for blurhash in clients, including akkoma-fe (which does use blurhash for sensitive images from Mastodon instances, like here: https://crimew.gay/notice/AQQSMvWmQsRK8DFzwe, but not for posts made on Akkoma)

Have you searched for this feature request?

I have double-checked and have not found this feature request mentioned anywhere.
This feature is related to the Akkoma backend specifically, and not pleroma-fe.

### The idea if someone uploads an image or video on an Akkoma instance, the width, height, and blurhash should be calculated for them, and get federated with other instances (as it already happens with incoming federation from mastodon) ### The reasoning - better API compatibility with Mastodon clients (e.g. https://github.com/mastodon/mastodon-android/issues/69) - better support for blurhash in clients, including akkoma-fe (which does use blurhash for sensitive images from Mastodon instances, like here: https://crimew.gay/notice/AQQSMvWmQsRK8DFzwe, but not for posts made on Akkoma) ### Have you searched for this feature request? - [x] I have double-checked and have not found this feature request mentioned anywhere. - [x] This feature is related to the Akkoma backend specifically, and not pleroma-fe.

lnl added the

feature request

label 2023-01-25 14:42:50 +00:00

floatingghost commented

2023-01-25 14:54:41 +00:00

Owner

this actually already works, you need to enable the AnalyzeMetadata upload filter

maybe that should be renamed

this actually already works, you need to enable the AnalyzeMetadata upload filter maybe that should be renamed

lnl commented

2023-01-25 16:27:00 +00:00

Author

hm, why isn't this the default behavior?

floatingghost commented

2023-01-25 18:23:45 +00:00

Owner

idk

just isn't

idk just isn't

ilja commented

2023-01-26 07:09:19 +00:00

Contributor

My guess is because the module requires ffprobe for width and height, which is an extra dependency[1].

One way to handle this, is to add it as a question to the instance gen task; Ask if people want blurhash and dimensions to be added and that ffprobe is required for it, detect if it's installed, let them know if it is or not, and default to yes when installed and default to no if not installed. This is how it's also done for stripping location data (where exiftool is required)[2].

I should add that this information is not required in AP, so there may still be remote images without this. Mastodon keeps a local cache of images, so they can add it then. Akkoma does not do this. If this is considered a problem (but is probably a different issue), I see a couple of solutions:

Don't mind that there are still some images who don't have it and use a fallback "warning" image
- Is now the case for akkoma-fe, but it gives an inconsistent look
Add a default blurhash for images where it's missing
- The look may be more consistent, but still not the actual blurhash
- This can be done BE-side or FE-side
- Actually, you could also not use a default blurhash and just use an image that looks more like a blur hash (:
Add this info through MRF when missing
- This means that all attachments (or at least images) will need to be downloaded to the server so it can be processed. I'm not sure if this extra load is worth it.
Maybe clients can determine this?
- Needs work on the clients, so not a good solution either. Also not sure how possible this is, if at all.

Personally I think having a default blurhash would make most sense all things considered, but I don't use this, so maybe I'm wrong.

[1] https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/upload/filter/analyze_metadata.ex#L64
[2] https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/mix/tasks/pleroma/instance.ex#L166

My guess is because the module requires ffprobe for width and height, which is an extra dependency[1]. One way to handle this, is to add it as a question to the `instance gen` task; Ask if people want blurhash and dimensions to be added and that ffprobe is required for it, detect if it's installed, let them know if it is or not, and default to yes when installed and default to no if not installed. This is how it's also done for stripping location data (where exiftool is required)[2]. *** I should add that this information is not required in AP, so there may still be remote images without this. Mastodon keeps a local cache of images, so they can add it then. Akkoma does not do this. If this is considered a problem (but is probably a different issue), I see a couple of solutions: * Don't mind that there are still some images who don't have it and use a fallback "warning" image * Is now the case for akkoma-fe, but it gives an inconsistent look * Add a default blurhash for images where it's missing * The look may be more consistent, but still not the actual blurhash * This can be done BE-side or FE-side * Actually, you could also not use a default blurhash and just use an image that looks more like a blur hash (: * Add this info through MRF when missing * This means that all attachments (or at least images) will need to be downloaded to the server so it can be processed. I'm not sure if this extra load is worth it. * Maybe clients can determine this? * Needs work on the clients, so not a good solution either. Also not sure how possible this is, if at all. Personally I think having a default blurhash would make most sense all things considered, but I don't use this, so maybe I'm wrong. *** [1] <https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/upload/filter/analyze_metadata.ex#L64> [2] <https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/mix/tasks/pleroma/instance.ex#L166>

No milestone

No project

No assignees

3 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#446

No description provided.