WIP: Portable image descriptions, part 1: Return image description from exif data

ilja commented

2022-10-23 13:14:31 +00:00

Contributor

During attachment upload Pleroma returns a "description" field.

This MR allows Pleroma to read the EXIF data during upload and return the description to the FE using this field.
- If a description is already present (e.g. because a previous module added it), it will use that
- Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract
- If no description is found, it will simply return nil, which is the default value
When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so

There was an Exiftool module, which has now been renamed to Exiftool.StripLocation

This is part of a broader "vision" where the ultimate goal is to get a following flow:

Person makes post with an image as attachement and fils in an image description
Someone else downloads the image for later use
That someone now re uploads the image
The original image description is pre-filled for this person

We do this by storing the description in the EXIF data. For this we need:

Part 1: Read the exif data on upload and return it to the front-end (This PR)
Part 2: Front end should propose this text to use as image description by pre-filling the field (Already works out of the box 🎉)
Part 3: When someone makes a post with attachement, we need to store the image description in the EXIF data when the post is made
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3803/diffs

During attachment upload Pleroma returns a "description" field. * This MR allows Pleroma to read the EXIF data during upload and return the description to the FE using this field. * If a description is already present (e.g. because a previous module added it), it will use that * Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract * If no description is found, it will simply return nil, which is the default value * When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so There was an Exiftool module, which has now been renamed to Exiftool.StripLocation --- This is part of a broader "vision" where the ultimate goal is to get a following flow: 1. Person makes post with an image as attachement and fils in an image description 2. Someone else downloads the image for later use 3. That someone now re uploads the image 4. The original image description is pre-filled for this person We do this by storing the description in the EXIF data. For this we need: * [x] Part 1: Read the exif data on upload and return it to the front-end (This PR) * [x] Part 2: Front end should propose this text to use as image description by pre-filling the field (Already works out of the box 🎉) * [ ] Part 3: When someone makes a post with attachement, we need to store the image description in the EXIF data when the post is made * [ ] <https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3803/diffs>

ilja added 3 commits 2022-10-23 13:14:31 +00:00

338612d72b Use EXIF data of image to prefill image description

During attachment upload Pleroma returns a "description" field.

* This MR allows Pleroma to read the EXIF data during upload and return the description to the FE using this field.
    * If a description is already present (e.g. because a previous module added it), it will use that
    * Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract
    * If no description is found, it will simply return nil, which is the default value
* When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so

There was an Exiftool module, which has now been renamed to Exiftool.StripLocation

f50cffd134 update moduledoc

ci/woodpecker/pr/woodpecker Pipeline is pending Details

66a04cead3 Descriptions from exif data with only whitespeces are considered empty

I noticed that pictures taken with Ubuntu-Touch have whitespace in one of the fields
This should just be ignored imo

nbsp commented

2022-12-07 23:09:51 +00:00

Contributor

what's the status on this PR? it looks very useful

ilja commented

2022-12-08 13:00:43 +00:00

Author

Contributor

@sfr Thk you :)

I assume the most relevant part will be when the third part (writing to images) is also done, so I was waiting until I finish working on that one. Especially since there are things I may want to change and that's easier when it's not merged yet (as in: I won't have to care about breaking changes if it's not yet in develop or stable).

But it looks harder then I originally anticipated and I'm not sure when I'll continue working on it. If someone believes this is already useful without the third part, then I can fix the current merge conflict and take it out of WIP. It's already merged in Pleroma, so it can work on it's own.

@sfr Thk you :) I assume the most relevant part will be when the third part (writing to images) is also done, so I was waiting until I finish working on that one. Especially since there are things I may want to change and that's easier when it's not merged yet (as in: I won't have to care about breaking changes if it's not yet in develop or stable). But it looks harder then I originally anticipated and I'm not sure when I'll continue working on it. If someone believes this is already useful without the third part, then I can fix the current merge conflict and take it out of WIP. It's already merged in Pleroma, so it can work on it's own.

timorl commented

2024-04-14 18:44:16 +00:00

Contributor

I recently learned about descriptions in EXIF data and decided this would be a cool thing to contribute, so imagine my joy when I noticed this PR doing exactly what I wanted! <3 I was less ambitious and only wanted to have the reading part without the writing, so in particular I think it would be quite cool to have this merged in the form it is in currently. I might even take a shot at finishing the writing functionality later (is there a reason to not have this extension get an additional feature in a separate PR?).

I merged the current develop into this in https://akkoma.dev/timorl/akkoma/src/branch/elseinspe – feel free to grab that commit, or I can open a PR from there directly if you don't want to keep working on this.

One question – when I started looking into this I though the default EXIF field should be just -Description – is there a reason why you preferred -ImageDescription? Or at least didn't add -Description as a possibility?

I recently learned about descriptions in EXIF data and decided this would be a cool thing to contribute, so imagine my joy when I noticed this PR doing exactly what I wanted! <3 I was less ambitious and only wanted to have the reading part without the writing, so in particular I think it would be quite cool to have this merged in the form it is in currently. I might even take a shot at finishing the writing functionality later (is there a reason to not have this extension get an additional feature in a separate PR?). I merged the current develop into this in https://akkoma.dev/timorl/akkoma/src/branch/elseinspe – feel free to grab that commit, or I can open a PR from there directly if you don't want to keep working on this. One question – when I started looking into this I though the default EXIF field should be just `-Description` – is there a reason why you preferred `-ImageDescription`? Or at least didn't add `-Description` as a possibility?

ilja commented

2024-04-15 06:11:36 +00:00

Author

Contributor

hey @timorl !

i'm glad to see there's other people interested in this. Sadly, i haven't found much time to work on Akkoma in the last year or so, and may not in the near future. If you are willing to create your own PR for it, please do! I'd be happy to see this in Akkoma as well.

For why I chose the fields I did, it's been a while. I did look around, but didn't came to a very conclusive conclusion. I chose those fields because it's what I found at the time and seemed to fit. I do remember one of the two fields I used only allowed a subset of characters, and even had a character limit. But I'm unsure. I made it in a way that a fallback is used, so it could still be changed/expanded on later, so feel free to do that if another way seems better to you. See also https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3535#note_89496

For why I was waiting for the 3rth part (write to media). It would've been a separate PR. I just didn't want to get this merged before knowing how easy the writing part would be because I didn't want an unfinished feature. But if anyone thinks this is already useful, then it can be merged for me because that means it's a perfect valid feature on its own. Maybe I was just worrying over nothing :)

If there's still questions I may be able to answer/help with, do ask! I'll try to answer as well as I can.

hey @timorl ! i'm glad to see there's other people interested in this. Sadly, i haven't found much time to work on Akkoma in the last year or so, and may not in the near future. If you are willing to create your own PR for it, please do! I'd be happy to see this in Akkoma as well. For why I chose the fields I did, it's been a while. I did look around, but didn't came to a very conclusive conclusion. I chose those fields because it's what I found at the time and seemed to fit. I do remember one of the two fields I used only allowed a subset of characters, and even had a character limit. But I'm unsure. I made it in a way that a fallback is used, so it could still be changed/expanded on later, so feel free to do that if another way seems better to you. See also https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3535#note_89496 For why I was waiting for the 3rth part (write to media). It would've been a separate PR. I just didn't want to get this merged before knowing how easy the writing part would be because I didn't want an unfinished feature. But if anyone thinks this is already useful, then it can be merged for me because that means it's a perfect valid feature on its own. Maybe I was just worrying over nothing :) If there's still questions I may be able to answer/help with, do ask! I'll try to answer as well as I can.

❤️ 1

timorl referenced this pull request

2024-04-15 15:45:36 +00:00

Read image description from EXIF data #744

timorl commented

2024-04-15 15:50:45 +00:00

Contributor

I created #744 then. :)

As for the fields, I looked up the main (official?, from here anyway) specification and Description by itself is not even an option there – this seems to be something made up by exiftool. The newest version of the specification allows UTF-8 and arbitrary length of ImageDescription, so it's the perfect default, you were right.

I created #744 then. :) As for the fields, I looked up the main (official?, from [here](https://www.cipa.jp/e/std/std-sec.html) anyway) specification and `Description` by itself is not even an option there – this seems to be something made up by `exiftool`. The newest version of the specification allows UTF-8 and arbitrary length of `ImageDescription`, so it's the perfect default, you were right.

Oneric referenced this pull request

2024-04-15 22:16:49 +00:00

Exiftool: Strip all non-essential metadata tags #745

ci/woodpecker/pr/woodpecker Pipeline is pending

Details

This pull request has changes conflicting with the target branch.

CHANGELOG.md
docs/docs/configuration/cheatsheet.md
docs/docs/installation/optional/media_graphics_packages.md
lib/mix/tasks/pleroma/instance.ex
lib/pleroma/config/deprecation_warnings.ex
lib/pleroma/upload.ex
lib/pleroma/upload/filter/exiftool.ex
test/mix/tasks/pleroma/instance_test.exs
test/pleroma/upload/filter/exiftool_test.exs

WIP: Portable image descriptions, part 1: Return image description from exif data #241