[bug] S3 Malformed URL #756

New Issue

Closed

opened 2024-04-23 10:39:59 +00:00 by amey · 2 comments

amey commented

2024-04-23 10:39:59 +00:00

Your setup

OTP

Extra details

Ubuntu 22.04

Version

Backend: 3.12.2-0-g26a91d5, Frontend: 7cc6c35

PostgreSQL version

What were you trying to do?

First off, thanks so much for supporting this project.

Given the recent changes to base_url (commit fef773ca35), I decided to move to S3 as a storage provider in place of local storage. This is an issue I have been meaning to make for some time, but I'm inexperienced with S3, so that might explain this. However, my issue is as follows:

What did you expect to happen?

I've entered all the S3 settings in AdminFE, and the images upload to the correct bucket. However, when serving the files, the URLs are generated in the form https://<base_url>/<bucket_namespace>:<bucket>/<key>. However, from my testing, AWS doesn't work with the <bucket_namespace>:<bucket> portion in the URL: only the <bucket> term should be there. The links work correctly when the <bucket_namespace>: part is removed (i.e. https://<base_url>/<bucket>/<key>).

What actually happened?

When the <bucket_namespace> value is left unentered, there is still a preceding colon in the URL. Is this expected behavior, or am I misinterpreting the settings?

Lastly, thank you again for reading this. My apologies if this is posted in the incorrect section or mis-formatted. Please let me know how I can correctly post this.

Logs

No response

Severity

I cannot use it as easily as I'd like

Have you searched for this issue?

I have double-checked and have not found this issue mentioned anywhere.

### Your setup OTP ### Extra details Ubuntu 22.04 ### Version Backend: 3.12.2-0-g26a91d5, Frontend: 7cc6c35 ### PostgreSQL version 14 ### What were you trying to do? First off, thanks so much for supporting this project. Given the recent changes to `base_url` (commit `fef773ca35`), I decided to move to S3 as a storage provider in place of local storage. This is an issue I have been meaning to make for some time, but I'm inexperienced with S3, so that might explain this. However, my issue is as follows: ### What did you expect to happen? I've entered all the S3 settings in AdminFE, and the images upload to the correct bucket. However, when serving the files, the URLs are generated in the form `https://<base_url>/<bucket_namespace>:<bucket>/<key>`. However, from my testing, AWS doesn't work with the `<bucket_namespace>:<bucket>` portion in the URL: only the `<bucket>` term should be there. The links work correctly when the `<bucket_namespace>:` part is removed (i.e. `https://<base_url>/<bucket>/<key>`). ### What actually happened? When the `<bucket_namespace>` value is left unentered, there is still a preceding colon in the URL. Is this expected behavior, or am I misinterpreting the settings? Lastly, thank you again for reading this. My apologies if this is posted in the incorrect section or mis-formatted. Please let me know how I can correctly post this. ### Logs _No response_ ### Severity I cannot use it as easily as I'd like ### Have you searched for this issue? - [x] I have double-checked and have not found this issue mentioned anywhere.

amey added the

bug

label 2024-04-23 10:39:59 +00:00

amey changed title from ~~[bug]~~ to [bug] S3 Malformed URL

2024-04-23 14:04:31 +00:00

Oneric commented

2024-04-23 14:39:12 +00:00

Member

When the <bucket_namespace> value is left unentered, there is still a preceding colon in the URL. Is this expected behavior, or am I misinterpreting the settings?

This sounds like you entered an empty string, but the setting is still set. You need to fully remove the setting (i.e. make its value nil). You can do so by clicking the garbage icon next to the setting in admin-fe

Alternatively, you can override the combined namespace:bucket value by setting truncated_namespace.

> When the `<bucket_namespace>` value is left unentered, there is still a preceding colon in the URL. Is this expected behavior, or am I misinterpreting the settings? This sounds like you entered an empty string, but the setting is still set. You need to fully remove the setting (i.e. make its value `nil`). You can do so by clicking the garbage icon next to the setting in admin-fe ![garbage-icon.png](/attachments/9e71075f-fce1-433a-a26c-06d3d4b7e800) Alternatively, you can override the combined `namespace:bucket` value by setting `truncated_namespace`.

garbage-icon.png

836 B

amey commented

2024-04-23 17:55:32 +00:00

Author

Thank you so much @Oneric! It’s working now. The steps I did were change the base url to https://<bucket>.s3.<region>.amazonaws.com, since deleting the bucket_namespace value resulted in the entire <bucket_namespace>:<bucket> string being left out of the url.

There may still be some unintended action here, but the issue is solved thanks to your help!

Cheers

Thank you so much @Oneric! It’s working now. The steps I did were change the base url to `https://<bucket>.s3.<region>.amazonaws.com`, since deleting the bucket_namespace value resulted in the entire `<bucket_namespace>:<bucket>` string being left out of the url. There may still be some unintended action here, but the issue is solved thanks to your help! Cheers

amey closed this issue

2024-04-23 17:55:33 +00:00

No Milestone

No project

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#756

No description provided.