Commit graph

91 commits

Author SHA1 Message Date
Johann150 ada577bde6
server: fix/document strange requirements for emoji packs
Some checks failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/test Pipeline failed
The change in the emoji export logic should fix the case where emoji
packs exported with Foundkey should be used in any other Misskey fork.

I've opted not to change the import logic and instead document the
strange behaviour because it would also not be accepted by Misskey.
2024-01-04 21:31:04 +01:00
Johann150 37658f5162
server: try to fix queue errors
Some checks failed
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline failed
Some errors in the queue are not properly handled. For example a blocked instance
will cause the respective queue job to be retried. this should of course not happen
and instead the job should be dropped. This is trying to correct that.

Changelog: Fixed
2023-09-24 16:29:27 +02:00
Johann150 a12debb7b6
server: replace unzipper with decompress
Some checks failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/test Pipeline failed
The unzipper package did not seem to work any more and was
mangling the meta.json file in its extracted form and potentially
other files which lead to the emoji import not working properly.

Changelog: Fixed
2023-07-02 00:09:45 +02:00
Johann150 f760426142
fix internal download in emoji import
Changelog: Fixed
2023-07-02 00:09:35 +02:00
Johann150 2ea6daaf7a
rename extractDbHost to extractPunyHost
Some checks failed
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/test Pipeline failed
2023-06-27 22:02:32 +02:00
Johann150 597de07465
server: refactor HTTP signature validation
Some checks failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/test Pipeline failed
2023-06-27 21:46:00 +02:00
Johann150 51a319e8ca
use extractDbHost
Some checks failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/test Pipeline failed
2023-06-23 22:00:31 +02:00
Johann150 7c9e118ff1
refactor checkExpired to use Promise.all
Some checks failed
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline failed
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline failed
ci/woodpecker/push/test Pipeline failed
2023-05-30 20:40:25 +02:00
Johann150 f54fa0ad02
server: remove unnecessary complex loop 2023-05-23 22:56:27 +02:00
Johann150 4312393169
Revert 'Revert "server: fix user deletion race condition"'
This reverts commit bb3ec8bafe.
2023-05-23 21:49:33 +02:00
Johann150 d6452795b0
translate more comments 2023-05-19 00:21:47 +02:00
Johann150 4fbbfff145
activitypub: also check incoming activity host for block
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2023-04-16 19:34:15 +02:00
Johann150 5f4aab6d46
translate yet another japanese comment 2023-04-16 19:33:50 +02:00
Johann150 75fd42b070
server: check for valid keyId URL before parse 2023-04-16 19:33:28 +02:00
Johann150 1bce487965
activitypub: allow to group elements for delivery 2023-03-11 09:42:14 +01:00
Johann150 78c93c5539
server: clean up activitypub deliver code 2023-03-11 09:42:14 +01:00
Johann150 2164fda2fb
server: do AP sent statistics in request function 2023-03-11 09:42:05 +01:00
syuilo af272ce358
fix(server): validate filename and emoji name to improve security
0d7256678e

Co-authored-by: Johann150 <johann.galle@protonmail.com>
Changelog: Fixed
2023-02-10 20:05:53 +01:00
Chloe Kudryavtsev bb3ec8bafe Revert "server: fix user deletion race condition"
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
This reverts commit cc83cbe523, reversing
changes made to 8abd3ebec7.

This changeset contains:
* multiple type errors
* a foreign key incompatibility
* breaks outgoing note federation (in at least two ways)
2023-01-30 14:59:24 +01:00
Johann150 b245d39b6e
server: delete records of fully deleted users
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-backend Pipeline failed
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-sw Pipeline failed
ci/woodpecker/pr/test Pipeline failed
2023-01-08 21:22:03 +01:00
Johann150 1eda1760d1
server: refactor to always use deleteAccount service
This should reduce code duplication around how deletion of an actor is
handled.
2023-01-07 19:46:05 +01:00
Johann150 0c8a3cfeec
server: fix lints
All checks were successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2023-01-03 03:51:38 +01:00
Johann150 9317d25078
server: expire notifications after 3 months
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-sw Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
closes #292

Changelog: Added
2022-12-21 21:46:45 +01:00
Johann150 fc36bb8880
server: reduce code duplication in check-expired queue job 2022-12-21 21:46:27 +01:00
Johann150 768d9bbdfb
refactor: remove default export for perform
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-12-11 18:23:19 +01:00
Johann150 3ef1a4b0f9
refactor: remove default export for Resolver 2022-12-11 18:23:07 +01:00
Johann150 14a9b9bedd
refactor: remove default export for request 2022-12-11 18:16:45 +01:00
Johann150 fde751df8f
fix: properly supply resolver (2)
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-12-08 19:06:55 +01:00
Norm c1a51547a9 BREAKING: server: remove wildcard blocking and instead block subdomains (#269)
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Co-authored-by: Francis Dinh <normandy@biribiri.dev>
Reviewed-on: #269
Changelog: Changed
2022-12-05 17:55:38 +00:00
Johann150 a421dd401c
activitypub: refactor to always apply recursion limit
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Refactor to remove as many "new Resolver" as possible.
2022-12-04 21:11:44 +01:00
Johann150 03b673165f
server: refactor "authUser" functions into separate file
They did not really fit into the DbResolver because they may fetch data
from remote instances even though DbResolver is only supposed to access
the database.
2022-12-04 21:11:35 +01:00
Johann150 418c88bb8f expire AuthSessions after 15 min 2022-12-03 10:38:32 +00:00
Norm 075e251822
server: add wildcard matching to blocked hosts
This adds in wildcard matching. For instance:
- `*.bad.tld` will match: `very.bad.tld`
- `bad.*` will match: `bad.something`
- `*.bad.*` will match: `very.bad.evil`

Changelog: Changed
2022-12-01 11:29:02 -05:00
Johann150 9e2553909e
server: use time constants
All checks were successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-11-20 23:15:40 +01:00
Johann150 5d23aa9e69
translate some comments to english
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-11-10 00:36:39 +01:00
Johann150 2aafe8fc9f
server: avoid adding suspended instances to deliver queue
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
This should reduce the performance hit when adding large numbers of
instances to the deliver queue by making the check for suspended and
dead instances a bulk operation.

Changelog: Changed
Reviewed-on: #215
2022-10-29 22:58:04 +02:00
Johann150 a0ef32f4f6
server: properly delete expired password reset requests
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Changelog: Fixed
2022-10-23 23:09:11 +02:00
Chloe Kudryavtsev d762143b89 backend: fixup missing deadTime and incorrect import
Some checks failed
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
2022-10-16 09:32:01 -04:00
Chloe Kudryavtsev 21c1e5c06c backend: simplify suspended and dead queries
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/test Pipeline failed
This should also have better latency due to being a single query.
Furthermore, it's no longer a linear scan, since host is indexed.
Would be cool to simplify it further to a single query for blocks also...
Why exactly are blocks not in the db?
2022-10-16 09:22:05 -04:00
Chloe Kudryavtsev 91a4f38871 backend: add automatic dead instance detection
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
It works by having a day-long cache of
"when did we last successfully communicate with this instance?"
Anything over a specified threshold (1 month) will act as though the instance
is suspended - all outgoing jobs are dropped on processing.
The day-long cache is in place because the ordering is necessarily a
linear scan.
Once an instance comes back online, we will detect that is the case as soon as
we receive an activity from them (which will update the "last communicated at")
field.

Potential future TODOs:
* Improve the caching system, it's actually pretty inefficient as it is.
  CacheBox with a call override?
* Think of ways to make it not-a-linear-scan, since the instances table can get
  pretty big. It's around 4500 on toast cafe.

ChangeLog: Added
2022-10-16 12:16:04 +00:00
Johann150 cb0b14ba2d
backend: allow to export only specific emoji 2022-10-12 01:49:53 +02:00
Johann150 5b7a06675f
refactor expiring data and expire signins after 60 days
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
closes #176

Changelog: Added
2022-10-02 00:18:07 +02:00
Johann150 36a0e48e49
bacakend: prefer absolute over relative imports
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
There are still many places where import paths with `..` are used and
maybe should use absolute paths also.
2022-10-01 14:40:30 +02:00
Johann150 7ceb96b148
limit id length of all incoming activities
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
2022-09-12 18:30:53 +02:00
Johann150 f0bdd9666f
remove empty import statements
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-14 12:26:18 +02:00
Johann150 09a7eabda1
backend: fix lint "no-throw-literal"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-04 11:00:02 +02:00
Johann150 c8f49bae76
fix lint "object-shorthand"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-03 14:58:24 +02:00
Johann150 37e47a257e
fix lints "import/order" and "import/no-duplicate"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
Also simplified some import paths by replacing relative with absolute paths.
2022-08-03 14:05:50 +02:00
Johann150 a6df127d3b
fix lint "quotes"
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-02 23:25:36 +02:00
yzhe819 9111d238c1
chore: fix lint errors (#8981) 2022-07-13 00:45:57 +02:00