Commit graph

78 commits

Author SHA1 Message Date
Johann150 fde751df8f
fix: properly supply resolver (2)
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-12-08 19:06:55 +01:00
Norm c1a51547a9 BREAKING: server: remove wildcard blocking and instead block subdomains (#269)
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Co-authored-by: Francis Dinh <normandy@biribiri.dev>
Reviewed-on: #269
Changelog: Changed
2022-12-05 17:55:38 +00:00
Johann150 a421dd401c
activitypub: refactor to always apply recursion limit
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Refactor to remove as many "new Resolver" as possible.
2022-12-04 21:11:44 +01:00
Johann150 03b673165f
server: refactor "authUser" functions into separate file
They did not really fit into the DbResolver because they may fetch data
from remote instances even though DbResolver is only supposed to access
the database.
2022-12-04 21:11:35 +01:00
Johann150 418c88bb8f expire AuthSessions after 15 min 2022-12-03 10:38:32 +00:00
Norm 075e251822
server: add wildcard matching to blocked hosts
This adds in wildcard matching. For instance:
- `*.bad.tld` will match: `very.bad.tld`
- `bad.*` will match: `bad.something`
- `*.bad.*` will match: `very.bad.evil`

Changelog: Changed
2022-12-01 11:29:02 -05:00
Johann150 9e2553909e
server: use time constants
All checks were successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-11-20 23:15:40 +01:00
Johann150 5d23aa9e69
translate some comments to english
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-11-10 00:36:39 +01:00
Johann150 2aafe8fc9f
server: avoid adding suspended instances to deliver queue
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
This should reduce the performance hit when adding large numbers of
instances to the deliver queue by making the check for suspended and
dead instances a bulk operation.

Changelog: Changed
Reviewed-on: #215
2022-10-29 22:58:04 +02:00
Johann150 a0ef32f4f6
server: properly delete expired password reset requests
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Changelog: Fixed
2022-10-23 23:09:11 +02:00
Chloe Kudryavtsev d762143b89 backend: fixup missing deadTime and incorrect import
Some checks failed
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
2022-10-16 09:32:01 -04:00
Chloe Kudryavtsev 21c1e5c06c backend: simplify suspended and dead queries
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/test Pipeline failed
This should also have better latency due to being a single query.
Furthermore, it's no longer a linear scan, since host is indexed.
Would be cool to simplify it further to a single query for blocks also...
Why exactly are blocks not in the db?
2022-10-16 09:22:05 -04:00
Chloe Kudryavtsev 91a4f38871 backend: add automatic dead instance detection
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
It works by having a day-long cache of
"when did we last successfully communicate with this instance?"
Anything over a specified threshold (1 month) will act as though the instance
is suspended - all outgoing jobs are dropped on processing.
The day-long cache is in place because the ordering is necessarily a
linear scan.
Once an instance comes back online, we will detect that is the case as soon as
we receive an activity from them (which will update the "last communicated at")
field.

Potential future TODOs:
* Improve the caching system, it's actually pretty inefficient as it is.
  CacheBox with a call override?
* Think of ways to make it not-a-linear-scan, since the instances table can get
  pretty big. It's around 4500 on toast cafe.

ChangeLog: Added
2022-10-16 12:16:04 +00:00
Johann150 cb0b14ba2d
backend: allow to export only specific emoji 2022-10-12 01:49:53 +02:00
Norm 8ccc22bb35 Merge pull request 'refactor expiring data and expire signins after 60 days' (#180) from expire-data into main
All checks were successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Reviewed-on: #180
2022-10-03 18:27:13 +00:00
Johann150 a7f9e244f3
merge: allow redis family to be specified as a string
All checks were successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
Reviewed-on: #165
2022-10-02 18:46:34 +02:00
Johann150 5b7a06675f
refactor expiring data and expire signins after 60 days
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
closes #176

Changelog: Added
2022-10-02 00:18:07 +02:00
Johann150 36a0e48e49
bacakend: prefer absolute over relative imports
All checks were successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
There are still many places where import paths with `..` are used and
maybe should use absolute paths also.
2022-10-01 14:40:30 +02:00
Norm 508748ac0d
backend: refactor Redis option loading 2022-09-30 11:05:49 -04:00
Norm 1b92f580cb
backend: fix imports in queue/types.ts
All checks were successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
2022-09-21 13:18:17 -04:00
Johann150 7ceb96b148
limit id length of all incoming activities
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-foundkey-js Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/lint-client Pipeline failed
ci/woodpecker/pr/lint-foundkey-js Pipeline was successful
ci/woodpecker/pr/lint-backend Pipeline was successful
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
2022-09-12 18:30:53 +02:00
Johann150 f0bdd9666f
remove empty import statements
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-14 12:26:18 +02:00
Johann150 6ce4b3fe2f
fix some lints
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
Many of these were fixed automatically with eslint --fix.

Some of them (e.g. adding return types to functions) were done manually.
2022-08-11 00:09:29 +02:00
Johann150 09a7eabda1
backend: fix lint "no-throw-literal"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-04 11:00:02 +02:00
Johann150 c8f49bae76
fix lint "object-shorthand"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-03 14:58:24 +02:00
Johann150 37e47a257e
fix lints "import/order" and "import/no-duplicate"
Some checks failed
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
Also simplified some import paths by replacing relative with absolute paths.
2022-08-03 14:05:50 +02:00
Johann150 a6df127d3b
fix lint "quotes"
Some checks failed
ci/woodpecker/push/build Pipeline was successful
ci/woodpecker/push/lint-backend Pipeline was successful
ci/woodpecker/push/lint-client Pipeline failed
ci/woodpecker/push/test Pipeline failed
2022-08-02 23:25:36 +02:00
Johann150 0e3a9d1e0d
refactor: use nullish coalescing / optional chaining
This seems to be more readable than ternary expressions.
2022-07-13 19:08:22 +02:00
MeiMei d4ac71593b
enhance: use ioredis everywhere, fix IPv6 support for redis
fixes https://github.com/misskey-dev/misskey/pull/8869
see also https://github.com/misskey-dev/misskey/pull/8869

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2022-07-13 10:40:39 +02:00
yzhe819 9111d238c1
chore: fix lint errors (#8981) 2022-07-13 00:45:57 +02:00
Johann150 568f14070d
remove unused logger 2022-07-12 14:44:19 +02:00
Johann150 ba0236700d
chore: remove unused imports 2022-07-12 14:41:10 +02:00
Johann150 3317e21b39
chore: remove unused imports 2022-07-11 15:45:12 +02:00
Johann150 c56e45ecef
fix: always remove completed tasks (#8771) 2022-05-31 17:54:02 +09:00
Johann150 ebc2566130
fix: add missing import
fix #8756
2022-05-29 14:33:42 +02:00
Johann150 9c80403072
use http-signature module that supports hs2019 (#8635) 2022-05-26 09:12:17 +09:00
Johann150 e27c6abaea
refactor: temporary files (#8713)
* simplify temporary files for thumbnails

Because only a single file will be written to the directory, creating a
separate directory seems unnecessary. If only a temporary file is created,
the code from `createTemp` can be reused here as well.

* refactor: deduplicate code for temporary files/directories

To follow the DRY principle, the same code should not be duplicated
across different files. Instead an already existing function is used.

Because temporary directories are also create in multiple locations,
a function for this is also newly added to reduce duplication.

* fix: clean up identicon temp files

The temporary files for identicons are not reused and can be deleted
after they are fully read. This condition is met when the stream is closed
and so the file can be cleaned up using the events API of the stream.

* fix: ensure cleanup is called when download fails

* fix: ensure cleanup is called in error conditions

This covers import/export queue jobs and is mostly just wrapping all
code in a try...finally statement where the finally runs the cleanup.

* fix: use correct type instead of `any`
2022-05-25 16:50:22 +09:00
syuilo ec3be7e4d3 enhance(webhook): add userId to payload 2022-04-03 22:42:01 +09:00
syuilo f8e6f3cc73 improve webhook 2022-04-03 22:36:30 +09:00
syuilo 8e5f2690f2
feat: Webhook (#8457)
* feat: introduce webhook

* wip

* wip

* wip

* Update CHANGELOG.md
2022-04-02 15:28:49 +09:00
syuilo 1c67c26bd8
refactor: migrate to typeorm 3.0 (#8443)
* wip

* wip

* wip

* Update following.ts

* wip

* wip

* wip

* Update resolve-user.ts

* maxQueryExecutionTime

* wip

* wip
2022-03-26 15:34:00 +09:00
syuilo ac8c66f5ab perf(server): refactor and performance improvements 2022-03-25 16:27:41 +09:00
syuilo 544b5ba838
perf(server): reduce memory usage of redis 2022-03-19 19:34:04 +09:00
syuilo caadc0978a feat: introduce bull dashboard 2022-03-19 19:08:55 +09:00
syuilo 2442592ef1 feat: アンケート終了通知
Resolve #4664
2022-03-06 16:06:27 +09:00
syuilo e68278f93e feat: 時限ミュート
#7677
2022-03-04 20:23:53 +09:00
syuilo c8070fc544 fix esm 2022-03-01 01:36:50 +09:00
syuilo 2e89b21544 fix esm 2022-03-01 01:34:53 +09:00
syuilo d071d18dd7
refactor: Use ESM (#8358)
* wip

* wip

* fix

* clean up

* Update tsconfig.json

* Update activitypub.ts

* wip
2022-02-27 11:07:39 +09:00
syuilo b6db709e02 refacator: fix typw 2022-02-19 14:53:33 +09:00