They did not really fit into the DbResolver because they may fetch data
from remote instances even though DbResolver is only supposed to access
the database.
This adds in wildcard matching. For instance:
- `*.bad.tld` will match: `very.bad.tld`
- `bad.*` will match: `bad.something`
- `*.bad.*` will match: `very.bad.evil`
Changelog: Changed
This should reduce the performance hit when adding large numbers of
instances to the deliver queue by making the check for suspended and
dead instances a bulk operation.
Changelog: Changed
Reviewed-on: FoundKeyGang/FoundKey#215
This should also have better latency due to being a single query.
Furthermore, it's no longer a linear scan, since host is indexed.
Would be cool to simplify it further to a single query for blocks also...
Why exactly are blocks not in the db?
It works by having a day-long cache of
"when did we last successfully communicate with this instance?"
Anything over a specified threshold (1 month) will act as though the instance
is suspended - all outgoing jobs are dropped on processing.
The day-long cache is in place because the ordering is necessarily a
linear scan.
Once an instance comes back online, we will detect that is the case as soon as
we receive an activity from them (which will update the "last communicated at")
field.
Potential future TODOs:
* Improve the caching system, it's actually pretty inefficient as it is.
CacheBox with a call override?
* Think of ways to make it not-a-linear-scan, since the instances table can get
pretty big. It's around 4500 on toast cafe.
ChangeLog: Added
* simplify temporary files for thumbnails
Because only a single file will be written to the directory, creating a
separate directory seems unnecessary. If only a temporary file is created,
the code from `createTemp` can be reused here as well.
* refactor: deduplicate code for temporary files/directories
To follow the DRY principle, the same code should not be duplicated
across different files. Instead an already existing function is used.
Because temporary directories are also create in multiple locations,
a function for this is also newly added to reduce duplication.
* fix: clean up identicon temp files
The temporary files for identicons are not reused and can be deleted
after they are fully read. This condition is met when the stream is closed
and so the file can be cleaned up using the events API of the stream.
* fix: ensure cleanup is called when download fails
* fix: ensure cleanup is called in error conditions
This covers import/export queue jobs and is mostly just wrapping all
code in a try...finally statement where the finally runs the cleanup.
* fix: use correct type instead of `any`