implement separate web workers #252
No reviewers
Labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Blocks
#249 Massive Performance impact when delivery Queue starts working}
FoundKeyGang/FoundKey
Reference: FoundKeyGang/FoundKey#252
Loading…
Reference in a new issue
No description provided.
Delete branch "web-worker"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
There are now separate web and queue workers.
The configuration entry
clusterLimit
has been replaced byclusterLimits
which allows separate configuration of web and queue workers.@ -147,0 +143,4 @@
const modes = ['web', 'queue'];
const total = modes.reduce((acc, mode) => acc + clusterLimits[mode], 0);
if (total > os.cpus().length) {
bootLogger.error(`configuration error: cluster limits total (${total}) must not exceed number of cores (${os.cpus().length})`);
Is this check actually needed? Running more tasks/processes than you have cores is fine. Syscalls are blocking and the scheduler exists for a reason.
I would have a warning if either type of worker had > 1.5x cpulength workers (different performance characteristics).
Certainly not an error.
@ -147,0 +149,4 @@
const workers = new Array(total);
workers.fill('web', 0, clusterLimits.web);
workers.fill('queue', clusterLimits.web);
This construction is a little odd and seems to me to be prone to off-by-ones.
I would write it as this:
Your example code wouldn't work because
new Array(n)
generates an array withn
empty slots andmap
basically ignores those.I've thought about it a bit and it's fine.
Could be a TODO for cleanup later, but this is fine and works as expected.
I'm going to just remove the check for now and merge it to my instance, see how that goes.
Time for wanton deletes!