Chart Datapoint creation slows down instance significantly

puniko commented

2022-11-17 07:36:43 +00:00

Contributor

I noticed that the creation of the datapoints for the charts slows down the instance significantly. It tents to slow down the worker queues to a near halt and posting takes sometimes up to minutes to go through. it was especially noticable during my database maintanace schenanigans the last 2 weeks.

so i think, chart creation may need optimization in some way

I noticed that the creation of the datapoints for the charts slows down the instance significantly. It tents to slow down the worker queues to a near halt and posting takes sometimes up to minutes to go through. it was especially noticable during my database maintanace schenanigans the last 2 weeks. so i think, chart creation may need optimization in some way

puniko commented

2022-11-17 14:52:53 +00:00

Author

Contributor

ok, i commented a couple of stuff out in https://akkoma.dev/FoundKeyGang/FoundKey/src/branch/main/packages/backend/src/services/chart/index.ts just for testing around a bit.

it indeed has a huge performance difference on my instance it seems

const charts = [
        federationChart,
        // notesChart,
        usersChart,
        activeUsersChart,
        instanceChart,
        // perUserNotesChart,
        // driveChart,
        // perUserReactionsChart,
        hashtagChart,
        // perUserFollowingChart,
        // perUserDriveChart,
        apRequestChart,
];

ok, i commented a couple of stuff out in https://akkoma.dev/FoundKeyGang/FoundKey/src/branch/main/packages/backend/src/services/chart/index.ts just for testing around a bit. it indeed has a huge performance difference on my instance it seems ``` const charts = [ federationChart, // notesChart, usersChart, activeUsersChart, instanceChart, // perUserNotesChart, // driveChart, // perUserReactionsChart, hashtagChart, // perUserFollowingChart, // perUserDriveChart, apRequestChart, ]; ```

Johann150 commented

2022-11-17 18:29:13 +00:00

Owner

I think there could well be many improvements to be had in the chart engine. But seeing that the core part of it alone (packages/backend/src/services/chart/core.ts) stands at 673 lines I'm considering if it would be easier to redo/redesign the whole system.

I think there could well be many improvements to be had in the chart engine. But seeing that the core part of it alone (`packages/backend/src/services/chart/core.ts`) stands at 673 lines I'm considering if it would be easier to redo/redesign the whole system.

👍 1

Johann150 commented

2022-11-25 15:06:15 +00:00

Owner

Already discovered that there is a setInterval which saves chart data that runs every 20min as well as the tick-charts background job that runs every hour. The tick-charts background job performance hit would be reduced by #252 but that seems to be the smaller part of the problem, if it is a part of the problem at all.

Already discovered that there is [a `setInterval` which `save`s chart data](https://akkoma.dev/FoundKeyGang/FoundKey/src/commit/c7255dbea09b0d4725d0219e55f19f084e30cac7/packages/backend/src/services/chart/index.ts#L46-L50) that runs every 20min as well as the `tick-charts` background job that runs every hour. The `tick-charts` background job performance hit would be reduced by #252 but that seems to be the smaller part of the problem, if it is a part of the problem at all.

👍 1

toast commented

2022-11-26 12:18:23 +00:00

Owner

for (const chart of charts) {
	chart.save();
}

This explains so much - namely why the performance hit is so obvious even I noticed it on a testing instance.
That does at least one postgres request for every chart individually.
These should be queued, it should check for changes, and it should batched.

Opened an issue to track (#253).

```js for (const chart of charts) { chart.save(); } ``` This explains so much - namely why the performance hit is so obvious even I noticed it on a testing instance. That does at least one postgres request for every chart individually. These should be queued, it should check for changes, and it should batched. Opened an issue to track (#253).

Johann150 referenced this issue

2022-12-03 13:29:59 +00:00

Massive Performance impact when delivery Queue starts working #249

Chart Datapoint creation slows down instance significantly #237