Explore optimizing CI tasks

norm commented

2022-09-07 18:12:18 +00:00

Owner

Currently the Woodpecker CI config is spread across 5 different pipelines, each having to run yarn to install dependencies. This causes unnecessary strain on the CI server, and might be avoidable if we can reorganize the tasks in each pipeline differently.

norm commented

2022-09-07 18:14:00 +00:00

Author

Owner

I think at the very least the lints could be consolidated into one pipeline.

norm referenced this issue

2022-09-07 21:50:50 +00:00

ci: combine lint tasks #126

Johann150 commented

2022-09-08 20:39:05 +00:00

Owner

I think at the very least the lints could be consolidated into one pipeline.

Sure they could, but it would not really help the point of separating them in the first place. Reality is that lints of the backend are passing but lints of the client are not. In a PR I don't want to see that the lint failed and have to look which of the lints failed, and if it was one that was failing before anyway. (Ideally we could easily see when there are any newly failing lints but thats on a different level.)

Quoting what I said in the PR:

Even if the other lints are run on failure of previous lints, it would be nice to have separate indicators for each of the lints. Because seeing which of the lints failed was the reason to separate them in the first place.

> I think at the very least the lints could be consolidated into one pipeline. Sure they could, but it would not really help the point of separating them in the first place. Reality is that lints of the backend are passing but lints of the client are not. In a PR I don't want to see that the lint failed and have to look *which* of the lints failed, and if it was one that was failing before anyway. (Ideally we could easily see when there are *any newly failing lints* but thats on a different level.) Quoting what I said in the PR: > Even if the other lints are run on failure of previous lints, it would be nice to have separate indicators for each of the lints. Because seeing which of the lints failed was the reason to separate them in the first place.

norm commented

2022-09-08 20:43:27 +00:00

Author

Owner

I think Toast's idea of using a cache for node_modules and yarn might be the best approach for both not having to do a full yarn install and being able to keep the lints separate.

All we'd really need now is a place to store that cache.

I think [Toast's idea of using a cache for `node_modules` and yarn](https://akkoma.dev/FoundKeyGang/FoundKey/pulls/126#issuecomment-3038) might be the best approach for both not having to do a full `yarn install` and being able to keep the lints separate. All we'd really need now is a place to store that cache.

norm commented

2022-09-08 20:48:09 +00:00

Author

Owner

As for a "forking" workflow, there is the group parallel execution for tasks within the same pipeline (which would aleviate the need for the cache).

Alternatively, if we do want separate lint pipelines, there's depends_on where we could have a single yarn install pipeline that can also refresh the cache, and then load the cache in the other pipelines.

As for a "forking" workflow, there is the [`group` parallel execution for tasks within the same pipeline](https://woodpecker-ci.org/docs/usage/pipeline-syntax#group---parallel-execution) (which would aleviate the need for the cache). Alternatively, if we do want separate lint pipelines, there's [`depends_on`](https://woodpecker-ci.org/docs/usage/multi-pipeline#flow-control) where we could have a single `yarn install` pipeline that can also refresh the cache, and then load the cache in the other pipelines.

norm commented

2022-09-08 20:49:02 +00:00

Author

Owner

Probably may be best to have a cache anyways since the CI would go faster regardless of the direction taken.

norm referenced this issue

2022-09-08 20:51:16 +00:00

ci: combine lint tasks #126

Explore optimizing CI tasks #123