AkkomaGang/akkoma
AkkomaGang/akkoma
16
67
Fork 105
Code Issues 167 Pull requests 13 Projects 2 Releases 19 Packages Wiki Activity

Optimise CI runs #938

Merged
Oneric merged 5 commits from Oneric/akkoma:ci-tweaks into develop 2025-07-11 22:23:30 +00:00
Conversation 6 Commits 5 Files changed 6 +36 -144
Oneric commented 2025-06-07 21:22:01 +00:00
Owner
Copy link

Not sure if this actually works, will need to experiment

Cuts down a bit on CI CPU time by removing superfluous redunancy between linting and testing.

Cuts down potentially much more on latency since CI runs no longer need to wait for skipped pipelines to be scheduled before concluding. When many CI runs are started in parallel (e.g. multiple PRs being checked and the runs from those already merged) this can take quite a while since the two available execution slots are rotated among all concurrent runs. After spending ~15min on running its tests a PR CI run might then have needed to wait number_of_other_runs * really_executed_pipeline_time before its final doc etc pipelines are scheduled — ust for those final pipelines to be noops anyway

~~Not sure if this actually works, will need to experiment~~ Cuts down a bit on CI CPU time by removing superfluous redunancy between linting and testing. Cuts down potentially much more on latency since CI runs no longer need to wait for skipped pipelines to be scheduled before concluding. When many CI runs are started in parallel *(e.g. multiple PRs being checked and the runs from those already merged)* this can take quite a while since the two available execution slots are rotated among all concurrent runs. After spending ~15min on running its tests a PR CI run might then have needed to wait `number_of_other_runs * really_executed_pipeline_time` before its final `doc` etc pipelines are scheduled — ust for those final pipelines to be noops anyway
Oneric added 3 commits 2025-06-07 21:22:02 +00:00 
When include is used only the specified combinations
are executed and the full list above is ignored
ci: merge lint and test pipeline
Some checks failed
ci/woodpecker/pr/build-arm64 unknown status
Details
ci/woodpecker/pr/test/2 Pipeline failed
Details
ci/woodpecker/pr/build-amd64 unknown status
Details
ci/woodpecker/pr/test/1 Pipeline failed
Details
ci/woodpecker/pr/docs unknown status
Details
e9851c3300 
The lint pipeline spent ~7 minutes downloading and compiling
and only a few seconds actually checking the style.
The former is fully redundandt with what’s done during test anyway.
If only steps are conditional the whole CI workflow
will be held up waiting until a slot is available to start them
just to then not do anything at all
Oneric changed title from WIP: try to optimise CI runs to try to optimise CI runs 2025-06-07 21:22:19 +00:00
Oneric changed title from try to optimise CI runs to WIP: try to optimise CI runs 2025-06-07 21:23:09 +00:00
floatingghost commented 2025-06-07 21:24:44 +00:00
Owner
Copy link

honestly if you wanted to combine the steps you may as well always lint instead of the environment variable check - before like 1.17?? linting didn't require compilation so it was quicker to separate them

honestly if you wanted to combine the steps you may as well always lint instead of the environment variable check - before like 1.17?? linting didn't require compilation so it was quicker to separate them
Oneric commented 2025-06-07 21:25:20 +00:00
Author
Owner
Copy link

The issue is, that different elixir versions will disagree on what is "correctly" formatted

The issue is, that different elixir versions will disagree on what is "correctly" formatted
Oneric changed title from WIP: try to optimise CI runs to try to optimise CI runs 2025-06-07 21:25:47 +00:00
floatingghost commented 2025-06-07 21:25:59 +00:00
Owner
Copy link

ah yeah I forgot about that one

fun

ah yeah I forgot about that one fun
Oneric force-pushed ci-tweaks from ea6484abdf to 74667e6dfb 2025-06-07 21:26:44 +00:00 Compare
Oneric force-pushed ci-tweaks from 74667e6dfb to e9851c3300 2025-06-07 21:27:05 +00:00 Compare
floatingghost reviewed 2025-06-07 21:27:28 +00:00
.woodpecker/test.yml Outdated
@ -6,4 +4,2 @@
matrix:
# test the lowest and highest versions
ELIXIR_VERSION:
floatingghost commented 2025-06-07 21:27:28 +00:00
Owner
Copy link

not sure you can remove this though? the matrix is generated from it and filtered by the include

not sure you can remove this though? the matrix is generated from it and filtered by the `include`
Oneric commented 2025-06-07 21:31:36 +00:00
Author
Owner
Copy link

yep, the docs use include without anything else and it works fine here in testing: https://woodpecker-ci.org/docs/usage/matrix-workflows

yep, the docs use `include` without anything else and it works fine here in testing: https://woodpecker-ci.org/docs/usage/matrix-workflows
Oneric marked this conversation as resolved
Oneric commented 2025-06-07 21:28:58 +00:00
Author
Owner
Copy link

Ideally I’d like to separate lint into a different step to make it more visible whether it fails in tests or during linting but I’m not sure how to check the value of a custom matrix variable in a step when condition

Ideally I’d like to separate lint into a different step to make it more visible whether it fails in tests or during linting but I’m not sure how to check the value of a custom matrix variable in a step `when` condition
Oneric added 1 commit 2025-06-07 21:33:07 +00:00
ci: make pipelines conditional
All checks were successful
ci/woodpecker/pr/test/2 Pipeline was successful
Details
ci/woodpecker/pr/test/1 Pipeline was successful
Details
0f0a66e688 
If only steps are conditional the whole CI workflow
will be held up waiting until a slot is available to start them
just to then not do anything at all
floatingghost commented 2025-06-07 21:35:30 +00:00
Owner
Copy link

I’m not sure how to check the value of a custom matrix variable in a step when condition

I don't think you can, but there is a really stinky workaround that could work, you could wrap lint steps in a script/bash -c step that exits if the env vars aren't what you expect

extremely stinky but would be functional

>I’m not sure how to check the value of a custom matrix variable in a step when condition I don't think you can, but there is a really stinky workaround that could work, you _could_ wrap lint steps in a script/bash -c step that exits if the env vars aren't what you expect extremely stinky but would be functional
Oneric force-pushed ci-tweaks from 0f0a66e688 to cfb967ed24 2025-06-07 21:38:00 +00:00 Compare
Oneric added 2 commits 2025-06-07 21:45:43 +00:00
ci: drop variables from files not using them at all
Some checks are pending
ci/woodpecker/pr/test/1 Pipeline is pending approval
Details
ci/woodpecker/pr/test/2 Pipeline is pending approval
Details
6c8ae2a4a1
Oneric force-pushed ci-tweaks from 6c8ae2a4a1 to d1612e2bb8 2025-06-07 21:46:52 +00:00 Compare
Oneric commented 2025-06-07 21:51:06 +00:00
Author
Owner
Copy link

ok, top-level when conditions work too, but they can't use &on-release variables, but on the other hand with the condition being at the top-level there’s less duplication and thus less need for variables to begin with

I don't think you can, but there is a really stinky workaround that could work, you could wrap lint steps in a script/bash -c step that exits if the env vars aren't what you expect

yeee, don't feel like doing that atm

but it seems to work fine from what i can test here in the PR. If it looks good to you this should help to cut down on CI latency

ok, top-level `when` conditions work too, but they can't use `&on-release` variables, but on the other hand with the condition being at the top-level there’s less duplication and thus less need for variables to begin with > I don't think you can, but there is a really stinky workaround that could work, you could wrap lint steps in a script/bash -c step that exits if the env vars aren't what you expect yeee, don't feel like doing that atm but it seems to work fine from what i can test here in the PR. If it looks good to you this should help to cut down on CI latency
Oneric changed title from try to optimise CI runs to Optimise CI runs 2025-06-07 21:53:06 +00:00
requested review from floatingghost 2025-06-07 22:11:08 +00:00
Oneric force-pushed ci-tweaks from d1612e2bb8 to ea5a2a9f21 2025-06-22 15:37:11 +00:00 Compare
floatingghost approved these changes 2025-07-11 19:09:45 +00:00
floatingghost left a comment
Owner
Copy link

i think this is okie

i think this is okie
Oneric merged commit e2987402b3 into develop 2025-07-11 22:23:30 +00:00
Oneric deleted branch ci-tweaks 2025-07-11 22:23:30 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
floatingghost
Labels
Clear labels   
approved, awaiting change

   
bug

Something is not working

  
configuration

This requires config changes

  
documentation

This is about human-readable docs

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
extremely low priority

   
feature request

Something new?

  
Fix it yourself

Support has been attempted

  
help wanted

Need some help

  
invalid

Something is wrong

  
mastodon_api

   
needs change/feedback

Issue or PR is stalled on response from the author or a third party

  
needs docs

   
needs tests

   
not a bug

   
planned

   
pleroma_api

   
privacy

   
question

More information is needed

  
static_fe

   
triage

   
wontfix

This won't be fixed

No labels
approved, awaiting change
bug
configuration
documentation
duplicate
enhancement
extremely low priority
feature request
Fix it yourself
help wanted
invalid
mastodon_api
needs change/feedback
needs docs
needs tests
not a bug
planned
pleroma_api
privacy
question
static_fe
triage
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#938
No description provided.
Delete branch "Oneric/akkoma:ci-tweaks"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?