TEMP: memory limit test (branch will be deleted)
Some checks failed
Memory limit test (temporary branch) / limit-visible (push) Successful in 3s
Memory limit test (temporary branch) / plain-step-uses-2gb (push) Failing after 8s
Memory limit test (temporary branch) / kaniko-run-step-uses-2gb (push) Failing after 27s
Memory limit test (temporary branch) / kaniko-normal-build (push) Successful in 6s
Some checks failed
Memory limit test (temporary branch) / limit-visible (push) Successful in 3s
Memory limit test (temporary branch) / plain-step-uses-2gb (push) Failing after 8s
Memory limit test (temporary branch) / kaniko-run-step-uses-2gb (push) Failing after 27s
Memory limit test (temporary branch) / kaniko-normal-build (push) Successful in 6s
This commit is contained in:
parent
21e961d4f0
commit
8341c98e38
4 changed files with 59 additions and 1 deletions
33
.forgejo/workflows/memory-test.yml
Normal file
33
.forgejo/workflows/memory-test.yml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
name: Memory limit test (temporary branch)
|
||||
on:
|
||||
push:
|
||||
branches: [test/memory-limit]
|
||||
jobs:
|
||||
limit-visible:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: |
|
||||
echo "SCOTTY_GIT_JOB_MEMORY_LIMIT=$SCOTTY_GIT_JOB_MEMORY_LIMIT"
|
||||
echo "kernel memory limit for this job: $(cat /sys/fs/cgroup/memory.max 2>/dev/null || cat /sys/fs/cgroup/memory/memory.limit_in_bytes)"
|
||||
plain-step-uses-2gb:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: node -e 'const a=[]; for (let i=0;i<40;i++){ a.push(Buffer.alloc(50*1024*1024, 1)); console.log((i+1)*50, "MB held"); }'
|
||||
kaniko-run-step-uses-2gb:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: ./kaniko-build
|
||||
with:
|
||||
context: kaniko-build/test-memhog
|
||||
dockerfile: kaniko-build/test-memhog/Dockerfile
|
||||
push: "false"
|
||||
kaniko-normal-build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: ./kaniko-build
|
||||
with:
|
||||
context: kaniko-build/test
|
||||
dockerfile: kaniko-build/test/Dockerfile
|
||||
push: "false"
|
||||
12
README.md
12
README.md
|
|
@ -100,6 +100,18 @@ repository, never at organisation or instance level:
|
|||
tool update can never change your builds without a reviewed change here.
|
||||
- **Versions:** use `@v1` to get compatible fixes. Breaking changes get a new major tag.
|
||||
|
||||
### Runner limits (all workflows on scotty-git)
|
||||
|
||||
Every job on the shared scotty-git runner is limited to **1.5 GB of memory**, so that no workflow can
|
||||
take down the git server it runs next to. Each job can read the limit from the environment variable
|
||||
`SCOTTY_GIT_JOB_MEMORY_LIMIT`.
|
||||
|
||||
**If a step fails with `exit code 137`**, it was stopped because it used more than 1.5 GB. Common
|
||||
fixes: run tests with fewer parallel workers, avoid loading whole datasets into memory, or split the
|
||||
job. `kaniko-build` prints this explanation itself. For other steps the runner can only report the
|
||||
exit code, so this is the place to look it up. If your workload genuinely needs more, ask the
|
||||
scotty-git admins (every memory stop is logged on the runner: `journalctl -t ci-memory-limit`).
|
||||
|
||||
### Changing this action
|
||||
|
||||
Open a pull request. The `Self-test kaniko-build` workflow builds a small test image with your
|
||||
|
|
|
|||
|
|
@ -71,7 +71,17 @@ IFS=$OLDIFS
|
|||
if [ -n "$EXTRA_ARGS" ]; then set -- "$@" $EXTRA_ARGS; fi
|
||||
|
||||
echo "Building $DOCKERFILE (context: $CONTEXT)$( [ "$PUSH" = "true" ] && echo " -> $DESTS" || echo ", not pushing")"
|
||||
/kaniko/executor "$@"
|
||||
# Keep a copy of the output: when a RUN step (not Kaniko itself) is killed for memory,
|
||||
# Kaniko exits 1 and only its message says "exit status 137".
|
||||
{ /kaniko/executor "$@" 2>&1; echo "$?" > /kaniko/exit-code; } | tee /kaniko/build.log
|
||||
RC=$(cat /kaniko/exit-code 2>/dev/null || echo 1)
|
||||
if [ "$RC" -ne 0 ] && grep -q 'exit status 137' /kaniko/build.log; then RC=137; fi
|
||||
if [ "$RC" -eq 137 ]; then
|
||||
LIMIT="${SCOTTY_GIT_JOB_MEMORY_LIMIT:-1.5 GB}"
|
||||
fail "Build stopped: this job hit scotty-git's per-job memory limit ($LIMIT) and was killed (exit code 137). The limit protects the shared git server. Reduce the build's memory use (e.g. fewer parallel steps in the Dockerfile, smaller dependencies), or ask the scotty-git admins for a larger runner."
|
||||
elif [ "$RC" -ne 0 ]; then
|
||||
fail "Kaniko build failed (exit code $RC). See the Kaniko output above for the failing step."
|
||||
fi
|
||||
|
||||
# ---- Output ----
|
||||
if [ "$PUSH" = "true" ] && [ -f /kaniko/digest ]; then
|
||||
|
|
|
|||
3
kaniko-build/test-memhog/Dockerfile
Normal file
3
kaniko-build/test-memhog/Dockerfile
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# Test only: a RUN step that tries to hold 2 GB in memory (tail buffers a stream with no newlines).
|
||||
FROM alpine:3.20
|
||||
RUN head -c 2000m /dev/zero | tail > /dev/null
|
||||
Loading…
Add table
Reference in a new issue