How to assess · For hiring teams
How to Assess CI/CD Skills When Hiring
The test formats that actually work for CI/CD, what a strong answer looks like, sample questions and a scoring rubric you can use as-is.
The short answer
Assess CI/CD with a task, not a conversation: review and fix a pipeline, design a pipeline from requirements, ai-scored assessment (e.g. cohesyve) or small practical. Score it against written criteria you fix before you see any submissions, and weight the criteria that the role actually depends on.
- Designs pipelines so that the fast, cheap checks run first and the expensive ones only when they matter
- Makes builds reproducible and cacheable, and knows where the time goes
- Treats flaky jobs as bugs to be fixed, not retried
- Separates build from deploy, and makes every deploy reversible
Paste a job description; Cohesyve generates a role-specific assessment and rubric. Ten candidates free, no card.
A CI/CD pipeline is the most-run piece of software most teams own, and the least tested. It is also where a lot of engineering time quietly goes: waiting for a forty-minute build, rerunning a flaky job, working around a deploy step nobody understands. The skill you are hiring for is not writing YAML. It is designing a delivery process that is fast, safe, reversible and comprehensible to the people who depend on it. This page covers how to assess CI/CD for DevOps, platform and senior engineering roles: pipeline design, speed, safety, secrets, and the judgement to know what belongs where.
Why CI/CD is worth testing
Pipeline problems compound. A slow pipeline means fewer, larger deploys; larger deploys mean riskier releases; riskier releases mean more manual gates; manual gates make the pipeline slower. A strong practitioner breaks that cycle; a weak one adds to it. Testing shows which you are hiring, and the difference is measured in how often the team can ship.
What strong CI/CD looks like
- Designs pipelines so that the fast, cheap checks run first and the expensive ones only when they matter
- Makes builds reproducible and cacheable, and knows where the time goes
- Treats flaky jobs as bugs to be fixed, not retried
- Separates build from deploy, and makes every deploy reversible
- Handles secrets through a manager with scoped access, never in the pipeline definition or logs
- Uses environments, approvals and progressive delivery deliberately rather than by default
- Writes pipeline configuration that a new team member can read and change
Ways to assess CI/CD
Review and fix a pipeline
Provide a pipeline definition with tests running after deploy, no caching, a secret in plain text, a flaky job with retries, and a deploy with no rollback. Ask what they would fix, in what order, and why.
Pros
Cons
Best for Mid and senior roles owning delivery.
Design a pipeline from requirements
Describe a service, its test suite, its environments and its risk tolerance. Ask for a pipeline design and a deployment strategy.
Pros
Cons
Best for Senior engineers and platform leads.
AI-scored assessment (e.g. Cohesyve)
Generate a CI/CD scenario from the job description — a pipeline review, a speed-up plan, a rollout design — with a rubric. Each candidate receives a different variant; reasoning is scored in writing.
Pros
Cons
Best for Screening a pool before interviews.
Small practical
Give a repository and ask for a working pipeline that builds, tests and produces a deployable artefact, with caching. Ninety minutes capped.
Pros
Cons
Best for Mid-level implementers.
Cohesyve
Run a CI/CD assessment on your next opening
Cohesyve generates a unique CI/CD task per candidate from your job description, with the scoring rubric attached. Questions are different for every applicant, so they cannot be shared or looked up.
What to test
Pipeline design
Whether stages are ordered and gated well.
Speed
Whether they can make a pipeline fast without losing signal.
Safety and rollback
Whether deploys are reversible.
Secrets and access
Whether the pipeline is a secure boundary.
Sample CI/CD questions
A pipeline runs tests after deploying. What is wrong?
EntryLook for Broken code reaches the environment before it is checked; tests should gate the deploy.
A job fails one time in eight and the team retries it. What do you do?
EntryLook for Find the cause — shared state, timing, an external dependency — and fix it; retries hide real failures.
Your pipeline takes forty minutes. Where do you start?
MidLook for Measure per stage, cache dependencies and builds, parallelise, run the slowest checks less often, and keep the fast feedback loop under ten minutes.
Design a deploy for a change you are not confident in.
MidLook for Canary or small percentage rollout, metrics watched, automated rollback, feature flag if available.
How would you restructure delivery for twenty services with one shared pipeline that everyone edits?
SeniorLook for Reusable templates or modules, ownership per service, versioned shared steps, and a way to change the shared piece without breaking everyone.
Red flags
- Cannot say how long their last pipeline took
- Fixes flakiness with retries
- Secrets in pipeline files or logs
- No rollback path
- Every check runs on every commit regardless of cost
Scoring rubric
| Criterion | Weight | What strong looks like |
|---|---|---|
| Design and ordering | 25% | Cheap checks first, gates where they matter, build separate from deploy. |
| Speed | 25% | Measured, cached, parallel, fast feedback. |
| Safety | 25% | Reversible deploys with observed rollouts. |
| Security | 15% | Secrets scoped and out of logs; access controlled. |
| Maintainability | 10% | Readable, reusable, owned. |
Mistakes hiring teams make
- Testing YAML syntax for a specific tool
- Not asking about speed — it is what the team feels every day
- Skipping rollback
- Accepting a pipeline with plaintext secrets because it works
- Confusing "has used GitHub Actions" with delivery design skill
Roles that need CI/CD
Common questions
Should I test a specific CI tool?
Only if the role is locked to one. Design judgement transfers between GitHub Actions, GitLab, Jenkins and the rest; syntax is learnable. Tell candidates to ignore syntax and focus on design.
What is the best single CI/CD question?
Give a slow, flaky pipeline definition and ask what they would fix first. The ordering tells you how they think about delivery.
How much CI/CD should a backend developer know?
Enough to read the pipeline, understand why a check failed, and know how their change is deployed and rolled back. Pipeline design is a platform skill.
How long should a CI/CD assessment take?
Thirty to forty-five minutes for a review exercise; ninety minutes capped for a practical.
Cohesyve · Skill assessments for hiring
Test CI/CD before the first interview
Generate a role-specific CI/CD assessment from your job description and see who can do the work before you spend interview time on them.
1,500+
assessments completed
50%
faster time-to-hire
90%
completion rate
5 min
from JD to assessment
No credit card · 10 free candidates · Plans sized to your hiring volume
From the blog