AZ-400 ยท Maintaining pipelines
Monitoring pipeline health
Failure rate, duration and test pass-rate reports, pipeline analytics, flaky test detection and quarantine, GitHub Actions usage metrics and workflow run history.
Key points
- Azure Pipelines Analytics tab, from run history:
- Reports: pass rate (failure trend, top failing tasks), duration (slowest tasks), test failures.
- Default range: last 14 days, filterable by branch.
- Flaky tests (different outcomes, no code change): system detection (fails, passes on rerun) or custom detection via REST API.
- They can be kept out of the pass percentage.
- GitHub Actions: workflow run history, usage metrics.
Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/reports/pipelinereport?view=azure-devops
Practice questions
Which Azure Pipelines report helps you find the tasks that cause the most run failures?
- Pipeline duration report
- Pipeline pass rate report
- Pool consumption report
- Release retention report
Show answer
Answer: B. Pipeline pass rate report
The pipeline pass rate report lists top failing tasks with links to their failed runs. The duration report is about time, not failures, and the test failures report lists tests, not tasks.
Which statements about flaky test management in Azure Pipelines are true?
- System detection marks a test flaky when it fails and then passes on a rerun
- Custom detection reports flakiness through a REST API call
- Switching between detection systems erases the stored flakiness history
- A test marked flaky is deleted from the test plan automatically
- The feature is available only on on-premises Azure DevOps Server
Show answer
Answer: A. System detection marks a test flaky when it fails and then passes on a rerun, B. Custom detection reports flakiness through a REST API call, C. Switching between detection systems erases the stored flakiness history
System detection uses reruns, custom detection plugs in through the REST API, and switching between the two erases flakiness history. Flagged tests are not deleted, and the feature is a cloud service feature, not on-premises only.
When you mark a test as flaky during a run, the result of that same run is changed immediately.
Show answer
Answer: False
False. Marking a test flaky or unflaky does not change the current pipeline run; the new setting is evaluated on future executions of that test.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
