Bitzu

AZ-400 ยท Maintaining pipelines

Monitoring pipeline health

Failure rate, duration and test pass-rate reports, pipeline analytics, flaky test detection and quarantine, GitHub Actions usage metrics and workflow run history.

Key points

  • Azure Pipelines Analytics tab, from run history:
  • Reports: pass rate (failure trend, top failing tasks), duration (slowest tasks), test failures.
  • Default range: last 14 days, filterable by branch.
  • Flaky tests (different outcomes, no code change): system detection (fails, passes on rerun) or custom detection via REST API.
  • They can be kept out of the pass percentage.
  • GitHub Actions: workflow run history, usage metrics.

Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/reports/pipelinereport?view=azure-devops

Practice questions

Single choice

Which Azure Pipelines report helps you find the tasks that cause the most run failures?

  1. Pipeline duration report
  2. Pipeline pass rate report
  3. Pool consumption report
  4. Release retention report
Show answer

Answer: B. Pipeline pass rate report

The pipeline pass rate report lists top failing tasks with links to their failed runs. The duration report is about time, not failures, and the test failures report lists tests, not tasks.

Select all that apply

Which statements about flaky test management in Azure Pipelines are true?

  1. System detection marks a test flaky when it fails and then passes on a rerun
  2. Custom detection reports flakiness through a REST API call
  3. Switching between detection systems erases the stored flakiness history
  4. A test marked flaky is deleted from the test plan automatically
  5. The feature is available only on on-premises Azure DevOps Server
Show answer

Answer: A. System detection marks a test flaky when it fails and then passes on a rerun, B. Custom detection reports flakiness through a REST API call, C. Switching between detection systems erases the stored flakiness history

System detection uses reruns, custom detection plugs in through the REST API, and switching between the two erases flakiness history. Flagged tests are not deleted, and the feature is a cloud service feature, not on-premises only.

True or false

When you mark a test as flaky during a run, the result of that same run is changed immediately.

Show answer

Answer: False

False. Marking a test flaky or unflaky does not change the current pipeline run; the new setting is evaluated on future executions of that test.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AZ-400 topics in this module