Bitzu

AZ-400 ยท Testing strategy for pipelines

Tests and code coverage in pipelines

VSTest/dotnet test, test tasks and PublishTestResults, publishing coverage, test agents and parallel test execution, results in Azure Pipelines and GitHub Actions, coverage thresholds.

Key points

  • Run tests: DotNetCoreCLI@2 (command: test), VSTest task or dotnet test --logger trx.
  • .NET Core and VSTest tasks publish results; scripts need PublishTestResults@2 (VSTest format = TRX), condition succeededOrFailed().
  • Coverage: Windows dotnet test --collect "Code Coverage"; Linux or macOS coverlet (XPlat Code Coverage).
  • PublishCodeCoverageResults@2: Cobertura or JaCoCo.
  • Parallel: strategy: parallel (up to 99 agents), VSTest slicing.

Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/dotnet-core?view=azure-devops

Practice questions

Single choice

In PublishTestResults@2, which testResultsFormat value is used for Visual Studio TRX files?

  1. JUnit
  2. NUnit
  3. VSTest
  4. CTest
Show answer

Answer: C. VSTest

The value is VSTest for historical reasons; it means TRX and also works for TRX produced by other runners. JUnit, NUnit and CTest are different XML formats.

Select all that apply

Which statements about running tests in parallel with the VSTest task are correct?

  1. A YAML job can use strategy: parallel to spread tests across up to 99 agents.
  2. Slicing can be based on the past running time of tests.
  3. Slicing by assemblies suits tests that use AssemblyInitialize or ClassInitialize state.
  4. Slicing by number of tests balances agents well even when run times differ greatly.
  5. Parallel test jobs run without any parallel job capacity.
Show answer

Answer: A. A YAML job can use strategy: parallel to spread tests across up to 99 agents., B. Slicing can be based on the past running time of tests., C. Slicing by assemblies suits tests that use AssemblyInitialize or ClassInitialize state.

Parallelism comes from strategy: parallel (up to 99 agents). Slicing can use past running times, and assembly-based slicing suits tests that rely on assembly or class initialization. Simple count-based slicing does not balance uneven run times, and parallel jobs still need available parallel job capacity.

True or false

True or false: by default, PublishTestResults@2 fails the task when the results file contains failed tests.

Show answer

Answer: False

False. failTaskOnFailedTests defaults to false, so the task just publishes results. Set it to true if the task itself should fail.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AZ-400 topics in this module