AZ-400 ยท Testing strategy for pipelines
Tests and code coverage in pipelines
VSTest/dotnet test, test tasks and PublishTestResults, publishing coverage, test agents and parallel test execution, results in Azure Pipelines and GitHub Actions, coverage thresholds.
Key points
- Run tests: DotNetCoreCLI@2 (command: test), VSTest task or dotnet test --logger trx.
- .NET Core and VSTest tasks publish results; scripts need PublishTestResults@2 (VSTest format = TRX), condition succeededOrFailed().
- Coverage: Windows dotnet test --collect "Code Coverage"; Linux or macOS coverlet (XPlat Code Coverage).
- PublishCodeCoverageResults@2: Cobertura or JaCoCo.
- Parallel: strategy: parallel (up to 99 agents), VSTest slicing.
Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/dotnet-core?view=azure-devops
Practice questions
In PublishTestResults@2, which testResultsFormat value is used for Visual Studio TRX files?
- JUnit
- NUnit
- VSTest
- CTest
Show answer
Answer: C. VSTest
The value is VSTest for historical reasons; it means TRX and also works for TRX produced by other runners. JUnit, NUnit and CTest are different XML formats.
Which statements about running tests in parallel with the VSTest task are correct?
- A YAML job can use strategy: parallel to spread tests across up to 99 agents.
- Slicing can be based on the past running time of tests.
- Slicing by assemblies suits tests that use AssemblyInitialize or ClassInitialize state.
- Slicing by number of tests balances agents well even when run times differ greatly.
- Parallel test jobs run without any parallel job capacity.
Show answer
Answer: A. A YAML job can use strategy: parallel to spread tests across up to 99 agents., B. Slicing can be based on the past running time of tests., C. Slicing by assemblies suits tests that use AssemblyInitialize or ClassInitialize state.
Parallelism comes from strategy: parallel (up to 99 agents). Slicing can use past running times, and assembly-based slicing suits tests that rely on assembly or class initialization. Simple count-based slicing does not balance uneven run times, and parallel jobs still need available parallel job capacity.
True or false: by default, PublishTestResults@2 fails the task when the results file contains failed tests.
Show answer
Answer: False
False. failTaskOnFailedTests defaults to false, so the task just publishes results. Set it to true if the task itself should fail.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
