AZ-400 ยท Testing strategy for pipelines
Designing a test strategy for pipelines
Test pyramid, local, unit, integration, UI, load and performance tests, shift-left, test environments and data, Azure Load Testing, flaky tests as a design concern.
Key points
- Test pyramid: many fast unit tests, fewer integration, fewest slow UI tests.
- Shift left: quick tests locally and on every PR; heavier later in a test environment with seeded (synthetic or masked) data.
- Load and performance tests before release.
- Azure Load Testing: URL-based, Apache JMeter or Locust; fail criteria such as response time or error rate.
- Flaky tests: isolate, fix or quarantine, no retry until green.
Source: https://learn.microsoft.com/en-us/azure/app-testing/load-testing/overview-what-is-azure-load-testing
Practice questions
Which two script-based test frameworks does Azure Load Testing support?
- Apache JMeter and Locust
- Selenium and Playwright
- JUnit and NUnit
- Gatling and k6
Show answer
Answer: A. Apache JMeter and Locust
Azure Load Testing runs Apache JMeter and Locust scripts (plus quick URL-based tests). Selenium and Playwright are UI automation tools, and k6/Gatling are not the supported script formats.
Which practices are examples of shifting testing left?
- Developers run unit tests locally before pushing.
- A pull request build runs unit tests before merge.
- The PR build also runs static analysis and dependency scanning.
- Load tests are postponed until after production release.
- The UI suite is run manually once per quarter.
Show answer
Answer: A. Developers run unit tests locally before pushing., B. A pull request build runs unit tests before merge., C. The PR build also runs static analysis and dependency scanning.
Running tests before push, validating every pull request with a build, and running static analysis in that same build all move feedback earlier. Delaying load tests to after release or running UI tests only by hand does the opposite.
True or false: automatically retrying a flaky test until it passes is a sound long-term fix.
Show answer
Answer: False
False. Retries hide the root cause (shared state, timing, environment) and slow the pipeline. Fix, isolate or quarantine the test and track it as a defect.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
