Bitzu

AZ-400 ยท Testing strategy for pipelines

Designing a test strategy for pipelines

Test pyramid, local, unit, integration, UI, load and performance tests, shift-left, test environments and data, Azure Load Testing, flaky tests as a design concern.

Key points

  • Test pyramid: many fast unit tests, fewer integration, fewest slow UI tests.
  • Shift left: quick tests locally and on every PR; heavier later in a test environment with seeded (synthetic or masked) data.
  • Load and performance tests before release.
  • Azure Load Testing: URL-based, Apache JMeter or Locust; fail criteria such as response time or error rate.
  • Flaky tests: isolate, fix or quarantine, no retry until green.

Source: https://learn.microsoft.com/en-us/azure/app-testing/load-testing/overview-what-is-azure-load-testing

Practice questions

Single choice

Which two script-based test frameworks does Azure Load Testing support?

  1. Apache JMeter and Locust
  2. Selenium and Playwright
  3. JUnit and NUnit
  4. Gatling and k6
Show answer

Answer: A. Apache JMeter and Locust

Azure Load Testing runs Apache JMeter and Locust scripts (plus quick URL-based tests). Selenium and Playwright are UI automation tools, and k6/Gatling are not the supported script formats.

Select all that apply

Which practices are examples of shifting testing left?

  1. Developers run unit tests locally before pushing.
  2. A pull request build runs unit tests before merge.
  3. The PR build also runs static analysis and dependency scanning.
  4. Load tests are postponed until after production release.
  5. The UI suite is run manually once per quarter.
Show answer

Answer: A. Developers run unit tests locally before pushing., B. A pull request build runs unit tests before merge., C. The PR build also runs static analysis and dependency scanning.

Running tests before push, validating every pull request with a build, and running static analysis in that same build all move feedback earlier. Delaying load tests to after release or running UI tests only by hand does the opposite.

True or false

True or false: automatically retrying a flaky test until it passes is a sound long-term fix.

Show answer

Answer: False

False. Retries hide the root cause (shared state, timing, environment) and slow the pipeline. Fix, isolate or quarantine the test and track it as a defect.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More AZ-400 topics in this module