In this episode of TMI, Chip and Tess dive deep into the frustrating world of flaky tests. These are the tests that pass sometimes and fail at other times, even when the code hasn't changed.

We discuss reference research from Trunk, who analyzed over 20 million CI jobs and found that flaky tests are a widespread problem, even among large tech companies like Meta, Google, and Uber.

We examine strategies for managing flaky tests, such as quarantining them. We also cover how to prevent flaky tests by ensuring isolation, using dynamic waits instead of hard-coded timeouts, and utilizing deterministic inputs.

Finally, we discuss the importance of the testing pyramid, controlled test environments, contract testing, robust error handling, retry mechanisms, and clear communication.

References:

• https://www.testrail.com/blog/flaky-tests/
• https://testsigma.com/guides/flaky-test/
• https://trunk.io/blog/what-we-learned-from-analyzing-20-2-million-ci-jobs-in-trunk-flaky-tests-part-1
• https://trunk.io/blog/what-we-learned-from-analyzing-20-2-million-ci-jobs-in-trunk-flaky-tests-part-2

Explore the three pillars of Agile estimating and planning: embracing change, prioritizing value, and effective communication—principles that resonate beyond software development.

• Learn why multitasking can hinder progress despite feeling productive.
• Understand the pitfalls of setting initial estimates in stone.
• Learn why providing estimation dates and team velocity as ranges -- rather than a definitive number -- helps manage expectations.
• Discover how properly prioritizing features can deliver 80% of benefits with 20% of the effort.
• We discuss using Story Points to estimate complexity, Ideal Days for estimating the time it will take to completer a task, and tips for splitting large user stories.
• Delve into planning poker for estimation and strategies for backlog prioritization.

Finally, we consider a sprint structure that includes dedicated time for technical debt.

More Resources:

• Agile Estimating and Planning