11:30 - 12:30 (UTC+02)
Talk (60 min)
Auto-testing for situational awareness
Why do you do automatic end-to-end testing? For keeping out regressions? For selecting and promoting release candidates? While we do those things at Neat, our ultimate aim for autotesting is creating situational awareness. Testing for situational awareness means moving beyond the spatial dimension, that of just tracking where in the system functional requirements fail. It introduces time, tracking when issues pop up in or depart from different code branches or release channels. It also involves moving beyond red and green, following the variations in other significant metrics over time, like test run durations and the numbers of actually executed tests. This talk will cover the gory details of CI yaml, dashboards and the practical strategies we use to see as much of the situation as possible, including how it evolves over time.