Testing

My first job for Microsoft was as a Software Design Engineer in Test (SDET), back in 2022 if I recall correctly. It's also one of the things I was somewhat surprised to see had a whole course for it while studying at UADE.

So testing is a topic that has been near and dear to my heart for a while, and I thought it might be useful to compile some resources and thoughts on it.

Types of test

TODO: fill this out; unit, integration, end-to-end, performance, resiliency, security, microbenchmarks, compliance, exploratory, long-haul, drain, stress, scalability, load, accessibility, usability, localization

Techniques

TODO: fill this out; isolating, mocking, data-driven, random generation, oracles, A/B implementations, code walk-throughs, debuggers, chaos monkeys, playtesting, fuzzing, visual, playback

Test strategy

TODO: describe what a strategy is and looks like, how test plans fit in that

Test matrix

TODO: describe the classic test matrix

Acceptance testing

TODO: describe the role of successive 'test levels' to roll out to developers, testers, production

Code coverage

TODO: discuss tools, pros/cons, block and edge, manual coverage with breakpoints

Working effective with legacy code

TODO: describe "legacy code is untested code you can't refactor/work with", mechanical refactoring

Refer to Working Effectively with Legacy Code by Michael Feathers. Great summary here.

Too many tests

TODO: describe how you inevitably end up in this situation, and some strategies to deal with it

Runtime tools

TODO: describe sanitizers, heap instrumentation, others

https://developer.apple.com/documentation/xcode/diagnosing-memory-thread-and-crash-issues-early

https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/tools-for-testing-drivers

https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/application-verifier

Conformance tests

TODO: talk about Android CTS, Windows Hardware Lab Kit

https://github.com/microsoft/WindowsProtocolTestSuites

https://learn.microsoft.com/en-us/windows-hardware/test/hlk/

Resources

Home