I’m currently debating how best to test core plans. Yes, @eeyun and I are working on an automated testing framework, but in order to inform how we would automatically test something we need to first understand how we would do it manually.
My current approach when reviewing a PR to core plans is that, if it adds new behavior or changes existing behavior, it must be tested manually before being merged and promoted (or in some cases after merge, but before promotion to stable). This does slow down the release of core plans changes quite a bit, particularly when it pertains to something without a ready made or easy to write sample use case.
One thought is to require, for all core plans PRs, steps on how to test the new or changed behavior, including a sample app when necessary.
However, I’d love to get more input on this, I will use that input to form an RFC around this if necessary.