having tests depend on user-supplied user-config.py is just
... broken.
I hear you!
So, still trying to find my way around the test suite, I see appveyor.yml. I don't know anything about Appveyor, but it looks like this is building a known user-config.py before kicking off a test run. Is there some reason this couldn't be part of the global test setup for all tests? I tried tracing back how the test suite works, but to be honest, I stopped reading once I found MetaTestCaseClass.