I find the opposite. TDD is great when you don't know what your program should look like. It gives you an opportunity to simulate how the program will be used on the outside, able to be quickly iterated upon until satisfaction, without having to write all the laborious internal code (often over and over again when exploring without TDD). Once you are happy with the result, then you just have to back and fill in the guts once.
If you know exactly what you need upfront, you can simply start coding, adding a sprinkling of acceptance tests to help catch mistakes. No need for TDD in that case.
When I get to a new database and don't even know what data is stored where I don't write a test first. I write a bunch of SQL scripts and then maybe take it into a python toolkit for stats stuff. When training a classifier and having to choose things like dimensionality I find that exploring what the dimensions actually express teaches me more about the dataset and the approach faster than starting with a test. Sometimes I don't even know what opportunities are in the data that I'm going through, so how would I even express the test?
That said, I'll try it your way next time and see how it goes. If it works for you maybe I'll learn how to make it work for me, since I love TDD.
As for knowing exactly what you need upfront, why start coding? Why not do TDD? I find the interfaces are more naturally expressed as the consumer than the implementer.
I rarely find myself writing unnatural interfaces when starting with the test, and by starting with the test it makes abstractions that must be faked / mocked easier to slide into the code that implements the feature without too much damage to the rest of the codebase. I avoid them whenever possible, but sometimes a network call must be mocked and it's better to do so with minimal collateral damage.
> I write a bunch of SQL scripts and then maybe take it into a python toolkit for stats stuff.
That sounds a lot like a test (or set of tests). It would appear that you are already doing something akin to TDD, although perhaps without the formality of turning your exploratory work into documentation for other developers to read about what you learned.
> Why not do TDD? I find the interfaces are more naturally expressed as the consumer than the implementer.
If exploration is necessary to find the right interface that may be worthwhile, but I find when the requirements are well defined the interface is obvious before writing either tests or implementation. It may be still worth writing unit tests up front, but it is not testing that is driving your development. The requirements are driving your development in that case.
If you know exactly what you need upfront, you can simply start coding, adding a sprinkling of acceptance tests to help catch mistakes. No need for TDD in that case.