Strongly agree, but also I think that the interesting thing is why this might be so and what it tells us about application architecture.
The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. Integration tests are a bet that certain aspects of the externally visible behaviour will not change.
Libraries tend to be smaller and with well-defined responsibility. Applications tend to be bigger and have many responsibilities. In general, I think it’s true that the requirements for libraries change less often than the requirements for applications. I think this leads us to expect that applications may need to be rewired “under the hood” and have their implementations changed as responsibilities are added, removed, or changed.
This, I believe, leads us to want to unit test applications less, because a unit test expresses implementation semantics, and we expect application implementations to change. No what about integration tests? Well, if we’re unit testing less in the application, we need to make up for it by integration testing more, otherwise where do we get our confidence?
Now if we throw the words “library” and “application” away, this suggests to me that those parts of the code that are small and tight and with a single, clear responsibility should be unit tested, while those parts that involve a lot of what the AOP people call ‘scattering and tangling,’ should be integration tested.
Unit testing didn't fully make sense to me until I played around with quickcheck (and eventually theorem proving in Coq). Unit tests vanish nicely to theorems and (empirical) proofs if your code expresses a succinct API. This is one end of the testing continuum.
I use this sort of stuff extensively when doing mathematical computing and statistics because there's usually a clear mathematical boundary. Once you're inside it, it's relatively easy to write down global properties (theorems) of your code's API.
The moment you cross that boundary your testing apparatuses have to get more complex and your tested properties less well-defined. Unit tests are hazier than quickcheck properties and integration tests hazier still.
This continuum seems to be precisely the same as the code reuse continuum. Highly abstracted, testable code with a shapely API is a highly reusable library whether you like it or not. Maybe it's being called by other code, maybe it's being called by your UI, maybe it's being called by the user themselves.
I view unit tests as a kind of proof by counter-example. You have a logical structure your program embodies. This structure is very hard to specify mathematically and prove deductively so you come up with key statements that must at least evaluate to true for this structure/theory. The tests are a bunch of counter-examples that should be false (test passed).
If a random testing framework is available in your language they really should be integrated as they are able to come up with some pathological examples.
For Haskell you can do even better than random testing: There's Lazy SmallCheck for exhaustive testing. (For some values of `better' and `exhaustive'.)
After playing with a language with a great type system (Haskell, not Java/C++), I've become more wary and borderline uncertain about my Ruby code.
My integration tests serve two purposes:
1) Runs the code in an repeatable and isolated environment to verify I didn't do anything stupid like misname a variable, or treat something nil as an object.
2) Validate my unique application logic.
I don't think #2 goes away with other languages, but #1 changes dramatically. I've written Ruby for years, so this isn't an "outsider looking in" opinion.
Whether conscious or not, I believe move Rubyists gravitate to testing because of both #1 and #2.
#2 can be satisfied with unit tests, but in my experience, the suite becomes a lot more flexible when validating it in terms of integration tests.
Note that, at least in python (which I assume is pretty similar to Ruby in ecosystem), you can get a lot of mileage for #1 by doing static analysis, for example Pylint.
On this, I tend to think that many (most?) libraries, especially where they don't have extreme performance requirements, should have run-time testing. I.e., pre- and post-condition checks, or contracts.
The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. Integration tests are a bet that certain aspects of the externally visible behaviour will not change.
Libraries tend to be smaller and with well-defined responsibility. Applications tend to be bigger and have many responsibilities. In general, I think it’s true that the requirements for libraries change less often than the requirements for applications. I think this leads us to expect that applications may need to be rewired “under the hood” and have their implementations changed as responsibilities are added, removed, or changed.
This, I believe, leads us to want to unit test applications less, because a unit test expresses implementation semantics, and we expect application implementations to change. No what about integration tests? Well, if we’re unit testing less in the application, we need to make up for it by integration testing more, otherwise where do we get our confidence?
Now if we throw the words “library” and “application” away, this suggests to me that those parts of the code that are small and tight and with a single, clear responsibility should be unit tested, while those parts that involve a lot of what the AOP people call ‘scattering and tangling,’ should be integration tested.
Thoughts?