I have a new interview question:
"What is the difference between acceptance tests and systems tests?"
Like all good interview questions, I have my own opinions but do not have anything like a correct answer. In addition, it gets information about the candidate that is not necessarily what they think I am asking.
Because I am generally the "good interviewer", and because I like people, the only way not to score points with me is to answer with no thought involved at all - "acceptance tests are done to accept the system", "there is no difference", "everything stops if the acceptance tests fail" - although now that I think about it, the last answer could lead to an interesting discussion, just not one that matches any reality I have worked in.
Extra points are scored by anyone who asks me a question back, especially a question about what I mean by acceptance tests: build acceptance tests, user acceptance tests, deployment acceptance tests, other kinds of acceptance tests...? This is one thing that I am trying to figure out, when we talk about acceptance tests shouldn't we be very clear about who is accepting what? We continue the discussion with me indicating that I am talking about user acceptance tests.
I get a lot of answers talking about business requirements versus functional requirements, which is okay but a bit flat for my tastes. I have worked for years with no written business or functional requirements, and for years before that with out of date and incorrect requirements, and I know that I am not alone in this. So I ask the candidate that if I show them a particular test case, how will they be able to tell whether it is a system or user acceptance test.
Now we're into the real mess. I often get answers about user acceptance tests being at a "higher level" than system tests, but this can work both ways - what about a system test that was written before design was finalised? Won't that be as high level? And for truly high level acceptance tests, what is the difference between them and the actual business requirements that they are supposed to be testing? I'm not interested in the exam board answers, I am not in a world where they are true, and people lose points all over the place for spewing those answers back at me.
This is my opinion on the matter. There is no way to tell whether a particular test is a system test or a user acceptance test just by looking at the test case definition. The real factors that determine it are who executes the test, when, and what they are testing for.
I have written user acceptance tests for a number of projects, and the who determines how high level they are - a business owner who has followed the development project needs fewer instructions, and so a higher level test, than a business owner who has never been near the system, and will probably need instructions at least to what login to use.
The when and the what for are kind of linked together, and actually I don't want to say any more about them right now - we'll end up on a rant about agile development methodologies and it is late on a Thursday night during my vacation. Let's get back to them some other time.
