Instead what I'd like to claim is that Java is not, generally, a hard enough programming language that it can be used to discriminate between great programmers and mediocre programmers.
Why on earth would you want an interviewee to pick a difficult language? That’s a bad sign.
As someone who has interviewed and hired a lot of programmers, let me counter with some advice of my own: If you give someone the task of digging a hole and offer them the tool of their choice, favor the one that chooses the auger over the garden spade.
Sure, seeing someone handle a nasty pointer issue tells you something. It tells you whether they can deal with nasty pointer issues. If you’re doing software that requires that sort of thing, by all means—force people to show you they can handle it.
Software development happens at many levels. I’ve known a lot of programmers that are good at the low level stuff but not so great at coming up with the high level architectures or the fluffy stuff that make a user love an application. And vice versa. I’m not convinced that all programmers can and/or will be great at both. I’d even go so far as to say that different parts of the brain and potentially different people are going to be right for these different layers and types of application development.
Joel feels that low(ish) level languages like C and C++ with their pointers and other various "hard stuff" are necessary to weed people out of the industry. The ones that can’t handle the hard stuff in the languages are the ones that were never meant to be programmers in the first place.
What worries me is that this will weed out the artists and the craftspeople, leaving us with nothing but a bunch of engineers. Sometimes you need an engineer. Sometimes you need an artist.
What you always need is someone capable of choosing the right tool for the job.