Now, the story as told seems to be apocryphal, but it does seem to have a grain of truth in it. Since recently in a couple of places I came across people mentioning (correctly) that the story as told is apocryphal, I did state what the grain of truth seems to be, and so I decided just to post one of the replies I made below, for others who may be interested or doing research on it:
Well, even though the story as usually told is apocryphal, there is a grain of truth in it. In 1905, the Daily News in London published a letter called "What's Wrong With the World" by "A Heretic" that generated a huge correspondence from readers for months. Chesterton was one of the people who responded, and while his response was much longer than the apocryphal story indicates, it did include among other things this passage:UPDATE: You can read the whole letter here.
"In one sense, and that the eternal sense, the thing is plain. The answer to the question , 'What is Wrong?' is, or should be, I am wrong.' Until a man can give that answer his idealism is only a hobby."
(-Letter to the Daily News, August 16, 1905, "What is Wrong")