From Maisie Ward's Return to Chesterton (1952), p.63:

A charming story is told by a man who saw Gilbert starting to chase his hat in the street. Plunging through the traffic, the passer-by rescued it to his own imminent peril, and was not pleased when Gilbert received it with the remark that his wife had just bought him a new one and would be sorry to see it again.
"Then why on earth did you run after it?"
"It's an old friend. I am fond of it and I wanted to be with it at the end."

The terrible danger in the heart of our Society is that the tests are giving way. We are altering, not the evils, but the standards of good by which alone evils can be detected and defined. It is as if we were looking at some great machine, say, a stonecutter's saw, and the saw was working briskly and the dust flying brightly. But when we came to look close, we found that the stone was unscratched and was wearing away the steel. The thing that should crumble is holding fast; the thing that should hold fast is crumbling [...] So the moral scales that were meant to weigh our problems are themselves breaking under the weight of them. The philosophical instruments which were meant to dissect existence are bent and twisted against the toughness of the thing to be dissected. Because it is very hard work to apply principles of judgment to anything, people are everywhere abandoning the principles and practically deciding not to test life at all, but only let life test them. They do not analyze the situation at all; they let their situation analyze them- which means, break them up [...] Instead of testing the passing institutions by the eternal institutions, we are nibbling away the eternal institutions and leaving ourselves with no test at all.

-March 25, 1911, Illustrated London News

"I know," said Father Brown, and his mouth took on again the twisted smile. "I sometimes think criminals invented hygiene. Or perhaps some hygienic reformers invented crime; they look like it, some of them. Everybody talks about foul dens and filthy slums in which crime can run riot; but it's just the other way. They are called foul, not because crimes are committed, but because crimes are discovered. It's in the neat, spotless, clean and tidy places that crime can run riot; no mud to make footprints; no dregs to contain poison; kind servants washing out all traces of the murder, and the murderer killing and cremating six wives and all for want of a little Christian dirt [...]"

-The Scandal of Father Brown (1935)

Fairy tales are the only sound guidebooks to life [...]

-Alarms and Discursions (1910)

By a curious confusion, many modern critics have passed from the proposition that a masterpiece may be unpopular to the other proposition that unless it is unpopular it cannot be a masterpiece.

-Generally Speaking (1928)

We have in this country all that has ever been alleged against the evil side of religion; the peculiar class with privileges, the sacred words that are unpronounceable; the important things known only to the few. In fact we lack nothing except the religion.

-All Things Considered (1908)

Trust the people and get the thing settled slowly. But in the name of all ancestral wisdom, do not trust the faddist and get the thing settled wrong. Do not trust the opinion of every chance person whose name you've heard in the newspapers as being somebody vaguely and irrationally important. Do not trust a man because you have heard of him as a cricketer or a journalist or a prize-fighter or a burglar or a millionaire.

-September 22, 1906, Illustrated London News

Most of us, of course, spend half of our time in abusing journalism, especially those of us (like myself) who spend the other half in writing it. But when we pass from abusing a thing to reforming it, we commonly pass from an easier condition to a much stormier one, for there is nothing more united than opposition, and nothing more divided than reform. When two men unite against a third with hearty and unanimous enthusiasm, it is generally because one thinks he is too far to the left and the other that he is too far to the right.

-January 6, 1906, Illustrated London News

Definitions

Well aware of how offensive I make myself, and with what loathing I may well be regarded, in this sentimental age which pretends to be cynical, and in this poetical nation which pretends to be practical, I shall nevertheless continue to practise in public a very repulsive trick or habit--the habit of drawing distinctions; or distinguishing between things that are quite different, even when they are assumed to be the same [...] I have again and again blasphemed against and denied the perfect Oneness of chalk and cheese; and drawn fanciful distinctions, ornithological or technological, between hawks and handsaws. For in truth I believe that the only way to say anything definite is to define it, and all definition is by limitation and exclusion; and that the only way to say something distinct is to say something distinguishable; and distinguishable from everything else. In short, I think that a man does not know what he is saying until he knows what he is not saying.

-As I was Saying (1936)

[An inscription in a book Chesterton gave to a young child]

This is the sort of book we like
   (For you and I are very small),
With pictures stuck in anyhow,
   And hardly any words at all.

You will not understand a word
   Of all the words, including mine;
Never you trouble; you can see,
   And all directness is divine-

Stand up and keep your childishness:
   Read all the pedants' screeds and strictures;
But don't believe in anything
   That can't be told in coloured pictures.

-Collected Works of G.K. Chesterton, volume X: Collected Poetry, Part I, p. 304

Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, "Let us first of all consider, my brethren, the value of Light. If Light be in itself good--" At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.

-Heretics (1905)

Utopia always wins best in what is, in another than the Wellsian sense,  a War in the Air. When the heavenly kingdom becomes an earthly paradise, it sometimes tends to be a hell upon earth. But it sometimes tends to be what is even worse, or at least weaker- a very earthy imitation of earth. So long as revolution is a failure, we all feel that it holds the promise of success. It is when it is a success that it is so often a failure.

-June 16, 1928, Illustrated London News