That is, I fancy, the true doctrine on the subject of Tales of Terror and such things, which unless a man of letters do well and truly believe, without doubt he will end by blowing his brains out or by writing badly. Man, the central pillar of the world must be upright and straight; around him all the trees and beasts and elements and devils may crook and curl like smoke if they choose. All really imaginative literature is only the contrast between the weird curves of Nature and the straightness of the soul. Man may behold what ugliness he likes if he is sure that he will not worship it; but there are some so weak that they will worship a thing only because it is ugly. These must be chained to the beautiful. It is not always wrong even to go, like Dante, to the brink of the lowest promontory and look down at hell. It is when you look up at hell that a serious miscalculation has probably been made.

* * *

Therefore I see no wrong in riding with the Nightmare to-night; she whinnies to me from the rocking tree-tops and the roaring wind; I will catch her and ride her through the awful air. Woods and weeds are alike tugging at the roots in the rising tempest, as if all wished to fly with us over the moon, like that wild amorous cow whose child was the Moon-Calf. We will rise to that mad infinite where there is neither up nor down, the high topsy-turveydom of the heavens. I will answer the call of chaos and old night. I will ride on the Nightmare; but she shall not ride on me.

-Alarms and Discursions (1910)

"Are you the new recruit?" said the invisible chief, who seemed to have heard all about it. "All right. You are engaged."

 Syme, quite swept off his feet, made a feeble fight against this irrevocable phrase.

"I really have no experience," he began.

"No one has any experience," said the other, "of the Battle of Armageddon."

"But I am really unfit—"

"You are willing, that is enough," said the unknown.

"Well, really," said Syme, "I don't know any profession of which mere willingness is the final test."

"I do," said the other—"martyrs. I am condemning you to death. Good day."

-The Man Who Was Thursday: A Nightmare (1908)

Thomas Paine invented the name of the Age of Reason; and he was one of those sincere but curiously simple men who really did think that the age of reason was beginning, at about the time when it was really ending. Being a secularist of the most simple-minded sort, he naturally aroused angry passions at the moment, as does any poor fellow who stands on a chair and tries to heckle heaven in Hyde Park. But considering him in retrospect, the modern world will be more disposed to wonder at his belief than at his unbelief. The denial of Christianity is as old as Christianity; we might well say older. The anti-clerical will probably last as long as the Church, which will last as long as the world. But it is doubtful when we shall see again the positive side of Paine's philosophy; the part that was at once credulous and creative. It is impossible, alas, for us to believe that a Republic will put everything right, that elections everywhere will ensure equality for all. For him the Church was at best a beautiful dream and the Republic a human reality; today it is his Republic that is the beautiful dream. There was in that liberalism much of the leisure of the eighteenth-century aristocrats who invented it; and much of the sheltered seclusion also. The garden which Voltaire told a man to cultivate was really almost as innocent as the garden of Eden. But the young men who saw such visions were none the less seeing visions of paradise, though it was an earthly paradise. Rationalism is a romance of youth. There is nothing very much the matter with the age of reason; except, alas, that it comes before the age of discretion.

-William Cobbett (1925)

This would be no place to enquire too closely why those bright youths who are so superior to eternity seem to be so subject and submissive to time; why they proclaim with such wild pagan gestures that they can pull down the Cross; but assure us, with such anxious and agitated motions, that we cannot put back the clock.

-July 13, 1935, Illustrated London News

..if a man thinks himself the pillar of righteousness, it is all the more spiritually dangerous for him the more it is approximately true. But if a modern artist thinks himself a beacon of blazing originality, that is less likely to be in any way flattered by the facts. That- as the Irish priest said to the lady who confessed she had thought much about her own beauty- that is not a sin, it is mistake.

-July 29, 1911, Daily News

Politics are eternal and politicians luckily are not.

-January 1, 1910, Daily News

According to the Wikipedia article for The Man Who Was Thursday:

In Neil Gaiman's The Sandman comics, the Library of Dream's castle contains every story ever written plus every story dreamed of but never written. Among the latter is The Man Who Was October by G. K. Chesterton, which is supposedly a sequel to his Thursday.