The Elements of Style · Rule 5

Do not join independent clauses by a comma.

If two or more clauses, grammatically complete and not joined by a conjunction, are to form a single compound sentence, the proper mark of punctuation is a semicolon.

Stevenson's romances are entertaining; they are full of exciting adventures.
It is nearly half past five; we cannot reach town before dark.

It is of course equally correct to write the above as two sentences each, replacing the semicolons by periods.

Stevenson's romances are entertaining. They are full of exciting adventures.
It is nearly half past five. We cannot reach town before dark.

If a conjunction is inserted the proper mark is a comma (Rule 4).

Stevenson's romances are entertaining, for they are full of exciting adventures.
It is nearly half past five, and we cannot reach town before dark.

A comparison of the three forms given above will show clearly the advantage of the first. It is, at least in the examples given, better than the second form, because it suggests the close relationship between the two statements in a way that the second does not attempt, and better than the third, because briefer and therefore more forcible. Indeed it may be said that this simple method of indicating relationship between statements is one of the most useful devices of composition. The relationship, as above, is commonly one of cause or of consequence.

Note that if the second clause is preceded by an adverb, such as accordingly, besides, then, therefore, or thus, and not by a conjunction, the semicolon is still required.

Two exceptions to the rule may be admitted. If the clauses are very short, and are alike in form, a comma is usually permissible:

Man proposes, God disposes.
The gate swung apart, the bridge fell, the portcullis was drawn up.

Note that in these examples the relation is not one of cause or consequence. Also in the colloquial form of expression,

I hardly knew him, he was so changed,

a comma, not a semicolon, is required. But this form of expression is inappropriate in writing, except in the dialogue of a story or play, or perhaps in a familiar letter.

In plain terms

If two clauses could each stand alone as sentences and no joining word such as and or but links them, a comma is too weak to hold them together. Use a semicolon, a period, or a comma plus a conjunction instead. The stronger mark tells the reader that one complete thought has ended and another has begun, so the sentence is not misread on the first pass.

BeforeThe rain finally stopped, the children ran out to the puddles.

AfterThe rain finally stopped; the children ran out to the puddles.

Today: Today this error has a name: the comma splice. Teachers and style guides treat it as one of the most common mistakes in student writing. Strunk allowed a comma between very short clauses that are alike in form, and fiction writers still do this for rhythm, but in essays and work writing, avoid it.