Anyone who has ever sat down to write fiction will know that dialogue is one of the hardest things to write.

That’s partly because speech is so complex. Language works at a variety of levels: phonological (at the level of accent), lexical (the words used), morphosyntactic (the grammar), prosodic (the tone, stress and rhythm), discourse (how multiple phrases and sentences are combined), pragmatic (how meaning is influenced by context), and even paralinguistic (the nonverbal signs and cues, such as hand gestures and facial expressions, that often occur alongside speech). When we talk, we convey meaning at all of these levels simultaneously – and not just in terms of what we say, but also in terms of how we say it.

Let’s take for example a very simple sentence, “you want a cake”. How I say that sentence to you will substantially change the information I convey. For example, if I increase the pitch of my voice towards the end of the sentence (a change in prosody), I will be asking you a question: “you want a cake?”. If I don’t, I will probably be making some sort of statement about how hungry you are. If I raise the amplitude of my voice, I might be telling you that I’m angry with you. If I say the sentence in a Scottish accent (a change at the level phonology), I will be telling you something about where I am from. If I speak in Received Pronunciation, I will be telling something about my social class. If I change the word “cake” to “petite madeleine” (a change, at the lexical level, in what linguists call “register”) I will be telling you something about my culinary knowledge, and so on.

In general, whatever choices I make across these various linguistic levels, when I say something as simple as “you want a cake”, I can’t help but tell you something about my identity – a topic which is still of great interest to linguists.

Things get even more complex when two or more people are speaking. First of all, real people don’t tend to talk to each other in complete sentences. Instead, they talk in fragments of sentences or phrases, or often just in single words. Moreover, real life dialogue doesn’t tend to be “she said, he said”. People do tend to take what linguists call “turns” in conversation, following some fairly fundamental rules of conversation, but quite often these turns overlap when multiple speakers speak at once.

It’s exactly this richness and complexity of everyday language that makes it such an interesting topic for researchers and linguists. And, as it turns out, capturing and representing the complexity of dialogue is as much a challenge for the linguist as it is for the novelist.

Novel writers take a myriad of approaches to rendering dialogue, differing in how they approach the degree of complexity. Some authors wilfully choose to ignore it. Ernest Hemingway, for example, takes a famously minimalist approach. In the dialogue of A Farewell To Arms, there’s little or no information about the levels of prosody or phonology. Instead, as a reader, you are left to create these for yourself from what you know about the characters:

‘You are all her dear boys,’ Catherine said. ‘She prefers the dear boys. Listen to it rain.’
‘It’s raining hard.’
‘And you’ll always love me, won’t you?’
‘Yes.’
‘And the rain won’t make a difference?’
‘No.’
‘That’s good. Because I’m afraid of the rain.’

In her novel The Keep, author Jennifer Egan takes this minimalism to another level. She eschews reporting phrases (like “he said” and “she replied”) entirely. Instead, the speech is stripped to the bare bones and presented like screenplay dialogue:

Danny: Shit. Where the hell is he?
Rafe: Could be right underneath us.

Here, the only information beyond the levels of words and grammar is the question mark, which indicates (at the prosodic level) a rise in pitch at the end of Danny’s utterance. You do get some sense of the identity of the speakers (his use of “shit” indicates that Danny probably isn’t a catholic priest), but you don’t know what accent the characters are speaking in, if they are speaking quickly or slowly, if they are whispering, and so on.

Punctuation in English is fairly limited. As a result, beyond marking questions (?) or exclamations (!), it’s pretty difficult to really show how characters are speaking. Instead, many authors resort to telling us how they are speaking. Although Stephen King is famously not a fan of the adverb, it is a convenient way to elaborate the manner of speech. For example, take this dialogue from Agatha Christie’s And Then There Were None. The adverb “decisively” suggests to me perhaps accelerated speech and a stress on the “not”:

The driver for the second taxi said:
‘Like to sit inside while you’re waiting?’
Vera said decisively:
‘Not at all.’

Older authors, however, do a lot more telling in their writing. In this bit of dialogue from Charles Dickens’ Hard Times, wealthy manufacturer Josiah Bounderby is talking to his housekeeper, Mrs Sparsit. Here, Dickens makes much more of an effort to capture the complexity of the speech, especially using adverb phrases (such as “in a highly superior manner”). Moreover, in Mrs Sparsit’s second turn, stress is indicated by the use of italics:

“I wish with all my heart, Sir,” said Mrs. Sparsit, in a highly superior manner; somehow she seemed, in a moment, to have established a right to pity him ever afterwards; “that you may be in all respects very happy.”
“Well, ma’am,” returned Bounderby, with some resentment in his tone: which was clearly lowered, though in spite of himself, “I am obliged to you. I hope I shall be.”
“Do you, Sir?” said Mrs Sparsit, with great affability. “But naturally you do: of course you do.”
A very awkward pause on Mr. Bounderby’s part succeeded. Mrs. Sparsit sedately resumed her work and occasionally gave a small cough, which sounded like the cough of conscious strength and forbearance.
“Well, ma’am,” resumed Bounderby […]

The dialogue here is clearly much richer, especially in its description of prosodic features. Still, because of Dickens’ use of standardised spelling, there’s still little information about what’s happening at the phonological level – that is, about accent and, for example, the regional or social backgrounds of the two characters. This is in contrast to this final excerpt, from Story In Harlem Slang by Zora Neale Hurston, in which the dialect (“de family”) is deliberately rendered using phonetic spelling:

“Sweet Buck, you fixing to talk out of place.” Jelly stiffened.
“If you trying to jump salty, Jelly, that’s your mammy.”
“Don’t play in de family, Sweet Back. I don’t play de dozens. I done told you.”

Of course, the more information writers include in their dialogue about what’s happening at the various linguistic levels, the harder it will be to read. There is an inevitable trade-off between accuracy and readability, detail and clarity. But, whether or not you like it from a stylistic point of view, it’s clear that Dickens’ and Hurston’s prose is richer linguistically, for example, than Hemingway’s.

It turns out that linguists have to a very similar problem to address when they are trying to transcribe speech data – both for the research itself and then, ultimately, for presentation in books and journals for other linguists to look at. Usually, what details linguists will record in their written transcriptions will depend on what linguistic features, at what linguistic levels, they are interested in.

For example, the following is a basic transcription of a conversation I recorded for my own Masters thesis. In it, a group of scientists are discussing potential research project ideas (and having a playful joke about the more existential aspects of quantum theory at the same time):

Ben:     Quantum states first maybe?
Anne:  Yeah.
Pete:    It’s a good one.
Sam:    We all want to be the author of ‘On The Ontology of Quantum States’
Pete:    Confirmation
Sam:    You need to be, like, you need to speak in French and smoke as you explain.

If you were, say, a lexicographer interested in word usage, this sort of transcription might be enough. However, it’s clear that this very basic transcription ignores lots of information about the other levels of language: of phonology, prosody, and so on.

For example, if you were a linguist who was particularly interested in accents and dialects, you could instead transcribe the data phonetically (using the International Phonetic Alphabet). Then, you’d be able to see from his pronunciation of the first vowel sound in “quantum” that Ben isn’t, for example, from North America.

Ben:     kwɒn.təm steɪts fɜːst meɪ.bi
Anne:  jeə […]

One of the most common transcription systems for analysing this sort of dialogue is called Jefferson notation. Used particularly by linguists and sociologists interested in “conversational analysis”, Jefferson notation is less concerned about phonological features. Instead, it is much more interested in capturing prosodic features of speech (of stress and intonation), as well as information about how the various speakers are interacting: that is, whether they are politely taking turns, constantly interrupting each other, all speaking at once, and so on.

By way of an illustration, this is what the dialogue looks like transcribed in Jefferson notation:

Ben:     quantum states first °↑maybe°
Anne:  yeah=
Pete:    =it’s a good one
Sam:    we all want to be the author of (.) >on the ontology< of quantum states
((laughter))
Pete:    confir er (.) confirmation=
Sam:    =you need to be like you need to speak in Fre:nch and smoke as you explai:n
((laughter))

Words in small circles (e.g. “°maybe°”) mean they are more softly spoken. Words in capital letters are spoken more loudly. Dots in parentheses indicate a small pause, but an equals signs (“yeah=”) means that there is no pause between one speaker’s turn and the next. Words in inward facing brackets (“>on the ontology<”) are spoken more quickly. A colon indicates a lengthening of the relevant syllable. And, finally, paralinguistic information (about laughter, for example) is put in double parentheses.

Inevitably, the transcription looks pretty complex, and is hardly easy to read. And it would get more complicated still if you tried to capture the phonological information in the same transcription! The point is that, whether you are a novelist or a linguist, it’s essentially impossible to capture all the complexity of speech on paper. And, even if you did manage it, the result would be essentially unreadable. All you can do is focus on the particular linguistic levels you are most interested in – or, if you are a novelist, the linguistic features of the dialogue your readers to focus on.

That’s not to say that, one day, some pioneering novelist won’t write all of their dialogue in Jefferson notation (and Jack Kerouac, for example, got some way towards that in Visions of Cody). Whether or not the result would be a bestseller, of course, is an entirely different question.

Incidentally, this is what Dickens passage would look like if a linguist had written it:

Spars:   I wish with all my heart sir that you may be in all respects very happy
Bound: well ↓ma’am (.) I am obliged to you (0.6) I hope I shall be
Spars:   DO you sir? (.) But naturally you do (.) >of course you do<
(1.2)
((Spars resumes work))
Spars:   ((coughs twice)) (1.3) ((coughs))
(0.5)
Bound: well ma’am

And, just for fun, here’s what the conversation between the four scientists might look like in a more literary form…:

They stood huddled around the flipchart. Pen firmly in hand, Pete looked to his colleagues. He wore a look of focus, ready as he was to capture their thoughts.
Ben spoke first.
‘Quantum states first maybe?’ he suggested, his voice trailing off meekly as he spoke.
‘Yeah,’ said Anne.
‘It’s a good one,’ agreed Pete. He began to write.
Then, Sam spoke. ‘We all want to be the author of… On The Ontology Of Quantum States,’ he joked.
A flurry of laughter erupted, and then subsided.
‘Confir, er, confirmation,’ suggested Pete, trying to stick to the job in hand.
But Sam continued. ‘You need to be, like, you need to speak French,’ he said, chuckling. ‘And smoke as you explain…’

(The names of the scientists in the transcription have been changed to maintain their anonymity.)