The Property, Not the List
In 1921 Roman Jakobson wrote a sentence that founded a discipline: the object of literary science is not literature but literariness: that is, what makes a given work a literary work. The move is deliberate. Instead of arguing about which books belong on the shelf, study the property that makes any stretch of language work as literature.
The Russian Formalists — Shklovsky, Eichenbaum, Tynyanov, Jakobson — proposed that the property is deviation with a purpose. Ordinary language is transparent: you look through it at what it says and forget the words. Literary language makes you notice the words, and the noticing is the point.
Defamiliarisation
Viktor Shklovsky's "Art as Technique" (1917) gives the mechanism, ostranenie, usually translated defamiliarisation or estrangement:
Habitualisation devours works, clothes, furniture, one's wife, and the fear of war… Art exists that one may recover the sensation of life; it exists to make one feel things, to make the stone stony. The technique of art is to make objects unfamiliar, to make forms difficult, to increase the difficulty and length of perception.
Two claims are packed in there. The first is about perception: habit makes us stop seeing, and art restores seeing by obstructing it. The second is about time: the difficulty is not a side effect but the method, because a perception that takes longer is a perception that happens at all. Shklovsky's example is Tolstoy describing a flogging or a property law entirely without the usual names, so the reader has to see the thing.
This gives a measurable, if crude, test. Take a poem and a prose paraphrase that says the same thing. The paraphrase runs twelve words and the stanza runs forty-eight: the stanza spends four times as long on the same content, and the surplus is where the literary work is happening.
The poetic function
Jakobson's later, sharper version comes from his model of communication. Any utterance involves six factors, an addresser, an addressee, a context, a message, a contact and a code: and each supports a function. When the utterance orients itself toward the message as such, the poetic function dominates.
His formula: the poetic function projects the principle of equivalence from the axis of selection into the axis of combination. Selection works by equivalence, choosing between child, kid, youngster. Combination normally works by contiguity, stringing words along. Poetry makes the combined sequence obey equivalence too: metre makes positions equivalent, rhyme makes sounds equivalent, parallelism makes phrases equivalent. That is what verse is, structurally.
What broke it, and what survived
The Formalist programme has a fatal objection: the same devices show up everywhere. Advertising defamiliarises, jokes foreground the message, weather forecasts fall into metre by accident. And texts move in and out of the literary category with no change to their language at all, a legal deposition, a diary, a set of letters. Stanley Fish's reply is that literariness is conferred by a way of reading, not found in the text.
What survived is the analytic vocabulary. Foregrounding, device, defamiliarisation and the projection principle remain the sharpest tools for describing what a text is doing, even once you stop believing they define what literature is.
Common pitfall: taking defamiliarisation as a synonym for "strange" or "difficult". Obscurity that yields nothing on the second pass has not renewed anyone's perception. Shklovsky's criterion is the recovery of the sensation of the thing, difficulty in the service of seeing, not in place of it.