In one of classes, my students were talking about the  Sean Penn’s article in which he interviewed the Mexican drug lord and escaped convict Joaquín Guzmán. For those unfamiliar with the piece, it was widely ridiculed after its publication. One target of that ridicule was the very style of the writing. As one outlet put it:

If you know only one thing about the Rolling Stone article itself, it should be this: The writing is terrible. Penn apparently fancies himself some combination of Bob Woodward and Hunter S. Thompson; he’ll tell you he farted when El Chapo shook his hand, but he calls it “expel[ling] a minor traveler’s flatulence.” It is the sort of self-serious dreck that no one but a celebrity could publish in a major American magazine, and that always inspires self-righteous grumbling from bona fide professional journalists.

So one of my students posed the question: If we did some basic corpus analysis, could we tell the writing is bad? This is a really interesting question. So I put together a small corpus for the purposes of comparison. Genre matters, of course. Comparing Penn’s article to famous works of fiction, for example, won’t tell us much, since the conventions of fiction are different than the conventions of magazine journalism. We need to compare apples to apples. I found a list of recommended pieces of journalism that included a set a celebrity profiles. Admittedly, this is hardly an unimpeachably objective list. But for the purposes of our small experiment, it works as a functional benchmark.

From that list, I put together the the corpus below. (Note there is an additional folder with versions of the files with the quotations extracted for checking their potential effect on the data.)

Sean Penn corpus 355.80 KB

Download

When we did a key word search with Penn’s article as the target corpus, and other profiles as the reference corpus, the pattern that jumped out was Penn’s relative overuse of first person pronouns.

Penn spends a good portion of his narrative describing his own thoughts and actions or on the combined actions of himself and a companion. This is clear in the following passage (first person pronouns in bold):

We quietly make our plans, sensitive to the paradox that also in our hotel is President Enrique Peña Nieto of Mexico. Espinoza and I leave the room to get outside the hotel, breathe in the fall air and walk the five blocks to a Japanese restaurant, where we‘ll meet up with our colleague El Alto Garcia. As we exit onto 55th Street, the sidewalk is lined with the armored SUVs that will transport the president of Mexico to the General Assembly. Paradoxical indeed, as one among his detail asks if I will take a selfie with him. Flash frame: myself and a six-foot, ear-pieced Mexican security operator.

Alternatively, if we use the profiles as the target and Penn’s article as the reference, we find a very different pattern.

Penn’s article appears to significantly under-use third person pronouns, as well as what are called reporting verbs — particularly the most common verb of journalistic attribution, said. (The gender skew of these pronouns is also eyebrow raising.)

The keywords, therefore, reveal an unconventional focus: away from the words and actions of the purported profile subject and toward the author himself. This doesn’t necessarily measure “badness.” However, it certainly points to a piece of writing that significantly strays from the norms of its genre.

I’ve had quite a few students who are interested in using Twitter data for corpus analysis. I understand why. Twitter is widespread, and it’s relatively new communicative technology. New technologies have often been drivers of language change. So this is an inviting source of data.

But even if we decide, “Yes! Twitter is for me!” collecting the data and putting it into a format that is friendly for commonly used tools like AntConc presents some very real challenges.

Given both the possibilities and the potential hurdles, I thought I’d walk though a coffee break experiment I did this morning. First, I want to share the results I generated and what I think they tell us. Then, I’ll show you how I got there.

One of the areas of corpus research that has really taken off lately is sentiment analysis from social media data (like Twitter). Sentiment analysis attempts to measure attitudes: How do we feel about that movie? Was it good? Lousy? To do so a score is assigned, positive or negative, to a text. And it’s easy to understand why this kind of analysis is popular. It provides people with interest in how things are being talked about in the wider world (like marketers) a number with an air of scientific objectivity that says, “Hey, this is getting great buzz!” or maybe, “We had better rethink this…”

So I thought I would give this a try (again more about the specifics in a minute). Given all of news regarding the draft of Michael Sam, the first openly gay player to be drafted by the NFL, I though that I would build a corpus of tweets containing the hashtag #MichaelSam, and apply sentiment analysis. These are my results.

The technique I used assigns a score of 1 to 5 for positive sentiment (5 being the most positive) and -1 to -5 for negative sentiment (-5 being the most negative. What I got was this rather perfect bell curve. I was a bit surprised by the number of neutral tweets, but many of them are like this one:

2014 #NFL Draft: Decision day for Michael Sam http://t.co/vR4t1i2mKb
#MichaelSam #PALMETTO #RECEIVERS #STARS

But others that scored a 0 were like these:

Men kiss men every day, all over the world.  Get. Over. It. #MichaelSam

I’m Not Forcing You To Be A Christian Don’t Force Me To Be Okay With Homosexuality Stop Shoving it down People’s Throats #MichaelSam #ESPN

These examples raise some interesting questions about how sentiment analysis works.

The particular technique I used is lexical (as most sentiment analysis is, though some researchers are working out alternatives). It applies lists of positive and negative words to the texts in the corpus and calculates its score. This raises a number of problems.

The first is polysemy. Words, of course, have multiple meanings and treating those meanings as a unified class can erase significant ambiguities. This is particularly true with contranyms (words that have opposite meanings). Contranyms are common in slang, where negative words (like bad) take on new, positive associations. This is one of my favorite examples from the Guardian.

Another issue arises from the creative respelling and punctuation that are important in technologically-mediated communication. In one of the examples above the author’s stance is communicated through the phrase “get over it,” and that stance is amplified by the insertion of periods between each word. That emphasis is going to be overlooked if we are only measuring words.

Finally, there is the difficulty of locating the object of sentiment. While a tweet may express a negative sentiment, and that tweet may contain #MichaelSam, that negative sentiment may be directed at something else. Here are a couple of examples:

People used to discriminate blacks and now people want to discriminate gays. Everyone’s welcome in the future. Just let it be. #MichaelSam

Why are parents struggling to explain #MichaelSam to their kids? In 2014 are we that afraid of homosexuality still? #GrowUpAmerica

In both of these it’s easy to see what words are being counted as negative. However, the negativity is being directed at a perceived discriminatory culture rather than at Michael Sam or the events at the draft. In that way, we might be inclined to classify them as positive rather than negative.

Sentiment analysis clearly presents challenges if you’re interested in this kind of work.

Now, how did I build my corpus? It’s not too difficult, but you’ll need to use R — a programming language that is often used in corpus linguistics. The good news is that you DO NOT need to know how to write code. You just need to follow the steps and make use of resources that have already been developed.

I based my analysis on resources provided by the Information Research and Analysis Lab at the University of North Texas. The link has tutorial videos and an R script that you can download.

You will also need R Studio (which is a programming interface), the opinion lexicon from University of Illinois Chicago, and you will need to sign up with the Twitter API. This last bit may sound technical, but it isn’t really. Twitter doesn’t let unidentified users to grab its data like it used to. You just need to sign in with a Twitter account, click a button to create an App. (You can call it anything; it doesn’t matter.) Then, you are provided keys, a series letters and numbers that gives you access to the Twitter stream. Don’t worry, the video tutorials will walk you through the process.

Finally, a couple of tips if you’ve never seen or worked with code before. First, the videos will have you assemble three corpora about baseball. A snippet of the code looks like this:

Rangers.list <- searchTwitter('#Rangers', n=1000, cainfo="cacert.pem")
Rangers.df = twListToDF(Rangers.list)
write.csv(Rangers.df, file='C:/temp/RangersTweets.csv', row.names=F)

See #Rangers in blue between the single quotes in line 1? That is telling your code to search the Twitter stream for tweets with that hashtag. Once you go through the process and get the hang of it, you can change what’s between the quotation marks to whatever you want. Be aware, however, that you must have something there. That variable is required.

Also, you can’t just run the code without doing anything. You will need to tell R where to find things on your computer and where to put them. This requires defining what is called the “path.” The path just identifies the location of a file. You can see a path in line 3 above between the single quotation marks after “file=”. You can also see it the example below in lines 2 and 3 between the single quotation marks starting with “C:/”.

#Load sentiment word lists
hu.liu.pos = scan('C:/temp/positive-words.txt', what='character', comment.char=';')
hu.liu.neg = scan('C:/temp/negative-words.txt', what='character', comment.char=';')

You are going to have to edit these paths in order to successfully execute the code. Fortunately, this is easy. To find the location of a file on Windows, right-click on the file and select “Properties.” The path is listed there, and I’ve highlighted it in yellow.

On a Mac, control+click on the file and select “Get Information.” Again, I’ve highlighted the path in yellow.

Note that with Windows, the path (as long as the file is on your hard drive) will start with “C:”. That is just the letter designation that Windows assigns to your main drive. On a Mac, this is different. There is no letter.

In either case, you can just copy the path and paste it into the code. The copied path, however, will not contain the name of the file itself. So you will need to type in a backslash (/) and the name of the file: RangersTweets.csv, positive-words.txt, or whatever.

Otherwise, you should be able to just follow along with the tutorial. If something gets messed up, you can re-download the code and start fresh.

Radiolab recently had an interesting interview with Daniel Engber. In it, Engber discusses the rise and fall of quicksand as a device in movies. (You can also read his Slate article on the same subject). His data shows a clear peak in the percentage of movies using quicksand in the 1960s (chart via Radiolab):

Engber suggests that the decline in quicksand in movies coincides with a decline in children’s fear of it. It has lost its allure as an image of terror. I was curious, then, if a similar pattern is evident in language. COCA shows the following distribution for quicksand over time:

The data seem to show an earlier peak (in the 1900s and 1910s), followed by a dip then a steady decline after 1940. One potentially complicating factor, here, is that quicksand is a relatively rare word, which could affect some of the fluctuations. So I thought this would be a good opportunity to try out BYU’s version of Google Book’s data, which Mark Davies discussed at this year’s Studies in the History of the English Language conference. The results are similar, but with a more clearly defined rise and fall:

As with the COCA distribution, the Google data suggest a peak in language use that precedes the peak in movies. The data also show a move toward metaphorization in recent decades. Early in the twentieth century, examples are typically like this one:

There was a ford directly opposite the cantonment, and another, more dangerous, and known to only a few, three miles farther up stream. Keeping well within the water’s edge, so as to thus completely obscure their trail, yet not daring to venture deep for fear of striking quicksand, the plainsman sent his pony struggling forward, until the dim outline of the bank at his right rendered him confident that they had attained the proper point for crossing.

Later, however, we find examples like this one:

Only their husbands believed the kids were their own. He missed his mother, too, whose quicksand love he’d wanted so badly to escape.

Or this one:

It may well be that in responding to recent Congressional language the N.E.A. has begun to have a chilling effect on art in the United States and it may be entering the quicksand of censorship.

Not that quicksand  as a physical entity disappears completely in more recent discourse:

Georgie was not popular in the swamp; in fact, the other members of the colony that inhabited this stretch of muck and quicksand, black water and scum, had banished him to the very fringe of the community.

While such examples exist, they are far fewer than the type in which some physical sensation is compared to moving through quicksand or something (like love or censorship) is compared to the ensnaring effects of quicksand, itself. Thus, the use of quicksand is not only declining, it is also undergoing a shift in meaning.

Which brings us back to our  earlier question: Why does quicksand peak earlier in language than in film? One possibility might be related to the relative durability of particular genres in different media. Adventure stories (like dime novels) had their greatest popularity in the early twentieth century. This genre appears to be an enthusiastic employer of quicksand as a conventional obstacle and threat, and the decline of the genre coincides with the decline of the word. How the rise of quicksand as a cinematic device relates to the rise of particular kinds of movies, I’m not sure, but the relationships among Engber’s data and the linguistic data pose some compelling questions.

On the BBC’s “Click” podcast last week, they discussed evolving voice technologies, and their increasing ability to incorporate complex prosodic and phonological properties like emotional emphasis and accent. It’s an interesting conversation and worth listening to, but the segment that caught my attention touched on Stephen Hawking and the preservation of legacy technologies related to his voice:

A 2001 Deborah Cameron article on “designer voices” provides some interesting context for Hawking’s choices regarding his synthesized voice:

Hawking faces an unusual choice here. It may seem obvious that a British accent would be more ‘authentic’ than the American one he has had to make do with up until now; this is a question of what linguists call ‘social indexicality’, the ability of the speaking voice to point to social characteristics like age, gender, class, ethnicity or, most saliently in this case, national origin. Yet voices are also privileged markers of individual identity. It would not be unreasonable for Hawking to take the view that his synthesised American voice essentially is his voice, in the sense that it is instantly recognisable as the voice of Stephen Hawking (indeed, so recognisable that Hawking can earn money doing advertising voice-overs). Some media reports noted that the scientist was having difficulty deciding whether to make the shift: he was quoted as saying that ‘it will bring a real identity crisis’.

There are a couple of issues here that I think are compelling. The first is one that both the podcast and Cameron note: Hawking’s choice speaks to the complex and powerful connection between our voice and our identity(s). In particular, it suggests that such a relationship can be an evolving one. In his early encounters with his synthesized voice, Hawking experiences a kind of distance from its American-ness. However, he ultimately chooses to retain it in the face of other possibilities. And there is no question how recognizable his voice is, inspiring a raft of amateur and professional parodies.

The other interesting issue is what that choice has meant technically. It has necessitated the preservation of legacy technologies in order maintain a powerful signifier of self. Which makes me wonder. As technologies continue to emerge that connect those technologies to our biologies, our identities and the signals we use to advertise those identities will change, too. Those inter-related changes, technological and indexical, may not move at the same pace, however. What might this mean? Could a voice, an identity, face a forced obsolescence?