Algorithm I: That Time Netflix Betrayed Me
To fully understand The Algorithm, we need to do some math.
For a long time, people who liked movies, upon hearing that I had a Film degree, would ask me my favorite movie. The implication (though it seems strange now, in a media-saturated culture) was that my favorite would carry more weight—that the discriminating taste granted me by my education would give me an affinity for movies that were of a higher caliber. Now, I have already expounded, in a previous post, on my tendency to separate the metrics of entertainment and quality—but I took this commission seriously, and allowed the blurring of that distinction just enough that I was always prepared to meet this expectation.
Objectives
This post is going to be the first in a series, about how AI is not a novel phenomenon ruining narratives—no, it is merely the culmination of factors that have been eroding our storytelling capabilities for many years now. The technology of LLMs, after all, is powered by the technology of The Algorithm, which, as I see it, has been a pervasive force within the media ecosystem, flattening not only the stories produced, but the ways in which they are disseminated and consumed and understood. The emergence of AI in the past few years, and the conversations it has produced, has had the remarkable effect of uniting many of my pet theories under a single, common umbrella—so this series will examine each of those theories within that context.
It seems only fitting for me to begin this series with the beginning of my own journey into despair—a journey that began in 2017, when Netflix betrayed me.
A Millennium of Movies
I have seen 1,000 movies. Literally. Perhaps you have, too—it’s not as impressive as it was fifteen years ago, when I started tracking (but thankfully I was around 800 when I started tracking, so you can still be impressed). I also have a lifelong obsession with lists, and I was in my late teens when it occurred to me that I should preserve some sort of catalog of all the movies I’d seen, just to know. Just to see.
I was also an early adopter of Netflix—or, rather, my family was, as I was not paying for DVD rentals. Around the time I started college, as the service (presumably) began preparing for its streaming transition, they rolled out an option to rate movies you had seen (including the rare and coveted options for Haven’t Seen It and Don’t Want To See It1). In addition to movies that were part of Netflix’s catalog, they had listings for movies that were not available through Netflix—listings, it seemed, for almost every movie you could think of. How do I know? I searched for those movies and rated them, and there were only a handful of times the search results came up empty.
In the course of my college education, I changed my major from English to Film—so I was starting to think more critically about Film and to try to build some sort of internal rubric for criticism. One Summer during college, I spent days on the Netflix website, going through pages of movie listings, adding ratings. Along the way, I was also building a list of all the movies I had seen in my lifetime, scouring every corner of my memory, to make sure my list was comprehensive. Once I completed this work, I continued to add new movies, dutifully updating my Netflix ratings every few months.
In the end, Netflix was the single source of truth for my entire history of viewing film, and the opinions I had formed as I built my critical framework.
Who Rates the Ratings?
As I write this, I am researching the events I describe—but before I check my facts, I want to recount the events as I remember them.
Some time around 2017, Netflix noticed that people weren’t watching the movies they rated most highly. As soon as I saw this, it made sense: my favorite movies at the time were Casablanca (which I think I have still seen less than ten times) and Million Dollar Baby—neither of which I had watched more than a dozen times. By contrast, I had racked up more viewings of The Lord of the Rings than I could count, along with other such notables as Titanic, How to Lose a Guy in 10 Days, and Brick. As I stated in my intro, those two favorite movies were chosen, in part, because they ranked highly in my personal, internal ratings system: 9.8 and 10.0 (out of 10), respectively. Titanic, I would rate somewhere around the 7.0 mark (7.2, maybe?—I’ve already discussed in this blog its fatal flaw of weak writing), and I would give a similar score to How to Lose a Guy in 10 Days, with something a little higher for Brick—8.5 or 8.7. Those are still pretty good scores, but you would be hard pressed to extrapolate from them an equation to predict whether I would watch something, based solely on rating.
pauses to attempt with a larger dataset…double-checks movie spreadsheet…struggles to find movies with high view counts and low ratings…
So it turns out that I don’t really rewatch truly terrible movies (<4.0/10). At the time, I would’ve held up Win a Date with Tad Hamilton! as my example—but it’s actually not THAT bad (and my view count fell off sharply as that phase of my life passed). I don’t know what to tell you; I like good movies. And, as usual, that’s another post.
Okay. Having done the pitifully-sampled spreadsheet and made a lil graph, I can conclude only that Netflix was right: my view count does go down at the higher end of the ratings scale. Looking over the data, in fact, there are patterns—but they would require, I think, a specialized algorithm to understand and utilize.
And Netflix can’t be making a whole algorithm for every user, i gUeSs.2
I do believe it’s worth noting that the ratings scale Netflix presented was not actually one of quality: the scale was entirely subjective, with options Loved It, Really Liked it, It Was Okay, Didn’t Like it, and Hated It (plus the essential aforementioned buckets to describe a lack of rating). Now, I don’t assume every person ran every movie they rated through this rubric—in fact, I recall reading, at the time, that a lack of user discrimination motivated Netflix’s decision, at least in part. The company wanted to be able to predict what other selections in its library users were likely to watch, and those predictions were harder to make when viewing habits were obfuscated by users’ own cognitive dissonance.
Case 2: Rotten Tomatoes
Having now done some research, I cannot say definitively why Netflix did this—I found three sources with three explanations from the company3—, and, frankly, I don’t really care. I will also go ahead and set aside my belief that they discarded all of the data I had given them through years of meticulous ratings. I can’t prove it, but that was the betrayal. It happened. It’s over.
I want to talk instead about why this was the worst possible solution for whichever of their propounded goals they were actually trying to achieve. And to do that…I need to talk about a completely different ratings service.
In 2018 I saw Robin Hood in the theater with my family. It was pretty much a 5 movie, not really good, not notably bad—a movie-shaped movie, with a coherent story and decent actors doing their thing—, but I would knock off an extra .5 because it was entertaining in its badness, and 5.0 means something specific to me (to wit, that a movie is wholly unremarkable and unmemorable—a 5.0 movie is almost entirely neutral, which is the worst thing a movie can be). The whole family enjoyed the movie. It wasn’t good, but we enjoyed it.
Then I looked up the movie’s score on Rotten Tomatoes—and this was when I stopped using Rotten Tomatoes. Robin Hood currently has a 14% critics’ score (which is, if memory serves, the same or similar to what it had at the time, seven years ago). This gave me tremendous pause: as I said, the movie wasn’t good—but it was nowhere near as bad as a 1.4/10.
Okay, so Rotten Tomatoes scores don’t map to quality ratings. Fine. But this was a medieval action popcorn flick starring a young actor whom many people find ridiculously handsome. It had, as I said, a coherent plot. You might say it was fluffy. Would 86% of viewers not want to see a movie like that?
Well, no—the audience score on the site is much higher, at 40% (closer to my own evaluation). I dislike this distinction, however, because I think it reinforces the idea that critical viewing is an elitist exercise—whereas I believe that the qualities that make a movie good also make it enjoyable. So rather than drawing potentially-distorted conclusions from the discrepancy, I want to dig into what a Rotten Tomatoes rating actually means.
It’s All Binary Now
Per their About page, this is how Rotten Tomatoes works:
“Tomatometer scores are based on the collective opinions of hundreds of film and television critics and serve as a trusted measurement of critical entertainment recommendation for millions of fans. The Tomatometer score represents the percentage of professional critic reviews that are positive for a given film or television show.”
I find it notable that there is no copy on the same page clarifying what makes a review “positive”, but I have a suspicion, which I think is confirmed by their Popcornmeter (audience rating) guidelines, listed on the same page:
“When at least 60% of users give a movie or TV show a star rating of 3.5 or higher, a full popcorn bucket is displayed.”
The above quote is referencing a ratings scale (displayed on each movie/show page) out of five stars. 3.5/5 = .7, or 70%. So already, we have a binary scale where the weight is not evenly distributed between both of its options.
Let’s dig into how this math is mathing. A 7.0 movie on the Dashcomma scale is a good movie—it is not particularly good, but it has a coherent plot, solid dialogue, competent production, and probably some light theming. In college, it’s a passing grade (although in college, the lower ends of the range aren’t really valid—if you got a 16% on that physics exam, you should drop the class before that deadline at the end of the week4). This seems more appropriate when you consider that many reviewers give academic letter scores—although less when you consider that ‘F’ is almost never used, so the scale would need to be adjusted (again, Rotten Tomatoes provides no insight into those calculations).
I invite you to think of a movie that is not good, but also not actively bad—something in the 5.1-6.9 range, out of 10. If your opinion of that movie is objectively true—if everybody shares that opinion—, every critic is giving that movie a grade of 5.1-6.9…which is not a positive review, if my assumptions are correct about Rotten Tomatoes’s system. That movie gets a 0% score on Rotten Tomatoes.
If you already knew this was how it worked—if you had already internalized that logic and have been making decisions accordingly—, congratulations on your ability to parse nuance. I did not understand it. Sure, I knew the scores were not one-to-one (The Dark Knight has severe story issues, y’all)—but I had not fully thought through how they actually mapped to movie quality. With this new understanding, I find Rotten Tomatoes functionally useless as a predictive model.
You see, the decision point for a user, employing this model, is just another question: a Tomatometer rating does not answer, “How good is this movie?” or even, “What did critics think of this movie?”—rather, it forces you to ask yourself, “How likely am I to agree with the majority of critics?” The model suffers not from a binary output, but from binary inputs—a flattening of all critical frameworks into the one metric of a positive review (yet again, whatever the fuck that means). A thumbs-up, if you will. Much like the one Netflix replaced its ratings system with.
The Algorithm
And it makes sense—because that’s what The Algorithm is. In the overlapping Internet biomes of streaming, posting, and reviewing, The Algorithm exists to match content with behaviors—to predict which behaviors lead to other behaviors, which content should follow which content, to keep the user engaged.
Even an individualized algorithm suffers from this flattening: if Netflix predicts I will only watch the best movies once or twice, they’re not incentivized to suggest the best movies to me, over the middling movies it predicts I will watch many times. Netflix just wants me to do more Netflix. It doesn’t care what I might want. There is no reward for nuance.
When I was a college Senior, I conceived one of my Pet Theories: The Death of Subtlety, which I later renamed to The Death of Nuance. If you have ever had (or seen) an Internet argument where the other party summarizes your position with an overbroad accusation (often about Nazis), you may instinctively understand what I mean; I’ll explain for everyone else. I saw nuance disappearing everywhere—from our discourse, our storytelling, our criticism, our science—truly, everywhere. I myself was accused of thinking Hurricane Katrina was good, because I posited that catastrophe could be a catalyst for positive social change (it was an argument in a seminar about revolution).
That was fifteen years ago—as you can imagine, the problem has since become a pervasive one. And I think I can point to The Algorithm as an accelerant, flattening discourse, narrative, and analysis in the dual ages of streaming and social media. Even when what you are consuming is criticism, the algorithm is feeding, offering you analyses that will keep you engaged—hot takes that inflame and reinforce, rather than nuanced breakdowns that edify or challenge.
And, because these different forms of content coexist in a single algorithm-driven ecosystem, what you consume affects the very nature of the stories you’re told.
But that’s next week’s post.
Someone working in accounts for a Saas company once told me that the worst thing she could receive in her inbox was a negative review with no comments—because a low rating, in itself, can mean so many things. Separating out the two metrics of Haven’t Seen and Not Interested is crucial because the former is a genuine lack of data, while the latter is a data point if you are measuring interest. Haven’t Seen was also useful to me as a reviewer, because it told Netflix to stop showing me the movie for rating—you have to understand that I wasn’t going through the Netflix library, finding titles to rate; I was scrolling through pages of movies, tiled across the screen, adding ratings in bulk.
I found a contemporaneous claim that they were actually doing just that, which makes this whole saga even more frustrating.
A press release from Netflix (April 2017), this article from Business Insider (April 2017), and the half of this Atlantic article that was in front of the paywall (March 2017).
Yes, it’s a real example.