CAPSTONE: THE MOVIE

This is the capstone project I did for General Assembly. Below are two presentations in one. One is technical, the other is nontechnical. Please note that every headline and slide title, with the exception of one or two, is a quote from a movie.

No Fate But What We Make

There is an ongoing, pernicious problem in the world: people have to wait for a movie to come out to know how good it is.

This is completely unacceptable.

Sure, we can wait for reviews to come in, or, better yet, wait for aggregates of reviews to come in on sites such as Metacritic or Rotten Tomatoes, but wouldn’t it be great to know that the upcoming superhero movie will disappoint you before you waste time getting excited for the trailers?

Movies are not black boxes (except for The Black Box (2005) and Black Box (2012)). They’re produced by people who have produced other movies, written by people who have written other movies, performed in by people who have performed in other movies, and directed by people who have directed other movies. Given that people who work in movies tend to be consistent in their quality of output (see: Michael Bay), we should be able to predict qualities of a movie with only the knowledge of how past movies have performed.

It’s easy enough to say that the new Jared Leto Joker movie will be awful. It will be. Every DC movie has been awful except for one, and Suicide Squad, the only other movie where Jared Leto played the Joker, was truly a crime against humanity. It’s easy enough to say that, but can we teach a computer to say it?

Some Fellas Collect Stamps

The first step in my capstone project to collect a large database of films was to first try and get a list of every movie I could. I figured that Wikipedia would have as many movies as I would need, and if a movie wasn’t on wikipedia, it was also unlikely to be one that provided me with any useful information in regards to Metascore. So, I used a Wikipedia Python package.

It was after this painstaking process that I found a huge list of movies on the site movielens (https://grouplens.org/datasets/movielens/latest/). However, instead of deciding to give up my life as a data scientist and moving to the woods, I noticed that this list was only updated as of August 2017, so I knew I had more movies from my scrape. Thus, I download the .csv then extracted titles and years from it.

Finally, I used a website that collects data from Metacritic, Rotten Tomatoes, IMDB, and a couple others and allows you to search through it. The API provided the following information:

– Actors
– Awards
– BoxOffice
– Country
– DVD
– Director
– Genre
– Language
– Metascore
– Plot
– Poster
– Production
– Rated
– Ratings
– Released
– Response
– Runtime
– Title
– Type
– Website
– Writer
– Year
– imdbID
– imdbRating
– imdbVotes

I didn’t take “Awards,” “DVD,” “Plot,” or “imdbVotes,” because all of those attributes are things you will never have access to before a movie comes out. I kept the rating values to use as my target variable.

I also didn’t take “Country,” “Language,” “Poster,” “Response,” “Type,” or “Website,” because none of these things gave any valuable information. Perhaps country or language would be somewhat illuminating, and I may take them at a future date.

My main issues with this API were that it restricted actors to the top four billed, had no other crew (Cinematographer? Hello? Composer?) and also didn’t have things like opening weekend box office, budget, months in production, whether the movie is part of a franchise, etc. There are other databases with this type of information and I plan to access those in the future.

All Clean. All Clean.

I collected quite a bit of data: 43837 separate movies. The actual cleaning of the data was as tedious and dry as the following paragraphs.

I wrote a bunch of functions to clean the data, even a few I didn’t need. This functions did things like turn data to floats or create new columns of data. Because of the size of the database, I didn’t spend much time imputing missing values. Because I were targeting Metascore, I went ahead and dropped all movies that didn’t have an associated Metascore, giving us a new DataFrame with 10192 rows. I also got rid of the other columns that weren’t going into this project.

After running a few models that performed poorly, I did a little feature engineering. I sought to weight the people involved in a movie by aggregating over their Metascores. To do this, I created three DataFrames where I isolated the directors, actors, and writers.

I pulled out my list of actors by using a count vectorizer on my features to get lists of columns and aggregate over those lists. I found every director, actor, and writer’s mean Metascore.

Finally, I saved a few versions of the DataFrame. One with directors weighted, one with directors and actors weighted, and one with directors, actors, and writers weighted. Then for each of those, I saved a version that was count vectorized only if a term appeared twice or if a term appeared three times. In total, six DataFrames to see which models the best.

Please Excuse the Crudity of This Model

For modeling, I took the practice of throwing everything at the wall and seeing what worked. I imported many different models, including linear regression, lasso, SGD regressor, bagging regressor, random forest regressor, SVR, and adaboost regressor, as well as classifiers including logistic regression, random forest classifier, adaboost classifier, k-nearest neighbors classifier, decision tree classifier, and even a neural network.

I then fed the dataframes through the following cell, which gave me three regressor scores, then transformed my y variable for classification (based on median Metacritic score) and fed that through three classifiers. Throughout this process, many models were attempted and thrown out. Dataframes were changed and had to be saved again and reloaded. At the end of the day I decided on the following models:

– Regression
– Bagging Regressor
– Random Forest Regressor
– LASSO
– Classification
– Logistic Regression
– Bagging Classifier
– Random Forest Classifier

Except for LASSO and logistic regression, there wasn’t much rhyme or reason for modeling choices. These just gave me the best relative scores (of the ones I tried), and also didn’t take a huge amount of time. Also, the bagging regressor and classifier, which didn’t seem to ever give me scores that were as good as the other models, still worked quickly and served as a veritable canary in a coal mine, warning me if something had gone wrong with the models.

My best classifier (logreg) accuracy was

.6945555555555556

using C = 10 with an l1 penalty.

And my best regression R2 score was

0.21850824761335874

with an alpha = .15

There is no reason I shouldn’t be able to achieve better than this given more time in the future.

Next Time It’ll Be Flawless

Future recommendations are numerous. There are many different ways possible to make this score better, the only constraint being time.

In terms of data collection, there are several other large databases to access, including imdb’s itself as well as Metacritic’s. It is entirely possible I have all the Metacritic scores, but I could always use more. Plus, Metacritic has statistics such as whether the movie is part of a franchise and how well the previous film did. I can, of course, make that data myself, but again, time is a factor here.

I would also like access to more of the cast and crew including producers, cinematographers, composers, editors, and more of the cast. After all, the theory underlying this entire endeavor is that people make movies and people are consistent in their product.

I could impute null values, especially with things like box office revenue, opening weekend box office revenue, Rotten Tomatoes scores, which could all replace Metacritic scores as the target variable. It would then be a simple mapping from one to the other. There could easily be more Rotten Tomatoes scores than Metacritic.

In terms of feature engineering, there are always more columns to make. I could use polynomial features on my numerical data. I could just use directors and writers. I could run more n-grams on the titles. I could change my min_dfs per column. I could sift down my list of actor weights. I could go back and try to get the actors’ averages like before.

Finally, there are more models for me to use. Several will allow me to tune hyperparameters to eek out better scores. There are models that work better with NLP. I can try a neural network for both classification and regression. I can try a passive aggressive classifier. And I’ll do all that and I’ll predict movie scores and eventually, they’ll make a movie about me.

What follows are the actual notebooks of the project you can find on my repository.

This entails a project I did in my data science bootcamp at General Assembly. This particular post will be heavy in the coding department and light in the writing department, but will give you a little glimpse into how I was thinking early on in the program. First I’m going to show you the notebook I used, followed up with the tongue-in-cheek presentation I made. I put the presentation last as it is not the most professional work, but I liked it.

And here is the presentation:

Here is the first project given to us at the General Assembly data science immersion program. We were given two databases of 2017 SAT and ACT average scores by state. We were given detailed tasks involving cleaning and exploring the data, and then told to make presentations to the College Board to suggest ways of increasing participation. Here is the presentation I made with my suggestions, followed by the notebook I used. You can check out the original repository here.

‘The ideal Evan Jacobs—the brash, confident stud I still want to be—existed for nine months of my sophomore year at the University of Chicago.’

 

 
It’s hard to go through life without regrets. Sometimes a decision that seemed perfectly appropriate at the time turns out to be a mistake. It’s happened many times in history: the Bush presidency, the XFL, Prohibition. For me, I realized that I should have been dating my girlfriend’s best friend a few weeks after I had become exclusive with my girlfriend.

Since college, my lack of confidence has betrayed me several times. I’ve acted too clingy with girls, I’ve tried too hard, and I’ve ruled out women just because they were attractive. The ideal Evan Jacobs—the brash, bold, confident stud I still want to be—existed for nine months of my sophomore year at the University of Chicago.

In September 2000 I lived in a dorm called Broadview, an architecturally beautiful building that had previously been a high-class hotel in the ’20s. All the rooms had their own bathrooms, and most were singles, though there were a few rooms with roommates. The lobby was large and inviting, and all the rooms had views of Lake Michigan, the Midway Plaisance, or the Chicago skyline.

Because I already had a year of college under my belt, I felt confident among the wide-eyed freshman who populated the majority of the dorms in broadview. They were a batch of children. One freshman in particular caught my eye. She was tiny, blonde, with a pointy nose and chin and large blue eyes. As soon as I saw her, I wrote her off as a girl that would scoff at any association with me. After all, she was hot—she wasn’t just cute or pretty. And damn, did she know it.

 

Perhaps it came from years of rejection and snubbing from the most attractive girls, but at the time, I had taken to heart this idea of “leagues.” Is she out of my league? Should I only date within my own league? To be frank, the idea of “leagues” was frustrating beyond the fact that it was a sports analogy. I wanted to prove it wrong. I wanted to overthrow it. Just because a girl was more empirically attractive didn’t mean that she shouldn’t date me.

Convincing her I was a worthwhile pursuit—that was another matter. This blonde girl was decidedly out of my league. When I ran into her in the elevator one day early on in the year, I made a bunch of rapid-fire jokes. I spoke to her in the way I speak to many people, to enjoy the attention and not expect it to lead anywhere. She laughed at everything I said, but it meant nothing. Plenty of people who would never speak to me otherwise, let alone date me, have laughed at things I said.

In truth, I don’t really remember what I said to her, or how it happened, or what the details were, but I do know that I offered to help her with her chemistry homework a couple days after the elevator incident. The offer turned into our talking through the night and into next morning, neither of us sleeping. As a crazy coincidence, we discovered that we were neighbors. I lived in the last door of a hallway, she lived in the second to last. I’m not sure if we kissed that night or if it was the next night, but before I knew it, we were together, naked in my bed.

♦◊♦

The world was inside out. Down was up. The sun revolved around the earth. She was mine. She liked me. A girl who was too attractive to be with a schlub like me had willingly joined me in a relationship that defied expectation and logic.

I soon learned that Christine was far from perfect. She was from a small town in Ohio and had grown up very poor in a family that didn’t place the highest value in education. She was the first member of her family to go to an institution with anything approaching the academic rigor of the University of Chicago. This made her insecure; she feared being stupid to the point of terror.

She was very kind, very nice, and although she was very intelligent, I found that there was something missing. She didn’t have the academic thirst that I had. She lacked the same cynicism, skepticism, and curiosity. Even a couple months into the relationship, though I was enjoying spending time with her, I had determined that I probably was not going to fall in love with her. The relationship was doomed to fail.

♦◊♦

Christine had three very good friends in Broadview. One of them, Kathy, was a skinny Polish girl with squinty eyes and a long nose. She didn’t have Christine’s sex appeal, but she was undeniably cute—and she had one huge draw that I found myself not only admiring, but lusting after: Kathy was majoring in physics and math.

Science has always been my true mistress. It has an inherent beauty in its absolutism; it shirks sentimentality and pretension; it embraces the universe beyond the insignificant dramas of everyday life; it explains reality and existence more than religion or philosophy ever has. It should be no surprise, then, that I majored in mathematics myself. To find a girl who was fascinated with math and physics was a twofold boon: 1) a girl who pursued these areas of study had similar interests to mine, and thus would almost surely understand my worldview; and 2) science and math, as academic fields, are male-dominated—so the women in the field struck me as brave, exceedingly intelligent, ambitious, and far above shallow pursuits, like obsessing over makeup or dancing to bad music—i.e., things Christine loved.

While Kathy’s looks made her cute, her interest in science made her hot. And her own insecurities fueled my lust even more. She was the inverse of Christine: confident in her intelligence, but unsure of her looks and social skills. And, as every socially awkward male knows, the only thing more attractive than a hot girl is a hot girl who has no idea she’s hot. Thus, even within a few weeks of dating Christine, I had fallen in love with Kathy.

I continued to date Christine. There was no obvious or overt reason to break up with her, so I decided to stay with it. At the same time, I would talk with Kathy every time I saw her, and even started visiting her in the lounge when I knew she was there studying.

♦◊♦

At some point, Christine screwed up. After the winter break, while Christine and I were in the middle of making out and reuniting, she began to cry. She had to tell me something, and I already had an idea what it was. Every time she went home, Christine would hang out with a group of local police officers. It was innocent, she said. They were her friends. They would get drunk with her and even smoke weed with her, but it was all in good fun.

I am far from stupid. They were men and she was an attractive female. There’s no such thing as innocence in those situations. Over the break, Christine had cheated on me with one of the police officers, and to make it worse, they had had sex in the back seat of his police cruiser.

I forgave her. Who was I to be angry? I didn’t love Christine. Sometimes I felt guilt for even continuing the relationship. Only now do I realize that this incident may have been her way of testing how I felt about her. If I truly loved her, I would have been much more upset. If I didn’t, I would have forgiven her. She had gotten her answer.

♦◊♦

It was only about a month later when Christine told me that she loved me, and I was put in the awkward position of not being able to reciprocate. I still don’t know how I managed to stammer through that one. I told her that I enjoyed that she loved me, and that one day I hoped to love her, but I just wasn’t sure, at that moment, if I was able to feel the same way about her, and I didn’t want to lie.

Though she said it was OK, two weeks later we broke up. I didn’t mind. All I could think about was Kathy.

To be sure, I felt somewhat guilty that I might hurt Christine’s feelings. But she had cheated on me; I’d done nothing wrong. I put Christine out of my mind. Nothing was going to stop me from advancing on the girl I knew I wanted.

♦◊♦

Over the next few weeks, I thrust myself into Kathy’s life. I asked her to hang out, although she seldom did. I studied next to her. I told her about what was going on with Christine. This was my plan. First I would become friends with her. Then I would tell her how I felt. Then I would date her. Then we would get married. Then we would have children. Then we would both die and be buried next to each other. It seemed foolproof, and for the most part, it was.

The confidence I had gained from dating Christine had altered my personality. No longer was I a loser with low self-esteem. I had become suave, charming, and enticing. I felt no concern that Kathy would resist my advances.

It took no summoning of courage to ask Kathy out—the confidence was already there. About a month after I had broken up with Christine, I was in the study lounge with Kathy, and I told her that there was something I wanted to talk to her about. I apologized for what I was about to say, and then I told her I wanted her—I had wanted her ever since the moment I met her.

“I don’t care about Christine and you shouldn’t either. You should let me take you out on a date.”

Sure enough, she agreed.

♦◊♦

I began holding her hand when we were alone, getting her used to touching me. I hugged her often. I took her out to dinner. I hung out with her in my room. Eventually, when I kissed her, I had no worries that she would give me her cheek or pull her head away.

Christine actually happened to live in the next dorm room over from mine, and I had to pass her room every time I went anywhere. I ran into her multiple times after the breakup while I was courting and later dating Kathy. I actually told her that I was dating Kathy, but she already knew—Kathy had told her much earlier on.

It wasn’t that awkward. Christine gave us her blessing, and started dating other men as well. She and Kathy remained friends. Maybe her guilt over cheating on me mollified any anger she might have (quite reasonably) felt toward us. Whatever it was, I had nimbly skipped my way through the minefield of dating two friends.

The thrilling thing wasn’t that I ransacked a clique one by one—it was the fact that yet again, the girl I wanted became my girlfriend. And for at least a month, dating Kathy was amazing. We would go to dinner, talk about physics, see movies at the campus theater, get coffee, meet each other on campus, etc. It was the ideal college romance. We even dabbled in marijuana. Of course, we also kissed each other and slept together—although therein lay some difficulty.

Whereas Christine had liked me too much, Kathy was more aloof. She never complimented me, never told me how cute I was, never told me how much she liked me. She didn’t try to hold my hand, she didn’t caress my back, she didn’t try to kiss me in public. When we did kiss, it was like I was introducing her to the concept of what kissing was.

She just wasn’t very affectionate. She wasn’t bursting to tell me that she loved me. She didn’t seem to understand how awesome I was, and in turn, I started feeling less awesome.

Kathy was shy. It wasn’t her fault. But I found my newfound confidence waning, and I flipped out.

If she didn’t think I was that great, shouldn’t she be with a guy who was? Someone she liked as much as I liked her? Someone she would want to be affectionate with?

So I did one of the toughest, stupidest things I’ve ever done: I broke up with her.

♦◊♦

So what if I thought I wasn’t good enough for her? So what if I thought she didn’t like me enough? She had agreed to be with me and that should have been good enough. If she had wanted to break up with me, she would have done so. But my newfound confidence had failed me. I was back to square one—I became the same neurotic, anxious, awkward, desperate jerk that I had ever been. And so she got away.

Like many mistakes in my life, I should’ve learned from it but didn’t. I second-guessed myself—and instead of vowing to trust my instincts in the future, I kept on doubting. I doubted my way through relationship after failed relationship. And it’s only now, 10 years later, reliving this disaster of a romance, that I’m figuring out where it all went wrong.

If I could time-travel and meet my 19-year-old post-Kathy self, I’d say this: You deserved her, dude. Fuck low self-esteem. The next time you meet a girl you really like—and trust me, there’ll be plenty—don’t give up on her. Because even if confidence can be a fickle mistress, regret will break your heart every time.

—Read Jacobs’s “Chronicles of a Phone Sex Addict” here.

—Photo quinn.anya/Flickr

 Reprinted from The Good Men Project