Courtney Ferguson Lee

This project wrangled location data from the OpenStreetMap project, then used data munging techniques to assess the quality of the data for validity, accuracy, completeness, consistency, and uniformity. The data consisted of XML files which were parsed using BeautifulSoup and regular expressions. After cleaning, XML files were converted to a relational SQL database for further querying and visualization.

This project identified which Enron employees were more likely to have committed fraud from public Enron financial and email data using machine learning. The dataset consisted of 146 observations across 21 features and a POI status label. Due to the unbalanced nature of the dataset, precision and recall were the primary metrics used to evaluate each classifier.

This project uses R to explore the variables, structure, patterns, oddities, and underlying relationships of New York City Uber pickups from January through June of 2015. The dataset consisted of 14 million observations spread across 12 variables. Exploratory analysis revealed the cyclical nature of rider behavior and that population density, not population size contributed most to trip frequency.

This project predicts the winner of a UFC fight with 88% accuracy. All data was scraped from fightmetric.com using BeautifulSoup, generating a total of 32 features and 8772 observations. Multiple algorithms are tested, including Perceptron, Random Forests, Decision Trees, SGD, Linear SVC, Gaussian Naive Bayes and K-Nearest Neighbors. It was a lot of fun putting this project together and development will continue over time.

This project communicates some of the main takeaways from the 2012 PISA assessment. Data is cleaned in Python and visualized in Tableau to explore gender, school and student home life differences. Students from 62 countries are represented in the dataset, which was comprised of 512,000 records and 636 features.

This project processes synthetic aperture radar images in Python and trains a machine learning model to distinguish icebergs from ships. The dataset was a JSON file consisting of 1604 observations. Each observation contained two flattened polarized images, an id, the incident angle used to capture the image, and a label. Averaging input vectors from each of the horizontal and vertical polarized image bands led to 75% accuracy.

This project uses natural language processing to analyze how twitter users felt towards Louis C.K. in the week following his sexual harrassment allegations. Data was queried through the Twitter API and analyzed in the cloud through the Google Cloud Natural Language API. The notebook below describes the process used to clean and aggregate the results and the Tableau link visualizes the data in a dashboard.

This notebook tutorial demonstrates how to use natural language processing to identify authors from text excerpts. The dataset is comprised of 19,579 records and 3 variables: id, text and author. Stemming, word counts and lemmatization are used to engineer new features and a variety of classifiers are tested. Multinomial naive bayes yielded the highest results, accurately identifying the author 85% of the time.