applied-ml

Curated papers, articles, and blogs on data science & machine learning in production. ⚙️

Figuring out how to implement your ML project? Learn how other organizations did it:

• How the problem is framed 🔎(e.g., personalization as recsys vs. search vs. sequences)
• What machine learning techniques worked ✅ (and sometimes, what didn't ❌)
• Why it works, the science behind it with research, literature, and references 📂
• What real-world results were achieved (so you can better assess ROI ⏰💰📈)

P.S., Want a summary of ML advancements? 👉ml-surveys

P.P.S, Looking for guides and interviews on applying ML? 👉applyingML

Table of Contents

1. Data Quality 2. Data Engineering 3. Data Discovery 4. Feature Stores 5. Classification 6. Regression 7. Forecasting 8. Recommendation 9. Search & Ranking 10. Embeddings 11. Natural Language Processing 12. Sequence Modelling 13. Computer Vision 14. Reinforcement Learning 15. Anomaly Detection 16. Graph 17. Optimization 18. Information Extraction 19. Weak Supervision 20. Generation 21. Audio 22. Privacy-Preserving Machine Learning 23. Validation and A/B Testing 24. Model Management 25. Efficiency 26. Ethics 27. Infra 28. MLOps Platforms 29. Practices 30. Team Structure 31. Fails

Data Quality

1. Reliable and Scalable Data Ingestion at Airbnb Airbnb 2016 2. Monitoring Data Quality at Scale with Statistical Modeling Uber 2017 3. Data Management Challenges in Production Machine Learning (Paper) Google 2017 4. Automating Large-Scale Data Quality Verification (Paper)Amazon 2018 5. Meet Hodor — Gojek’s Upstream Data Quality Tool Gojek 2019 6. Data Validation for Machine Learning (Paper) Google 2019 6. An Approach to Data Quality for Netflix Personalization Systems Netflix 2020 7. Improving Accuracy By Certainty Estimation of Human Decisions, Labels, and Raters (Paper) Facebook 2020

Data Engineering

1. Zipline: Airbnb’s Machine Learning Data Management Platform Airbnb 2018 2. Sputnik: Airbnb’s Apache Spark Framework for Data Engineering Airbnb 2020 3. Unbundling Data Science Workflows with Metaflow and AWS Step Functions Netflix 2020 4. How DoorDash is Scaling its Data Platform to Delight Customers and Meet Growing Demand DoorDash 2020 5. Revolutionizing Money Movements at Scale with Strong Data Consistency Uber 2020 6. Zipline - A Declarative Feature Engineering Framework Airbnb 2020 7. Automating Data Protection at Scale, Part 1 (Part 2) Airbnb 2021 8. Real-time Data Infrastructure at Uber Uber 2021 9. Introducing Fabricator: A Declarative Feature Engineering Framework DoorDash 2022 10. Functions & DAGs: introducing Hamilton, a microframework for dataframe generation Stitch Fix 2021 11. Optimizing Pinterest’s Data Ingestion Stack: Findings and Learnings Pinterest 2022 12. Lessons Learned From Running Apache Airflow at Scale Shopify 2022 13. Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training Meta 2022 14. Data Mesh — A Data Movement and Processing Platform @ Netflix Netflix 2022 15. Building Scalable Real Time Event Processing with Kafka and Flink DoorDash 2022

Data Discovery

1. Apache Atlas: Data Goverance and Metadata Framework for Hadoop (Code) Apache 2. Collect, Aggregate, and Visualize a Data Ecosystem's Metadata (Code) WeWork 3. Discovery and Consumption of Analytics Data at Twitter Twitter 2016 4. Democratizing Data at Airbnb Airbnb 2017 5. Databook: Turning Big Data into Knowledge with Metadata at Uber Uber 2018 6. Metacat: Making Big Data Discoverable and Meaningful at Netflix (Code) Netflix 2018 7. Amundsen — Lyft’s Data Discovery & Metadata Engine Lyft 2019 8. Open Sourcing Amundsen: A Data Discovery And Metadata Platform (Code) Lyft 2019 9. DataHub: A Generalized Metadata Search & Discovery Tool (Code) LinkedIn 2019 10. Amundsen: One Year Later Lyft 2020 11. Using Amundsen to Support User Privacy via Metadata Collection at Square Square 2020 12. Turning Metadata Into Insights with Databook Uber 2020 13. DataHub: Popular Metadata Architectures Explained LinkedIn 2020 14. How We Improved Data Discovery for Data Scientists at Spotify Spotify 2020 15. How We’re Solving Data Discovery Challenges at Shopify Shopify 2020 16. Nemo: Data discovery at Facebook Facebook 2020 17. Exploring Data @ Netflix (Code) Netflix 2021

Feature Stores

1. Distributed Time Travel for Feature Generation Netflix 2016 2. Building the Activity Graph, Part 2 (Feature Storage Section) LinkedIn 2017 3. Fact Store at Scale for Netflix Recommendations Netflix 2018 4. Zipline: Airbnb’s Machine Learning Data Management Platform Airbnb 2018 5. Feature Store: The missing data layer for Machine Learning pipelines? Hopsworks 2018 6. Introducing Feast: An Open Source Feature Store for Machine Learning (Code) Gojek 2019 7. Michelangelo Palette: A Feature Engineering Platform at Uber Uber 2019 8. The Architecture That Powers Twitter's Feature Store Twitter 2019 9. Accelerating Machine Learning with the Feature Store Service Condé Nast 2019 10. Feast: Bridging ML Models and Data Gojek 2020 11. Building a Scalable ML Feature Store with Redis, Binary Serialization, and Compression DoorDash 2020 12. Rapid Experimentation Through Standardization: Typed AI features for LinkedIn’s Feed LinkedIn 2020 13. Building a Feature Store Monzo Bank 2020 14. Butterfree: A Spark-based Framework for Feature Store Building (Code) QuintoAndar 2020 15. Building Riviera: A Declarative Real-Time Feature Engineering Framework DoorDash 2021 16. Optimal Feature Discovery: Better, Leaner Machine Learning Models Through Information Theory Uber 2021 17. ML Feature Serving Infrastructure at Lyft Lyft 2021 18. Near real-time features for near real-time personalization LinkedIn 2022 19. Building the Model Behind DoorDash’s Expansive Merchant Selection DoorDash 2022 20. Open sourcing Feathr – LinkedIn’s feature store for productive machine learning LinkedIn 2022 21. Evolution of ML Fact Store Netflix 2022 22. Developing scalable feature engineering DAGs Metaflow + Hamilton via Outerbounds 2022 23. Feature Store Design at Constructor Constructor.io 2023

Classification

1. Prediction of Advertiser Churn for Google AdWords (Paper) Google 2010 2. High-Precision Phrase-Based Document Classification on a Modern Scale (Paper) LinkedIn 2011 3. Chimera: Large-scale Classification using Machine Learning, Rules, and Crowdsourcing (Paper) Walmart 2014 4. Large-scale Item Categorization in e-Commerce Using Multiple Recurrent Neural Networks (Paper) NAVER 2016 5. Learning to Diagnose with LSTM Recurrent Neural Networks (Paper) Google 2017 6. Discovering and Classifying In-app Message Intent at Airbnb Airbnb 2019 7. Teaching Machines to Triage Firefox Bugs Mozilla 2019 8. Categorizing Products at Scale Shopify 2020 9. How We Built the Good First Issues Feature GitHub 2020 10. Testing Firefox More Efficiently with Machine Learning Mozilla 2020 11. Using ML to Subtype Patients Receiving Digital Mental Health Interventions (Paper) Microsoft 2020 12. Scalable Data Classification for Security and Privacy (Paper) Facebook 2020 13. Uncovering Online Delivery Menu Best Practices with Machine Learning DoorDash 2020 14. Using a Human-in-the-Loop to Overcome the Cold Start Problem in Menu Item Tagging DoorDash 2020 15. Deep Learning: Product Categorization and Shelving Walmart 2021 16. Large-scale Item Categorization for e-Commerce (Paper) DianPing, eBay 2012 17. Semantic Label Representation with an Application on Multimodal Product Categorization Walmart 2022 18. Building Airbnb Categories with ML and Human-in-the-Loop Airbnb 2022

Regression

1. Using Machine Learning to Predict Value of Homes On Airbnb Airbnb 2017 2. Using Machine Learning to Predict the Value of Ad Requests Twitter 2020 3. Open-Sourcing Riskquant, a Library for Quantifying Risk (Code) Netflix 2020 4. Solving for Unobserved Data in a Regression Model Using a Simple Data Adjustment DoorDash 2020

Forecasting

1. Engineering Extreme Event Forecasting at Uber with RNN Uber 2017 2. Forecasting at Uber: An Introduction Uber 2018 3. Transforming Financial Forecasting with Data Science and Machine Learning at Uber Uber 2018 4. Under the Hood of Gojek’s Automated Forecasting Tool Gojek 2019 5. BusTr: Predicting Bus Travel Times from Real-Time Traffic (Paper, Video) Google 2020 6. Retraining Machine Learning Models in the Wake of COVID-19 DoorDash 2020 7. Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow (Paper, Code) Atlassian 2020 8. Introducing Orbit, An Open Source Package for Time Series Inference and Forecasting (Paper, Video, Code) Uber 2021 9. Managing Supply and Demand Balance Through Machine Learning DoorDash 2021 10. Greykite: A flexible, intuitive, and fast forecasting library LinkedIn 2021 11. The history of Amazon’s forecasting algorithm Amazon 2021 11. DeepETA: How Uber Predicts Arrival Times Using Deep Learning Uber 2022 12. Forecasting Grubhub Order Volume At Scale Grubhub 2022 13. Causal Forecasting at Lyft (Part 1) Lyft 2022

Recommendation

1. Amazon.com Recommendations: Item-to-Item Collaborative Filtering (Paper) Amazon 2003 2. Netflix Recommendations: Beyond the 5 stars (Part 1 (Part 2) Netflix 2012 3. How Music Recommendation Works — And Doesn’t Work Spotify 2012 4. Learning to Rank Recommendations with the k -Order Statistic Loss (Paper) Google 2013 5. Recommending Music on Spotify with Deep Learning Spotify 2014 6. Learning a Personalized Homepage Netflix 2015 7. The Netflix Recommender System: Algorithms, Business Value, and Innovation (Paper) Netflix 2015 7. Session-based Recommendations with Recurrent Neural Networks (Paper) Telefonica 2016 8. Deep Neural Networks for YouTube Recommendations YouTube 2016 9. E-commerce in Your Inbox: Product Recommendations at Scale (Paper) Yahoo 2016 10. To Be Continued: Helping you find shows to continue watching on Netflix Netflix 2016 11. Personalized Recommendations in LinkedIn Learning LinkedIn 2016 12. Personalized Channel Recommendations in Slack Slack 2016 13. Recommending Complementary Products in E-Commerce Push Notifications (Paper) Alibaba 2017 14. Artwork Personalization at Netflix Netflix 2017 15. A Meta-Learning Perspective on Cold-Start Recommendations for Items (Paper) Twitter 2017 16. Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time (Paper) Pinterest 2017 17. Powering Search & Recommendations at DoorDash DoorDash 2017 17. How 20th Century Fox uses ML to predict a movie audience (Paper) 20th Century Fox 2018 18. Calibrated Recommendations (Paper) Netflix 2018 19. Food Discovery with Uber Eats: Recommending for the Marketplace Uber 2018 20.