Spark ML pipelines under the hood I Dynamic Talks RUS

Vitalii Monastyrev, BigData Engineer, Grid Dynamics “Modern IT companies actively develop the Data Science stack in their projects to predict profits for subsequent quarters, configure targeted advertising, build a recommendation system, and much more. Quite often, the data used to build Machine Learning models weighs hundreds of gigabytes or more. In this case, many questions often arise: - How to work with so much data - How to generate features - How to train models - How to integrate work between Data Engineer and Data Science teams