Harsh Singhal

The modern Data Scientist or Machine Learning Engineer is increasingly being called upon to build data products end to end. In this talk we try to do just that. While it is obvious that building a complete product isn’t the job of a single individual, it is critical that data professionals are aware that it is indeed possible to build end to end data products with modern data technologies. We will examine open data sources such as Common Crawl internet corpus, a data asset that has found widespread use in the development of corpora for language models. We will follow the process of converting raw data into various structured formats and finally build a data product, all within the Python ecosystem. For a more engaging read, do check out the book https://buildadataproduct.netlify.app where I go into a lot more detail about building a data product.