Executive Director, GDS Consulting Data & AI team at EY
All Sessions by Kunal J Turakhia
Day 1
Feb 5, 2025
15:40
Practitioners guide to accelerate data transformation using LLMs
15:40 - 16:10
Advanced data transformations accelerators, such as data lineage powered by GenAI are becoming increasingly critical, as industries prioritize data modernization, transitioning from traditional legacy systems to advanced data technologies. This paper delves into the application of LLMs in tracing the data lineage. We will share our direct experiences in addressing the issues, obstacles encountered and the effective strategies we implemented to overcome them. The proposed approach offers several advantages, including improved efficiency, accuracy, reliability and enhanced data security by keeping data knowledge within the enterprises cloud environment. The approach involves custom chunking strategies, hybrid semantic and vector search capabilities, selection of embedding model. Overall, this paper presents an efficient and secure solution for accelerating data lineage task leveraging LLMs.