Sathish K S

Data observability builds upon the principles of Infra/Software observability as well as practices of SRE to address nuanced challenges thrown in by modern data-driven systems. In this talk we explore the foundational concepts of Data observability and the landscape of multi-tenanted Data/AI products and additional requirements needed from observability. We delve into the specific challenges posed by multi-tenancy, including tenant-specific processing pipelines, usage patterns, data lineage, operational complexities, and the management of comprehensive data catalogs. Furthermore, we discuss the impact of evolving privacy regulations and compliance requirements, which underscore the necessity for robust, tenant-aware monitoring and auditing capabilities. Finally, we examine emerging trends, including AI-driven observability solutions, that are shaping the future of data reliability and scalability. These innovations aim to address the intricacies of multi-tenant environments while ensuring performance, transparency, and compliance at scale.