Machine Learning Developers Summit 2024

Rajat tells the story off google's journey into an AI-first world! The story talks about chance interaction,  curiosity about the human mind and above all the googleyness spirit that helped Google create differentiated product offerings by being amongst the very first to leverage AI and ML in their products. He also talks about how you can leverage Google's learning in your own organisation to re-create the same magic!

In recent years’ video action recognition has become a hot research topic due to a wide range of applications such as video surveillance and video analytics. Large volumes of training data, which have more than 56,000 video sequences, are required for training a decent action recognition neural network. However, such large-scale video recordings, which are supposed to capture the dynamics of every action category, are not only prohibitively expensive to gather but also impractical. Training samples are few and rare (e.g. when the target action classes are not present in the current publicly available datasets). For the proposed frugal pipeline, 10% of labeled data is good enough for training an initial neural network to predict the labels for the unlabeled data. Initially, we started with only 10% of the data and passed it through the augmentation pipeline. Then, we used that data to train a neural network and used the best neural network weights to perform pseudo-labeling. All our unlabeled data was labeled during our experimentation at the 4th iteration of the data through our pipelines. It increased the model's accuracy by 10\% on a dataset with no augmentation. This approach consists of two pipelines, the first is a data augmentation pipeline, and another is a pseudo labeling pipeline; types of augmentations can be changed according to the use case.

Data entered into QuickBooks has real financial implications - a single data entry mistake can be expensive for the business requiring significant effort in undoing the impact. Customer research highlighted the following top concerns: Error in entering invoice data, Error in paying employees, Fat finger a bill payment to a new vendor.. We needed a system which will flag errors on key fields while entering transaction data, and nudge the user towards the correct range. AI solutions are now norm for delivering smart product features to enhance customer experience but most solutions are focused on tailored use-cases that need resources from ideation to deployment and subsequent integration, every time we adopt a new use case. Given the scope of multiple use-cases across data input fields in Intuit products, the error-detection system had to be extensible so that it allows developers to build tailored AI models for specific error detection by providing information about the fields and data sources only. FEDS employs novel methodologies for error detection based on Empirical Bayesian Test, Likelihood Ratio Test and Deep Learning approaches. It decouples the efforts of Data Scientists from the underlying implementation layer, which facilitates the product developers to easily adopt across multiple use cases.

In this paper, we describe a novel case study on the application of recommender systems to generate sales leads for merchants in a B2B environment. A user-based collaborative filtering algorithm is used on corporate payments data to generate new client recommendations for the merchants based on the existing clients of similar merchants. Client recommendations or prospects, if generated accurately, can have huge business potential for the merchants by aiding them in expanding their business. Hence the paper describes an application of recommender systems with significant business impact and value. Despite finding lots of use cases in the B2C environment, recommender systems have till date, very limited applications in the B2B arena. By providing a concrete step-by-step account of how to apply this powerful machine learning technique, our paper aims to firstly bring to light the significant business impact the use of recommender systems may have in the B2B space and, secondly, serve as evidence to be referred to for further applications.

Computer vision AI systems trained on medical images such as X-rays, CT scans etc. can be used to identify, classify and locate specific health conditions, like severity of knee-joint-space, location of vertebrae slip etc., with increased speed and accuracy. Leveraging medical imaging AI tools can provide clinical reviewers with images annotated with relevant clinical parameters, which can aid decision-making across the utilization management value chain, driving improved operational efficiency, accuracy and consistency in decision-making and delivery of health care.

Inventory or stocks are the goods held by any business organisation for the purpose of consumption, production, or sale. These are the non-capitalized assets of the organisation. Inventory can be classified into four categories - finished goods, raw materials, work-in-progress inventory and spare parts inventory (also called as plant maintenance, repairs and operations (MRO) inventory). Spare parts inventory is leveraged in the running and maintenance of the plants or the manufacturing units. In general, inventory management is the process of balancing the inventory thereby - having the right amount of inventory in the right place at the right time. Though it looks easy, it is one of the most challenging problems that industries deal with. In this paper, we will discuss what are the different challenges organisations face during inventory management and how we solve them using artificial intelligence (AI). We have built an AI-enabled dynamic inventory management application across all the inventory categories. Dynamic inventory for finished goods application optimises the inventory using stock, order, product, location and historical sales data across the business. The application can accurately forecast demand to get the right level of inventory, in the right locations, to meet service-level targets. Similarly, dynamic inventory for raw materials application applies AI models to inventory, sales, bill of materials, supplier, work-in-progress and finished goods data from across your business – so you can optimise raw material inventory levels to meet demand and reduce costs. The dynamic inventory for spare parts inventory management application applies AI models to plant data like location and capacity, and inventory data like stores, spares and consumables across the business. We predict demand across stores, spares, and consumables on a granular level. Then, we use forecasted demand to optimise inventory across each plant which will save cost while meeting the required service levels. We will also show how businesses can achieve a good ROI by deriving value from our AI-based inventory management applications using some use-case examples.

Google has been widely recognized as the industry leader in AI/ML. Apart from building Google's own planet-scale consumer applications like Gmail, Photots etc. which are powered by AI/ML, Google has also been a strong supporter of and contributor to the open source community. Open source is indeed at the heart of Google—it’s what we’re built on and at the core of everything we do. From sharing datasets to creating platforms like Tensorflow for building AI/ML models and container orchestration systems like Kubernetes that help scale ML, Google's contribution to the open source AI/ML community has been immense. Join this session to learn more about Google's AI/ML contributions to the open source community, that you as ML Developers can take advantage of, to innovate, build and scale, just like we at Google do.

Preparing an efficient and effective staffing roster is a challenging task for retailers. An efficient staffing schedule optimizes payroll cost while meeting desired customer service levels. Doing this while complying with labour laws and organizational policies makes the schedule effective. While compliance with legal requirements is mandatory, evaluating the impact of alternate organizational policies helps formulate a suitable staffing strategy. These policies present themselves in a combination of factors around shift duration, standardized shift start times, number & length of breaks during the day, off-days, etc. In this paper, we describe our evaluation of alternate organizational policies on the payroll costs of a leading luxury retailer in the UK. Each candidate policy was evaluated using Mixed Integer Linear Programming (MILP), following a two-step approach. In the first step, we forecast the number of customers visiting the store at half-hourly intervals and translate it into the expected staff count required on the shop floor. In the second step, we infuse this, together with legal and organizational policy constraints, into the MILP. Our experiments revealed that flexibility in shift start times, and the mandated minimum staff strength for a store, were the primary drivers of payroll cost. More than three shift start times a day allowed the flexibility of having a higher mix of part-timers. This translated into a 10-12% reduction in paid hours. This benefit can then be considered in the context of the overhead of managing a large part-time population and a potential shift in organizational culture. Further, the driver of minimum staff strength highlighted the need for reviewing the store’s physical layout.

Digital advertising heavily relies upon cookie-based solutions to identify and target the prospective audience. With third-party cookies being deprecated in the near future, advertisers will have to look at alternate strategies, such as geo contextual for targeting. In this paper, we have proposed cookie-less solutions for audience expansion by suggesting look-alike postal codes (for geo and audience targeting) and domains (for contextual targeting). These solutions could be used as a part of pre-campaign planning and use customer affinity and geo-contextual datasets along with a universal ad feed as historical learning. We have used deep autoencoders and self-organizing maps for low dimensional latent space representation and probabilistic methods for arriving at insights. A penalized similarity function is applied to this latent space to suggest look-alike postal codes and domains. To overcome the challenge of estimating how good our unsupervised learning models are, we also have developed a framework to threshold the number of postal codes/domains suggested and validated the results.

An intelligent combination of Data-Centric and Model-Centric approaches that not just focus on procuring the relevant data and on algorithmic tools and techniques but also focuses on generating information by emphasising on business relevance, responsible AI, sustainable AI and explainable AI, which collectively help to solve the business problem.

SOTA computer vision algorithms play a large role in addressing business problems across verticals such as Manufacturing, Retail, Insurance, etc. In this talk by Praveen Jesudhas, learn how advancements related to active learning, synthetic data generation, few-shot learning, etc., can be consolidated efficiently to enable rapid evaluation and deployment of Computer vision-based solutions.

This paper explores the impact of brand trust on commercial consumer behavior and marketing management in the technology industry, specifically in cloud services. In this study, we examine factors influencing consumer loyalty in products and service businesses leading to trust and love and offer constraints-driven recommendations to promote it. Research focuses on the purchasing behavior of organizations buying goods and services for use in the production of other products or services that are rented, sold, or provided to others. Testing the hypothesis and constructing the model begins with exploratory factor analysis and structural equation modeling. Results indicate that the factors mentioned have an impact on consumers' trust in brands and are presented as a visual network graph. In order to measure the relationship, an evolutionary equation is derived utilizing controllable factors such as service history and surveys and uncontrollable factors such as socioeconomics and market factors. As a result, each account in service is assigned a rank-based time-driven risk profiling index.

Retail manufacturers are grappling to predict customer demand for each product at superior forecast accuracy levels with the ability to refresh the forecasts at weekly intervals due to computational expenses. The overall cost of demand forecasting is determined by adding product shortage cost (loss of missed customer demand), excess product cost (inventory holding costs), product pilferage cost (expired goods cost), and computational cost (involved in forecast models training). While the first three cost components can be attributed to the forecast accuracy levels, the computational cost lever can be controlled through computationally light forecasting systems. The motivation of this research is driven by very limited empirical evidence on the adoption of unsupervised techniques to identify groups of products with similar levels, trends, and seasonality patterns for the development of computationally light forecasting systems. This study aims to decrease the forecast model training cycles by leveraging unsupervised techniques like K-means Clustering and Hierarchical Clustering to identify clusters of products with similar customer purchasing behavior. This research introduces a Clustering-based Demand Forecasting Framework that determines the best forecasting algorithm for each cluster. The experimental approach utilized this framework to predict the customer demand for more than 500 dairy products for the next 8 weeks. A comparative study on computational time across product-level model training and cluster-level model training is presented for better realization of relaxation in computational costs. The Weighted Accuracy Percentage Error (WAPE) based product level forecast accuracy from the Clustering-based Demand Forecasting Framework was found to be superior to the Standard Demand Forecasting Framework. Further, this research provides better interpretability of clusters by providing a product category-specific name to each cluster through Natural Language Processing (NLP) techniques. Holistically, this research effort provides empirical evidence to retail manufacturers for enterprise-wide adoption of the computationally light short-term forecasting system.

According to a study by Mckinsey, 71% consumers expect personalisation and 76% get frustrated when they don't find it. As we see a surge in digital behaviour, personalisation has become a necessity for most businesses. However, while most people lay stronger emphasis on the recommendation engines powering personalisation, design and engineering play an equally critical role in delivering maximum value through personalisation, along with data science.

Traditional numerical methods of classification of both univariate and multivariate time series have shown to be challenging due to their problematic features, including the inability to preserve temporal correlation, lack of pre-trained models, difficulty in training Timeseries models and finally, Timeseries tend to act incorrectly when presented with multiple input. Decades have shown immense importance in applying deep learning methods to image and video data to solve real-world issues. The authors are motivated by recent accomplishments of supervised learning approaches in computer vision, so they have explored various time series image encoding techniques to enable machines to visually identify, classify, and learn structures and patterns by leveraging state-of-the-art Deep learning and computer vision. The Authors demonstrated rapid spot groupings of series with specific features. Later, these series were classified and subjected to comparison analysis using multiple cutting-edge deep neural networks to their maximum performance potential. Finally, the comparison results were demonstrated.

Lookalike audience generation is an effective way to increase the audience base in online advertising. Segregating the lookalike audience into multiple priority levels gives greater flexibility to the advertiser in selecting their user reach. This paper explains a novel approach to lookalike audience generation in multiple priority levels on large-scale data with millions of users and thousands of audience segments. An automated system combining custom models to generate similar audience segments and group lookalike audiences into priority levels using Spark Scala and the Hadoop ecosystem is developed. The experimental results comparing different approaches show that our proposed model outperforms others in reach, scalability, and speed.

In information extraction, not only extracting the entities but also identifying the relationships between the entities can lead to better hidden insights. Triples are used to represent the relationships between clinical entities. Triples are made up of two concepts and a relationship between them: . Graph data is prominently used to extract vital hidden insights by node-edge analysis. This research studies different graph embedding algorithms to predict the relationship between nodes. The paper examined the state-of-the-art in clinical relation prediction and created a graph-based model to predict the relationship between two clinical concepts. The relation prediction model is trained using the proprietary Graph dataset, maintaining the concept and relation triples. The work implements and compares the results of link prediction algorithms in the clinical domain. The ComplEx graph-based node embedding algorithm has been found to be promising in predicting semantically enhanced clinical relationships. The paper discussed the potential use cases of clinical relationship prediction in detail. This research will be critical in determining the causal relationships between medical concepts.

As more organizations look to undergo a transformation, it is critical to understand how data and analytics are the thread that carries the project through—start to finish. The stakes are high for any organization coming out of the last few years. As the marketplace faced unforeseen disruption, it became imperative for businesses to digitize how they engage with both employees and customers. Whether the goal is to sustain, compete, or disrupt—organizations of all types, sizes, and industries have had to reimagine their business models and processes. The phrase that’s been on the forefront of every business-centric conversation for a while has been “digital transformation” and for a transformation to be successful, you need to implement the right data strategy, technology, and infrastructure to modernize your company and become a more competitive and agile player in the marketplace. It requires technology, digital processes, and organizational change. It also requires a focus on data and analytics because the two are key accelerants of any transformation effort. As technologies and consumer expectations continue to evolve, businesses need to keep pace. A company’s ability to compete and thrive will be determined by its ability to make quick, informed data- driven decisions. Where data provides the information, analytics provide the insights that lead to informed decision-making. Using data and analytics as the thread that carries your transformation—before starting and throughout—allows you to overcome key modern business problems that can be roadblocks to successful data initiatives.

Temporal disaggregation is a method of disaggregating low frequency time series into higher-frequency time series. A major challenge faced by many when working with time series data is the non-availability of data at the required granularity. One might come across instances where the data is monthly or quarterly, but the preference is for the data to be weekly, daily, etc. Commonly used techniques to estimate the missing values often fail to capture the behavior of the underlying high-frequency data. This paper introduces a novel approach to tackling this problem. We tried to use a statistical gaussian-interpolation technique to tackle the problem. Gaussian Process is the non-parametric method that tries to model the functional form of time series using a correlation matrix known as the kernel function. It not only models various crucial components of time series which are trend, seasonality, and stochasticity but also can be tuned to avoid overfitting and smoothen the time series using appropriate kernel functions which could be a problem with the naïve approaches such as interpolation which tries to fit a certain polynomial function. Since it is a statistical model every predicted/filled value there is an associated probability which gives the confidence of being an instance of the parent time series. We will also highlight how our method compares to the other commonly used method and instances where it can be deployed without risk. Often, some time-series datasets are rendered useless in our analysis/model due to the lack of availability of data at the preferred granularity. It is also sometimes not feasible to capture the data at a given frequency or only some sample data at the frequency it is available. Our method deals to tackle exactly those problems by providing an accurate estimate for the underlying data.

Patients’ increasing digital participation provides an opportunity to pursue patient centric research and drug development by understanding their needs. Drug development can benefit from including patient and caregiver social media listening (SML) as part of patient-focused drug development (PFDD) strategy. This work introduces an Artificial Intelligence (AI)-enabled SML concept to identify and understand patient and caregiver experience to inform characterization of unmet medical need, target product profiles, value demonstration strategies, evidence generation strategies, definition of patient Centered outcome (PCO) endpoints, and selection of PROs in trial design.

From Una in Himachal Pradesh to Tuticorin in Tamil Nadu and Mundra in Gujarat to Guwahati in Assam, the 15,113 km based massive network of pipelines in IndianOil is engaged in the safe and sustainable transport of crude oil, petroleum products and natural gas. Apart from the natural vulnerabilities, including corrosion and leakages, the pipelines are also subject to various intrusions by miscreants in an attempt to steal products. Each of these vandalism attempts could account for damages in crores. Apart from economic losses, any pilferage-led spill endangers human lives and the ecological viability of the surroundings, along with threatening the energy security of the nation. Catering to the prevention needs of pipeline integrity against pilferages, a machine learning-based model, the Pipeline Vulnerability Management System (PVMS), has been built. PVMS is an ensemble-based classification model that identifies certain stretches of pipelines to be more vulnerable (Red Zones) to pilferage attacks than the rest. The limited security personnel can be optimally deployed in the Red Zones to prevent pilferage events. The solution assures extensibility to various other industrial applications (Heatmapping Conveyer belt maintenance, logistic supply chains in FMCG), and further research can help enhance assessment at a more granular level.

Large scale Machine Learning solutions tend to fit the “head” - needs of the more engaged audience. The performance can be poor for the under-represented populations whose votes get lost among the vast data. In this talk, we will discuss the initiatives underway in the internet product industry to democratise machine learning models for creating value for the whole population.

Today’s data centres have thousands of servers that belong to multiple vendors, and they vary vividly in terms of technology, model, generation and numerous peripherals like firmware, software, and other hardware components. It is a challenge for the data centre administrators to track activities being executed across these servers and the resource consumption at various stages of deployment, delivery, installation, and upgrades. This leads to a subsequent ripple where the data centre administrators find it tough to foresee GPU availability and/or consumption in their data centres. This poses a risk when they need resources to be available for critical activity in the future, but because of an overburdened data centre, the resources are unavailable, and the critical tasks fail. On the other hand, if the data centre always remains underutilized, the unused servers in the data centre become a liability and bring the ROI down. This not only leads to unmitigable risks but also has a monetary impact on the profitability of the organization. This also leads to poor resource planning because of the lack of visibility of future needs. This paper focuses on designing a forecasting solution for data centre admins. The solution includes performing data explorations and developing enough understanding of the various metrics of the servers in the data centre like CPU, IO, Power etc., and developing univariate/multivariate forecasting solutions using the time series data sets of these metrics. This paper proposes to explore TDM as well as GAM models for predicting behaviors of various factors of data centres and CSPs. This solution would assist the data centre admins to have clearer visibility of the future load/availability of the GPU in their data centres. By having so, they would be in a better state to understand the utilization metrics for their data centres and better financial planning for future investments based on the clear visibility of under/over-utilized resources. This will also lead to better data centre resource optimization and rationalization.

Businesses increasingly focus on brand loyalty and customer engagement as they move towards more customer-centric strategies. Besides end consumers, sellers also form a key customer base for eCommerce marketplaces. Bad customer experience can lead to negative brand perception and drive down NPS (Net Promoter Score). This study was to help an eCommerce Marketplace scientifically identify key experience areas that were leading to a drastic decline in Seller NPS scores across regions over eight months. This would help businesses focus on the most important levers, prioritize budgets and improve NPS. In this study, we modelled customer survey data on brand perceptions and experiences, with historical transactional data and company policy information, to determine drivers of NPS. It was important to understand the inter-relations among various aspects of customer data, such as brand perceptions, issue resolution, customer service and macro factors, to provide holistic and actionable recommendations. We used a path model approach with a Bayesian network to determine causality between the key factors driving NPS and the interrelation among each factor. Structural Equation Modeling followed this to quantify the relative impact of each factor on NPS. The study found that sudden policy changes without effective communication, poor customer service and lack of protection for sellers from false buyer claims led to a poor selling experience. Sellers perceived the site as unfair to them and unsafe to sell. These insights were actioned upon by the company in an organization-wide cross-team initiative which helped address the seller’s dissatisfaction and eventually brought NPS back on an upward path.

Today AGI is changing the world and Human expectations from the machine. Increasing variety and complexity of human-machine networks is failing non-deterministic behaviours of both the end. We may need to focus on two major aspects: Trust building in the new agency method Perceived risk and probable mitigation

Executives across all industries have recognized the impact that Artificial Intelligence and Machine Learning can have on optimizing every part of their businesses. So why are so many companies struggling to capitalize on the power of these capabilities? The answer often lies not in the effectiveness of the algorithm but in the disconnect between the teams that manage customer experience and the data science/engineering functions in large organizations. In this talk, we will explore a few examples of successful and challenged deployments of AI and offer a perspective on how to deliver game-changing AI by starting with the customer, not the math.