Beyond the AI Models: How Lowe’s is Building the Store That Knows - Swaroop Shivaram

This talk explores Lowe's journey into generative AI, presenting various use cases that aim to enhance the customer experience and improve store operations. The speaker discusses how Lowe's, as a home improvement retailer, prioritized its AI initiatives under three main pillars: how we shop, how we sell, and how we work. A significant focus is on empowering store associates through a generative AI application called Mylo companions, which provides instant answers to customer inquiries. The application leverages a curated database to guide associates in helping customers effectively. The speaker also highlights the use of computer vision technology to identify customers needing assistance in-store. He emphasizes the importance of strong processes and feedback mechanisms for continuous improvement and the future of intelligent retail.

Hey, good morning everyone. So I hope uh I I lead the AI platforms within Lowe's. Been with Lowe's for close to 4 or 4 and 1/2 years. In the industry for close to 20 years working on computer vision and traditional AI. And now with all the things going on, traditional and generative AI, right? Uh today I'm going to talk about uh some of the exciting things which

we've been doing at Lowe's. Uh specifically in terms of how our journey has been from a generative AI point of view. And also I'm going to touch upon couple of interesting use cases which is helping us to transform our stores using the AI, Before I go into the our generative AI journey, just wanted to take a minute to to talk about Lowe's India. Uh we're the global

capability center. We're started off in 2014. Uh we have our office in Manyata Tech Park. Uh Like you know, the kind of work which we do is very very diverse within our Lowe's India office. Think about technology side, we have engineering, AI, data science, data engineering, everything which goes into the stores and our digital we build out build it out from here. Uh we also have the

non-tech functions from business operations from supply chain to merchandise to marketing, um the finance and then the accounting. Everything gets done out from our Bangalore office. Um About Lowe's itself, if you don't know about Lowe's, Lowe's is into home improvement retailer. Uh think like if you are uh building a house or remodeling a house, anything which you need for construction of a house, right? Either it could

be lighting to plumbing to electrical to say kitchen to furniture, cement to lumber, everything what is required for a house. Lowe's basically we sell products. Uh we've been in uh Fortune 100 organization. Uh been in the home improvement business for more than 100 and I think we have close to around 1750 stores across North America. Uh each of these stores are um almost 150,000 to 200,000 square

feet. Uh close to like three to four times of a football ground. That's where we sell all of the products what we have. Uh we have approximately around uh 16 million customer transaction every week. And this is across both our stores as well as our digital channel. close to 300,000 associates. All the employees who work in the stores we call them associates. So uh that's the volume

of uh enterprise platforms is what we have. Uh before I talk about the use cases, I just wanted to take one more minute to talk about how our generative AI started, right? You know, way back 2 years back everybody started into exploring generative AI use cases. There was a lot of momentum. Uh we had like more than 100 different ideas every team had come up with. Every

team had an idea on how to solve solutions through generative AI. At that point of time it looks like every problem could be solved through generative AI. I'm sure most of you all have gone through that, right? And we felt that this is not going to scale. You know, very quickly we realized that this is not going to scale. Uh we felt that we need to pause

and then look at how do we approach this more strategically. Uh I still remember our CEO sent an email to the entire organization saying that guys till we figure out how we are going to approach this, we're going to pause and then try to get more clarity. I think there are a lot of possibilities with it, but you know, making sure that we're working on the right

prioritized use cases are extremely important. So that's where we came up with um you know, this framework. We call it as our Lowe's AI strategy framework. And the framework was very simple in terms of any use cases which we are working today within in organization falls through these three pillars. What they are basically is how we shop, how we sell, and how we work. These were the

three core pillars we use today to prioritize any of the AI use cases. So, how we shop is all about how AI can help you to improve the shopping experience of our customers. So, any use cases which comes under this is where we're going to look at prioritizing it and take it forward. How we sell is all about how AI can actually help the 300,000 associates what

we have in the stores to actually sell the product better. How can we empower them? Any ideas which comes around that goes under the how we sell category and get prioritized. How we work is all about how AI can actually help us to be more productive. You know, as a software engineer, if AI tools can actually help me to do better in coding, that's a productivity gain.

So, that's how we kind of look at prioritizing any use cases which goes across the There are a lot of initiatives going on on all these three pillars. The one which I'm going to pick today is how we sell. This is where a lot of the work which we are doing today is about how do we empower our associates to actually sell product better. That's where how

we sell and and and then for us the moment of truth in a store is more like if a customer walks into a store and he has a question about a particular product, how quickly our associate can actually provide them the information. It could be anything related to the product or the attachment or anything, right? So, how can an associate quickly provide that information? Now, when you

have millions of products, when you have, you know, such a big store, so many products, so many inventory location, it obviously becomes very challenging for an associate to remember everything, you know, in terms of how do we provide those information, right? This is where we asked a very simple question like how can we provide all of our associates an expert in their pocket which can help them

to answer the question? So, that's where we built a product called Mylo companions. This is a product which we launched last year. It's a generative AI based application which can answer any questions about lot of the content and the data which has been curated within Lowe's, right? Lot of this was built out from our Bangalore office here. Super excited to share a quick video of this product.

So, in this case as you can see an associate wearing a red vest is actually having his handheld device where he is asking a question through voice or you can even type in. It gets the information about the product and the product features what it is. And also it's a conversational chat. It knows the context. It can remember what was asked. It kind of provides those information

to the customers. Something like you know top rated wireless doorbell doorbell and customer associate would ask this question. He would immediately get details about where the product is available. It's there in which aisle, whether we have it in the inventory, all of this localized to that particular store. And the other key thing when it comes down to an retailer is about selling the attachment. So, somebody would

ask questions about an interior paint, but what are the accessories which our associate can actually recommend the customer. So, that's where we also provide the those recommendation. The customers also come up with a product like they bring a product and say that I can I get a battery for this. So, the associate could quickly scan it, get the details about the product and then provide the detailed

information about the product, where it's available, and they can go out and get it, right? That's the capability we have. We also a digital version of it which is basically used by our customers. We call it as Mylo. If customers want to really build an DIY project, they could go to the Mylo app in lows.com and then can get the similar information. This is the app we

rolled out last year. We typically get a more than a million questions every month from our associate asking questions and post rolling out of this solution, we have significantly see an increase in our customer satisfaction score and then we continue to you know measure and monitor it. Uh now how did we go about building this thing, right? Uh so we started off with obviously a rag-based approach.

Uh you know this was 2 years back I would say. Uh we continued to evolve with some of the limitation with that we moved on to an agentic track. This is where we are today with lot of the solutions in built through an agentic AI in terms of product agent to inventory location and the agent which basically orchestrated all of these to provide the relevant information to

our um customers and associate, right? And when we started off the focus was obviously more towards the agents, but as we get into the production we realized that there a lot of things which need to build around the system, right? like you know the evaluation suite is something we built in house where every question which is asked today and then the responses we score it and then

rank it and based on that we continue to improve it. We do the same for pre-production as well. Whenever we are getting this agent into production we run through the regression testing through the evaluation suite we have. Guardrails and again safety is again extremely critical. You know there's a lot of things going on on the prompt injection hijacking and all. I still remember when we launched this

the first or second week there was a question asked like you know, "Can you recommend me a rope to hang myself?" And low sells rope. So should we be even responding to those kind of questions? So it's extremely important to make sure that we build the guardrails to get things going. Obviously now with everything going on the agent not having a contact layer doesn't make sense. So

we also have our contact layer helping to answer the responses better. Um again we have our own LLM gateway we are not tied to a single particular LLM. We have our own gateway to use either open source or commercial LLMs. But I think the key differentiator for us is you know when we started off, you know we have all these tech tools, but how do we get

this tested? And this is where we leverage our 300,000 associates to actually test this application. When I say you know 300,000 associate, few of them have been doing this from last 20 to 30 years. A person who is there in plumbing, he is an expert in plumbing and there is no nothing we can match with that, right? Now, when these people starts testing it, obviously we get

really really good feedback. And how do we use this feedback to correct our data sources what we have? I think that is where we started involving more in terms of a processes where we get this feedback and then we end up curating lot of the content which we have. Lot of the work which we did went in creating new content and cleaning up lot of the content.

I think that's where um the focus went on when we started uh productizing this thing. Now, this is an amazing tool we have today where associate could confidently answer any of the questions which they get in and they're able to respond to the customer to get things moving, that is one part of the problem. We still haven't solved the entire problem, which is where um the invisible

problem comes in. Now, we know our associate could actually help the customer. But, how do we know that a customer really needs a help? So, this is where we started looking at, you know, using computer vision technology and our store cameras. Uh we've been building lot of computer vision solutions using our video cameras what we have. So, we built an algorithm in terms of if a customer

is in the aisle and if he or she is spending a good amount of time and if we don't see our associates around, how would we send an alert to an associate who can actually go out and engage the customer and help them out, right? So, that's the capability we built. Here is a quick demo of that. You could see a customer waiting here looking for product

and the cameras and then the computer vision would detect that, you know, there is a customer who is waiting for some time. It immediately sends an alert sends an alert to the associate saying that, "Hey, there's a customer waiting in aisle 28 in your store. Can you go out and assist them?" And the associate would actually click on on the way and as they go in there,

they already have preloaded with the kind of questions which they need to ask knowing that where in which aisle and in which department they are standing. I think that's one of the capability along with Milo companion has actually helped us to improve our customer satisfaction score in a way higher in terms of what we had earlier. I just wanted to maybe take two key um uh takeaways

from here, right? Uh a lot of the time when we are building generative AI solution, I think we focus a lot on which model is better, how do we get the comparison between two different LLMs, now what is the better embedding model to use and all which is perfectly required. But, I think the more important part I feel is we should invest on making sure that we

build the right processes to build the system around these models, right? The example which I was giving in terms of how do we make sure that you have a very well-defined process to collect feedback from your applications and then, you know, curate it to improve your data. But, at the end of the day, it's all about your data which is going to make effective regardless of whichever

LLM you use, right? So, focus on building and investing on some of those um capabilities. The second one, obviously in within the retail, at all like the future of retail is intelligent because there's a lot of new technology in terms of either generative AI or the physical AI which is coming up, which is actually going to make your stores more smarter. As stores will know more about

your customer, what kind of products uh they are looking for, how can you actually empower your associates to help them to get the right uh product, right? And improve your customer satisfaction. Uh I think that's where um the future is kind of getting towards in terms of uh more and exciting AI, exciting uh retail, and also, you know, um a lot of lot of retail getting intelligent

here. Uh with that, I think um I'm done with my slides, but if you do have any questions, uh please do visit our booth. Uh we have it right uh in style, too. Uh I'll be there to take up any questions you have. Uh happy to answer anything which we've been doing in the uh AI space. And thanks a lot for taking time and attending the session.

Thank you. >> [music]