Listen in as Sridhar Raghavan, a Senior Director of Product Management at Salesforce, shares his journey from working in speech recognition two decades ago to generative AI today. We explore his early memories of playing Pac-Man on his dad’s computer in the mid-90s, his break from Salesforce to pursue his own startup, and his strategy to get kids away from digital devices and into reading with AI.

Discover how Salesforce is pushing the boundaries of AI-assisted code generation for developers. We discuss parameter sizes, Salesforce’s open-sourced 16 billion parameter model, and the importance of customizing and updating language models. Learn about the GPT (Generative Pre-Trained Transformer) acronym and how it is used to create the wiring necessary for a language model to become a learning engine. 

Don’t miss this fascinating conversation with Sridhar Raghavan!

Show Highlights:

• Sridhar’s strategy to get kids away from digital devices and into reading and how AI can help make reading more fun and engaging.
• How Salesforce open-sourced a 16 billion parameter model.
• The importance of customizing and updating language models.
• What the open source project CodeGen is and how it is used to generate code in six popular programming languages.
• How Salesforce is enhancing CodeGen to generate Apex code, LWC, and flows.

Links:

• Sridhar on LinkedIn: https://www.linkedin.com/in/sridharraghavan/

Episode Transcript

Sridhar Raghavan:
Yeah, so I did my masters in electrical and computer engineering. I specialized in speech recognition, trying to make computers understand speech, and this was almost 20 years ago.

Josh Birk:
That is Sridhar Raghavan, a senior director of product management here at Salesforce. I’m Josh Burke, your host of the Salesforce Developer Podcast, and here on the podcast you’ll hear stories and insights from developers, for developers. Today we sit down and talk with Sridhar about generative AI, its past, where it’s going and how it’s going to intersect with developers, but we were agreeing to pick up where we left off there with his early years.

Sridhar Raghavan:
Back then, our state-of-the-art systems were being judged based on how well they could detect a string of digits being spoken by users.

Josh Birk:
Really?

Sridhar Raghavan:
It was challenging to do this in quiet environments. You can imagine how it would be if it’s in a noisy environments. And this was pre-deep learning era, obviously 20 years ago. So we were using techniques like hidden Markov models, and it came a long way mean. But with deep learning, six to eight years ago, things changed quite dramatically in terms of accuracy, robustness and everything.

Josh Birk:
What’s your earliest memory with the computer?

Sridhar Raghavan:
So my earliest memory with the computer was playing Pac-Man on my dad’s computer. This was in mid ’90s. That was when I got hold of my first computer. My dad had it for his account software back then.

Josh Birk:
Nice.

Sridhar Raghavan:
So this was a game that I had access to, and I literally tore it apart.

Josh Birk:
The number of times the answer to that question is, “I got to play games on my dad’s computer.” It’s so great. I also love the fact, because I have this memory of Space Turkeys, my dad got this, saw this ad about Space Turkeys and it stuck in his head that that’s what all computer gaming was like. And so he told us, me and my brother, we can get a computer, but you’re not allowed to play Space Turkeys on it. You know the biggest problem my brother and I had using the computer? Getting my dad off of it because he kept playing games.

Sridhar Raghavan:
Yep, totally understand.

Josh Birk:
When did you first get introduced to Salesforce?

Sridhar Raghavan:
So I joined Salesforce in 2018, when we were starting out the initiative to bring voice to our applications. And that was a small set of teams where we were evaluating the use cases where voice could really empower our customers. Obviously, service was a big use case. We worked with Sales Cloud as well. We in fact launched a product called Einstein Voice Assistant back in 2018-ish at Dreamforce, which was to improve productivity of salespeople who are on the move, to be able to take notes and get them into the CRM fairly easily, without having to fill in forms or go through most clicks on the UI. So that was the project I was working on back then. That led to other things. I worked in the search team for a brief period, and I did take a break actually from Salesforce as well, before I rejoined just a few months back. This time, I’m part of the AI team, and it’s exciting stuff that’s going on. We can talk more about it.

Josh Birk:
Nice. I noticed your boomerang, what led you away from the mothership for a while?

Sridhar Raghavan:
Yeah, so I left Salesforce to pursue my own startup. So to be frank, I did not have a specific idea in mind. I just wanted to do something on my own. So I kind of decided to take the plunge.

Josh Birk:
Got you.

Sridhar Raghavan:
And while I was evaluating many problem areas, there was one problem that was sort of personal, and in a way I could see that being personal to so many other friends of mine, which is to get their kids away from watching digital devices and maybe read more books.

Josh Birk:
Really?

Sridhar Raghavan:
And this was a big problem, especially during COVID when it had just set in, and parents were all in the house and screens were the only way of entertaining. I mean there were books too, but there’s no competition between books and the screen when it comes to kids. So, that was the problem I was trying to solve.

Josh Birk:
Nice. At a high level, what was the strategy to try to… First of all, noble effort, love it. What was the strategy to try to solve it?

Sridhar Raghavan:
Yeah, so I did some survey of basically reaching out to my daughter’s friends parents, and also including our teachers. I got a lot of feedback on what would be useful and some of it was all of the plays. As you can expect, this is a very open problem. But one thing that resonated with a lot of our parents is, and kids as well, I was also trying to interview children in that age group. This is mainly K-6, not older kids. One of the things that resonated with a lot of them was reading material that’s on paper or just a piece of text is not necessarily fun if you’re not being… There are two ways to make it fun. One is collaborate with your friends or gamify their experience, where they could earn some kind of points or they can be evaluated, but not in a judgmental way, but in a fun way to make them feel like, “Wow, I nailed that report. I really understood what is photosynthesis.”
So my approach was, okay, I mean, my background is in AI, so I thought, “How can I also apply AI?” Part of my background kind of helped bringing a proof of concept in to some of these parents, where they could literally take any material. It could be on a piece of paper, or it could be a PDF file, or it could be a handout given by your teachers, or from a textbook. You could just take a picture of that piece of text, and it will basically create a flashcard for kids to play with, and evaluate them on that content that they just took a picture of or uploaded.

Josh Birk:
Okay, dude, that’s really, really cool. I love all of that. However, we are happy to have you back at Salesforce. How would you describe your current role here?

Sridhar Raghavan:
My current role, I couldn’t be more excited about where we are today with respect to AI at Salesforce.

Josh Birk:
Nice.

Sridhar Raghavan:
It’s essentially become, I wouldn’t say… It’s always been our core competency, but now that is the central focus of everything we do with AI data and CRM. And my current role involves working, I’m in the AI research team, so my team has basically full of smart PhDs and scientists, some of them professors who did professorship for five years, and now they’re taking up a job for the first time. So it’s a very diverse group, and my job is to funnel the business perspective ideas to them, and kind of guide them on the most impactful applications of AI that we should help our partners, which is our clouds, and sometimes even direct to customers in certain scenarios.

Josh Birk:
Got you. You’re a smart person wrangler. I like it, I like it. And it’s interesting that you describe it like that, because I know all of a sudden AI is in the forefront of the conversation, but as you have said, this is not entirely new to us. How would you describe the current public state of AI usage with Salesforce?

Sridhar Raghavan:
So at Salesforce we launched Einstein, which was our AI fabric across clouds about 10 years ago, if I’m not mistaken. And at that time, our goals were fairly modest. We wanted to hit maybe a few thousand predictions a week.

Josh Birk:
Got you.

Sridhar Raghavan:
Fast forward to today, we’re making close to, the last count I heard was we’re heading close to a trillion predictions a week.

Josh Birk:
Whoa.

Sridhar Raghavan:
And that’s pretty amazing. And this is the classic machine learning or AI. So we are moving this to the next level, which is the generative AI.

Josh Birk:
Wow, okay. So, let’s cut that, because I want to move into describing what generative AI is. And so the current state of AI is, tell me if I’m using any of this incorrectly, but it’s natural language processing associated with deep learning, so that people can have conversations with their data. Sorry, not conversations. Interactions with their data, but that data is not, or that interaction is not conversational nor generative. Is that accurate?

Sridhar Raghavan:
It’s partially correct. The way I would maybe phrase it is the data has, we’ve always had data, but what you could do with data prior to the generative AI type models was to make predictions, or build models that are highly tuned for a very specific type of use case or domain, or vertical.

Josh Birk:
Got it.

Sridhar Raghavan:
With generative AI, basically generative AI is just a broad name as they are powered by these models called as large language models. For those who are not familiar with what language models are, in very simple terms, it’s like the auto complete we have on our phones, where you’re typing in something, it tries to predict the next word. That’s exactly a language model. But the large language models are much larger in size, because they’re built with a lot more data. When I say built, they’re trained with lot more data and that gives them the ability to not just predict the next word, but they can predict entire, they can generate an entire email, for example, or a text message that an agent wants to send to their customers, or it could be a piece of code like Apex.
And these large language models are also very flexible in the sense you can fine tune them for very specific task. For example can, if you want a conversational bot like ChatGPT, you can fine tune that with a much smaller set of data than the original model was trained with to get it to really address your specific problem. So both the power of being able to generate full content and the ability to fine tune them for these very specific use cases in a very, I would say in a very light way, gives us this power to really take it to the next level.

Josh Birk:
Because when we say, well, okay, so that’s interesting and it kind of changes my next question a little bit, because when we say large language model in the terms of the currently publicly available version of ChatGPT, or I have not double checked my math on it, but I believe Bard is probably a similar size. When we say large, we mean really, really large. Not millions of nodes, but trillions of nodes.

Sridhar Raghavan:
That’s correct. And it’s relative. I think five years ago, millions of parameters was considered large, and in present day we’re hitting close to, I think a trillion parameters. And by parameters, I mean to keep it simple, let’s assume it’s simple, you can relate it to the size of the model. These are like the neurons in our brain kind of a thing, if we can relate it to that. So the large, it is large, to make it even more relatable, the largest language model that Salesforce open sourced sometime last year that generates code is 16 billion parameters in size.

Josh Birk:
Wow.

Sridhar Raghavan:
That’s the base on which our developer tools are being developed.

Josh Birk:
Got it. And so Bard and ChatGPT, they kind of need these massive neural networks in order to handle the fact that they’re asking anybody in the world to ask for a cooking recipe, or what are good trails in Colorado, like anybody in the world asking everything. But what you’re describing is that your problem set is much more focused, say Apex, properly coded and therefore your parameter size can be much smaller.

Sridhar Raghavan:
That’s exactly right. Our parameters, we can have a much smaller set of parameters and give our customers the same or better quality than what they could get with a much larger language model that is not only more efficient, but it’s also easy to customize and fast to roll over new updates.

Josh Birk:
So that was a conversation actually, I just had with somebody recently because they asked me, because a lot of people don’t understand that ChatGPT and Bard, they know the world in like 2022 because that’s when their data dataset was a snapshot in time. And so the question was, “Well how easy is that to refresh?” And I’m like, “Well, it’s a lot of data,” but what you’re saying is that with the models we’re looking at, it’s not that hard, because you can modify that delta of parameters and update it on something. And I’m just throwing this out here, something probably closer to what is like R3 release cycle.

Sridhar Raghavan:
Exactly. In a much shorter time cycle. That is exactly how you said it. Yes.

Josh Birk:
Nice. I’m going to ask a question on that project really quickly, but I want to get this out just so that people can get on their brains. What does GPT actually stand for?

Sridhar Raghavan:
The GPT stands for generative pre-trained. So it has three things going on describing what it is. Generative means it’s a model that will take some input and generate an output. And it’s a language model inherently is always generative in the sense, just like the auto complete example I had mentioned before, you give two words, it gives you the third word that follows as a prediction. So that’s generative. Pre-trained stands for the fact that these models, if you consider traditional machine learning, we required a lot of labeled data to build these models. Simple things like if we want to build a intent detection model based on some freeform text, we needed data which looked like utterances on one column, and the corresponding intent category in another column.
And these were all hand curated, and then a model was trained on that. With pre-trained models, we would just build this large language model from all sorts of data without having any human having prior inspected this model in any way. So just by crawling web or repositories with code, blogs, some models are multimodal so they can even process images and audio. They all go into this model as part of the pre-training stage. And this builds the wiring. It really cannot do anything useful, but it builds the wiring necessary for the model to become a learning engine as we start fine-tuning it. And the last part, which is generative pre-trained, the last part I think I didn’t say the last T, T stands for transformers.

Josh Birk:
Pre-trained try to sneak in there as the transformer. Yeah, exactly.

Sridhar Raghavan:
So T stands for transformers, which is nothing but the architecture of these large language models, how they’re able to learn, and are able to be fine-tuned for specific tasks. So that’s what it stands for.

Josh Birk:
Got it. Now tell me a little bit more about the open source project you were talking about. Is that CodeGen?

Sridhar Raghavan:
That’s correct. CodeGen is the open source project.

Josh Birk:
Okay, so the assumption is our AI to build things, develop tools, correct?

Sridhar Raghavan:
Yes. So we are building on top of CodeGen.

Josh Birk:
Okay.

Sridhar Raghavan:
CodeGen was released sometime last year. So, we’ve made a lot of enhancements or we have grown the model to solve more use cases. Like the CodeGen model we released, which is CodeGen 1 last year, generates six different programming languages, the six popular ones. But the version that we’re using internally for our developers, we want to also give them the ability to customize Salesforce, be able to write Apex, maybe LWC. And also we are working on making it generate flows as well, like the metadata for flow so that you can converse in natural language and be able to automate the processes that you’re trying to accomplish. But the base foundational model is CodeGen.

Josh Birk:
Got it. And there’s a video that’s kind of in a demo, I think is making the rounds right now in a few different places. There’s a podcast, we only have audio. So walk me through a theoretical example in Visual Studio Code where if that’s CodeGen enabled, then I’m an Apex developer, how am I going to interact with the AI to say build out a class?

Sridhar Raghavan:
Yeah, so we did evaluate different ways to bring this capability into our developers ecosystem. And one simple, no fuss way was to just bring it into the IDE. Yeah. And it’s not something new. I mean, Copilot has already done that. There are other tools that are doing something similar. So we’re bringing to our developers a plugin within this code that will allow them to do simple things like auto complete. When I say auto complete, it’s going to fill in the entire function, not just the line as they’re typing.
We can also generate entire functions by just stating what you want in natural language within the IDE. So when I say within the IDE, there is like a command pallet at the top where you can go say SFDX colon, and then type your question and that’s how it generates the whole code. But when you’re using the auto complete feature, it’s there as part of the editing canvas. So it keeps filling in as you’re typing.

Josh Birk:
Got it.

Sridhar Raghavan:
And I mean, this is just a start. We are bringing in additional features, they will all be seamless, seamlessly integrated into the same kind of a user experience.

Josh Birk:
So where I’m doing my work today, you’re just basically adding in new layers where the AI can come in and, in an assistive way, generate or fix code.

Sridhar Raghavan:
Exactly. And by no means this code is going to be something that would indicate itself as a complete code that is ready to ship, but a human would need to make sure that the API keys are correct, and there will be placeholders for the human to intervene and fill in.

Josh Birk:
Yeah. I think this is one of the most important conversations I keep having with people is that what the AI is not applying here is critical thinking about how your code should be shipped. That’s still the job of the human, that’s still the job of the expert developer who knows the difference between boilerplate code that gets them a start versus the code that’s actually going to go talk to the API, get the data, put it in correctly in Salesforce and not break anything along the way.

Sridhar Raghavan:
Absolutely, absolutely. So this is built around the human being the central consumer of this, and this is not going to do the job that a human is intended to do, but it’s just assisting the developer in a way where the routine, like the boilerplate code example you mentioned is something that, so searching on Stack Overflow, I mean, we can generate something that’s more customized to your schema, for example, with the right column names and stuff. It makes it a little bit easier.

Josh Birk:
Yeah. It’s an ongoing joke on the show that I consider my real IDE is Stack Overflow, right? And so that’s why, that’s the easiest comparison to me is now I don’t have to go search through 4,000 other people’s code, it’ll just be there in Visual Studio for me to actually play with, and I love that. So one of the interesting things about Bard and ChatGPT is their conversational angle, right? They know what your intent was, but they also kind of know what was being said before. So you can nudge directions, you can ask it fresh questions, it’s all based on the history of the conversation. Are you looking into tools like that that could be helpful to a developer?

Sridhar Raghavan:
Yes, the short answer, but the way we’re thinking about that is bit more specific to things that our Salesforce developers are asking us, for example things like customers, some of the customers sit on a lot of legacy code. And in order to keep supporting those with new developers, there’s a lot of uncertainty about, “Hey, what is this piece of Apex doing? This was written four years ago.” So, explaining that and making those core changes less risky for new developers is an important thing. Along with understanding a repo in its entirety and knowing, “Okay, this function exists somewhere.” So if you’re asking for something, I can probably recommend you reuse that function or refactor it. Because some of these pain points are probably common for all developers, but for Salesforce, some of them are slightly more of a priority.

Josh Birk:
Right, [inaudible 00:22:34].

Sridhar Raghavan:
And also to add, it’s important to note that at Salesforce for us, we want to make sure that the code that is generated is also very efficient. And in our case, efficiency goes hand in hand with your database size and schema, and being able to effectively use all that knowledge to generate that recommendation code is important.

Josh Birk:
That makes a lot of sense. Tell me, in the back of your head, do you have concerns, or do you feel like there’s risks and ethics when it comes to this particular corner of AI? Or is it a relatively safe place for both you as somebody who’s helping bring this to market, as well as developers who might be using it?

Sridhar Raghavan:
Yeah, that’s a great question. And to be honest, we breaking new ground here, so we’ll have certain roadblocks that we’ll have to work and solve for. However, we work very closely with our ethics team, and we also have our legal and security team working together on projects such as this, and we have put together guidelines or a rubric for ourselves to keep us in check and make sure that we have all the items in the checklist satisfied before we launch the product.

Josh Birk:
Got it.

Sridhar Raghavan:
But I do foresee there will be bumps along the way, not just for us at Salesforce, but in general in this field, because this is new stuff for a lot of us.

Josh Birk:
Yeah, that makes a lot of sense. Safe harbor and totally theoretical, ballpark, guesstimate, etc., etc. What kind of timeline do you think you’re looking at here? When do you think developers will actually be able to play with it?

Sridhar Raghavan:
Yeah, so we announced that these new capabilities I was talking about as part of the developer productivity suite at TDX, which was two months ago. These are going to come fairly soon, and by fairly soon, I mean in months. So stay tuned. Yeah, we’re right now testing it internally, so it’s in that stage.

Josh Birk:
Okay, and that’s our show. Now before we go, I did ask after Sridhar’s favorite non-technical hobby and trust me everybody, I don’t even have a good pun for this one. This one came out of left field and I absolutely love it.

Sridhar Raghavan:
I don’t know how many of your audience would know this, but I like to grow corals in my reef tank. So that’s my side hobby.

Josh Birk:
What?

Sridhar Raghavan:
Yeah, I try.

Josh Birk:
That is so cool.

Sridhar Raghavan:
Yeah, that’s very time-consuming. My family hates it, but I love it.

Josh Birk:
How? That is a first, my friend. I’ve never heard that before. How did you get into that?

Sridhar Raghavan:
Well, I was always fascinated by the oceans and the underwater world in general, but I’ve never had the privilege of owning a piece of the ocean in my home. So I kind of have a small little tank with saltwater corals, the lighting, and all that good stuff.

Josh Birk:
I want to thank Sridhar for the great conversation and information. And as always, I want to thank you for listening. Now, if you want to learn more about this show, head on over to developer.salesforce.com/podcast where you can hear old episodes, see the show notes, and have links to your favorite podcast service. Thanks again everybody, and I’ll talk to you next week.