Computer Programming is Dead; Long Live AI-First Programming - Stephen Chin, Cassandra Chin

This talk explores the challenges and opportunities associated with using AI in learning, particularly in programming and development. The speaker discusses the differences between effective human teaching and the limitations of large language models (LLMs). They emphasize the importance of grounding prompts with accurate context to avoid misinformation and increase learning efficacy. Through examples like coding tasks and word problems, the speaker illustrates how LLMs can misinterpret instructions without sufficient background. The session also introduces an AI-first learning methodology designed to enhance the educational experience by leveraging AI tools more effectively. The speaker advocates for a return to traditional learning methods like books while integrating AI technologies responsibly.

Great to be back at the GIDS conference. I I think that the best way of describing this conference is, you know, GIDS with a the huge audience and with all these technologists. Um this is this is kind of like um Devoxx Belgium, but then in India. And you just you just look at Devoxx Belgium. >> Yeah, I did. And there's definitely a lot of people like this

and a really great friendly community. >> So, we're going to chat a bit about AI first development and what all of you can do to be better learners using AI technology. But before we get started, Cassandra's going to talk a little bit about what it means to be a good teacher. >> So, like before we really get into learning with AI, it's really good to go back

to our roots and think about what really makes a good teacher. So, this is a photo from my elementary school. Uh my elementary school teachers are some of the best teachers. I remember learning a lot from them. I had a fourth grade teacher, Miss Gray. And I remember she taught me how to write cursive, and I still sign my name in cursive. So, I think what she

taught me really least has stuck to me. And great teachers also make learning very fun. Uh teachers make learning fun by doing projects with students. So, rather than just reading a book, maybe you do an art projects where you create something like a scene that happened in the book, and that really helps you to stick that memory to you. And even my sister can learn from teachers.

I think that means something. She did this aquarium picture. It's actually really good. She even had the same teacher, Miss Gray. And she's a little bit older now, and now she has her own aquarium. >> Uh is that an aquarium? >> I would like to think it is. Actually, she dumped out this tank and made a new one, so >> How did she learn? >> And great

teachers also bring their own knowledge to the classroom. They don't just read the textbook. They really teach from their own knowledge. And I think a great example of this is in the US, where I'm from, where I'm from, they teach history about Christopher Columbus. They're saying he's the guy who founded America, landed there, great did things in the textbook. But in reality, he was actually quite a

terrible human, and he destroyed the habitat of the local Native Americans there. Thankfully, they actually changed the holiday and renamed it from Columbus Day to Indigenous Peoples' Day. But back when I was learning this, they didn't quite change it yet. So, I think teachers add their own little extra, which is better than the textbooks. And teachers also give a great amount of care to the students. So,

they don't just read the content and give it to you. If you're stuck, they will spend that extra time with you. Uh in my chemistry class, I found it very difficult. There's these unit conversions, and it's very math-heavy. So, my teacher actually spent extra time during lunch to really teach me the concepts. I think that's unique to And I think we're also very lucky to be here

at Gids today, cuz the speakers are amazing teachers for developers here today. These are just some of the speakers. Like, Mary is a community jug leader. She teaches runs the Java user group, which is really educational. And Daniel, oh, he gives I see him everywhere. He's always teaching people, always at conferences speaking. And you guys all just saw Venkat. He's absolutely amazing. He's actually a professor himself.

He teaches his own students. And he's giving probably the most talks here. >> And he just finished his own conference last week. So, give another round of applause to Dr. Venkat, who's amazing. And flew all the way out here after his own conference to make sure that all of us here have amazing content. So, thank you, Venkat. But, those are amazing speaker teachers. LLMs are really bad

teachers. And we use them. We use them for coding. We use them for automating tasks. But, how much are you actually learning from using LLMs versus just unlearning and having it do things for you instead of actually learning the process. And to kind of show some of the um capabilities we'll be able to do with LLMs, we're going to talk about an AI-first methodology. And the first

step of the AI-first first methodology is grounded. So, if you want to learn from LLMs, you have to give it the right grounding, the right information, so you can actually get the right responses back rather than getting information which is inaccurate or possibly flawed. It has to be reproducible where we can get the same results time and time again, so we can learn. This is how we

learn as humans. We learn by repetition, we learn by doing things. This is how, for example, you learn math or you learn different sciences. Academic, where it actually is grounded in real facts, real information. Personalized, so that it understands about you, what your knowledge base is, what you've learned, and then it carries that information over multiple exchanges or it's episodic together with us. And to describe the

first one for grounded, we're going to first talk about hallucinations. Who Who here is familiar with hallucinations in LLMs or has experienced LLMs? Okay, almost almost everyone in the room here. But we're going to exemplify this with a word problem, and it'll be the kids audience versus OpenAI, and we'll see how you all do versus the LLM. So the word problem is a school has an elective

course choice of computer science school sports or home economics. We have a class of 36 students um in a grid of desks. Um there is no girl sitting next to another girl. No girls are sitting diagonally. What's the maximum number of girls this class layout allows? Okay, so given this problem, how would you all go about solving this? And um does anyone Anyone have any like a

grounding or how you would start thinking about this? And if you if you have an idea, just just shout it out or um raise your hand if you want to if you want to say something quickly. >> Sounds like a graph problem. >> S- Sorry? >> Sounds like a graph problem. Okay, so you you definitely want to work for Neo4j, and we love everything with graphs. Now

um this actually I I I think it might be more of a a row and column problem because we need to know the class layout and we need to know what what we want to do. And there's a bunch of different layouts that this sort of class layout can allow. Um the most obvious one is it could be a square, it could be 6 by 6, it

could be 3 by 12, 4 by 9, 2 by 18, maybe even 1 by 36, but that's more of a row. I I I don't know how that would reasonably work in a in a classroom. And then using this, we can calculate the number girls. So, if we, you know, kind of leave a space horizontally and vertically, um and diagonally, we can fit in a different number

of girls in different class layouts, and it seems like if we if we did the 3 by 12, that's pretty good, right? 12 12 girls? How How's the human consensus? Do we think Do we think that's the answer? Yeah? Okay. So, I think I think we have consensus with the humans. Now, let's see how the OpenAI response did. Now, it's interesting when you look at the response

details, um because it grounded on a specific problem, which is a non-attacking kings paper. And basically, this is a known problem where, you know, that we set this up so it behaves kind of like the king attack movement in chess. Chessboards are always square. I mean, typically 12 by 12. And therefore, it assumed that it was a square chessboard that it was solving for. It calculated the

maximum number of girls, and it got a lay an answer of nine. So, you can see the grounding on the information makes a huge difference in the responses you can get. And if you don't ground what you're asking the LM, this is a huge problem. Let me bring it back to a real-world example. So, this is um one of my Christmas projects. I was I was building

a new computer. Um I thought it would be clever to set up a a RAID array, and then to do a fancy um hyper-virtualized >> Why do you need two graphics cards? Isn't that too many? >> Uh yeah. Yeah. It So So, when when you're an engineer, more hardware always solves pretty much every problem. Faster hardware, more hardware, and um I was solving for this problem, which

is how I could optimize storage. So, I'm using ZFS file system. You need to figure out the right checksum. I asked the Open AI um what's the FS flags I should use for an encrypted file system. Now, it gave me a response. It says, "Use SHA-1 or Blake3 checksums for better data protection." And if I blindly took this, it would be it would be fine, but this

was non-optimal cuz cuz it was confusing encryption so with your data is encrypted versus encrypted checksums. And in this case, my I I what what I needed is I needed something which would give me high performance. And Blake Blake3 is the optimum algorithm for that, whereas instead it recommended I use an encrypted checksum, which won't actually help with the data integrity. And so, whenever you ask a

question, it's very important to ground in the right information to get the right answer, or it will confuse things. It will confuse things which are similar, similar words, similar technologies, but not actually the type of problem you're solving. >> Now, that's very interesting, but you almost burned down our house. When you were applying the thermal paste on the CPU chip, you forgot to remove the packaging. >>

Um let's let's forget about that. >> No, no, no. I have a video to prove it. >> Oh. Yeah, okay. So So this never happens. Don't try this at home. Um but that's definitely not the way to apply thermal paste to your to your >> Now, LLMs are also very non-deterministic. If you type into the ChatGPT window, you'll never get the same response back. And this can

be very difficult when you're trying to learn. So I remember when I was younger, I have this math homework assignment where I'm solve five math problems online. So you think I should just be done after solving five math problems, but for each problem I get wrong, they add a new problem and it's completely random. So, that's extremely frustrating for learners. And you also see a lot of

non-determinism in AI. So, with this example, I said, "Write a simple program to sort donuts." And I ran this twice after flushing the context window. So, the first program gives me return sorted donuts and the second program says donuts.sort. So, these both work. They will both sort lists. But for a learner, this is very confusing. Why are there two sort functions and they both do different things?

So, the AI is not the best way to learn. It's better to have curated content. And you can also see that AIs are very random when it comes to images. I put in the prompt, "Draw a cat jumping over the moon." And it looks like the first image is more of a kitten and the second more It's more of an adult cat. >> So, did you just

add this in cuz you like cats? >> You know that cats are the best in the world, especially my pet Mochi. My fluffy cat. >> Yeah, he is pretty cute. >> And dealing with non-determinism in AI can be very frustrating. it's important to tell the AI exactly what you want. A poor prompt would be, "Teach me how to build a UI." And the AI doesn't know what

framework, what technology, or what programming language to use. So, it'll give you something A better way better way to specify with AI is to give it more context and information. You basically have to know more than the AI before you even prompt So, a good prompt is teach me how to build a responsive web UI using Spring MVC. So, with this, you'll get more reliable results. And

LLMs are better at coding than teaching. They're really good at just doing the work for you. They'll always come up with something, but if you ask it what it just did or try to explain it to you, it is really bad at that. So, if you ask the AI to write a program to do a selection sort, it will happily give you a classic selection sort algorithm.

But, if you're a learner seeing a selection sort algorithm for the first time, it's very difficult to look at this and piece it together and try to figure out what's going on. So, you're thinking you must just ask the LLM to teach you. So, you write a prompt, teach me how to code a selection sort algorithm. So, this is the output for the AI. It kind of

explains what a selection sort does. Uh selection sort repeatedly finds the smallest element in the unsorted portion of a list, and moves it to the correct position at the front. And it continues down and tries to explain. But, as you go down, it gives you examples of lists. And eventually, it just codes for you without much explanation. >> did it for you. Why would you read the

rest of that? >> Good point. I'll just submit that for my homework. I teach a lot of kids workshops, and I sometimes this is close to what LLMs do. So, I have a lot of volunteers who help the kids out with workshops. Uh this is a Minecraft modding workshop. It's a bit too hard for the student, so the volunteer is He has his hand on the mouse

and he's doing the workshop for the kid. So, I think this would be the equivalent of an LLM just giving you the answer. Now, I think a better way of learning and how we humans actually learn is by teaching. So, Hippy Hacker is a really good teaching. He has his hands on the can, not on the mouse, and he's verbally explaining to the kid how to do

it. So, this way she is doing it herself and even when the volunteer walks away, she'll remember what she just did and be able to continue on. But, I don't think LLMs are quite capable of teaching us like this. So, uh we need to stop LLMs from doing the code for you. So, with this prompt, teach me without showing the answers so I can learn. So, obviously,

the LLM won't give us the answer this time. It'll teach us properly. It'll be great. There you go. >> Okay, so this is the actual feed prompt it the feedback it gave. Now, it's a lot of text. I'm not sure who can actually learn from this. >> Yeah, so this is essentially the equivalent of pseudo code. It tries to describe exactly the algorithm in words. So, still

didn't do it. >> So, clearly the LLM can't quite do it. >> But, Dr. Venkat can do it. Here's Here's an amazing book, Cruising Along with Java. >> So, I think we're really back to reading books. If you remember before the internet age, the only way to learn programming and any new technology is to go to a library, bookstore, and buy a But then uh the search

engine came along and it got easier to find information and tutorials online. But I think we've actually regressed a bit because now the internet is basically impossible to use with all the AI-generated content on it. So I think books are especially important now. If you want to learn a new technology, the latest frameworks and things, just pick up a good book. So we have really good books

by professors and technologists like Kai and Venkat. And JavaFX books for anyone who's still learning it. >> All right. So give a big round of applause if you love JavaFX. All right. See, I'm vindicated. >> Yeah. >> Okay. And I'm going to talk a bit about how to personalize LLMs and and get them to understand you. And I I think that part of this is understanding how

LLMs work, right? So they they really don't know anything about us because they're designed on a very simple premise. So they're first of all, they they encode things in word vectors. Um this is a good way of describing how word vector does in in vector space. So you can see an example of um cat in English and what it's related to and it's relatively close to domestic

animals like like dogs and um pets. Cat is relatively far away from zoo animals like like pigs. the way that it encodes the vectors is by going through a bunch of transformer phases. Here's an example, a developer's best friend to write code is AI? Maybe. And then the maybe one transformer phase which figure out parts of speech so it's okay developer and AI or are both nouns

or actors writing is an action. Then the next phase might start to pull out the relationships so the um the developer is the one who's helping to write the code the AI is the friend that also is writing the code so you can see now we're building relationships and LLMs do this by a large number of transformer phases a huge amount of GPT technology and um neural

networks that they kind of feed the next token through to um answer the question. So when you go through inference it'll basically take the the token series or the question you asked it in word vectors um and then convert that to the next most likely outcome. Now the reason why these give extremely good results is not because it's a complicated algorithm actually the GPT paper is very

easy to read and approachable if you want to understand how LLMs work it's because of the the amount of training. So GPT-3 was trained with 500 billion whereas the average human at the age of 10 is exposed to only 100 million words. So they just have a much larger context library of information at their disposal what this feels like when you're a human talking to an LLM

is kind of like pick picking a random book out of the library opening it up in the middle and starting to talk to it. It doesn't know anything about you it doesn't know anything about their your problem space about what you're doing but it can give very good generalized information from a knowledge set. >> Actually I see you do that a lot. You just open a book

and read the middle and you're done. >> May maybe I I think like an LLM a bit myself. a a poor prompt a non-personalized prompt would tell me about the new Java simplified syntax and if you if you try this yourself, you get very generic information about the modifications to the the main block, some of the automatic static imports, but it's not helpful to you, and it's

it's also not targeted. If you're an enterprise developer, and you need to use this on building systems, actually understanding the underlying class format changes and how it works is not the first answer it'll give you. But if you give it a more detailed prompt, I'm a senior Java developer working enterprise systems, tell me about the new simplified syntax, then it will give you more details about the

JVM changes which went into different versions of this JEP feature. So, the context of like what you're working on and how you're doing problems is extremely important when working at ends. LLMs are also episodic. So, they don't remember what they taught you before other than what gets fed back into it as previous context, and they're very very bad at this process. And when you look at it,

it makes sense in the transform and how LLMs are structured cuz when you give it a user prompt, it's basically just taking that prompt plus some information which may be your your IDE or the interface is feeding us on previous instructions and system messages, and it's feeding it through a transformer, the attention loop, and then giving you a response back. If you modify the input to the

LLM, and you give it the same seed, it will always give the same response back. So, it really only understands the context which you present to it. So, how do we the right context? How do we actually get the LLM to understand us better and not treat us like a substitute teacher who just walked into the classroom and knows nothing about the students? So, there's a few

ways we can do this. we can ask ask we can give it information about us from our profile. So, for example, in in this example I'll give you an example right off my LinkedIn and hand it to the We can answer the question based on other things we already learned, so we can keep maybe a Google Doc or a spreadsheet of information about us. Or best yet,

we can use a knowledge graph and populate that with information from our learnings and use this with as a context graph where we learn something new, it automatically populates and adds that to our memory. And that's actually one of the things which has been super popular this year is to um turn knowledge and memory systems into graphs that LMs are really good at writing and also really

good at querying. >> I think we need more graphs. Lots of graphs. >> Lots Lots of graphs. Should I wear my graph t-shirt today? >> Okay. So, as an example, the first one where we give it information. So, the question here I asked it was teach me how to use Angular based on what I already know from my LinkedIn profile. And rather than just giving me a

generic answer because it knows, well, Steve wrote the JavaFX book, he knows quite a bit about Java development, about enterprise systems. It said, "TypeScript Java for the browser." And it talks about, you know, Angular's written TypeScript, familiar if you come from Java, and it's comparing it to things I already know. So, this is a very effective way to learn new systems, to upskill on new technology is

first teach the LM what you know, and then it can give you better answers, it can shortcut a lot of that knowledge, and help us to learn better and faster. >> So, even though LMs are quite bad at teaching us, we still need to use AI to stay competitive in the workplace. So, employers are really demanding gen AI skills with developers. They A lot of your managers

got told by upper management to use AI cuz it'll magically make everything better and Job postings for generative AI engineers are up by seven times. Job postings requiring generative AI skills in other IT roles are up by 35 times. And computer science postings that mention at least two skills of to give a salary premium of 260%. So, I think more money's always good. >> And the fact

you're here at Gads learning about AI technology hearing from the experts, this This is really good for you, right? >> So, the computer science students aren't actually learning generative AI in the schools. So, this is the computer science placement rate 6 months after graduation. You can see that the students have a fairly high placement rate. You can see that the chart is going up. But then chat

GPT got introduced. And the computer science placement rate started tanking and students are not finding jobs. And it's actually worse than this graph shows because it doesn't include information about students who go to take a master's degree. There's also a increase in computer science masters after chat GPT. So, 209% students rather than going to get a job, they're pursuing a master's degree. But like problem with students

not getting jobs just gets delayed a little bit. So, the computer science masters students are the placements have dropped. Before it was 91%, but it's dropped to 71%. So, a full 20% less in job placements. >> Yeah, and this I mean this has been repeated by a whole bunch of folks. Um Ben Lorica, other folks who are looking at the AI industry where junior jobs, jobs for

developers who are just coming out of school are disappearing. Whereas senior developers and particularly senior developers who up skilled on AI actually have a lot of job opportunities. >> But next thankfully you guys are in the right place. There's a lot of AI talks at the conference. So now we have a lot of students who they really high engineering skills but they don't have a way to

make money so they'll go to hackathons to have pizza. We are writing a AI first programming book series to introduce this new AI first teaching methodology. And this is really how you can learn from the AI. So we have the cycle where you first prompt the AI to generate some And then you look at that code to learn from it and that's how you can grow as

a person. And since the AI always generates different random response when you ask it, we created this AI first extension. It's open source. But what this does is we've curated a curriculum with different prompts so that when you put prompt in, the AI first learning model will automatically pick up that prompt and give you a curated response. This makes it very consistent for a learner so that

you always get the same response back, the same code back so you can analyze it and really learn from it. >> Yeah, and one of the other challenges a lot of professors who I talked to were teaching students is you know, as professionals working for companies typically you have access to AI tools, you have maybe even a token budget you can use for using AI technology and

it's fairly accessible. But for students the free version of co-pilot like the tools which are available to you without paying are very limited and schools have still not caught up making this a priority to give their students access to AI technology. So, using something like this where you have a a free model that's backed on data that's repeatable makes AI technology much more accessible for people who

are learning to program. >> One of my friends goes to Stanford University. They're one of the top schools in the US and they only just recently added a base level AI tools beginning course and that's for a very advanced school. So, I think the schools need some catching up. So, we are writing the book series AI first programming. I am writing the Python edition and Stephen and

Jennifer are writing the Java edition. So, please look forward to it in the fall. >> Okay. So, thanks so much for having us at the Gids conference and enjoy the rest of the technical sessions.