Panel Discussion: The AI Revolution in Software Development: From Code Writing to Quality Assurance

This panel discussion examines the role of artificial intelligence in software development and quality assurance. The speakers, seasoned professionals from AWS and other organizations, share their perspectives on how AI tools have transformed coding practices and productivity. They discuss the use of various AI-driven coding assistants like ChatGPT and AWS Q, which can simplify the coding process and aid in tasks such as generating code snippets and providing explanations. While there is enthusiasm for AI's potential benefits, the conversation also addresses limitations, quality concerns, and the implications for junior developers. The panel suggests that AI will not replace developers but will alter their roles, requiring new skills such as prompt engineering. In conclusion, the speakers emphasize the importance of continuous learning and adaptation in the evolving landscape of software development.

[Music] ladies and Gentlemen please welcome to panel discussion about the topic the AI revolution in software development from code writing to Quality Assurance hello and good morning everyone to the second onsite day for conference we will start the day with in this Hall with a nice panel and I would say we just don't want to waste time and uh I will get you folks started uh hi

morning everyone welcome today we will talk about Ai and software development and QA uh in general about software development because it's it's entire topic so say welcome to my guests I will uh give them time my name is Antonio sorry and I will give them some time to introduce themsel because I don't want to pronounce some someone's wrongly so I'll go good morning everybody my name is

Boaz I'm a leading the developer relations team inia for AWS and I've been in the software development world for a bit more than 20 years so I think uh it's an interesting perspective of where we are right now hopefully we'll go talk about it hello my name is Kristof conl and I'm Solutions architect and chief of data engineering department in uh CIA Poland uh I'm also uh

AWS Ambassador and Community Builder uh working quite a lot with with the cloud uh and I'm pretty much involved in gen topics and gen uh new ideas and future uh prospects with with this within this uh huge AI uh world and geni world so I'm looking forward to have a discussion my name is Roy waser uh study computer science Once Upon a Time in beginning of the

century then decided to do Master studies uh in philosophy of Technology with specialization in AI the study was all about what is going to be possible with a I what isn't so it's really interesting to see how some of those ideas evolved over time kept track of the AI field basically in my entire career a little bit less than 20 years um and I'm co-founder of the

software development consulty called open value thank you one more notice we we asked the moderator to to give us the questions here on the screen so you can pause the questions and we will try to respond in a real time to have some engagement with the audience so that could be interesting try it so the first questions for for for the crew here is like let's stay

for a while in a present like how are how are you using AI tools today and how that has impact in your everyday work comparing to for example two years ago you can start first but okay so coding is not my day-to-day job but I do code from time to time and part of my job is to try new things uh play with new toys new technologies

new stuff and uh since I don't know every language out there and unfortunately I don't have enough time to follow all the new features and new libraries and new tools I think that gen changed the way I've been working for many many years so I always say that the best code that I wrote was something I copied from a pick a website that you've been going to

uh to find the code examples or GTH up for somebody else today the best code I write is basically with uh some llm helping me to understand uh how to control this lead light with an iot library or how to integrate the latest JavaScript front end inside one of the demos I'm doing so yeah I I completely changed uh going to look for code online to look

for help from uh gen tools and I I don't think it's a it's a dramatic change because uh looking for Solutions was something we've been doing as Developers for very long time uh I started with books if any of you try that it's pretty interesting to look for coding books and then we move to website and forums and other things I just changed the tool it's faster

it's not more reliable I have to say not yet and uh this is something that I'm sure is going to progress a lot use more chat GPT or or Google now so I actually use uh surprise surprise AWS tools so we released a q developer several weeks ago it's basically an integrated tool inside your ID uh if you're using vs code or intellig it's basically sitting inside

your ID it looks at your code and I can just chat with it and it will give me the the missing parts I I like it a lot mainly because there is the layer of explanation not only throwing the code at you and now try to figure out what's going on you're getting a very nice layer of explanation of why it was suggested co-pilot competitor or uh

I assume yeah but with a slightly different experience okay same question for you too uh to be honest I'm on the same page as as as bu and the thing is that uh since a Solutions architect I'm don't really code uh dayto day uh but I'm doing quite a lot of PC's prototypes and different kind of things so usually this is the same experiences as yours uh

the thing is that whenever I would like to write something uh which is a boring tedious job to find out some code uh instead of doing the research on a Google or using stack Overflow I use the stack overflow on steroids so probably some some tools like CH GPT or similar uh but to be honest I would like to say it because you mentioned uh and and

the question was uh about using chat GPT or Amazon q and like if there are alternatives or co-pilot but these are very different Tools in fact and we sometimes mix uh mix the like what is a given tool in fact like people say for instance we are using chat GP so this is just a large language model but it's not it's just an application on top of

a large language model which is called gp4 or GPT 3.5 the same thing is for Amazon Q so it's based uh uh on it's built on top of of a model that is exposed in Amazon but Amazon exposes multiple models like anthropic clo or Amazon Titan or coher uh metal Lama 2 and these kind of things so we have to be very careful what we are actually

talking about this has also the impact on how do we use these tools because in fact if you consider using copilot in your day-to-day job it means that co-pilot is basically taking your open tabs or open code or the whole code as a context of the query to the large language model it's nothing more than that of course there are lots of different like we call them

guard rails to to pick the or to propose a good code for you but in fact it has to use a very uh long available context window so the prompt to the model so it's just like you uh deal with chat GPT or asking a short question and get a long answer this is a typical usage so this is the different thing you get a very long

question with a whole context and the very short answer with which is a piece of code but in fact I'm just avoiding in my day-to-day job uh tedious tasks or find things on Google and I if I want to run Aurora on AWS and want to make a quick module on terraform I don't really care I I'm just getting the code from whatever tool and then fixing

it but I don't have to write all the lines myself saves a lot of time okay um so actually I want want to ask the audience the same question kind of um how many people here use something like co-pilot in the day today job H quite a lot two firsts maybe that's quite interesting I also put out this question in our internal slack and of people that

responded half of them were using it in the normal coding uh job and I also asked them how satisfied are you with with the tool that you're using and uh they could rate it from one to three three is happy and one not so happy and but I was quite surprised I I always hear a lot of skepticism about uh well how useful it actually is especially

if you're a senior I mean you're better than the co-pilot kind of suggestions uh uh pop up all the time and there's probably some truth to it as well I think but most of the people were actually quite happy with u uh with co-pilot and how it helped them be more productive like if you ride a service for one domain class it's really easy to have co-pilot

write the other implementations uh based on what you already did for one domain class so a lot of lot of really good examples there and I also think the funny thing about this question is uh it falls really fast I mean if you look uh maybe 18 months ago um six months ago six months ago three months ago right yeah um any anything we say now about

the quality is probably uh different uh even in a few weeks but uh it's definitely definitely really really useful so if you need to do I need to do something with a language I'm not familiar with um so I didn't really do anything with python for instance and I thought that that's right um an easy uh Comm line chess game with with python U by only using

chat GPT in this this case and by the way also benchmarked with Google Bart at that time so this is like six seven months ago and the funny thing was with jbt it took me like 60 Minutes to create a really good working chess game that uses the fisser engine and I could just play with it come on line Google Bart was basically um it was using

uh um python syntax that didn't really work uh using of mentioning libraries that didn't existed and so on um probably B isn't even around now it's of course rebranded it's probably doing a better job now as well so but I think in those scenarios especially if you're not really familiar with maybe a certain library or language um well it can save you ton of hours of maybe

researching all all kinds of documentations uh so for those kinds of things uh super useful and for a lot of other things as well I would say can I ask a follow-up question on your survey so I I assume a lot of people are are happy with using those coding assistants do you have any insights regarding the quality of the code U because I I know some

internal data from AWS one of our customers uh one of our first customers with Amazon Q uh one of their engineering teams start using Q for the last four months they use it between 10 to 15 times a day which I think it's it's it's a nice number and the acceptance of the code suggestions is around 37% I think it's pretty High what's your experience with that

yeah so uh what I uh also saw actually actually uh discussion was started under the poll I did and uh what people said it's uh primarily useful to like uh write write the tests the standard stuff and and like do the implementation for other domain objects if you already did one for one surface um or to uh like bootstrap your your project for the scaffolding part and

uh maybe you do it a few times times and then uh after that you complete the code yourself so uh it's really helpful to get started basically I would say I think the quote code quality is not on the top of the senior Developers for example but it's it's logical that it's very useful for senior developers because they can understand difference between good and bad and basically

you are getting your personal assistance that will allow allow allow you code faster that means you will be more productive and that's win-win situation but having some Juniors who doesn't understand difference between good and bad quality it will end up as as disaster on the long term if they are just using llm models today uh on the other side as you said starting some MVP and bootstrapping

some projects or playing for your homework or whatever it's amazing tool that's basically where I'm mostly using like here's the prompt give me something and then I just adjust the code in a way I need but two months ago I needed to I wanted to test some uh we had some idea about it investing in cryptocurrencies it doesn't matter to go in detail so I want to

back test the idea of the of the of to back test the idea on the price of the Bitcoin so I I asked C GPT like can you give me the graph with everything blah blah and when I started asking more advanced questions to edit to the very simple application from the position of someone who understand the coding it was unable to reach the level that that

that I wanted so it end up that I just took the data put it in the Excel and solve the issue with 15 minutes but basic the start like connecting the apis to the coin gecko Etc it saved me like probably 2 3 hours of reading and reading documentation and testing the testing the ways how to do that and basically I'm not a JavaScript developer so it

was not my field of the expertise so also save me some time there so I think we are in we are in the area where uh where we have some really cool assistants but they are still not able to take over the entire software development and I'm not sure that that very even close to that so um one of the things I'd like to respond to is

about uh junior developers using jet GPT coding systems there's actually a nice uh paper uh written about this from the beginning of the year it's called refactoring versus refactoring nice title um don't know that who the artist were exactly um but what it basically says is um yes it's very very helpful if you're a junior developer to to uh helps your ability to write the code but

there's one uh one one problem here um you don't really learn the details and of course you can easily relate to this maybe if I don't know how many people here still write their own regular Expressions um I mean why would you there are good websites that can help you with that and jbt does a great great job and the effect is of course that you well

kind of kind of lose your own touch with writing these kinds kinds of things and this is just one example and if you're a junior and you never learn these uh coding details um or how to actually Implement certain things you're probably not going to internalize that it's going to be maybe quite hard to become eventually a meteor or senior developer that can that can maybe be

be a problem with u with using DD kind of things when we are also when we are talking about the problems we have in interesting question it's talking about the the using Code assistance tools is I share my sensitive code with the third party that will not be allowed what's your what's our take on this if I can try to answer the uh the thing is that

we always share the code with someone and it's not something specific and I always make an analogy with these kind of uh problems or issues with the customers which say for instance some of the customers some of our customers say we cannot use co-pilot because it's dangerous or it might be dangerous this is absolutely the same story like 8 n years ago with a cloud so everyone

wanted to say we don't want to go to Cloud because it's someone else's computers it's data center we don't control we have this kind of like fear like of going into Cloud but why we actually did why we migrated into Cloud it's because we trusted and why we trusted the cloud it's because you like Amazon has all the certifications ISO things all the guarantees and Sh responsibility

models and these kind of things and we learned that it works so this is the same case if you go into the details for instance I'm I'm a fan of Amazon bedrock and I'm using this service quite a lot for uh doing the applications for the users and uh it has all the certifications and guaranties needed that my data or the users data or the customers data

will not be used to train the models once you get that then it's the same thing as using any cloud service because in fact you're using GitHub to put your code there uh you uh so this is someone else's uh service databases computers and everything then you put your code onto cloud machines which is someone else's computers so you always share your code with third party the

only thing is at the end of the day is the trust and that the customers actually read all the uh legal things which are behind the guarantees that the given cloud or given uh model vendor gives you and that's basically all because at the end of the day the customers pay attention to compliance not to the safety of your code in fact and I think for the

for the co-pilot that you can turn off sharing your code so that's a nice feature to have and I think also uh um if you're really worried about this there are quite some really good local uh models you can use with with coding systems as well they based on llama they work quite well apparently I haven't done too much with them myself but uh there was also

one thing that was mentioned in our discussions a few people were actually doing that they say it's almost as good the scope viot so that's another way maybe to solve this issue if you think that's the case for you but you have to purchase a server you have to install that or you make your on your local machine but it then takes your resources so it's a

trade of oh but if you're on Lama locally it's okay it's surprisingly uh uh well it's it's using resource but I would say it's difficult uh to with an M3 it's it's working nicely but uh I I think I I fully agree with what was said here I'm just want to say that uh eventually you need to separate between sending the code to the vendor to perform

the ask you just did and then what happens with the code afterwards and this is basically make sure you read the end user license agreement make sure you understand what the vendor is doing with that obviously if you don't want to do that at all if you don't want to send anything working locally is an option but uh I think you're losing a lot of the advantages

of those online tools the fact that they're being constantly updated the fact that they're distributed performance if paying by local resources or being able to distribute it across larger teams so I don't want to start managing 50 developers installing Lama on their laptops and making sure it's up to dat and so on I I forgot about it many years ago I don't want to go back there

but once again if you have to do that if it's compliancy if other reasons uh it's absolutely possible today and and it will get better we're going to get smaller and smaller models for local usage I know people running uh llm models today on Ras Pi so they're small they're tiny they're doing very very specific specific things but it's possible I'm waiting for the first one for

esp32 or something like that but there is a trade of here if we use the local model they usually or pretty much always they are smaller and they have smaller context windows so less code can get into the prompt into the question to uh to the model which might increase the quality of what's actually uh returned because we cannot forget about that that gen well large language

models is in fact a stochastic machine so it's like returning you something that is most probably uh in the most probably in the given context it's it's indeterministic it doesn't understand the code we we we we have to remember that it doesn't understand the world anything it's just a probabilistic machine nothing more yeah so I I would uh like to say that it is a really good

thing that we're not 100% so maybe you guys are both from AWS but it's also a good thing that we're not 100% depending on like large companies that offer service that we are going to need in our day-to-day job that we can still do it without uh uh well um AWS or Google or Microsoft in the end it should be one option it should be a tool

that you're using based on on the use case yeah okay we have another uh interesting question so because we will face the wrong results or fake results from the artificial intelligence will that make us unable to trust real intelligent people I can jump on this so there are there is one problem with llms currently that that I noticed so basically the llms are trained on the data

that people created but now more and more data is created by AI so at some point AI will start learning from itself and I don't know how and will that be a quality content so my point on this is that actually real intelligent people will be more worth in in the future comparing even to today because actually they will be the people who can resolve what's actually

true what's what's fake so that's I think a great value because we are still intelligent people like and AI from my point of view today is not intelligent it's just great like it's not intelligence but but the name the name is stupid by itself but it is it is not even new technology it is technology that exists from the 80s and it's just pure math and today

we have enough computer power to actually use it and we have enough data to actually use it so it's not intelligent system that's the that's when I said when you are building MVPs you still need a person or experienced developer to to take things to the better place to make it better I tend to disagree here also maybe for the sake of discussion a little bit so

um what is intelligence then um the thing is if you were really good at math under 20 years ago everybody would say that's a really intelligent person then the calculator came along and we know we basically stopped using how well somebody can calculate do do divisions multiplication as a way to determine how smart somebody is how intelligent somebody is okay um bunch of other technology was introduced

then we were able to beat of course Gary Kasparov deep blue in the '90s before that everybody would say somebody that's a grandmas at Jess that's probably one of the most intelligent persons in the world no longer using that as an analogy to what's really what intelligence is really about so we we shift the definition of what is intelligent as it suits us we would like it

to be something that we as humans are good in and as soon as a computer can do it ah no that's that that's not really intelligence that's that's just well raise you pure math applied statistics and so on so the question is if uh we now come up with something and we say yeah but we can do this or that and a computer can do this I

I would say if we would have done this panel like a year ago we would say yeah but a computer cannot generate like good-looking video or maybe create a music uh that's creative well you've seen I think it everybody has seen the the cool examples and has played around a bit with Sora and other Technologies the music sounds really good good um sometimes um so yeah um

I I'm not so sure if if the system is not intelligent also because we don't really have a good definition of what intelligence actually is there is actually good definition about intelligence so uh if you can solve problem that you never saw in your life that means intelligence and calculator for example cannot solve anything except the math as you and you as a person you can solve

something that didn't exist in the past and that's what llms cannot do today how can you prove that that llm cannot cannot solve a problem that has not been solved before I mean it can use the entire context of the internet it can make correlations that maybe never been made before but they can I'm 100% guys I think it's a little philosophical then I'm not saying it

will not be changed in the future because like if you have better knowledge and a background of course you can solve more issues but currently I don't see that as as something that is real that's just my person I think that that that that's why uh the data scientists and and the scientists that are actually involved in artificial intelligence treat usually the term artificial intelligence as a

marketing buzzword rather than really useful term because nobody really knows or can by engineering or in in engineering terms define what intelligent is then probably uh the better term is usually machine learning but machine learning is like an old term and everyone says uh if you're an ml engineer then you are a boomer probably then but in fact this is pretty much the same the only difference

and I always when when speaking with with my customers there is always a case uh we would like to speak about Ai and they don't they say AI but they mean gen Ai and I say there are different skills to uh Implement things connected with machine learning and gen because in gen most of us like I think 100% of us don't create gen models because it requires

lots of money lots of uh resources uh lots of data and everything it's super difficult but many of us can create machine learning models using the old good old uh algorithms even the easiest one or to neural networks it doesn't really matter so I say if we are speaking about gen then we are speaking about the application and using the services like Amazon bedrock or the models

that exist in the world but if we are speaking about the ml then we are looking someone that is able at least to multiply matrices one by one and and and know you know the details of the algorithm this is a different thing and uh but this is again this is a marketing world and we have to deal with it and we have to explain to the

customers again and again what what is I'm sorry what is the difference between I just want to that the question was about trust and and I think uh well a very nice discussion here and eventually I think we are constantly changing our intelligence uh so yeah there is the the definition but it's in in terms of what people perceive out of it is changing constantly and we

will change it a lot I just want to remind you we are in very very early stage somebody told me a few weeks ago that uh the stage we are at right now if anybody was was here around when iPhone was launched you remember the flashlight the flashlight app it was a app it was not a built-in it was an app it was an amazing app like

first time seeing that who oh my God this is where we are right now with generative AI we are in the flashlight app era just think about the flashlight and what you're doing with your iPhone today and try to imagine how it's going to look like so this is how it's going to change we are going to change the way we think about things we are going

to change the way we use things and it will change our definition it will change our trust pattern as well so we need to be ready for that I don't think there is much we can do because nobody can guess how this panel is going to look like in a year yeah let's talk about future a little bit okay uh okay we have one more question if

AI code assistant tools reach a level that that it does the most do you think that that div one will decrease that was the question for the end of the panel but we can skip to that so how do you see the developers jobs in the future uh are are we going to replace them with AI or or we will get some new roles in in our

industry or not just developers software Architects uh qas whatever like we we usually when we are talking about software development we are talking about developers but there is much many things in software development not just developers so how do you see software development in the future can I be optimistic yeah it's like I've you know I'm a developer from uh I think 1998 or something like that

and I've seen a lot of revolutions uh the first one was intellisense in net and everybody said well we have intelligence now the developers will not be needed anymore or less of them because it's easy to develop now we've seen a lot of I've seen a lot of new Frameworks new new capabilities new tools new devices and with every such a small or big revolution there was

something like de developers will not be needed anymore and apparently it was like the contrary uh there was more and more developers needed and I'm optimistic because gen is not a threat for us or at least it's not a threat for those of us who are uh who want to learn and apply new things new ideas and are creative uh but it gives us a lot of

new possibilities like I have always one story from my company when we had couple of years ago we uh discussed about uh a tool that would be very nice that will record our meetings make uh like a um make notes from these meetings send them with an email with a summary an action points and we said oh it would be great but it's not doable it's impossible

to do that well just give me two days I will do this tool and this is built in in teams I think it's it's a plugin because it now is feasible to do that but how many possible tools are how many tools are possible to be developed with the possibilities of gen uh and there are a lot of needs that we are not even aware of it's

like with smartphones they entered the market and everybody said why do we need this a phone without flashlight yeah with a and without the keyboard it will be difficult so many apps why I need them now everybody takes a smartphone and installs 30 ups at the first for the first the first day so yeah but uh the fact that it hasn't um that all those no Technologies

uh haven't uh caused haven't been a cause for the declining needs of developers so far doesn't mean it can't change I mean the calculators we just talked about they have been replaced by a calculator so uh it could happen who knows it's always fun of course to predict the future I think uh um in the end it probably won't be the case because um you need a

human to interact with a system we as humans have uh beliefs desires things we want at this moment in time but anybody that has done an IT project knows that it will change probably the next couple of days already what you actually want and how you want it as soon as you see the first results and I think it will be nearly impossible for any um any

well AI based system to understand our our true needs and biases and create the right kind of code you need like a we as as a developer also make an interpretation about what our clients wants us to do I can imagine that maybe um we get new programming languages so um if you look 60 years ago um maybe we doing C C++ if you were uh back

in the days the punch card operators we had assembly and we we started adding F section layers on top of it right and if you work with Java you have the jvm and nowadays most people write functional code and is also kind of an extraction layer on top of decorative languages you could say um so maybe there's there's a new exraction layer where well you you give

a different kind of instruction um as a developer that translates deep down into into code that's being compiled somewhere I don't know that could maybe could maybe make sense I can just add that every technology Revolution change something in the employment market and I think that AI will change things that we are not aware it's not only developers I think many other professions are going to change

I do think that we are in a very early days of a huge wave of digital transformation we see it all around us most of us I I we're in a technology conference we are working in technology companies or technology roles so we see it around us every day but very big parts of the world very parts of the industry are not there yet in terms of

this digital transformation and machine learning Genna will help to increase the pace there were all kind of events recently like like covid that did a huge boost in digital transformation I think this is another another opportunity for that and I think developers will have a very very center role in this process I do think we are going to change as as you mentioned we're going to change

the way we're working I don't know if it's a new language I don't know if the path into the developer Persona is going to change probably yes probably people will feel more comfortable to start we'll have better tools and I think we'll have to learn new skills and I hope you're doing that already prompt engineering is something that I think everybody should invest time I mean to

understand how those machines are working behind the scenes to improve their P pumpt skills and this is changing a lot like you can sit next to somebody doing good prompts or doing very basic prompts and you'll see the change in the results and it's pretty amazing so invest time in that it's it's it's a great point with prompt engineering because uh uh for instance Amazon Bedrock has

a feature which is called agents uh this is just a pattern to use llms when you would like to uh have the dynamic results in your response like for instance you can imagine that you are asking your application or your Bot built on top of llm about the weather in Tokyo for instance and it cannot has have the knowledge the current knowledge so we have to request

make a request to an external service that holds the uh results for uh for the weather forecast or these kind of things uh but whenever you you you can run an agent in Amazon bed within a couple of clicks but then you can observe what kind of prompt is actually generated by uh by Bedrock uh behind the scenes and it's absolutely amazing so you I I could

not even write that myself so it's super complex and prompt engineering is really uh very very good very very valuable skill right now but I don't think that only prompt engineering will be a new skill like managing these agents or managing the environments for them so a kind of new let's say a devop like role for managing the models uh or even evaluating them because there are

a lot of ways to evaluate the results of the model if you're asking a model who is the president of the United States you can validate that by if it's a good answer or not but if you're trying to generate a marketing text about some products you cannot evaluate that using any objective measures you can say it's good or not but you have to have a human

in the middle that human in the loop that will read that and say it's okay or not uh or maybe you can employ some additional automation tools to say it's toxic for instance model to check it yes use another model to check it is a good idea as well but not not always returns the good results and the more sensitive text you're generating the less valuable it

is but then you have a lot of new potential ways of developing things uh for instance there are a lot of tools right now to make so-called llm Ops so all the oper like operationalization around llms there are new skills new new things so what I could say if we learn and we're creative then we don't have to uh be afraid of the future I think you

covered everything just just one just one small notice that I have to edit here like if you have two developers of the same skill let's say senior mid level doesn't matter and one is using AI assistant and another is not using there is obvious who will be the faster and more productive it's just you have your personal assistant helping you there trying to figure out some problems

that you maybe never saw or something like that so it's it's like steroids in a sport like you just get Advantage so if you not using start using it that's my that's my advice we have another question we have like seven eight more minutes we have another question about chat GPT here like do do you use custom chat gpts and any tips on creating them have you

tried scraping documentation and feeding them to the GPT so are we referring to different models I'm not sure like like I I did try scraping some documents and feeding them to the that's very easy to do with a cad GPT basically you can just ask it if you have premium CAD GPT 4 you can do it out out of the box just don't feed it with h

with some privacy documents or ET because in in in in a terms it says that they will use that for for learning the model so maybe at some point it will be public but if you want to keep it private use the apis so that's how you can solve that it's brilliant uh so um I um we we had a client a large International client and they

gave me a gigantic contract and uh well we're a rather small company and uh we don't have a a legal person available we could of course go in the market and find somebody for a few hundred EUR an hour to read read it so I was a bit worried so i' I took out indeed some of the sensitive parts and I gave it to GPD said can

you please highlight the things you think that might be important for a company such as ours and explained of course what kind of company we are it was really fantastic it it highlights everything uh you want to know in in complex contracts I also asked how should it be changed give me some suggestions gave that back to the legal department of the company they actually changed it

so that um yeah that's really really brilliant so yes I try that and I really like it but that that raises another risk over Reliance on AI like you can trust the AI to the extent that you actually do not have any doubts of the quality of what you return but again we have to keep in mind that it's just an in that in deterministic model that

Returns the words not the whole context it doesn't understand the document at all yeah well I would say I tend to be a bit philosoph philosophical again here so what's understanding them yeah is my question I I agree that it is so if you know how to ride a bike it's something you cannot put into words right it's uh something that you know how to do um

embodied some somehow um and this because you're Dutch yeah soon as be born probably but but um I I would say uh uh so yes this is something uh LM cannot do because it is trained on text so it doesn't embody something while it does recognize pattern so that is kind of you can do some kind of emergent understanding by doing correlations right so that is kind

of the same as understanding when a human understands something we also ah now we understand it we can link this to something I already know and now now I understand it it's is also kind of a correlation so it's it's hard to say if that's really a different kind of understanding or not it's probably not the same thing but is it not understanding I'm not so sure

it is not sorry I have to answer that because this is super important it is not understanding for the simple fact that what uh I'm always asked uh or maybe not always sometimes ask why the large language models actually are named like that uh and we can uh imagine that like it's a big mesh of words and all of them are connected with each other it's not

like all with the other with all others but most of them are connected within a numeric value every connection has a numeric value and every numeric value depends on the context in which these two words actually existed in the context so in fact what the model is is like a big mesh of connected words that's why it's large language model because it models the language but it

models it in a really stupid way because it just learns the connection between words and if we put some text like a question it tries to find find the optimal way from one word to to the other one it's like optimization mathematical optimization nothing more while we when understanding things we understand why we are connecting these words together and this is a difference so we know what's

the purpose of what we do the large language model does not know what's the purpose it just connects words that's the difference for me of course I think you just uh put into words in a really good way what the Chinese room Ford experiment is about yes exactly yeah so I don't know if the audience familiar with this it's like a famous for experiment from 1980 already

from John surl and he says uh imagine a person that doesn't understand Chinese he's locked in in a room without uh uh without any glasses he cannot look outside and um he has has an instruction book on how to manipulate symbols in his own language and symbols are being put in in the room and these are Chinese symbols and in the instruction book he sees oh if

I get these symbols in I should put other symbols out so he has a box of symbols that he can return and apparently by doing that he's answering questions in Chinese so does anyone here now thinks that that man or that person is able to to understand Chinese of course not and that's kind kind of the same thing that you're saying but from an observer there's no

difference exactly so I think that we should like discuss more philosophical questions afterwards so we have two more minutes so what's your final thoughts for today uh I I just want to come back for a second to the previous question and say that documentation is a is a great example for generative AI really really helping helping me on the dayto day um Q is integrated now in

the AWS console if you haven't seen that look on the right corner there is an ation for Q you can ask them any question about your environment and get it immediately uh so that works working really really well for the licensing question uh my my answer would be read the manuals and read the LI the license agreement uh because it's very very tricky I know some lawyers

dealing specifically with that it's it's pretty tricky uh Last Words bright new future yeah thank you guys