Sara Tandowsky - Fireside Chat on Documentation

In this talk, Sarah Tandoski, CEO of Gitbook, discusses the ongoing challenges of documentation, emphasizing its importance in a fast-paced tech environment where outdated information can hinder user experience. The speaker highlights how rapid advancements in AI have exacerbated the need for up-to-date documentation, shifting the focus from traditional text in documents to interactive, machine-readable formats. She explains that various teams, including marketing and support, are increasingly engaged in the documentation process as it plays a critical role in user activation and success. The discussion explores how documentation is evolving into a collaborative, living system that integrates feedback from multiple sources, optimizing it for both humans and machines, and includes the integration of AI tools to facilitate writing and maintaining high-quality documentation.

The next topic is going to be super exciting because it's a topic we all kind of hate and love at the same time. We we love it when it's not our own and we hate it if it's our own. It's documentation of course. And we're going to speak with someone that is an expert in this area. It's the CEO of Gitbook. It's Sarah Tanoski. Tadoski. Ah that's

the one thing I wrote down. Tandoski. Yeah. Give a big hand everyone. >> Nice. >> Yeah. Thanks for uh yeah speaking with us about this super exciting but still always like annoying and often outdated um topic. Uh I mean not topic but our documentation is outdated. Like how how come that documentation is so frustrating? Yeah, that's a great question. Well, first, thanks for having me. Um, it's

been a problem and continues to be a problem and even more is a problem now when it comes to up-to-date documentation. Um, because things are moving at a much quicker speed with AI uh documentation, it's more noticeable when things are out of date, right? and and things have moved from search to uh chat experience and people expect correct answers quicker. Um so that problem is yeah top

of mind and for a lot of teams now. >> Yeah. And how like how come it's still often outdated like is is it the incentives? Is it the tooling? Is it Yeah. the culture? >> It's a bit of everything. I think when you think of traditional documentation, it was a tech writer or someone in a product team who thought of it as checklist item. So product of

chip and okay were the docs updated check yes deploy to documentation side and and that's the end. Um now that's not necessarily the case. So I think it comes into culture um what's impacting it's not just impacting humans now it's also impacting machines. Um so other teams are involved as well um and seem to care about what the documentation is for um who's using it and how

it affects their KPIs within the business as well. >> Okay. What what teams do you mean? >> Uh marketing is a big one that we see uh because especially for technical products documentation is really the first experience that users have with a product. That's how they're discovering that's how they're using um activating first with your product. So adoption activation are really key and they're really involved. We

see them often joining. It's not just product teams or engineering teams, but it's also marketing teams because yeah, it impacts what they're working on. Another big one is support and customer success. Of course, documentation can play a key role in their in their jobs as well and making sure that they can successfully help users be successful with their product. And as Gitbook, how do you deal with

that? Like how what does it changes for you? Like having all these other stakeholders suddenly involved, having documentation become even a marketing tool. >> Yeah, I mean it's great. I think the conversations has changed of, hey, maybe your docs don't look so great or maybe your search doesn't work so well or um yeah, the experience could be better to totally different conversations. people are coming in bought

in in how important documentation is. They know they need it uh for discovery of their tool. They know they need it for better experience for their their users. Um and so it's great to have those people in the conversation and it makes our job easier but also we're learning from the people that are actually working daytoday with users and customers and providing them with the tools they

need. >> Okay. So yesterday we heard from the Netlifi CEO that talked about going from user experience to agent experience since so many agents are using the web these days like do you see that as well and how do you deal with that? How do you anticipate that? >> Yeah, absolutely. Um an interesting trend that we've seen is documentation hosted across gitbook has uh it's accessed by

more than 50% of machines. So the shift has moved from just humans accessing it and reading it to really machines and that's kind of our our core user group. Of course humans are very much important as well. Um but there's a big shift. >> Okay. And have you optimized your product for that? like can they read markdown now and >> yes definitely. So um optimizing the structure

of your content making it readily available in markdown lm txt um mcp servers for your your your tools that's all out of the box and then I think that's table stakes I think for any documentation >> that's just standard now for everyone. >> Yes. >> Wow. And yeah, I think also a lot of people are now more tempted to kind of vibe coat their own docs solution.

Um, like what do you think about that and what are people perhaps underestimating? >> Yeah, we don't really see that yet within teams um at Gitbook. Think of course teams of one or you know an open source project maybe they're they're testing that out a bit more but I think documentation is not a one person problem. It's really about context and high quality documentation which requires lots

of different teams um lots of different tools. It's not just what comes from the code as well. So building that entire connected system while also allowing users to contribute in an easy way within their workflows is still remain key. >> Okay. And what what tools are you talking about then? >> Um so if you think there's context that lives in Slack, there's context lives in linear GitHub

um intercom support tooling analytics um all of those different places are things that impact the docs and will make them higher quality. So being able to to tap into that is is critical. >> And about uh writing documentation, like what what what do you see is a big shift that's happening? And can agents now write all of our documentation? Can we basically fire all the technical writers

>> Yeah, it's an interesting one. So we run this state of docs report um at Gitbook and it was our second year this year. So One thing we asked is how much technical writers and product team product managers are actually leveraging AI in both years and the jump was significant. It I think more than tripled in terms of how much people are actually leveraging it for writing

um and obviously of course editing and maintaining documentation but that has been a massive massive shift and change. So we've seen the role of technical writers or people working on documentation really change from pure writing and publishing to documentation site to content strategy and building more living systems, living knowledge systems that um they don't spend their day just writing. They can use that for a first draft

but accuracy still remains key. So having a human in the loop is is important. >> Yeah. Yeah. Living system, knowledge system sounds also more exciting than documentation. Is there also besides a nicer name some more content that would not be in normal documentation? >> Yeah, I think where documentation is moving is kind of becoming more of this company brain. So there's a lot that goes into the

way product's built um to what actually customers and users need to know about the product. That comes from code that comes from product briefs that comes from customers and feedback you have there and all of that playing into each other and being accessible in different places and being um fed information from those channels as well. And do you give agents now additional information like skills.mmd how to

navigate the docs and maybe like I I love making giant monor repos with way too many features and API. but how yeah how do you optimize also for agents? Yeah, the skilld file is an interesting one and one we're seeing a lot of adoption and people building in Gitbook directly alongside their docs. So you can easily build that file um within book if you can do it

in a nice way with permissions and versioning and all of that which is is important to make sure that they're accurate and having it distributed alongside your documentation so that agents and people that want to use your product have that information readily at hand. And about agents writing documentation, is it possible to have this completely automated? You see that people are doing this in every pull request

that they also write some documentation. >> Yes and no. So not on full autopilot. Um we we think there's a lot proactively that agents can do and can help with and can gather from um code and other other channels. But the way we structured it and why we think it really works is that it ends up in a change request still that is reviewed by someone and

making sure that tone consistency accuracy is all there so at the end of the day the end published documentation or knowledge that is available is accurate and correct. And do you see that it's moving towards a standardized like protocol or format that you think oh this is what agents love the best or this is what works best for marketing and this or external users and this is

a version for internal users and do you see different versions perhaps also coexisting at the same time of documentation? Yeah, I think this is what a lot of teams are trying to figure out now. Um, and us included. We ran a small kind of research project building a new API and and testing on a bunch of different platforms and then testing it with cloud and codecs and

other tools to see how they would um how they would be successful. And it it is there's differences there. And I think that's something that we're trying to figure out of how does that act, how can we actually make that easier for our teams that we work with to create different types that help AI become quicker when using their documentation. >> And is this something you can

also automate the testing for it or benchmarking it and tracking the performance of that? >> Yeah, that's something we're working on as well. Um, I think that's really important. We're very focused on kind of the analytics aspect. So figuring out users and machines are running into errors, what are they not finding? Um where are they getting stuck and kind of building that loop around that? And I

think the next step of that is okay what exactly how can we automate that and make sure that we fix what's where it's going wrong. >> And do you use L&Ms as a judge to also read it or have a testing suite for this? there's some open source tooling available. I think that's something we'll want to look at building in directly to Gitbook in the future. Um,

yeah. >> And what do you like if you would join any of our startups or like pet project teams? Um, what would you change as one of the first things in this fast-paced VIP coding environment? Yeah, I think the mentality and the if you start from day one, you're not going to end up in the situation that a lot of teams are moving away from now of

thinking it as the end document the end uh place you want to visit is the documentation site. So thinking of it from day one as this living system that goes both ways. Um you're learning as you go, but you're also being able to access that information in a quicker fashion. If you can set that up from day one, I think you're going to be in a much

better place going forward than having to kind of change both culture, tools, thinking process later on. >> Okay. So you'll you'll set some process in place that everyone is going to neatly spend two hours after two hours of fiber coding doing two hours of documentation or like >> no definitely not definitely not that you should >> if I have eight agents working parallel at the same time

just sketching out like my user interface or my app like it goes pretty quickly. like how how how how would I deal with that? >> Yeah, that should live alongside like we should your knowledge system should be learning as you're going through that and as you're building >> and should we all centralize all of the documentation or also use inline comments um in the code. >> I

think there should be some central place. Um where that lives is still to be determined. I think this concept of the company brain or brain of some sort is a big one. Um, and where the format that fits in, I think we're still seeing. >> And where do you see the future of Gitbook going? Do you see your be Yeah. >> Yeah, it's it's a great question.

Um, right now it's moving away from traditional documentation and we're really trying to help teams get there into a more modern setup. But what we're also seeing with AI is knowledge is not going anywhere. Context is key, knowledge is key and that remains critical. So building alongside that and making sure that companies are set up with high quality accurate knowledge for their systems, humans and machines is

yeah what we're here to do. >> Yeah. And about the um about the humans like reading it, I'm sure Gitbook is amazing for that. Um because you offer like the markdown version, you offer the MCP. Um but how do they in the first place find the right documentation or how what should we think about in terms of GEO or AEO like the SEO for agents? Uh how

how should we optimize our documentation for that? Yeah, having the right format available is important and not all documentation is available in markdown or LM txt formats. Um, so that's definitely one, making it easily indexed and accessible. I think that's probably the most critical thing um for now. And yeah, a lot of the same rules apply from SEO, traditional SEO, and the way you write content and

how it's formatted and things like that. Those would probably be the top two for now. >> Okay. And uh does LLM text still play a big role in this or do you see it's not that important? >> Yeah, it's funny because it got a lot of hype. Um and in actuality, we don't see much usage or adoption of it. It seems that the markdown files are the

most used and the most efficient. So what we're seeing is they're mostly reverting back to that. >> Yeah. And talking about usage like in you wrote this state of the documentation report like what are the big trends that you have witnessed besides the the things we've already discussed. >> Yeah. So definitely the adoption of just using AI for writing has been a massive massive shift. Um we're

also seeing a lot in that the way documentation is being viewed for uh adoption and activation. I think we see that a lot of products are more technical and typically that means they go directly to the docs. Um when users are evaluating a new tool they don't go read necessarily the marketing site. Um they go directly to the docs or they ask in cloud does this product

do exactly what I want it to do? Is it going to solve my problem? And that's where docs are key. And we see that being acknowledged and recognized by companies and teams a lot more globally this year. >> Yeah. I know that from my own experience now, if I look at some SAS tooling, the first thing I'm going to do, okay, can my agent interact with this?

You know, can I >> Does it have the request that I need? Um, but how Yeah. How how do you see that people interact differently with dogs? Like are people just using search or um is is there also a way I mean I know that a lot of docs have now copy paste to markdown which is pretty okay but um are there are there other ways that

people now exploring documentation and reading it? >> Yeah, I mean I would say the ones you listed are quite common. um being able to connect them or just look through askclaude um astrbt in general um building alongside them obviously in cursor uh things like that are also common things we see um but most documentation or good more modern documentation these days has some form of its own

chat experience as well directly right so if you're discovering a tool you want to know what it does you go there you ask the AI chat that's on the documentation like is this going to solve my problems Can I use this? Um, still being quite >> And you see it's now also more connected to AI agents that live on the website like customer service agents or uh

that are then directly connected into the >> Yeah, definitely. So, we see a huge shift. Um, typically there'd be some form of knowledge base or something that maybe lived with support teams in different tools. We see a lot more centralization of that alongside the docs because ultimately especially for product that information lives should live together in one place be searchable. So the connection and how much docs

is impacting support teams and answers they can get to customers faster is is huge. >> And um have you actually have you ever updated docu the documentation of Gitbook itself? >> Do you love documentation? Uh that's a good question. So I don't come from an engineering background. So it was less of a direct issue um for me personally but I think it it's critical when I mean

when I look at products when I go to discover things you have to look at the docs. >> Yeah. No gitbook looks very nice. I uh I enjoy it. Even the state of the state of documentation was completely made in Gitbook as well. Uh yes, a good portion of it. >> That that looks quite good. what what were the most funny things or biggest mistakes that you've

seen in this state of documentation? Um like uh report I don't know if there were necessarily mistakes. I think everyone's learning as they go. Um, but the interviews are super interesting because we see such a wide range of approaches and and ways that they approach documentation. I don't know if there's anything specifically that people are doing wrong. I would I would call out. I think it's everyone's

learning together and I think that's why we did the report in general is having it surface these different approaches what they're learning and so that we can all get to the end goal of high quality >> Um, one thing I really like about documentation is to be able to preview like how a request would look like, what my data would look like. Um, and now less and

less I have to do this myself, but um, is there an option to enter like your API key and your agents can already preview kind of what the request return data would look like? >> Yeah, you can build that out. Um, one thing that we also believe is that the there's not one sizefits-all of documentation. Um, so one thing we have is kind of this adaptive form

of of content that you can kind of customize to your customers. So some customers you want to see certain pieces. Um, other people have access to a beta feature that don't. You can connect into kind of feature flagging and customize the end experience. The same applies for API keys. Um, and that is definitely a shift we see as well. People don't want the 500,000 page long documentation.

And even though you can ask a chat experience, you can customize that experience as well. Um, and then on the API side, you can directly try and kind of connect in within Gitbook. Um, yeah, you mentioned before about this um like new type of prompt that we actually going to need to write coherent uh documentation across all the people that are contributing to the docs. um like

a clawed MD or an agents MD file for writing documentation. Is that something you have some tips for of how we can write a good one or where we should store it or is it possible to store this within GitBook? >> Yeah, so that's something we have now. It's an early kind of alpha feature, but most companies have their own form of a style guide. Um so

that's a concept that's quite familiar to technical teams. Um, and so we're working on building that in directly into Gitbook because we see that sometimes, you know, a lot of tools have agents that can help you write, but often it doesn't necessarily match the right tone, company brand, um, formatting style, everything around that. And so having that built in for your agent directly into Gitbook um, is

important now. and any types any tips on or any changes that you see happening towards like what the actual schema or structure should be. Will we annotate more already within like an example request for example what the expected outcomes are or the options are to or how are we ma having to make it more context efficient right because yeah documentation can be be kind of long to

read for humans but now also for agents any practical tips I think it's heavily dependent on the product that you have and we see a lot of different teams experimenting here and I think that's why going back to your question on kind of the tests uh for agents and how they're actually how successful they are uh being with your documentation becomes important because that's where we can

kind of tailor things and I think there's a version where um there are maybe different formats available um and different information kind of lives for different readers >> yeah, I'm kind of out of questions. You've answered a lot of great given us a lot of great information about where it's heading and what we can do. um how how would you compare yourself for example against uh Mintify

and other uh players in the market? >> Yes, it's a great one. I think what's exciting about the space is that there are so many new tools popping up. Like I think the recognition for documentation, the need for it um with AI has drastically changed which is super exciting to see. And what's nice is that everyone is kind of taking a bit of a different angle. Of

course, we have the same goal of great high quality documentation. Um, but Gitbook, I think we're really embedded in both solving the cultural workflow aspects as well as well in a c cultural um, flexible way. So, you can work on documentation in GitHub or alongside your code editor. Um, that syncs by directly with what's happening. And then gitbook itself is also has you know a typical what

you see is what you get editor with a built-in agent and everything for nontechnical people to easily contribute and work on the docs and all of that to be in sync that's kind of our core bread and butter which I think doesn't change with AI of course what we have and channels feeding into it proactively looking at things like um support tickets or changes in uh tickets

in linear for example And kind of bringing in some change requests for that is really important. And that's kind of all built in there. And the more we can add on top it, the more becomes this living kind of connected knowledge system. And a knowledge system, living knowledge system would go also more towards becoming a second brain for your organization as a whole, including perhaps a branding

kit. like do you see much more information being put in the >> Yes, I think the format that it's it's in and where it's surfaced will change. I think one thing we see now is that the product and documentation experience are one like they're not separate things anymore. And so, for example, now with GIO, you can already embed your documentation directly in your product. is yeah, key,

right? like you need that information where users are unblocking them alongside that. And so that's yeah >> And and in terms of docs becoming like your kind of landing page for both agents also for humans, right? Mhm. >> Like is there even a docs to interactive onboarding video of your tool or a docs to having some >> more fancy pictures to make it or custom icons to

just make it a little bit more fun and good looking? >> Yeah, for sure. We have a full video all of that you can build out in Gitbook. But I think what's nice about embedding in your product is you can design that first experience, right? you can kind of onboard customers in the way that you want prefill kind of questions that live in that chat experience so

you know when they're stuck. And I think there's a lot more to build on top of that, right? Because we know a lot more information about customers now and how can we design that experience and make them successful with your product. And do you also see a problem with having faster iterations of for example API versioning or just in general way more code being updated and written

all of the time? Uh and how to make sure it's actually being that it's actually up to date. >> Yeah, I think it makes the first problem your first question of >> Yeah. Yeah. Let's go back. Let's go full circle. Yeah, it was on how do you update and maintain. I think AI and the speed and the amount of updates that are happening makes the problem worse

if you don't build better processes around this and a way to actually tackle this problem. Um because there's so much change happening and the docs become I mean out of data so you're almost writing them at this point. So yeah, that's I think why there's a lot of other tools, ourselves included, trying to tackle this problem now. Uh because it is even greater and more impactful. I

think when you um when you see agents going wrong with your product, it can often be traced back to incorrect or out ofdate documentation. So you'll yeah you mentioned already that agents cannot fully write all the documentation. We need to have a human in the loop because it's so crucial. Would we would get book then perhaps create agents that nudge and remind developers to update their documentation

or to kind of real time consistently check with the code base if things are up to date. Is that kind of the future where we're heading these AI managers? >> I don't think it's necessarily the nudge route. I think it's more the how do we reduce the friction and do this for them and then have someone make sure that it's correct. >> Yeah. No, thank you so

much uh for giving us some very practical tips of how to actually make something that to make documentation that is correct uh agents for external users that are curious. Hey, does this product actually have the API um options that I need? and for um our internal team of course to easily onboard new people. Yeah, thank you so much for sharing the state of documentation and yeah, I

hope we all feel more motivated to actually write that good documentation. Don't forget it. And uh yeah, thank you so much. Give a big hand to Sarah Tendoski.