Future Proofing Training

Hi. How's it going? It's going well. So I think, yeah, this we're hoping that this will be an informative yet informal chat, and I think introductions are the first order of business. Is that right? Let's do it. Okay. One second. Oh, it's the other way around. There we go. They told me. You wanna start? Sure. My name is John as, the gentleman said, and, I'm the CEO of a company called Adminstrate. We have a training management system that's built solely for enterprise and that solely for enterprise bit is not always been the case. And we'll kind of get into that a little bit. My background is as a software engineer and, built a healthcare technology company before administrate, but, I've been administrate for quite a bit of, my career now.

And a lot of it has been working with Ping and before that ForgeRock. So. Yeah. We've been a customer for a decade now, I think. So that's a while. Hi. My name is Leonie. I'm director of customer training at Ping Identity. Been there for six years now. I think most of you know me, so I'm gonna keep those relatively short. And most of you know Ping as well. But I think what's special about this session is, we have a really fantastic partnership with Administrate. And we're gonna show you a little bit about how Administrate and Ping Identity have really influenced each other's business. Not just as, you know, as vendor and customer, but as as actual partners throughout this journey.

Yeah. So while we dive into it, I've actually mentioned this many times. We were talking about this before, but, prior to ForgeRock, coming on board as a customer, we had a a great SMB focused business. And I would say that that was about four or five years into our journey and that and and now when we look back, the first four or five years of Administrate's journey was really all about trying to find one customer, which is ForgeRock, which opened our eyes to just how much of a challenge and how big of a, you know, when when you scale up to enterprise training level of things, everything breaks and goes weird and it's much more difficult and so forth. But that journey that ForgeRock kinda took us on, it's sometimes kicking and screaming, along the way was, has turned us into the company we are today.

So, you loved it. Don't pretend. And and a lot of it is around this concept of, like, how does data and looking at data help you on that journey and helped us. Shall we jump in? Yeah. Cool. Alright. I wanna take you back to 2021, which is, when we first started talking about a really big problem we had at ForgeRock, now Ping Identity, which was all around our self-paced learning. We had a really big issue, how people were accessing our self-paced learning, how much time it took us to get them access to it, how much time it took us to give them everything they needed for it, the materials for it.

It wasn't scalable for a really, really small operations team. It wasn't user friendly. It made our on demand completion rates suffer. And we weren't utilizing our product to the best of our ability because we weren't selling it. It was all free. So it was a massive issue for us. Yeah. And I remember we got the call and it was kinda like middle of the pandemic and stuff. And I was like, I don't have time for this shit. You know, like, ForgeRock trying to do something new and innovative again, you know. But it it was actually it it played into a real sweet spot for for both of us as we kind of went went down that journey. Just you wait. Fast forward to 2025 But before we go there, let me let me show you because like it was insane.

So a learner had to go to our customer portal to a self paced learning section. They had to be a partner. They had to be part of a partner support set. Then they had to find their course. Then we will get a ticket. Now I have an EMEA-based Ops team. We have partners who are based in Americas. We had to give them access to a whole bunch of stuff. Namely, they got books, they got labs, they got the registration email from us, three emails. And only once we've actually manually issued books and labs to them, which when they were based in Americas, you know, that could have been an eight hour delay sometimes. Right? And we all know that the more delayed your access to self paced learning is, the less likely the learner is actually going to complete it, the less likely are they engaged with it.

Massive issue. And you could see it, as well with, like oh, sorry. I don't know where to point. Go this direction. You could also see it with, like, how long it was. Like, a video of ours was half an hour long. Like, nobody's going to stick around for a half an hour video anymore. Right? Like, we all know it's five to ten minutes. Like, it needs to be fast. It needs to be one topic. No wants to do that anymore. And then we had, like, so many of them. Like, I think we had fifty videos on some courses. So it's just, like, it just didn't make any sense. And you could really see that an hour before numbers. So this is where the analytics are coming in. Shall we talk about analytics? Yeah. So so yeah. And this is this is one of those things that, I've always admired about the the crew at Ping is they and and just by way of background, there are probably four or five major features of our platform that have pings, paw prints, footprints all over, hand prints all over them.

Right? And some of this was the team would build out what they needed in advance of us, you know, if we couldn't get it done or put on a roadmap in time, and then we would come along later and fill it in. Sometimes it was a joint thing, whatever. But what I really admired about the team at Ping was when they came to us with a project, it wasn't just I wanna do this for this reason or this idea or because it looks cool or it feels cool or whatever. It was always very data driven, and that's been a theme kinda throughout all of these these projects. Yeah. I I appreciate that. I I feel like, that's gonna be, like, the theme here as well. I feel like it feeds them really nice from what Richard did earlier because once you have all of that data, what are you actually gonna do with it? Right? It's like a now what? What are the decisions that I'm making on it?

And then it's this. It's recognizing that there was a problem. So I wanna remind you, this is still 2021. Right? Like, we, we had a look at our self paced learning. We had high intervention courses and we had low intervention courses. But all of our courses, we needed to do something. With a high intervention course, it took me five minutes. I needed to go to three different vendor systems. I needed to send books from Gilmore. I needed to send labs from Cloudshare. I needed to check on Administrate that that partner was on Salesforce or partner. And then I had to put everything in my student notes so I couldn't find it again. Five minutes per registration, with some of our low ones, two minutes. But all of that added up five weeks per year.

We were spending just on registration management. Five weeks per year. Can you, like, can you believe that? And that's just for two thousand three hundred registrations. And you could see it in our on demand completion rate as well. It was thirty percent. Thirty percent is is alright. Like, massive open online courses usually have, like, a twelve point six percent completion rate, so I'm fine with thirty percent. But you're gonna see the new numbers in a bit as well. And that's I think that's telling a story. Yeah. Oh, yeah. The solution.

Yeah. You you wanna take us through it? I can come over. Yeah. This was I think it's important too. This is like a a multi year planning exercise and journey as well. So sometimes when we're talking about what has been achieved here, people are like, well, we can never do that. Well, you can. You just need to go on a step by step process. And the other, I think, important learning for us and that we actually talk about this with prospects in our sales process or with customers that are thinking about coming on board is they'll say, you know, we're using SurveyMonkey today and, you know, this for that or whatever. And it's like, all of that is gonna change over the next five or six years of of working together. Right? You'll change out your survey tool to Qualtrics or whatever it is because the needs of the business will change.

ForgeRock came to us when they were, like, post series a funding, and I think the training team is, like, one, maybe one and a half people and and, you know, very, very small company and then ended up going all the way to public and then merging and all that. So it's it's definitely this type of thing has a sequence and has a way to approach it. Yeah. For sure. And I I think a lot of you have seen this particular slide a couple of times now. Right? Sometimes things change because we have a composable tech stack, as Richard mentioned earlier. So, it's really easy switching things out. But I I think what John said is really important. Like, do you anybody else remember the composable tech stack talk that that we did last time? Yeah. I see some nods. Yes. Awesome. So what we really did, we we did this in phases, but that's what we we wanted to do with administrate.

We wanted to move to composable tech stack that was, helping us with our registrations, our self paced registrations, moving from self paced to on demand. And we didn't just go from monolithic to composable. Right? We went first monolithic. We were using administrators, the TMS, and administrators, the LMS. Then we split out our LMS part, and we moved it to our customer portal. And then we started after we had this really strong foundation of the core of our UI piece together with our financial piece, we started plugging other things into it. Right? Making it completely invisible to the customer, how all of these integrations were actually working in the background because we had the UI layer in in the front end. Alright. And that was a really good thing for us.

So I remember it was when talking with Kevin at one point, but also with I'll never forget this. I was, like, out in, West Coast of the US, and I got this email in from another customer of ours, Maersk training. And they're like, the CEO, the CFO, the head of operations, or whatever needs a call with you tomorrow. Like, you know, we're we're really got an important thing to discuss and all that. Fuck. We're getting fired. Right? Like, this this customer's churning for sure. Right? So I wake up, and it's, like, six in the morning my time, and I'm, like, roll out of bed, and I try to put put a shirt on or something, try to look presentable, like, get get on this call. And I'm like, what's up, guys? And they're like and they're like, oh, well, we just wanna let you know we're gonna we're gonna use a different LMS than what we call our portal. Right? And I was like, great. Excellent. Because whenever that happens, that gets depressurizes the relation.

We don't we don't want to be in that business. Right? The learner experience front end thing is not something that we care about. We care about the back office, managing of training, and so forth. And so they're like, we're gonna change this. We just wanted you to know, and we still support you. So I was like, oh, great. Amazing. I was like, can I get off the phone now? It's early. And they're like, they thought it was a big deal for us. But because this is this is one of the ways that we learned how to focus down on what we're good at and embrace this composable or headless path with our customers, and it's been transformative for us, particularly in the enterprise space because we are just not gonna be the the best at everything. We wanna really be good at what we do and let and let you all pick and mix and match and whatever along the way. I think we always call administrate our engine of our really, really shiny car, which is our customer portal.

So Yeah. It's been working really well. And, to be honest, you're gonna see it in a second when we go into the analytics. But, But, maybe I fall in love with your reporting engine. Like, it's really good. We every time we compare it to, like, another vendor's reporting engine, it's just like, but where all the things that I want to report on? And then like being able to combine it with like the APIs because everything you can report on, like is in the APIs. So that's fantastic. Anyway, shall we go to the next? Yeah. So to now, how has it how's it looking now? So if you remember back then, two thousand three hundred registrations. We are at fourteen thousand now. Fourteen thousand on demand registrations.

That's in the last, well, three years really. Right? Like, twenty twenty five now. Yeah. Six times increase in registrations in the last three years. And we have an eighteen percent increase of our on demand completion rates. And we're not just open to partners anymore. We we are now selling this. And we're gonna show you the revenue in a second as well. But I think what I found quite interesting when I ran the numbers, if we were still doing the same thing, I would spend eight months doing only registration man management. I would have to hire a person whose sole job it is to do registration management and send out books and send out labs, and that's just not scalable.

So, yeah, we wouldn't have been able to be here without that, integration. And we also wouldn't have been able to be here because we are selling it now. And from twenty twenty two to twenty twenty five, we have tripled our revenue because we're selling it. And because it's not just been, we're doing it for, an operational enhancement, we have also completely changed our business model with it. We went from something that was training credit based and, like, quite, like, ad hoc sellable to a training subscription model because we could now sell it. And that can really be seen in our revenue split as well.

So if you look at our, composition of our revenue from twenty twenty two, you can see we started selling training subscriptions in twenty twenty two with twelve point four percent. Over seventy percent were training credits. That's the lovely TC on the left side. On the right side, seventy five percent of our revenue is coming from training subscriptions now. I know we have a chat tomorrow about, why you should ditch training subscriptions, and you should go with training credits instead. So I'm I'm interested to see it. But for us, this this definitely was the right move. Tripled revenue in three years.

Oh, okay. Alright. Yeah. And what I what I love about this too is this is, like, an important principle that we talk about a lot, which is you don't necessarily have to go all in or pick which business model you want to be using. You can mix and match them. You can there's different, you know, reasons and applications of this. And sometimes the market will swing you one way or the other. Right? And it's it's great. So we really just, you know, find a platform that can embrace all of those different models, and you can swing back and forth as the pendulum of the industry goes. Right? Alright. Next one. I think Yeah. So this is, like, kind of the the four parts that I think I'm personally really interested in, which is, you know, that efficiency and scalability piece was kind of the driver for this whole thing.

Right? If you hadn't had that, you know, my my accent is American. And sometimes I'll be in a meeting with Europeans, and I'll be like, well, if you had a gun to your head, what do you choose? You know? And and they're like, what? Why would you have a gun to your head? You know? So but you had a gun to your head kind of. You had this problem that had to be solved. And but then that unlocked the next pieces of this this puzzle. And and throughout the whole thing was this current of data and decision making and decision support. Yeah. Absolutely. And I'm sure there are, like, a whole bunch of other areas where we're using our data, but I really liked how you saw it as, like, this little stepping stone. Right? So yeah. Yeah. I mean, it's really Maslow's hierarchy of needs. Right? You you can't do the the stuff at the top of the pyramid, like get your first world problem solved, your free Wi Fi, and things like that until you've got, like, water and, you know, infrastructure and so forth put in place.

Yeah. Yeah. I agree. And I I think maybe this is this the section where we're gonna talk a little bit about how you find the story within your analytics, which, like, I think that's something that, like, whenever you have a whole bunch of data there, you don't really know what to do with it. How, like, how are you actually reading it? What are you gonna do with it? And that's something that I always tell my team is like, well, it's great that you got the data, but what does it mean? Why? Why are we doing it? What does it mean? What are you gonna do with it now? And why does it look the way that it does? So, we're looking at it from, like, four perspectives. Scalability, and efficiency. You already saw an example of that, but we're gonna get another example going.

And and actually, do we have a couple of microphones around? Because I think we're gonna need that in a second. Then the other one is external needs and issues. So having a look at what our customers are actually doing, what can we see? Then it's all around, demonstrating value and enabling conversations. So what is the value of training? And this is one that we're still struggling with. I think I saw Lynn somewhere here around. So she has a great she has a great new programme, the VantagePoint programme. So if you haven't checked that out, join the SIG, because I think that's gonna talk a lot about that. And then the last one is, how do you justify your initiatives and your objectives? How do you justify your outcomes?

And I think that's a really important one. Talk a lot about that with a lot of people when you say like, I don't get that much direction. I, I find it difficult justifying my initiatives back up. No? Let's jump in. So, alright. Gonna stand up for this, actually, if you don't mind. I gotta leave you sitting. Okay. So this is an example of, some of our analytics. This is deliveries, event deliveries, and student deliveries. I wanna know from you, well, you can see we got twenty twenty four in, like, the dotted line, and then, the current twenty twenty five numbers are a nice shiny red. Looks great. Right? I mean, we're we're doing better than twenty twenty four.

Love it. Thirty percent increase in classes. What's the issue? Anybody have an idea what the potential issue could be when we're looking at this? When we're talking about limited head counts?

Sorry? Business impact is a good one. I think about scalability and efficiency. I have a limited headcount. I have the same operations team that I had three years ago.

It takes me around fifty five minutes per class to set it up. If I have a thirty percent increase in classes, how am I gonna do that with the same team that I have now if that continues down for the rest of the year? If I continue having that next year as well. Oh, I'm pushing up on my water. That's, forty two hours for q two alone that I spent on setting up classes. If that continues like that, I'm spending six weeks per year to set up classes. And that's only gonna keep on increasing as my business keeps on being more successful. Right? And and we wanna have more ILT classes. We wanna be more successful. We wanna get more people in.

But I can't scale that. Right? So I need to find a way to look at my vendors and reduce the time that I'm spending on them to manually set up all my classes. And that's something that we're currently doing with administrate. We're looking at our tech stack. We're looking at how can we enhance our integrations. So I'm going from fifty five minutes to nine minutes per class.

And then how do we track it? We've already done our first integration. Our first integration was with Gilmore, our ebook, provider. We already saved fourteen hours. Fourteen hours that my team can spend on being better with our customer service, on, pushing new initiatives ahead, which we're gonna see in a second as well. I think by the end of it, when we have everything done I'm I'm looking over to my senior ops manager. Shout out to Nia. But I think by the end of it, we are going to save five weeks of time. We're reducing a whole bunch of six weeks of manual effort to one week across the whole year because we're reducing it from five, from fifty five to nine minutes per class.

Yeah. It's it's probably worth, it's probably worth pointing out as well that that's a straight line Mhmm. Kind of analysis of what could happen in the future. But, actually, all the research, when you look at ATD and so forth, all the research from ATD basically shows that as companies get larger, their training operations actually get more inefficient. And so what you see is the very largest training delivery models are the most inefficient. It's not a straight line. It actually gets worse. Which one? There you go. This is a very warm mic. So he had very warm hands. It wasn't mine. Yeah.

Alright. Well, we're back here. Let's talk about external needs. Because, like, obviously, you don't just apply that to your to your own part of it. Right? We have a whole bunch of external information. Some of the student days and deliveries that we got, that's all pulled from administrate. I wanna show you a couple of things now that are around our ticketing system. So this these are our ticket numbers. We have a training ticketing system. I imagine a lot of you do as well. But one of the issues that I immediately see I'm not gonna quiz you again. Sorry, guys. Again, dotted line twenty twenty four, red twenty twenty five. I have I went from two hundred tickets to over three hundred on average in a month.

Why is that? What happened? Because, yes, it's great that I'm having more students. That's that's all good. So I can account for that a little bit. Right? But surely something is going on when I have such a large increase of tickets. And I'm sure that's what you do as well when you look at your data and you ask, oh, there's such a massive jump. What happened? Right? At least you should ask the question, why? Because my CSAT is great, hundred percent. My response rate is alright, with six percent. My resolution time and my response time is fantastic, better than our support team. But what happened? And if we look into it, you can see a massive red block in the middle. Sixty three tickets sixty five tickets on exam issues.

All exam issues. And that's not just for q one. These are some of my q three and q four data. Exam issue. Exam issue. Exam issue. Exam issue. Exam issue. Exam issue. Exam issue. Consistently around the thirty percent mark. So what do we do? We have a composable tech stack. We go and switch out our certification vendor. And hi, by the way, to the newest member of our vendor family, Criterion. Thank you so much to have you on board. Yeah. And we'll see how that's going to improve our certifications now because, of course, we then had a further look into what our certification issues are. We set goals around what we want with our new vendor, and we'll track it.

That's how we look at it in terms of what we're seeing from our customers. And that's just one example. We get, you know, NPS data. We get comment analysis data. I hope you all have a survey tool where you can check-in on your students how everything is going. Right? Right. Yeah. And so I think another key thing here is you need to have multiple different ways of solving these problems with the composable tech stack. Right? We don't have to get too geeky. But when you know, so most most applications out there, a lot of lot of products will have an API. That's great, but it means you gotta code something and build something, whatever. You also wanna be able to plug in maybe an integration platform as a service, and maybe there needs to be out of the box integration.

So we we never see one customer of ours solving everything in the exact same way for every situation. This kind of annoys our sales team because it means we have to have solution consultants on a lot of conversations to get the right solution. But I think that's something to keep in mind as you're going through this journey, which is, okay, we need to swap this vendor out. How are we gonna do that? What are the mechanics behind it? So Yeah. Yeah. And I I think we've we've become very, very, efficient as at this. You know? Like, I I know vendor integrations are a scary, scary thing sometimes, especially when you don't have developer resources yourself. But, you know, we did the last one in two months. So It is a skill, and you get better at it. And the more that you practice that as a team, then the better you'll be set up for the future.

So, yeah, then I think it's was it the the third bucket, right, which is so first one, we've gotta do this. Second one is like, okay. Let's take stock of our environment and so forth. And then it's also, alright, now that we have the the power of being able to do that, and we feel like those two things are relatively under control. And believe it or not, we talk to very large companies that we've my mom has heard of, and, you know, like, the that's an important test, by the way, if you're really an enterprise vendor or not. Has your mom heard of most of your customers? But a lot of them won't even get to that kind of first or second step. Right? They're struggling. They're caught right around there. But once you break through that, then it's kinda like, okay. Now we can actually use data to go in with a business case and plunk it down. And, you know, this funny thing happens. Right?

Our our we've got a great CFO at our company, but she has the nickname of CF No. Right? Because she says no a lot. Right? But what's interesting about that is the dynamic changes so rapidly if somebody can come in and say, if we put a pound or a dollar into the top of the sausage machine and turn the handle, we'll get three out, right, or four out. She'll actually just be like, well, we have to do this. There's no I don't even have a decision to make here. And I think that's what we're talking about. How do you make that case to the rest of the business, particularly when times get turbulent or there's a lot of change happening? And I think that's that's the really difficult part. At least we are still struggling with that a little bit. And I know that probably a lot of the members in here are still struggling as well.

How do I, how do I show the return on investment on training in terms of decrease of support tickets, increase of renewals, decrease of churn, better customer health metrics, increase customer NPS, and so on. Right? There's a lot of data that goes into it. And especially when, like, you have a composable tech stack and you need to combine all of your data as well. So I'm starting here again with it because that is my comfort zone and you have all seen it. And we also talked about this, tackling the data challenge. And I think Richard showed something really interesting how you can tackle the data challenge. We went down a different route. We went down the route of the data lake.

And we are still waiting. So, you know, we're halfway through. But, what you can then do, and I I think that's the first step that I'm going to take is, first, I wanna visualize, my training data for the people who are engaging with my customer. So I wanna be able to provide something like this to my customer success managers to say, hey, you have your account has, fifty percent of their subscriptions assigned. They have taken a whole bunch of training in these quarters, and these are the trainings that they've taken. And if you go down, you can see who in particular has taken the training. And by the way, you have a new tab where you have all our comment analysis, all of the NPS.

You can see how many learner you learners you have active right now, how many courses they have taken per learner, and so on. So that's our first step. That's what we're currently building, and it's been taking a little while. But it's it's super exciting, and I cannot overestimate how transformative this will be within your business. We went on a similar journey just within administrate because I I must have been a couple years ago. I had this little app, and it was I don't even remember what it was, but they would email me every week and say, you've saved nine hours with our app. You know? And I remember, at one point, somebody said, why aren't we doing this with our stuff? You know? And so we put up quite a bit of time and we are surveying customers and be like, you know, how long would this take you normally? And we built out a a dashboard just like this that we share with you and and whatever.

And what we found was really surprising on because it was really effective and it was great and whatever. But what we found was that then our customers will take that information that we're preparing for them and use it to say, this is why we cannot get rid of administrate, or in your case, this is why we need to do more training. And then you have this culture of coming in prepared and justify you know, the data to justify what's going on. It it's really transformative. So It it really is. Maybe we have taken some inspiration. Yeah.

But I think we that's that's where you need to start. Right? You need to give, like, the the transparency. You need to provide, like, a little bit of, like, a level of understanding of what training actually is and, like, what we actually do. You need to provide the visibility. And then you can talk about, like, some of the value. And I I I think that is the part that we are still working on because our preferred version of this is I want all of these other composable tech stacks from my support, from my community, from, my renewals team to feed into the same data lake and then I want to be able to pull all of that data. Now that takes forever, and you are dependent on IS resources or IT resources. So we're actually going to go down route b, I wanna say.

But we're hoping we're gonna make it better and do it a route c. Yeah. Yeah. We'll we'll have a chat. Okay? Cool. Awesome. We need some help. So we can understand how our training has helped decrease support tickets, how it has helped increase renewals, decrease churn, helped increase customer NPS. Because that is the language that is being spoken by our exec leadership team. It's not necessarily about training NPS, right, or about, our survey response rate. It's it's about the overarching customer health score. How is this customer doing on a holistic level? It's about how much ARR is attached against that customer and how healthy are they and are they likely to churn.

And, hey, by the way, if they're trained, they're three times less likely to churn. Those are the kind of data points that we wanna get to. Yep. And that will unlock massive investment when you can get to that point. And, like, one of the things that that we say a lot is it's kinda like a negotiation. Right? The first person that throws out a number, then you're anchored there, and then it's very difficult to unanchor yourself. The first data wins. Right? So whoever comes out with the first data point, that's where the discussion is gonna revolve around. And so you need to be on the forefront of that and and really or you don't even have the discussion, and that's even probably worse. So. But I I would love to, like, connect with some of you afterwards as well, see how some of you are doing that. So yeah.

I'm I'm hoping the Vantage Point program is hopefully gonna be really useful for that. I'm putting a lot of pressure on you over there. I wanted to make a comment, Leonie, and thanks for the plug on vantage point. But you have I think in one of your previous slides, you already have that data about the learner engagement and what they've completed, etcetera. And what you can do is you can use a simple,

what is it, like the user ID, for instance. And you can use user ID. You can extract data from Gainsight for your NPS. And then I think you have a CRM in there somewhere. You can use that same user ID, grab all that data, and then start to do that's where your business analyst comes in and he does all this. Which is exactly what we're going to do, with the difference that we don't have a business analyst. But I have a fantastic ops manager, so no pressure.

But, yeah, that's exactly oh, we we're not we're not waiting anymore. Right? Like, there are so many data sources. I I think one of the issues is as well access to the data sources. Right? Because some of them are locked behind, like, where I can't get access to. So I need to find the right people who can give me the right reports that are then landing an email somewhere unless they give me access to it to run it myself. And then you need to correlate the data. You need to compare it. You need to do a right? And that's fine. The easier way would have been our data lake. But hey ho. You know?

Alright. Last one. Measure success and impact. Yeah. So we were talking about this a little bit before, and, you know, I think one way that we tend to think in inside of our business is we're may we always wanna be making bets here and there. We wanna make a bet. Right? And it's it's basically just scientific method. So you you hope. You have a hypothesis, you know, and then you wanna either use data to prove it or disprove it. And it's a super simple thing, but it it tends to, not be the the norm for for a lot of folks. And, you know, when you're when you're smaller, less complicated organization, you can go we were talking about this. You can go with your gut for quite a long time. You know, they can carry you through for years. But at some point, you become unattached, and and you need you need to really get into the habit, particularly around training, I think, of we've made this bet that this class is gonna do something.

What is it supposed to do? What what behavior is it supposed to modify? Because if you haven't modified your behavior, you haven't learned anything. So yeah. That You need to justify. Right? And that's really what the the last part is all about, justifying the things that you are doing in your organization. And that's why I'm closing the loop back over to, like, people who tell me, well, I don't get direction. I say, well, we'll just give yourself direction. What is the data telling you? And then justify it. It's better asking for forgiveness than for permission, especially when you have the data to back you up. So these are the initiatives that we are running. I'm really letting you have, like, a little bit of a look under the hood here, but, this is what we're doing this year.

We have thirteen initiatives. We got a whole bunch of KPIs underneath it, which we are reporting up to exec level, which are around NPS, response rate, CSAT scores. We're tracking subscription assignment rates, renewal rates, revenue, bookings, and so on. Right? But I think the the interesting part really of this is this is what it looks like when I put thirteen initiatives against what I'm going to do in a quarter and how I'm going to track if they are successful, which is this part over here. Why am I doing it? Because that is going to be what somebody is going to ask you.

Why are you spending your time on this? What is the business outcome of your initiative? And why aren't you focusing your time somewhere else? How does it connect back to your business objectives? And then I think it's really important to then, think about your metrics, and how you actually measure success. We talked a little bit about token metrics earlier. I so I I think, you know, like, something that I wouldn't use is I wanna complete five enablement sessions per week. I mean, that's that's great, but why do you wanna do it? What is the impact of your enablement session? Is your support engineer who you're going to give those enablement sessions to going to increase their CSAT score?

Is it going to increase the NPS? Is it going to reduce ticket response times because you provided those enablement sessions? Because in the end, those five enablement sessions that you gave them, if you don't have an outcome to it, they have no value. They are just a vanity metric that you've done something. It's a checkbox exercise.

So if you don't have the metrics, make the metrics. So one of the things that we didn't have I'll go back. So one of them was around, learning UI and design and learning engagement enhancement. We didn't have any metrics around this, so we added new metrics into our survey for it. We combined already existing ones. We added a new survey to it. And right now, until we are implementing, we are establishing a new baseline. So we're collecting data for the next couple of months. We're probably going to start seeing some change on this probably end of q two. By that point, I'm going to have six months of data that I can compare myself against. And I can say, okay. My metric on this increased by ten percent because of the changes that I have made.

And those changes, I can directly relate back to this particular initiative. Same with on demand completion rate. We didn't track that, before. Now we are. So, you see our registered versus passed and our completion rate here on the side did the exact same thing. So how are we tracking this here? And you can see, how we're tracking against our benchmark as well. Just some examples of what we're using. And, if you don't have it, within your team, I would highly encourage you to put a monthly business review in place with your team. It doesn't have to be up to your exec leadership team. Right? Make members of your team responsible for numbers.

Every single one of your team members has a particular role. Right? They, they are responsible for their numbers. Let them report it up to you. Let the team have an MBR and show each other what they're doing and give each other visibility and tell them they own it. Because if they own the numbers, they also will see how they can approve it. They also will understand the story behind it. Right? Eric? Yeah. No. I I think that's such a good tip. We've we've put that in place, administrate, and, there's all all kinds of different things. Right? Some people use OKRs, and we use EOS, and there's all kinds of different frameworks or whatever. But just having something there to really, really dial in and and look at. I mean, you'd be amazed.

You put a number on there, and if it's green, you know, or red, people really pay attention to it. So, it's it's pretty transformative. And I think too, like, the the backdrop of all of this, it's easy to forget when Leonie's talking about this that, like, you have an incredible amount of change happening at an incredible rate. I mean, just as a partner, watching the just meteoric ascent of ForgeRock as a company and going from just tens to thousands of employees, it's I mean, their business was growing like crazy. You're putting out legacy getting rid of legacy products and introducing new products. And then there was going public, and that involved a ton of compliance training and all kinds of stuff. And then it was, being being merged and then taken private again.

And, I mean, that amount of change, even over an eight to nine year time span is incredible, and yet this stuff was all needing to happen. And and so those numbers were changing all the time. And it's like, we talk a lot about it's not so much. There's no scary numbers. There's just trends that you need to to watch and monitor and so forth. And I think this stuff becomes so important when we have huge amounts of change because training is usually in the crosshairs for one reason or the other, good or bad. You know? And having this stuff at I know for sure probably gives you a ton of confidence and your team a ton of confidence that you're making right decisions and the rug's not gonna be pulled out from underneath you when all this change is happening, probably unlike most other teams within your business. Yeah. For sure.

And I John and I talked a little bit about, like, has this always been the case for us as well. Right? Like and and it hasn't. And, like, I, so I I took over this team one and a half years ago. And look. I had to deal with a whole bunch of new executive leadership, right, that I hadn't dealt with before. Also, I'm a woman in, I was thirty when I took over. I was talking to people who had a lot more experience than me, who maybe had a certain perception of a young woman leading a team. And I had to justify every single decision that I was making.

And I was justifying it with the data. And I wanna repeat. Right? Like, tripled the revenue in three years. Okay. And I I remember you I think you we spoke about this a couple years ago. You're like, I wanna double the revenue. And I was like and I went, oh, that is that is a huge lift. And, yeah, you guys smashed it way, way better than I think anybody anticipated probably. Yeah. It was good. Feels good. But I I think it you know, the necessity of it and having to do that, which which is fine. You know? It's whatever.

But I think it has really given us such a strong standing to understand our own business and to understand where we're going and to get the credibility and the buy in from our executive leadership team. Because when when I come now with an initiative and when I say I wanna do this, they are not questioning why anymore. Right? Because we have proven that we know what we're doing, and we know what the output is going to be, and that we know our business. I think it's a good spot to leave it at, isn't it? Yeah. Alright. Let's do it. We got questions, I think. Yeah. There you go. Yeah. Any any questions or and and maybe before we pop in a question or two, just encourage everybody, like, that this was a journey. Right?

It doesn't it's not a switch you can flip, and you can go from, like, nothing to to what you see. You just gotta break it down to bite sized pieces. And so a lot of times, you know, yeah, if you want tips and, you know, thoughts or commiserations, whatever, you can talk to us afterwards as well. But, yeah, I don't know if we have time for a question or two.

Look. We covered it all. Alright. The data says that, you know, we did a good job. The data says it all. Thank you, everybody.