3 Steps to Make Skills-Based Hiring Predictable with Alessandra Pegnim, Udemy

Skills-based hiring only works when teams can consistently define, assess, and develop the right skills.

Alessandra Pegnim, Head of Global Talent Acquisition at Udemy, joins Shannon to share how her team built a more practical approach to skills-based hiring. She explains how Udemy validates skills with managers, identifies patterns of top performers, and uses structured interviews to make hiring decisions more predictive.

Alessandra also discusses how this work extends beyond hiring. She walks through training interviewers with AI role-play, widens the talent pool beyond traditional backgrounds, and uses onboarding to close skill gaps early so new hires can ramp faster.

Key takeaways:

1. Start with top performers: Analyze what your best employees do differently and build hiring criteria from those patterns.
2. Validate skills with data: Combine internal insights with external benchmarks to confirm what actually predicts success.
3. Train for consistency: Use rubrics and practice sessions, so interviewers know what strong answers sound like.
4. Hire for mobility: Track skills beyond hiring to unlock internal moves and reduce the need for external hiring.

Timestamps: 

(00:00) Introduction

(00:54) Meet Alessandra Pegnim

(02:25) Why skills-based hiring matters right now

(04:04) What skills-based hiring actually looks like

(04:58) Building job descriptions around validated skills

(05:57) Three steps for skills-based hiring

(07:57) How to validate skills for hiring

(09:48) Turning skills into structured interview rubrics

(12:21) Training interviewers with AI role play

(16:33) Widening the talent pool through skills-first hiring

(18:19) How skills visibility supports internal mobility

(19:54) Adding AI proficiency to hiring criteria

(21:31) Using onboarding to close skill gaps faster

(22:32) Where to connect with Alessandra

Transcript

Alessandra Pegnim (00:00):

We often hire people with 70% of the required skills and ignore the remaining 30% for a long time. But at Udemy, we're really bringing that in on their onboarding and really the beginning period of time by identifying what skills folks have and which ones are the areas that they need support and getting on top of that right away. So creating learning paths and allowing our tool to really help bring them up to that 100% that's needed.

Shannon Ogborn (00:29):

Welcome to Offer Accepted, the podcast that elevates your recruiting game. I'm your host, Shannon Ogborn. Join us for conversations with talent leaders, executives, and more to uncover the secrets to building and leading successful talent acquisition teams. Gain valuable insights and actionable advice from analyzing cutting edge metrics to confidently claiming your seat at the table. Let's get started.

(00:54):

Hello and welcome to Offer Accepted. I'm Shannon Ogborn, your host, and this episode is brought to you by Ashby, the all-in-one recruiting platform empowering ambitious teams from seed to IPO and beyond. I am so excited to be here at Transform with Alessandra Pegnim. She is the head of global talent acquisition and employer brand at Udemy, which there's a lot of confusion around the name, but we've confirmed that's how it's pronounced, where she leads talent strategy for a rapidly evolving global workforce with over 20 years of experience in recruiting and 15 years leading teams.

(01:27):

Ali has built and scaled talent organizations from Fortune 50 companies and high growth startups. Over the past four and a half years, she has been at Udemy and helped support the company's global expansion, navigating remote work, of course, international hiring and shifting talent, mobility, and a changing world of work, a lot of which we're going to get to today. She is known as both a connector and a coach and brings really practical people first perspective to conversations about the future of global talent. Thank you so much for joining us. Yeah,

Alessandra Pegnim (01:57):

Thank you for having me. I'm so excited to be here and transform this year is amazing.

Shannon Ogborn (02:02):

Yes. The energy is always very good at the conference, so I'm super excited to have our guests here at Transform. But like I said, we're going to get a little bit into the future of work and what that means. And Udemy, of course, as a company, focuses a lot on skills-based work and hiring. So just kind of taking a big step back to give people context, why do you feel like skills-based hiring actually matters and is so important, especially today?

Alessandra Pegnim (02:29):

Yeah. I think it's important to understand that we've always been hiring for skills. This isn't something new. We're not changing the entire hiring process at the moment, although it's evolving.

Shannon Ogborn (02:40):

Yes.

Alessandra Pegnim (02:40):

And so I think it's important that skills be at the front end of how we're hiring and how we're evaluating talent as we're in a changing workforce right now.

Shannon Ogborn (02:51):

And I know that there's some interesting stats on 60% of organizations say they're effectively using skills, data, and hiring and talent decisions, despite most claiming they're hiring for skills. What are your thoughts around that?

Alessandra Pegnim (03:07):

Yeah. So I have a huge thought on why do you post a job? And you get hundreds of resumes, thousands of resumes of all these people raising their hand saying, "I can do this job. I read the job description. I see what it is that needs to be done and I can do it. " Yet we screen them out from that process based on either years of experience or education or companies that they work for. And so by having skills be really at the center of how we're doing it, it allows us to pull the thread through on what that experience is and really what these candidates can do.

Shannon Ogborn (03:45):

It is very frustrating, especially now in the candidate landscape, people are getting very frustrated by this. It's like, "I know I can do this. I'm not getting any calls back." And so I think just acknowledging where that gap is and being able to sort of resolve it is going to be a huge thing, especially in the future. I guess taking kind of a step back, when it comes to skills-based hiring, how are you actually defining that? What actually is it? Because I feel like there is some confusion around what does this actually mean? Are these just buzzwords or is it actually something tangible?

Alessandra Pegnim (04:21):

Yeah. So you have to structure an interview. And so it's really structuring an interview around the skills that are needed within that specific team or function. And so it's creating a rubric, it's creating questions that go around that, and it's understanding what kind of answers you want to have, and it's following that process too. So again, nothing new to the interview process, but it's really honing in and focusing on what are those skills that team then needs to hire for.

Shannon Ogborn (04:49):

A lot of people, when they hear skills-based hiring, they're like, "We're going to have to make really fundamental changes to our process and the way we're doing things." That's not really actually the case. A lot of companies have the building blocks in place. They just haven't maybe executed it to the level to make it true skills-based hiring.

Alessandra Pegnim (05:08):

Yeah. And we started kind of two different directions at Udemy, and we have created a job description builder to help us create our job descriptions and pulling in data that we have from external sources as well as validated skills internally to make sure that those skills actually are validated by our workforce of what we need to have to make someone successful as well as to do that job. And we continue to pull that through the talent management side too. So I think it's important that you're not just stopping at the front end of the job description, but you're also bringing that from the interview into the actual work itself.

Shannon Ogborn (05:46):

Yeah. It's really the elongated experience between being a candidate and then just becoming an employee because that doesn't stop once the person starts. And obviously we're in an ever evolving, changing world, especially when it comes to AI. I know that you all have three key steps that you've been following to make skills-based hiring a reality and successful. The first of which is starting with what success already looks like. And I know that you all have an interesting perspective on the data here that backs it up. Can we start with that?

Alessandra Pegnim (06:27):

I'd say don't start with the job description, but really start with a persona. So thinking about who is your most successful person. Top performers can be up to eight times more productive than your average employee. And so it is important to take a look at why are they successful and what are they doing? So we took that persona and we took a look at the skills that those people have and helped bring that into really our validated

Shannon Ogborn (06:54):

Skills

Alessandra Pegnim (06:55):

That we were using in the job description.

Shannon Ogborn (06:57):

I guess in terms of where to start, where do people start? Like what types of roles, what types of personas? What is going to be, I guess, the lowest hanging fruit that's going to have the best long-term impact if someone were to be trying to implement something like this?

Alessandra Pegnim (07:13):

Yeah. So I think it's important to start where you have buy-in from the organization and typically high-volume positions are that, because they're doing hiring all the time. So starting in your sales and CS orgs where there's quantifiable predictive metrics once they join the organization and are performing well is easy to do. So thinking about your sales teams is a great place to start and looking at your top sales performers and understanding why are they successful.

Shannon Ogborn (07:40):

Yeah. I think sales and even CS to a degree, they have, especially sales, it can be pretty black and white, but there's kind of like multiple roads that lead to Rome, I think in this case. Obviously, you all had thoughts about what skills might be relevant to carry in. How did you go about validating those skills with the team, with anyone external? It was

Alessandra Pegnim (08:06):

A close partnership with our L&D team, but what we did is we brought a third party data validator in. We took those skills as well as the skills that we were already thinking about and using in our job descriptions, and we validated with managers who were hiring all the time. And so we took that, we took real life experience of actually having done this for a while and also the third party validation to kind of bring everything together. And then on the inside, continuing with talent management and being able to take a look at those skills.

Shannon Ogborn (08:39):

What was sort of an interesting outcome here, especially when it comes to industry? Because a lot of people make assumptions that in order to be the best person you can be in this capacity, especially maybe in sales and customer success, you must have experience in, let's just say at Ashby for talent tech or HR tech and maybe for you guys ed tech. How did that resonate when you actually looked at the skills?

Alessandra Pegnim (09:05):

Yeah. I think what we found was it wasn't just about the experience they had coming from specific industry, being in learning or ed tech wasn't necessarily a predictor of whether they were going to do well, as well as education. And so I think it was a combination of realizing and opening up and really being more inclusive of a wider range of skills and backgrounds that we could hire for.

Shannon Ogborn (09:30):

And that only does good because it just increases your talent pool. I feel like if you're looking in like, we only can hire people from ed tech or from education backgrounds, they get it and that's great, but there's also different layers of what people can bring and sort of filling the gaps in the team. And sort of on that note, once you have the skills, then you have to build them into the process as the next step. How did that resonate with Udemy and how did that go?

Alessandra Pegnim (09:57):

Yeah. I think what made it easier was tying it back to this is going to help make this process more predictable and you can trust in the decisions you're making through this

Shannon Ogborn (10:11):

Process.

Alessandra Pegnim (10:12):

And so it was identifying what skills we really just needed. It was creating a rubric that they followed and it was really helping to give back time to the managers because they were able to do this in a way that they felt confident and secure in making those decisions.

Shannon Ogborn (10:30):

Structured hiring is so important because we know that structured hiring works and that rubrics help interviewers understand what's going on and how to evaluate what's good, what's not good. Are there any other things that come to mind for you with that?

Alessandra Pegnim (10:46):

Yeah. There is some research with LinkedIn and the Harvard Business Review that states that hiring for skills is five times more predictive of performance than just looking at education and experience alone. So it's proven and it is, I think, important that hired managers understand and trust the process.

Shannon Ogborn (11:08):

That also kind of goes back and ties into the experience that you all found with hiring or not hiring for ed tech. It can be a path, but it's not the only path. And I think that's where that alignment is really important to get on the same page with the hiring managers so that everyone knows we're rowing in the same direction. We all have the same goals, right? We want to hire the right person for the right role at the right time. But I think having the data to back it up can really help influence, especially hiring managers who are more data oriented.

Alessandra Pegnim (11:42):

Yeah. And that's where once you have the case study internally, you can start to pull that across to other functions in the organization and it really helps get hiring managers and even recruiters on board.

Shannon Ogborn (11:54):

It's funny that you say that because I do think that sometimes recruiters can be the hardest people to bring along because they've seen different processes work at different companies and in different cultures. And it's not always easy to align talent people because they're market experts and they've seen this level of expertise across several companies. So it is an interesting thing to try to bring recruiters along for the ride. But when it comes to interviewers, I guess how are you training interviewers to really understand what they're looking for? How do they know that they're looking for these skills? How do they know that they've heard the right thing?

Alessandra Pegnim (12:37):

Yeah. So it is creating a rubric which would include what you want to see and hear and having them trained in that. So we have used our Udemy platform to create learning paths for our hiring managers, as well as our role play function to have them practice that as well. So in real time, they're able to practice and ensure that they're listening for the right things, that they're prompting in the right way, and that they're able to then just do a good job at the interview.

Shannon Ogborn (13:09):

Tell me more about the role play, because I feel like when people hear role play for training, they're like, "I hate that. " I feel like that's a sentiment across. How do you get people bought into this? And I guess how has the team reacted to the role play setting?

Alessandra Pegnim (13:26):

Yeah. I mean, it is a function within, so you are role playing with an AI person, and so I think it's fun and really the future of how we're all probably going to be interviewing at some point. And so having it be part of our product as well as really thinking about a new way of doing it. And just I think for hiring managers that don't hire all the time, so not on the sales teams, it is useful for them because it's not something that they're doing all the time. And so it's helping them get comfortable when they're in an interview.

Shannon Ogborn (14:01):

I truly believe this for both candidates and managers and interviewers, getting the reps in is the one thing that's going to make it better. Candidates usually perform better in interviews when they're, maybe not to the exhaustion place, but later in their processes or further down the line in their job search, they've got the reps in, they know what's going on. I feel like same thing with interviewers. It's like they get the reps in and it actually helps them get calibrated before the interviews start instead of like, "Okay, now we're midway through the interviews with X stage of this role and now we've got it all figured out. " It kind of gets it figured out at the jump instead of in the middle of the process. That's

Alessandra Pegnim (14:44):

All on the planning. I mean, I think the more you can, again, identify the persona you're going after, find those skills, and you can really think about the skills that are needed. We have about 10 skills per job function, but you're not going to put 10 skills on a job description. You're going to identify the top four that you need right now, and those will be the four that are sort of brought through that interview process. It is really helping those hiring managers just feel comfortable, again, in the decision making. If you build the right process, if you follow that process, I still use a template and a script and make sure I'm asking the same questions

Shannon Ogborn (15:22):

To each

Alessandra Pegnim (15:23):

Candidate. You want that same experience so that you are actually listening to the different answers you're getting and understanding which candidate is going to actually be successful in this company right now. It doesn't mean that some of the other candidates won't be successful other places, it's just

Shannon Ogborn (15:37):

What

Alessandra Pegnim (15:38):

Your team needs in that moment.

Shannon Ogborn (15:40):

That's another thing on the candidate side too, that I've talked to a lot of people about. And even one of our recruiting folks, Anna had talked about this in her podcast episode, like Deep Cuts call back from way long ago, that just because a candidate doesn't get a job doesn't mean that they're not qualified for even that job or a different job, but it's really just like, we think that this other person is, we're predicting that they're going to perform better. The quality of the person is not in question, right? It's like if you get to a certain stage, you might be qualified, we're just sensing that based on this predictive stuff that this person might do better.

Alessandra Pegnim (16:24):

Yeah. Yeah. It's an environment. It's the system

Shannon Ogborn (16:26):

That

Alessandra Pegnim (16:27):

They're working in. We're looking for people who can work in that system right now.

Shannon Ogborn (16:32):

Yes. Yes. What's changed since you started doing some of these things more programmatically?

Alessandra Pegnim (16:38):

Yeah. I mean, the net's wider. So we did change really the type of candidate that we were looking for when it came to our sales and CS teams. We opened up the aperture to include all types of tech sales versus just learning tech. And I think in general across all the teams, this has really been the case where we've seen just a wider net be drawn and it's more inclusive of folks coming from all different industries.

Shannon Ogborn (17:06):

You mentioned to me previously, even on your own team, you've seen some kind of exciting things come out of this.

Alessandra Pegnim (17:12):

Yeah. So I mean, you take those skills, they become an employee, you then have those skills that you're developing and the ones that they're really strong at. And so I feel really proud, but the recruiting team has seen a lot of internal mobility. So I think when you hire for skills, it just increases the chances of internal mobility because you now know what skills that that person has. But yeah, we've had folks on the recruiting team move into compensation, move into our L&D, which is our learning and development team, as well as into events marketing. So it's been awesome. And also technical program management too. Oh, wow.

Shannon Ogborn (17:50):

That's amazing. Yeah.

(17:51):

I think it's a really important time to focus on skills-based hiring in tandem with internal mobility because especially there's roles that are, not say being deprecated, but they're changing a lot and you may need more people in this area and less people in that area and you're wanting to utilize who you have internally. And I think skills is one of the best ways to do that because there's all these new roles popping up, right? Especially in people, system analytics as it relates to AI. And it's like, how can we actually use skills to see who would be predictably best to move into that instead of hiring externally? Because a lot of these roles, they've never happened before. There's no one who has a lot of experience in this. So I think that the combination of that is going to become increasingly more important.

Alessandra Pegnim (18:37):

Again, you could take a look at the persona of someone who is in that sort of AI job that you're

Shannon Ogborn (18:41):

Trying

Alessandra Pegnim (18:42):

To hire for now in another company and understand what were they doing before and what skills did they have previously to getting into that job as a predictor to what kind of skills you would want to hire for that job now. And that's a great way of seeing that happen a number of times in my career too. And you've got to go back and kind of take a look at someone started somewhere at some point. They haven't been doing it for years and years. And so I love that sort of predictive looking at skills and knowing that someone's capable, what their capabilities really are.

Shannon Ogborn (19:16):

I was just having this conversation with someone about how there's probably going to be more of a need internally for like an AI auditor because you are responsible for the output of what you're building and all of these tools and Claude code and everything. And it's like, oh, who might be good at that? Maybe a forensic accountant that AI has actually kind of taken a large part of their job, but maybe because they've been doing auditing for their whole career, maybe that's the input. So it's interesting to think about that. I guess looking forward because the job is never finished, what does that look like for you all?

Alessandra Pegnim (19:54):

Yeah. I mean, it's not a set it and forget it. You constantly have to reevaluate what skills you have on each team and within the job descriptions. In fact, we're going back right now and we're adding a layer deeper of AI proficiency into our sales roles, our sales and yes roles, because it's important that there's a level of understanding, a willingness to use it and the ability to kind of jump right in. So there's obviously a scale of what that proficiency looks like, but it's important that we're testing for that in the interview process now.

Shannon Ogborn (20:29):

Yeah. AI proficiency is an interesting one. Thankfully, you all are a learning based company, so it will be cool to see how that evolves internally. Tell me a little bit more about the longer term vision.

Alessandra Pegnim (20:43):

Yeah. We often hire people with 70% of the required skills and ignore the remaining 30% for a long time. But at Udemy, we're really bringing that in on their onboarding and really the beginning period of time by tying that into, we do use Workday, but Workday Skills Cloud and identifying what skills folks have and which ones are the areas that they need support and getting on top of that right away. So creating learning paths and allowing our tool to really help bring them up to that 100% that's needed.

Shannon Ogborn (21:17):

Exciting stuff. I'm really keen to see how this plays out, especially in the next year because the AI proficiency, we're a little bit all over the place as an industry trying to figure out what do people actually need and where are the actual gaps? And what is the actual skill? Is a skill AI or is the underlying skill something else? So I'm curious to see how that plays out.

Alessandra Pegnim (21:42):

It's interesting that you mentioned that though, because we just had a part of a new product within Udemy called Altus that was just released and it's to help you understand what skills and where the gaps are and then help managers help to kind of close this. So this is that bringing to life, not just saying we're going to do it, but actually bringing it to life and figuring out where people learn best. So it isn't just like, go take this course, you set it, forget it, and not try and see whether it's incorporated into the work, but this is really trying to see whether there's output and real business impact from it.

Shannon Ogborn (22:21):

That's great. We just need more data, more validation because I think that's what helps build the business case. But we are going to get to our last three questions. For those who are regular listeners, you know that these are on YouTube. For those who are a newer listener, we have moved these questions to our extended version on YouTube. So if you want to hear what hiring excellence means to Ali, what her recruiting hot take is and what one thing she would tell her earlier career herself, head there. We are coming up on our time. Where should people go to learn more about you and your work?

Alessandra Pegnim (22:56):

Yeah, awesome. Well, you can follow me on LinkedIn and then also Udemy.com. We have a blog post that has a lot of great information.

Shannon Ogborn (23:03):

Amazing. Well, I think this is going to be super helpful for folks. Like we've reiterated several times, you're already probably doing a lot of these things. It's just more executing them in a way that's going to be more predictive for your job process. So I think people are going to get a lot out of this and thank you so much for joining us.

Alessandra Pegnim (23:21):

Thank you for having me. This is awesome.

Shannon Ogborn (23:25):

This episode was brought to you by Ashby. What an ATS should be, a scalable all- in-one tool that combines powerful analytics with your ATS scheduling, sourcing, and CRM. To never miss an episode, subscribe to our newsletter at www.ashbyhq.com/podcast. Thank you for listening and we'll see you next time.