In this podcast episode, Max learns from Alooba Founder, Tim Freestone, how to effectively hire data professionals. According to Tim, CV screening is not enough and there is definitely a simple and more objective way to screen candidates.
Watch the entire episode below.
Or listen on your favorite platform:
Don't feel like listening? You can read the entire transcript right here. 👇
Max: Hello, and welcome back to the Recruitment Hackers Podcast. I’m your host, Max Ambruster and on today's show, I'm delighted to welcome Tim Freestone, who is the founder of Alooba, a tech startup based in…well Tim is based in Australia, but I gathered your team is spread out all over the world, which is specialized in helping companies, hire data scientists, engineers, architects and analysts, and so all the people that deal with data. And if you live in the same world that I do, that share of the employment workforce is always growing, and every company needs them. So, I'll be asking Tim about how to attract and how to interview this talent, and welcome to the show Tim.
Tim: Thanks for having me, Max. It's great to be here from a very sunny afternoon in Sydney.
Max: Great! Great to be connected. So Tim, tell us a little bit about yourself to begin with. How does…how did you end up in the…or maybe we'll start with your company, Alooba. Did I describe it okay? Is it an attraction methodology? It's more of a screening methodology or tool, right?
Tim: Yeah! So we basically assess people skills in analytics data science and two main use cases for that. One is definitely that hiring use case which you mentioned and so companies would use our product typically either as a very short initial screening quiz that they would send to every applicant who applies for one of their data roles. And there'll be a customized assessment on our platform assessing things like ,I don't know, statistics machine learning visualizations, really depends on the role. And that's kind of one half of the company. The other half is really around assessing people skills internally within a business trying to find the strengths and weaknesses. And that's most often being used in conjunction with the data literacy strategy. So it's becoming bigger and bigger these days that you know, you might be a data scientist and have really advanced skills, but what about the 99% of the company, who aren't data scientist? What kind of data skills do they need? And so a lot of businesses realize that everyone needs some basic data literacy. And so we often get involved at the starting points of putting in place that learning and development plan. We really come in as that measurement tool to understand okay what's our current benchmark, and then keep measuring through time to see hopefully that they've had some improvement in their data literacy.
Max: Hmm! Yeah, makes sense. I was advised for my business to put a portion of our account management team on things like learning how to use SQL and I'm getting training like that, so I guess I've put it out to my employees as a good recommendation but I haven't enforced it, but you know in bigger companies you're seeing data literacy being enforced at the corporate level and pushed across departments. Is that an example? Like an SQL training?
Tim: I'd say SQL would probably fit into the relatively advanced part of data literacy. So they'd be things that are even more basic or simple than that would normally form part of that program. It could be things like, hey, you know what metrics should I be looking at to answer these types of problems. Understanding basic ideas around like sample size. So, you know if you're reading a report and you see that I don't know the number of bookings in England went from… went up by 50% but you know, to look at that and actually they went from two to three so that doesn't really mean much if the book has gone from two to three right? And just having that kind of understanding of the basics of data, really.
Max: Yeah! I suppose there is such a wide gap between, you know, the experts and the beginners that you gotta lift people so that they don't say anything stupid to begin with, like use two decimal points on a percentage when your basic…like you said on a sample size of two or three, stuff like that. Great! Well, how did you end up, you know, launching Alooba. I suppose this is a problem that you…it sounds like this is something personal, apparently, that you want to do for yourself.
Tim: It definitely is a selective confluence of the last 10 years of my life, really is this business. So, the last role I was in was at a tech company, I was leading an analytics team. And so, I noticed two big themes, while I was at this business, so one was anytime I went to hire any kind of data professional, so, data analysts, data engineers, data scientists, I found it personally a massive pain in the ass trying to hire. So, the process was you know you put up a job ad on LinkedIn or in Australia. We have seek like the big job platforms. You get all these applications through and you basically get a CV. And then from that CV trying to pick through quickly and figure out, who are the best candidates to interview.
What I found consistently was that it was very hard to predict, based on a CV alone, who the best candidate was to speak to. So, that meant that I have to do a lot of interviews to hire one person. I'd often get five minutes into an interview and realize the candidate who said they had X, Y said advanced skills, obviously didn't have those. And so, I really wanted a more efficient, simple, a more objective way to screen candidates that was one origin. And the other piece was looking around at my colleagues and realizing that in a company of 150 people, we had maybe I think six or seven data professionals. But then, there are at least another 30 or 40 people, all the product managers, all the online marketers, the senior managers of the business where I looked, what they actually did day to day, it was basic analytics, even if they didn't think of themselves as analysts. And it was very clear to me that this data literacy thing was becoming more and more important.
Max: Yeah, yeah! So, it's like…it should be like a mandatory step in the journey for a good portion of the job be…well, can you handle data? Do you know how to extract it? How to use it? How to interpret it? That makes sense and the other point you made, which is you know the resumes are lying, right? That if you look at a hot space like the one you're in where it…we know that salaries are inflated and that there is not enough talents, and so it's going to potentially attract people who are trying to find a shortcut to a better life, you know and good. But I will guess there are some resumes that are kind of like packed with keywords that don't belong there.
Tim: Yeah! There's definitely some keyword stuffing. There's some inflation there's also just… you know, we're not the best judges of ourselves, and a really interesting data point that we collect directly on Alooba to kind of master this is that, before a candidate starts a test, they rate themselves on a scale of one to ten for each skill that we are about to assess them in, and then we compared their self-rating to their actual performance to come up with what we call the self-awareness index, and to cut a long story short, all our data shows that people consistently overestimate their skills, and that's true for any gender. It's like more or less for men or women, like everyone does it so.
Tim: We're not generally the best judges of ourselves, that's part of it, but then it's not actually just the person writing the CV. It's also the person reviewing the CV and what we've done in a really interesting experiment actually was we contract that whole bunch of recruiters, gave them all the same set of 500 CV, it’s the same job description, and basically asked them, okay, shortlist who you think we should interview. And, it was amazing. They all came back with different suggestions, like almost like complete randomness. And, behind the scenes, we also had these candidates test performances, which we didn't expose. And, so what we found was the CV screening often missed the best performing candidates anyway so actually, if you would almost flip a coin that might have been a more accurate predictor than having someone look through the CV. And it could be any number of reasons for that. You can think of bias like there's been some interesting studies around. You know, callback rates and application depending on your surname and ethnicity but most all those kinds of things, but in general I think a CV is like a really, really weak data set to figure out who to interview.
Max: Yeah! Of course, and this experiment I think would prove the randomness will be even higher if you go towards younger people, I mean if somebody's got 20 years of experience. You know, and the track record to show for it and professional references, and I think the resumes a little bit of a stronger indicator right? But you know if you're looking for people in their 20’s, and assessments is the way to go. So what's the… you know, for those who are not experts and who heard the word, you know data science and data analysts and all that, but they don't know how to categorize this domain, could you give us a quick kind of summary of what are the different profiles within the data world so that people can organize their thoughts around that around that talent?
Tim: Yeah for sure, and it has changed a lot, even in the last few years, so it was only I'd say a few years ago where they were almost all one person look…everyone's looking for sort of the Holy Grail unicorn person that do everything, but now it's been split out so, yeah, definitely. A data engineer, I would say, it's basically the person responsible for the data pipelines, so responsible for getting data from A to B, and getting it in a state that's easily analyzable so they've cleaned it up. They've aggregated, they've rambled it, that’s kind of ready to go.
To data analysts will then be responsible normally for visualizing the data, putting together reports, digging into like negative trends that they find in the data, and being probably the closest person to the business. So, they'll be the ones, actually, you know, working with I don't know let's say the tech company, the product managers, the marketers, and helping them understand the data and defining all those metrics.
Tim: And then data scientists often work on more complicated problems. There's an expectation they'll have what I'd call like proper data science knowledge, so machine learning, you know, maybe some more deep learning natural language processing. And, they're basically using more complicated statistics often to do more like forecasting and predictions. Sometimes, as part of a product, sometimes as part of marketing, doesn't really matter. And, there's let's say often a little bit further away from the business, maybe slightly less expectations of business acumen, doing less of the day to day stuff and working more on projects. That's the way I split them up.
Max: One, two, three, and that sounds almost also like a chronological line you need to have your data pipeline in order before you can analyze it. And then only after you've got a large enough data set, can you bring in machine learning, NLP, and that data science.
Tim: Yes, and you would think that and I think a lot of the businesses only discovered that over the last few years where they initially built out advanced data science teams only to realize the things you just pointed out, which is you're going to have the data pipelines and the data is not clean, then, it doesn't matter how good your algorithms are if the data is not correct and you know…
Max: … I made that mistake.
Tim: Speaking from bitter experience, then.
Max: Absolutely, yeah! We're loaded on data engineering now, and I'm still actually still have to sort out through a lot of data, because you know, at Talkpush we process like millions of data points every day, and it's just a tremendous cleanup job. And, you can extract anything meaningful out of it. And, so, well let's imagine somebody is not a customer of Alooba, but still wants to interview a data engineer, a data analyst, and figure out who's right for the job. What are some good interview tips and methodologies that you can offer our audience?
Tim: Yeah! Yes, I think interviews are a really interesting space and we're going to start working on some products in that area soon. From our research, what we tend to think is that interviews tend to be done in quite an unstructured way on average which makes them potentially very, very subjective, and our vision for interviews is to try to help companies do them in a structured and therefore as an objective way as possible. Just as a quick anecdote, like I've had I've done many interviews and last year myself, sometimes with someone else in our business. It's really fascinating that you get to the end of the interview, and if you sit there and do your own evaluations of the candidate independently, you'll straightaway see the problem. And that is, that you'll often have very divergent opinions over how the candidate performs on any particular thing. So you might say, oh, I thought they were great on this field, and your colleague might said, oh terrible. So trying to find a way to make it as objective as possible is really what we're going for. I think that starts with just defining very clearly before you even start hiring, like what exactly you're looking for in the candidate. What are the requirements of the role? And then, setting up each stage the hiring process, be that an interview, the tests, some other process to just match exactly that and nothing else.
Tim: So, I think where companies often fall down is they just add extraneous steps that are you know, this is gone really well, but you know speak to Joe first. I'll have a quick chat with this person. There's so much moving the goalposts we call it in Australia. So, I think you know just trying to have as clearly a structured process as possible, down to the question level I'd say. If you're saying we're going to interview this candidate, define ahead of time what are the ten questions you're going to ask him and why. You know these three questions related this skill which we've identified as essential. These three questions relate to this trait which you've identified as essential. Then, within that try to have a scoring methodology, so you might have let's say for a particular question, you're going to ask like what would be a good answer, what would be an okay answer, what would be a bad answer, and try to think of that ahead of time, if possible, so, you can more objectively categorize the candidates and assign to each of those buckets. And then hopefully…
Max: …It's all those things on the fly right because you're trying to have empathy towards your audience and to read their answers and listen. So, you can think of your questions during the interview. You really have to prepare for it.
Tim: Yeah, exactly.
Max: So, let's get into a little bit more granular then. What are some interview questions that we can ask? For data engineers or data analyst, I think…Am I right in saying that data analyst is perhaps the more junior role of the three or the one where there's the most number of talents? And then data engineers, a little bit harder to find, data scientists is the hardest?
Tim: Possibly, yeah I'd say there are more entry level data analyst roles, and I'd say the skill set is a bit less technical than a group of data scientist or engineer. I think that's a fair comment , yup.
Max: Okay, so let's hone in on these guys on the data analysts that are a little bit more junior, because those are the hardest interviews, the ones with people who don't have a lot of experience.
Max: Yeah! What… I mean, I know you’re go to move would be… well let's move them to an assessment as fast as possible, and if somebody doesn't again that is not a customer of yours, what kind of assessments can they build at home?
Tim: Yeah, for sure so, I'd say any good hiring process probably has a combination of things in it, I think, absolutely you should interview candidates. I just think there's also some value add to testing this skill somehow, whether that's through a take-home assignment, whether it's through something like a platform either way. And, I think it's a case of thinking about what things you're trying to evaluate and whether or not they're better to evaluate an interview or a test. So, for example, anything that's more of a soft skill I'd say you're better off trying to evaluate in an interview. It's a lot easier to test someone's communication skills, that decision making, those type things in an interview. Anything is a bit more technical and a bit more black and white, for example, like writing sequel or Python coding or those types of you know, basic statistical and machine learning knowledge, I'd say put that into a test for a couple of reasons. One is you'll be able to assess a lot more things in the same period of time as what you could interview. Like in an interview, it's going to take at least a few minutes to go through each question, or in a test, you could ask three or four questions in a couple of minutes and gather more data points. And, do that in a completely apples for apples way, where every candidate has identical amount of time, they've been asked the questions in the same way like everything's held the same.
Tim: So, I'd say yeah, figure out what you're trying to assess and split it into a test or an interview. For the interview part of things, yeah, so for those kind of softer skills and what not, again I try to think ahead of time of like how you're going to categorize like what good communication skills are as an example. Something like that is actually quite vague in terms of what good communication skills are, like are you looking for they easily understood you and answer your questions? Are you looking for them being really articulate? Are they meant to be charismatic? And you know when we talk to companies and who we have worked with, and they said, oh, their communication skills weren't good enough. Like you dig in a couple of layers and you realize that that made so many things that turned different to many different people. Same for the cultural fit thing, like those types of things I'd be really careful with anything that's like very, very subjective and I just try to find a way to define it.
Max: Yeah, and back to your earlier point about people not being able to assess themselves, even if you ask somebody how your communication skills, there's a certain set of people who will rate themselves poorly even though they have impeccable vocabulary grammar fluency, understanding comprehension, but because they're not the popular kids in high school, they just think my communication skills aren’t good. So that makes sense, yeah. So, the assessment part where you're asking them to write sequel or do some Python coding, and so on. Is that something that is easy to… for a small company to put in place without you know, a professional tool which is time-bound, controlled, password-protected, or you know, does that open the gates to some cheating? How do you feel about like sort of like homegrown testing on these things?
Tim: Yeah! I think it depends on the scale of the company and who's already there. So, you’ve already had an analytics team built out with the skills to create those assessments and evaluate them, I think, potentially, it makes sense to do that. The other upside is you can use your own data sets and craft exactly what you need. And, if you're willing to wait, you know, for the candidate to be able to complete this, often don't take like a week, to go through and to evaluate it, if you're willing to do that, then you can get some quite deep insights because you can basically give them effectively real work to do. I don't think anything will ever be as stronger predictors as that. Like just giving them literally what they would work on and seeing how they go. Downside is really the time it takes the candidate and yourself to evaluate it, especially in this current market that is going to probably deter some candidates from completing it. The other thing is that you'll be struck down with scale, like you probably can't assess more than five candidates in a week, you know you probably want your data analysts doing data analysis, not grading assessments. So, there's probably some upside to using a platform because it's all kind of set up there waiting for you, the questions are created, the tests are ready, and all platform works, etc.
Max: Okay, well, I normally give this question to the end but I ask you now, so if somebody wants to work with wants to find out how the Alooba can help with the selection of data professionals, how do they get in touch with you? Who's your ideal customer and how do they get in touch with you?
Tim: Yes, so we normally work with a large enterprises or high-growth tech companies, and that's probably because they're the ones who hire the most in analytics and data science, so it makes sense, that they have the biggest problem to solve. And they can get in touch with us at our website alooba.com that's a-l-o-o-b-a.com and there's just a form that you can fill in to book a discovery call. Just start with a quick chat, understand basically your hiring process, your pain points, and then potentially to just tell how we could help.
Max: Great, great! And now what is…actually my last question, which is the one I asked all my guests is… to walk us back through a hiring mistake that you personally made, and without giving out names, recall a time when you hired somebody, made a hiring mistake, and walk us through the steps of that mistake so that we can learn from your mistake, and try to avoid making one ourselves.
Tim: Yeah! That's a great question. Let me contemplate on like…
Max: Are you seeing some ghosts flash before your eyes?
Tim: Some very recent ghost as well. Yeah, I mean, I think we…to be honest getting people right for our businesses probably been one of our biggest challenges, so this is really hard and I… everyone has my full empathy is on how hard it is to find the right people. I can think of a few people. So they fit into quite different categories, and some would fit into the category of us maybe over-indexing on skills, maybe drinking a bit of our own cool-aid right, like saying okay, maybe skill is all that matters and neglecting called…what I call values alignment let's say. So, after we went through that stage, this is about 18 months ago we got really, we sat down and talk really quickly about like what our values are in the company, like what we want our people to be like from a value perspective. We've got them incredibly concrete. And after that point we then embed them into our hiring process, so we now ask hiring questions at various stages of the process to cover up those values, so we weren't just focused on… Could they technically do the job? So, that was I guess one big trend. Another one was probably over-indexing on people that I’d known already and I was comfortable with and assuming that they can transition into quite a different role in a different type of business environment, and maybe also underappreciating how different is to sell different types of products. Yeah, and you know learning that new domain of software, sales and software marketing, and how much of a transition that can be if you've been in other environment.
Max: Okay, yeah. I'll latch on to the first one, you said, and this reminds me of a chat I recently had with the time Dr. CEO who said that… he mandates the culture fit assessment before the technical assessment because otherwise, in a market where technical skills are in such high demand, it excuses the rest of the hiring process. You have somebody who aces you know the assessment and it was like super great on a technical level, then you're going to kind of ignore all the red flags that come afterwards.
Tim: Yeah, absolutely and it's such a capital bouncing up, I think, because you know some companies can use cultural fit to mean anything, like you can exclude anyone in any hiring process for cultural fit, and yet we know that it matters like, you know, having certain types of people to work in a certain way is really important. I think it's just about can we more objectively concretely define what cultural fit is... to then make sure we evaluate candidates as fairly as possibly.
Max: You almost name like a Chinese wall between the person who evaluates values and technical skills so that you know they don't influence each other. They can both come up with their own independent scores regardless of you know, in which order you do it well. Tim it's been really educational, and thank you for sharing your knowledge on how to hire data professionals, and all the best to Alooba.
Tim: Awesome! Thanks for having me Max. It's been a great afternoon.
Don't forget to subscribe on your favorite platform for new weekly episodes.