Are you interested in pursuing a career in machine learning? Professor Rahul G. Krishnan shares details about his research on ML in healthcare, giving examples and advice along the way for students about courses, undergraduate research, and more.
[1:53] What is the relationship between traditional Artificial Intelligence and Machine Learning? Professor Rahul G. Krishnan explains.
[4:24] Rahul discusses his research at the intersection of Machine Learning and Healthcare.
[9:40] How to do you get started in machine learning as an undergraduate student? Rahul discusses relevant course knowledge and gives examples of undergraduate research projects he has supervised.
[14:30] Rahul talks about grades, how they may impact graduate school applications, and contrasts work done in research with work done for an undergraduate course.
[18:00] Mario, Diane, and Rahul give some parting advice.
Diane: Everyone wants to know about machine learning, and we see it in the massive demand for ML courses. In this episode, Professor Rahul Krishnan offers advice for students who want to get involved in machine learning, and tells us about his research at the intersection of machine learning and healthcare.
Mario: You know, Diane, one of my favourite questions to ask students is why they chose to pursue computer science. And, more often than not, the answer I get includes the words “machine learning”.
Diane: Yeah, you know, machine learning has been getting a huge amount of attention in the media. And it’s just exploded since large language models like ChatGPT became publicly available.
Mario: In this episode, we’ll be talking with Dr. Rahul G. Krishnan, a professor at the University of Toronto whose work lies at the intersection of machine learning and healthcare. In this episode, Rahul gives us more insight into machine learning, his research, and how to pursue machine learning in and beyond an undergraduate degree.
Mario: Diane, do you know what my follow-up question is for students who answer with “machine learning”?
Diane: No, do tell.
Mario: Well, I usually ask “Why machine learning?”. But what I’ve found from these conversations is that, maybe, students coming into university have some misconceptions about what machine learning is.
Diane: I can definitely see that. The media often uses the terms machine learning and artificial intelligence almost interchangeably, when, in fact, machine learning is one approach to AI – it's a subfield of AI. Rahul talked about the relationship between machine learning and some more traditional methods.
Rahul: I think there's many, many similarities and many, many differences between them. But if I were to sort of highlight some of the key differences between them, I'd say that traditional AI, at least historically, has been referred, has been used to refer to the class of methods that are involved with symbolic manipulation, where the world or the variables of interest within the world are represented by symbols. And the goal is to get an intelligent agent to try and complete a certain task by manipulating the symbols that it has access to.
I think machine learning as, as it's often used in, in today's day and age, I think refers to the class of methods that aims to learn these symbols from data. And use the symbols that are learned internally within the model to complete tasks. So in the context of predictive problems, whereas perhaps, prior work might have taken the approach of handcrafting, a set of symbols that are correlated to a predictive task of interest, a lot of modern machine learning methods instead, take the view that one should use data alone, along with a model to try and automatically learn both the predictive task at hand and hope that the model and its internal representations captures the relevant symbols that are necessary to solve this task.
Mario: So, Rahul, it sounds like in artificial intelligence, the knowledge that is represented can come from a domain expert, for instance, someone with expertise in linguistics, but when we take a machine learning approach to AI, the knowledge is inferred from the data itself?
Rahul: That is a fair depiction of I think one of the key similarities between these methods, which is, where that knowledge is being represented and how it's being represented.
Diane: And of course, if the knowledge is inferred by the machine, and is implicit in the structures that it builds, rather than being explicitly represented, that leads to all the challenges that people are working on in terms of explaining what machine learning algorithms are doing.
Rahul: That’s right, I think what's been realized, at least in the community is that allowing machines to form their own internal representations often leads to very superior predictive performance. But it comes at this cost as you that you no longer have human interpretability into how the machine is making its decisions.
Mario: It’s easy to get lost in the terminology when we talk about things abstractly. Let’s try to make things more concrete by looking at how we might use machine learning in the domain of health care. Rahul, can you tell us about the research you do in machine learning and healthcare?
Rahul: So I spend about half my time thinking about how do we take some of the advances that we've seen in machine learning, and use that to solve clinically meaningful problems. I work with clinicians at one of them, at some of the many hospitals around the university to try and solve predictive problems that they might have in their own clinical practice.
And the other half of my research is on developing novel machine learning algorithms that are motivated by some of the unique problems that exist in healthcare. And so healthcare data is in many ways, interesting, because unlike data modalities, such as clinical text, or computer images, they have unique characteristics. And those unique characteristics often require the need for developing new models in order to solve predictive tasks with them.
Diane: Okay, so Rahul’s work goes in both directions: applying the latest machine learning results to clinical problems, and also using those problems to drive research on new machine learning algorithms.
Mario: And a key feature is that he works closely with clinicians to solve the problems that are actually helpful to them.
Rahul: Yeah. So often, what my colleagues and I do is we go out, and we talk to clinicians. And this is, I think, one of the fun parts of my job, because it's, it sort of gets me out of sitting in front of a computer all day. And I think the goal really in machine learning and healthcare is to work with clinicians to come up with and curate a dataset of clinical variables and outcomes of interest. And so the emphasis, at least in the first step of research in this space is to talk to clinicians to make sure that you're actually solving a clinically meaningful problem, which is to say, if you build a machine that can make predictions, someone might care to actually use that machine.
Diane: Let’s focus in on that. We asked Rahul to share an example of a predictive problem in healthcare.
Rahul: When a patient actually goes to the doctor, a lot of the doctor's decision making is being driven by this question of, can I take a look at the patient's clinical records, and leverage the patterns within them to come to an understanding of what the patient's underlying physiological state actually is. And then doctors use that information to guide treatment decisions, or to guide next steps, in terms of how the patient, whether the patient should stay on in the hospital or whether it should they should be discharged.
Mario: For our listeners, one of the key aspects here is leveraging patterns, something that computers and machine learning can do very well. Let’s listen to more from Rahul’s example to understand what kind of patterns we might be looking for.
Rahul: So these predictive problems are really checkpoints in the journey of a patient through the hospital system. And so one such example might be assessing the severity of a patient's tumor. If there's a patient who is suffering from some form of cancer, he or she may go to the hospital, and a surgeon might take out a portion of that tumor and put it under a microscope.
Pathologists, for example, would take a look at this tumor section and make an assessment of, what are the unique kinds of patterns that these kinds of cells exhibit. And so this is a very magnified image of a tiny, tiny portion of the tumor tissue. And by looking at these large images of the tumor tissue, pathologists have been trained to identify the severity of the cancer that is underlying within the patient. And so that's an example of a predictive problem where they're assessing the grade of the cancer. And that's one that I and many others are hoping that we can use tools from machine learning to automate and help accelerate treatment decisions.
Diane: And so that's predictive in the sense of taking this information and trying to predict how the tumor is likely to evolve.
Rahul: That's exactly right. So, one predictive task here might be the grade of the tumor. But another one might be the likelihood of the tumor getting worse or progressing.
Diane: Earlier, we talked about how machine learning can infer symbols from data on its own, but there was no mention of symbols here… we asked Rahul to clarify.
Rahul: In healthcare data, the idea of symbols is not one that was often used. And so it's interesting because clinicians, for example, if you were trying to build a predictive system of trying to understand which patient might need to be readmitted into the hospital, you might build a machine learning model based off of some of their clinical covariates. Now, their clinical covariates, on its own, have been specifically chosen as being representative of the patient's underlying physiological state.
Mario: For our listeners, this is the data that is provided to the model. Next, Rahul talks about what the model does with this data to produce a result that is useful to clinicians.
Rahul: What we often see in many cases is that the predictive model might latch onto combinations of those features that are relevant to solve some predictive task of interest. And so although the clinician knows that, individually, each of these lab tests is important, he or she may not know what combination is best predictive of an outcome of interest. And I think in those kinds of scenarios, that's really where the model might help.
Mario: For those who don’t know, Rahul teaches CSC311 – Introduction to Machine Learning. That gives him some unique insights into what undergraduate students need to know, and where there may sometimes be gaps.
Diane: That’s right. So, I asked Rahul: if I were a student interested in getting into machine learning, what prerequisite knowledge would I need?
Rahul: I think machine learning is sort of, as my colleague once put it, a field of immigrants, there's just a lot of, it's a blend of different kinds of tools and techniques all coming together with the common aim of extracting patterns from data. And so I think every investment in a foundational area often goes a long way before starting a career in machine learning.
It is incredibly important to invest in your foundations, your mathematical foundations, and so topics like linear algebra, probability and statistics, random stochastic processes and even advanced probability theory.
Mario: That takes me back. I remember that, during my masters, I was just blindly applying all these machine learning models to this data I had generated. But the results were meaningless, at least to me, which is a very bad thing when you are trying to write a paper and analyse the results. I basically had to teach myself a lot of statistics and probability before the discussions in my paper started making any sense.
Diane: Yeah, it takes me back, too. Linear Algebra is my like, undergrad, I don't know what's the word for it. But the course that I had the worst time in as an undergrad, and if I had known it was going to be relevant, in a specific way to a specific thing I was really excited about, like machine learning, it would have been so helpful.
Rahul: It's interesting, because when I was taking linear algebra, at, in the ECE department at UofT here many years ago, I think we had a phenomenal instructor at that time, who really used pictures and gave us a ton of geometric intuition and how you go from real valued numbers into vector dimensional spaces, I think a lot of that is so useful to me in terms of how I think of, what these matrices actually do as functions in vector spaces. And so I think, the foundational years and making sure that you actually understand the foundations, when you learn them, is incredibly important for a career in machine learning.
Diane: It's really great to have that specific list of topics that are relevant to ML. I know some of our listeners are interested in doing research in machine learning at the undergraduate level. What might that look like? Rahul, do you have an example you could share?
Rahul: Students in my lab, what they often end up working on is a sub part of one of a much bigger research project. So, for example, I have a graduate student who is interested in using machine learning to develop risk scores for patients on the transplant waitlist.
As part of that project, a crucial way in which we need to evaluate any risk score is through the use of a simulator. And what the simulator actually does is that it takes characteristics of a cohort of patients. And then it simulates what would happen if you were to use risk score A or risk score B, who might get a liver or who might not. And by looking at the characteristics of who actually receives the liver, and who does not, we can compare the differences between different risk scores.
I had a summer student who was an undergraduate, from UofT working on helping get an open source version of this simulator up and running in my lab. And that's something that we're using right now to evaluate the risk scores that the graduate student and the undergraduate student built together.
Mario: That’s a great, concrete example of doing applied research. Are there other kinds of research you have undergraduate students do?
Rahul: Students interested in more methodological research, what I often recommend that they do is start by reading some of the research papers that they've found on the internet through their own studies, or they've seen in the media and been really interested by and come to an understanding of the underlying methods in those problems. And often, there's interesting extensions that are suitable for an undergraduate student to work on if they've gotten the original paper or reproduced in their own spare time.
Diane: This segues nicely into another topic our listeners may be interested in. Any undergraduate student who has had the opportunity to contribute in any of those ways in your lab is going to have a real leg up in applying to grad school, no doubt. What advice you would offer to students who want to do graduate school in ML in particular?
Rahul: So I'd say that graduate schools often look at a student's grades, their letters of reference, and then research experience, those are the three things that graduate schools and admissions committees often use to evaluate a potential student who is applying to the department to continue to do graduate school there.
Mario: Okay, but Rahul, can you tell us exactly how important grades are? Compared to other things you listed, are grades above all?
Rahul: You can often interchange them, which is to say that, if you feel like your grades aren't as great as they could be, you can often make up for that by doing excellent research over the summers that you're an undergraduate student. And often having high quality reference letters might even make up for an odd bad grade here and there.
Mario: No, but seriously. Grades. They are super important right? Like, if I don’t have a 4.0, then I won’t be good at research, right?
Rahul: So, what I’ve found is that there's not necessarily a tight correlation between grade and quality of student researcher, because research is inherently very different from doing work as an undergraduate student. In your undergraduate degree, you're given homework where you know that there exists a solution to that homework, and it's your role to find it. In research, you're not even sure the problem is interesting. It's a much more curiosity driven frame of mind that students need to have. And it's one of the things that I recommend students try and spend at least one summer on the undergraduate doing to know if they are interested in it or even have a knack for it.
Diane: I think it's really interesting that you're talking about grades as an imperfect signal, students understand grades more than they understand the other parts of the application. And sometimes they get very fixed on, I have to have grades at a certain super high level, and they lose sight of the importance of the other elements. Do you think of grades as being like you need to be above a certain level? And then I'll look closer at you. And then I'm not that interested in your grades at that point? Or do you, do you see it differently?
Rahul: I do think that is, there is some minimum bar that a student needs to meet in terms of competence, particularly in the foundational subjects, you want to be able to go to graduate school and hit the ground running, you don't want to be relearning a lot of the foundational concepts that you expect it to know from your undergraduate degree. And so there's certainly a minimum bar that I look for.
But I also think that at least when I interview students, what I often ask questions that test their applications of the foundations that they have. So like, did they, you know, when they were learning about PCA did they really have a strong geometric intuition for how PCA works? Or were they just looking at the equations and writing them up and just thinking about them for the duration of that class? And after that, that was it?
Mario: So, not, not all of our students want to go into research. For those who are just finishing their undergraduate degree, but still want to get into machine learning, are there opportunities out there for them?
Rahul: I think one of the many benefits of this boom that we've seen over the last decade in the use of machine learning in industry, is that there's a need for people who have advanced technical skills and know how in the development and use of existing machine learning methods.
And so you'd see a lot of job opening for data scientists. And the goal there really is to take some of the knowledge that you have learned about in your undergraduate degree in CSC 311, CSC 412 and 413 and use it, and apply it to the problems that a potential employer might care about. Because if you're working for, say, a company that was interested in assessing the future price of goods, they might be interested in how well these machine learning models can be used for time series forecasting. Your role there would not be one of coming up with new research problems or coming up with new methodology, but rather taking methodology that you're familiar with, and applying it to a new set of data.
Mario: Wow, this episode was just jam packed with solid pieces of advice and information regarding machine learning.
Diane: I agree! Rahul was a wonderful guest. And, for our undergraduate students, he gave one more bit of advice that really resonated with both Mario and me as teachers. He talked about the value of building a mental map of the concepts you learn throughout your undergraduate program. Here’s what he has to say.
Rahul: Some students do a really good job at this. They're sort of able to pull in topics and concepts that they've learned in very disparate courses and understand how they link with each other. And I think if I have any advice, it's to do that process consciously as you go through your undergraduate degree, because it really does help solidify how the foundations that you learn relate to each other. And note that you know, like a topic like machine learning is a combination of linear algebra, probability and statistics, optimization, and numerical methods, and all of these put together.
Mario: And, Diane, that advice is not limited to machine learning. A lot of the subfields in computer science are a combination of several foundational topics. It’s really important to, at the end of a week, a semester, a year, reflect on how all those concepts fit together for the subfield that you’re interested in.
Diane: Well said. We hope that our listeners found this episode helpful, and we want to thank Rahul for his time and insights.
Mario: Absolutely. Thank you, Rahul, for being a part of this podcast.