Episode 12.2: Building AI infrastructure for ambitious research

Luke: I think University of Idaho has a really, you know, interesting and multifaceted strategy to AI. A lot of it centers around this concept of sovereignty. You know, we have mechanisms for people to access the frontier models, whether that's through Microsoft Copilot or Claude Enterprise or other methods when that's needed. But now we have the ability to run a lot of this infrastructure ourselves and control our destiny in that way. And so I think that sets us apart from a lot of universities that don't have that mindset, that are engaging fully with the large cloud providers and aren't investing in their own university controlled AI infrastructure in the same way that University of Idaho is.

Danae: Hi, everyone. Welcome to The Vandal Theory, a podcast about science and research at University of Idaho. I'm your host of the day. I'm Danae Lenz, external communications coordinator at U of I. And I am joined by my lovely co-host. Hi, Leigh.

Leigh: Hello, Danae. My name is Leigh Cooper and I'm a science writer and editor here at University of Idaho.

Danae: Today we're talking about a very timely topic that's at the top of everyone's mind, and that is artificial intelligence. So to kick us off, I want to ask you if you have ever used AI to help you solve a problem.

Leigh: Many, many times. But I'll use an example from work that came up recently. I'm part of a team that's looking at how AI talks about University of Idaho when the subject comes up. So for example, you type in, you know, "Who's got an engineering degree in the Pacific Northwest," does it talk about us? Does it talk about us nicely? When I started this, I didn't have a very good idea of how does it get that information? How does it learn that? What influences the AI's answer? So I've actually been asking AI how it works and how I can influence its answers, and it's been teaching me about how it works.

Danae: That's very meta. I also use it for a variety of things. The number one thing recently is I had to start a doctor mandated diet and I couldn't eat things like onions and garlic. And so I went to ChatGPT and was like, "Hey, how can I, like, make my usual recipes but without this?" And it's been really helpful.

Leigh: It can do that.

Danae: It can do that. So today we're going to get into all sorts of artificial intelligence stuff that's going on around campus. And we're talking to Luke Sheneman. He's the director of research computing and data services in U of I's Institute for Interdisciplinary Data Sciences. His team helps researchers get the computing tools and support they need for ambitious projects. We'll hear about how that work connects projects as different as the Deep Soil Ecotron and wildlife research, and what becomes possible when researchers have AI resources at their fingertips. Are you ready to get into it?

Leigh: I'm ready to go.

Danae: Hi, Luke. Welcome to The Vandal Theory.

Luke: Hi. It's good to be here.

Danae: Can you just start off by introducing yourself? Tell us a little bit about the work that you do at U of I.

Luke: I am the director of research computing and data services. We're organized under the Institute for Interdisciplinary Data Sciences, but we serve the entire campus. So we provide research computing solutions for faculty, staff, students across all colleges and departments, and even to some extent, external collaborators that want to engage with our services and are willing to pay for them. What that means by services, everything from high performance computing, which means like supercomputers, we provide access to those, access to special infrastructure to support research, whether that's large scale data storage, again, special servers that have, for example, lots of memory or have special hardware like GPUs or graphics processing units for doing AI and machine learning. So we provide that infrastructure component at the university. In addition to that we also provide services like web application development, mobile application development, just websites, things like that.

And we host those things here locally on our own university infrastructure. And then we also do data management. So we have a couple of full-time data managers on staff that work on funded research projects that are large scale, that are generating lots of data, and that need to manage that data effectively and disseminate it to other researchers or to the public. And so those are the kind of the things that my group does. And again, we support everybody across the university that's doing research. Lately, we've been doing more with AI and kind of getting into other things beyond the research envelope, but we can talk about that.

Danae: I'm excited to get into that, but just to start out, I want to make sure that everyone's on the same page. So artificial intelligence is a big umbrella term. For the purposes of this conversation, what exactly are we talking about?

Luke: Yeah, so artificial intelligence has been around as a discipline for a long time, since probably the 1950s, but it's changed a lot since then, obviously. What we mean by artificial intelligence in the common language today mostly involves these large generative AI models. So we think of them as large language models like ChatGPT or Gemini. Those are common. We also talk about AI in terms of the models that can generate images or video or music from texts. So those are the generative AI models that are sort of large scale where they generate something that humans are intended to consume, whether that's words or pictures or text or whatever. And so that's typically what we mean by AI today, and definitely our group is involved in spades with that kind of work. We're also working with more traditional, smaller scale, what we call machine learning projects.

And I think you and I will talk a little bit about some of those. So we support from the really small computer vision projects that can identify the difference between a cat and a dog to the ones that can actually write complex software.

Danae: U of I is really aiming to be a leader in AI. Can you talk a bit about what makes our approach unique?

Luke: I can talk a bit about, from my perspective, what makes our approach unique. And I think, so it really comes down to this concept that is called AI sovereignty. And so what's meant by AI sovereignty is that organizations, countries, institutions, and even individuals should have the choice of when to use AI and how to use AI and what kinds of governance and constraints and costs they want to associate with that as well as risk and benefits. And so when I talk about AI sovereignty, you'll notice in the news that, for example, European countries, this is really an important national thing for them because they're interested in developing their own capacity to control their destiny with respect to AI. They want to have their own models that they develop centered from Europe with European sensibilities that don't necessarily benefit American big tech companies.

And I think the University of Idaho is doing something similar to that in the sense that we're developing a layer of AI infrastructure that we manage ourselves, that we control, that we invest in from a hardware perspective, but that we can now use for things that we deem important for the University of Idaho without sending our data to the large big tech companies when we don't want to, in a way that we have full control and importantly, we have full observability. What that means is we can see how the resources are being used, how much energy they're consuming, what people are using them for, and we can sort of dial that in to maximize the benefits for everybody at the university. I think the University of Idaho has a really interesting and multifaceted strategy to AI. A lot of it centers around this concept of sovereignty.

We have mechanisms for people to access the frontier models, whether that's through Microsoft Copilot or Claude Enterprise or other methods when that's needed. But now we have the ability to run a lot of this infrastructure ourselves and control our destiny in that way. And so I think that sets us apart from a lot of universities that don't have that mindset, that are engaging fully with the large cloud providers and aren't investing in their own university controlled AI infrastructure in the same way that University of Idaho is. Part of it also comes down to cost. AI models can be expensive when you use them at scale, especially with the new, what we call agentic tools. If you're being metered on a token by token basis, those can get very expensive and we can control costs on our end by doing this fixed investment in hardware that we can then reuse for many years.

And over time we find that based on our token usage, that at some point we reach a breakeven point financially where we're no longer sending money to the big tech companies as much. We've already committed to our own fixed cost investment and it begins to pay for itself. And so we're already seeing that happen here at the university. And I think that really sets us apart not only from other universities in the country, but especially universities our size. I think it's a relatively unique approach and it seems to be working well.

Danae: Yeah. And speaking of something else that sets us apart, we do work with this supercomputer. Can you just get into that a little bit?

Luke: Oh, sure. Yeah. So the Idaho National Laboratory in Idaho Falls has a large data center, which is called the Collaborative Computing Center. And in that facility, there is a very large state-of-the-art data center where Idaho National Laboratory, which is a Department of Energy lab, can deploy their large supercomputers. Now, they've agreed as part of this arrangement for the mission of the building and the mission of the Collaborative Computing Center to coordinate with University of Idaho, Boise State University and Idaho State University, the three research universities, to coordinate with them to co-manage one of their supercomputers. So they've essentially handed over, originally it was a supercomputer called Falcon, which was, I think, commissioned back originally in 2014, I think. It was upgraded again in 2017, and we just decommissioned it last year and replaced it with a new supercomputer called Lemhi.

So Lemhi has 20,000 CPU cores over a terabyte of RAM, petabytes of disc space. It's a very large scale supercomputer, certainly for universities our size, and we are managing that along with Boise State and Idaho State. University of Idaho is the lead institution in that arrangement currently, and we're providing access to all researchers across our universities to access that supercomputer collectively. It's been a great relationship and I'm hoping that it continues for many years to come. I think the Lemhi supercomputer, for those researchers that need access to computers of that scale, provides a resource that's free of charge that they can use at any time to really enable research that wouldn't be possible otherwise without incurring additional costs or bureaucratic headaches. It's been an accelerant for research. I think it distinguishes us as having this substantial cyber infrastructure capacity for research.

And I think it helps attract students and faculty who see it as infrastructure that they can use and rely on. It's got lots of benefits from the perspective of research outcomes that wouldn't be possible otherwise, and also attracting and retaining high quality faculty and students.

Danae: Let's get into specifics now. What are some of the coolest things that you and your team have been working on recently?

Luke: I'll start with some of the large engagements that we have, which are pretty interesting, and then we can maybe talk about some of the AI specific things. So we're involved with the IAMP project, which is a USDA NIFA project. It's the largest grant that the university's ever received. It's operated through the College of Ag. So something on the order of $55, $60 million grant to work with growers to collect data, to work with them on marketing crops and commodities that are grown in a specific way. But there's a large data collection aspect to that, and it's a very big project that covers the entire state, and it's a multi-year thing with lots of reporting requirements. And so we've got a couple of people on our staff that's been working with that. It's super exciting to be supporting the largest grant that we've got, providing really critical infrastructure and software development to support that.

Another project that we're really involved with is the Deep Soil Ecotron. So the Deep Soil Ecotron is a facility that also run out of the College of Ag. It's a National Science Foundation mid-scale award, around 18, $20 million award to develop a large instrument, which will be a national resource for researchers to study soil chemistry and geochemistry at depth. It's kind of a unique facility where they're out actually able to study roots and soil biochemistry up to three meters deep, which is unprecedented, not only in the US but across the world. And it collects an enormous amount of data, right? So each one of these 24, what they call eco units in the Ecotron have dozens or hundreds of sensors that are continuously collecting real-time data. And our team is helping provide the infrastructure and the IT support to that, collecting and aggregating all of that data into one place, creating dashboards and tools that researchers can use to monitor their experiments that are running on that.

And we're working closely with Mike Strickland, Zach Kayler, and Ben Harlow on that project. Now moving a little bit towards the AI side, I'm a co-investigator on a project that we call AI for RA. So that's AI for research administration. This is a large, almost $5 million grant that we got from the National Science Foundation to use generative AI to help streamline administrative processes inside research administration. So the idea is to use large language models and other kinds of generative AI tools in a specific way to make the process of managing grants and contracts and that sort of thing much easier to do. Right now, today, that is this strangely arcane process that involves a lot of repetitive rote work, usually having to do with reading documents, comparing PDFs to each other for things like compliance or completeness. There's a lot of manual search, replace, cutting and pasting.

I call it the "swivel chair problem" where people have to look at one screen, get a little bit of data, swivel, look at another screen, get a little bit of data, another screen, copy and paste, and they do that all day long. It's important, valuable work, but a lot of that can be automated. This is one area which generative AI can actually be made to be really consistent and useful. We're making good progress with that. We have a tool called Vandalizer that is an outcome of that project that is receiving not just a lot of interest within the university's research administration unit, but nationally. So there's a lot of universities that are trying Vandalizer right now. They're working with it, trying to figure out if it works for their particular university. There's a lot of people who want to collaborate on co-developing it.

And so it's become, on the national stage, a very prominent output of this project. I'm involved in another project that we call Wild Marker. And so this is a National Science Foundation project. Scott Nuismer from biology is the principal investigator, and it's a multidisciplinary thing. So we have folks from mechanical engineering, biology, computer science and then me from research computing. And what we're doing is we're actually designing and developing and deploying a baited enclosure to help study small mammals out in the field. And so with our device that we're creating, it's a baited enclosure where animals can enter because there's some sort of food in there. It's this peanut butter-based thing that we invented. They enter the box. We've got cameras that are pointing either forward and also top down, and a small computer that's deployed in the box that's running AI models that can detect whether or not an animal's present.

They can detect the species of the animal, whether it's a Colombian ground squirrel or North Idaho ground squirrel, chipmunk, whatever, we can detect that. And then, in theory, take some immediate proactive action. So we could, in theory, mark the animal, do some sort of mark recapture kind of thing where we could apply a small amount of biodegradable paint to them so that we know if they come back, we know that we've seen them before, or in the future, maybe apply vaccinations. So we have a long-term interest in managing zoonotic disease in small mammals, and this might be an approach for doing that. We're also collecting data, so not only are we taking pictures of them, there's also a scale so we can weigh them. And there's a pit tag reader so that if they've been microchipped, we can read that as they enter into the dated enclosure.

This AI tool turns out to be really helpful for data collection in a passive way, so it's easier on the animal. They don't have to be trapped, stuck in a trap, and then blindfolded and then manhandled. All of this can be done passively, which is better for the animal, but it can also be done sort of at scale where we can deploy a lot of these boxes with these computers in them and collect an enormous amount of data with a smaller amount of staff that can be focused on maybe deploying the boxes and then managing that. It's an experiment. It's working out really well. We just finished our first field season. We've got two more in the grant, and we're working now on the next generation of box that we'll deploy.

Danae: Yeah, dude, all of this stuff is so cool. It's so exciting to see U of I at the forefront of all this stuff.

Luke: Oh, man, I have the coolest job in the world. I get to touch all these things; deep soil, small mammals, physics. I mean, I get to touch a lot of cool things. And then with AI as kind of a unifying glue in the middle, for me at least, it's been great.

Danae: Okay. So you are obviously super passionate about AI, U of I as a whole is super passionate about it, but what limitations, risks, or ethical questions should people keep in mind when they're using AI?

Luke: I would say that I have a nuanced sort of relationship with it. As far as ethics, there's a number of things, right? There's everything from, if you're using the big AI tools and models, everything from Google Gemini to OpenAI's ChatGPT to Anthropic models, those kinds of things, those all run in very large, massive data centers that are problematic from an energy consumption, land use, and water use perspective. I'm certainly cognizant of all of that and concerned about it. And I study it to some extent, right? So I understand a little bit in more nuanced way the water situation with data centers. The environmental impacts of data centers certainly concerns me. I think that some of the dialogue around it is overblown and some is underrepresented. It's a complex topic. I helped facilitate a tribal AI sovereignty workshop on campus back in July, and data centers were front and center to that discussion because the impact of data centers on Pacific Northwest tribes in particular who are really concerned about land use and water and things like that, it's on the top of their mind.

It was rewarding to talk to the tribes about their perspective on data centers and their concerns, and I learned a lot in the process. There's the environmental concerns that are important. There's issues of bias, right? All AI models are biased because it depends on how they are trained, who's training them, what decisions were made during the training of those models, and then how they're, and how they're used. It's impossible to remove bias from the models. You have to be aware of it and work with the models in a specific way if you want to mitigate that effect, but it certainly exists. I do think that the labs that build these large models are aware of this and they try their best to mitigate it, but at the end of the day, it's impossible to get away from the fact that you have a small number of people, usually in Silicon Valley, who are relatively detached from the rest of the world, who are making choices about how to handle bias in these large models.

And it doesn't always come out fairly for everybody, and it never will. It's just the nature of how these models are trained. When I talk about AI sovereignty, one of the things about that is that you can provide context, guardrails, and in some cases, fine-tuning training around these models to get them to behave more in the way that you need for your particular institution or tribe or state or whatever. So there is some flexibility if you're able to run your own AI models, and fortunately here at the University of Idaho, we can do that. I'm really concerned, honestly, about cognitive offloading. These models have become so good and they know so much that it's really easy to allow them to think for you, right? I definitely fall into that myself occasionally, and I try to pull back from that if I realize I'm just offloading too much of my own thinking to these models.

But it sometimes is the path of least resistance, especially when schedules are tight and, you know, whatever. And so that's a risk. That's one that I think is important for people to personally sort of try to address to whatever extent that they can as an individual, their relationship to AI and how they use it. But it's something that I definitely grapple with. There's a long list of ethically questionable things about AI. That said, I do think that AI can be used genuinely for productive, meaningful work. I have seen it advance science. I've seen it take monotonous workloads off of people so that they can focus on more important things. There are ways to deploy it that are meaningfully good. And you're starting now to see with the large AI models, starting to make, you know, fundamental discoveries, especially in mathematics, but you're going to start seeing that more and more in other areas, I believe.

You know, it's a complicated topic. There's pros and cons, and again, I have a very nuanced kind of complex relationship with AI.

Danae: I appreciate that the people leading this charge are aware of all of these things going on, and it's not just full speed ahead without taking all these things into account.

Luke: Yeah, exactly. And I think that's really important. AI is not going away. It's here to stay. We have to learn how to work with it, both as individuals and as an institution. We need to know what that means for students who are entering a world in which AI is going to be pervasive, but we have to grapple with the ethical issues of it too. I believe that part of the answer to that is what the University of Idaho is doing with respect to AI sovereignty. I think that that gives us certain dials that we can control that other universities can't. And so I'm really excited about that.

Danae: So we're almost done here, but what do you most want people to take away from this conversation?

Luke: I think we live in a very exciting time, right? For sure. I think everybody would agree with that, both on the AI front, but also just on how computing is getting used to enable and propel and accelerate science and research. I think research computing and data services, my unit here at the University of Idaho, is kind of at the epicenter of that right now where we're providing the critical infrastructure and the services around all of that, whether it's AI or not. But I would encourage anybody who's interested in talking with us about their research, their projects, their needs, you know, to reach out and talk to me and, you know, we can see what we can do to help.

Danae: Awesome. Thank you so much for being on the podcast. This has been great.

Luke: Yeah, thank you so much. It was super fun.

Danae: Okay, Leigh, what did you think about that conversation?

Leigh: That was great. There's just so much to talk about, so many layers and so many different types of projects that he's working on. Probably the first thing that stood out to me was the concept of AI sovereignty. I hadn't heard that term before. It's such an interesting and I think pertinent part of the conversation that's going to happen around AI. How much control do you as a person, organization, unit, country give up and hope that somebody else answers or solves or creates a tool to dictate AI's use? Or do you take the responsibility of doing that yourself? And I think that's a conversation that, as I said, is going to happen at every level and probably is extraordinarily pertinent. So yeah, I think that was a very interesting idea.

Danae: It's cool that U of I is kind of at the forefront of all of this as well.

Leigh: Agreed.

Danae: And I just, I think it's so cool, like how many different things we have going on on campus that are also related to AI. We've talked about quite a few of them on this podcast before, but you know, everything from the administrative backend stuff that he was talking about that saves a bunch of time and the ground squirrel research, but we've also talked about in the past about how they had a project that digitized all this old music so that people could use it. And it's just really cool how many different things, how many different areas this can touch.

Leigh: And I can't say that I haven't put some thought into the idea of cognitive offloading my thought process to AI. I've certainly found myself doing that at times. I hope that for the most part, I use it as a learning tool or to save time on a project that I could absolutely do myself and wouldn't necessarily gain any, you know, greater understanding or, or knowledge or practice. But I know I haven't threaded that needle perfectly at times.

Danae: Me too, Leigh. Me too. Thank you for listening to The Vandal Theory. Be sure to rate, review and subscribe on your favorite podcast platform to help others discover conversations about science, research and innovation at University of Idaho. Email us with research topics, questions or issues you’d like explored in a future episode at vandaltheory@uidaho.edu. To learn more about research at U of I and listen to past episodes of The Vandal Theory, visit go.uidaho.edu/vandaltheory. I'm Danae Lenz,

Leigh: And I'm Leigh Cooper.

Danae: Thanks for listening.