Herding LLaMaS: Using LLMs as an OS Module
Aditya K Kamath* and Sujay Yadalam*(*Authors contributed equally to this work)
Presented at ASPLOS 2023, Wild and Crazy Ideas Session, 2023
Sujay and I found ourselves chatting about the future of research one day. It was around this time that an explosion of interest in Large Language Models (LLMs) had started, sparked by the release of ChatGPT by OpenAI. Our discussion inevitably veered towards this topic, and we became curious about how LLMs could revolutionize the field of operating systems.
Our brainstorming led us to an intriguing idea: leveraging LLMs to enhance the decision-making of operating systems when confronted with a growing heterogeneity of devices. Imagine you have a system with a new type of memory that can give you amazing performance when reading from the device, but terrible performance when writing to the device. To get the best performance from your programs, you’d need to rewrite every application, manually keeping data that is heavily read from but rarely written to in this new type of memory. This just isn’t practical.
Instead, you can add a new component to the operating system which decides which program data to keep in which memory. This is what’s done today. The operating system makes all the hardware-dependent decisions for the program so that it doesn’t have to be rewritten for every type of device. These decisions could range from selecting the most suitable memory device to place data in to effectively scheduling processes on different types of compute devices. This requires a customized solution for every type of system, e.g. Non-Uniform Memory [1, 2, 3], systems with GPUs [4, 5], NVM [6, 7], or CXL memory [8, 9].
Instead of tailoring these solutions to specific systems, we envisioned that LLMs could act as an intermediary layer. The system administrator would provide the LLM with specific details about all the devices contained in the system. The LLM would then provide guidance and hints to the operating system about how to manage these devices. Any time a new device is added to the system, we only have to inform the LLM about it; no OS modifications are needed.
We performed some feasibility tests using ChatGPT and found it to be quite reliable. We eventually documented everything in the form of a talk which I presented at ASPLOS 2023’s Wild and Crazy Ideas Session.