Yes, both paths are real, and most people who try this start with the second.
Writing a whole OS from scratch is possible. Hobbyists have done it for decades, and the OSDev wiki (wiki.osdev.org) walks through the stages: a bootloader, a kernel, memory management, a file system and a shell. A small system can boot and run programs in a few months. The hard part is hardware: drivers for graphics, Wi-Fi, audio and power management are where most of the years go.
Building on Linux is the practical route. Linux already handles the hardware, so you work above the kernel:
- Remix an existing distribution: Debian live-build or Arch's archiso can produce your own bootable image with your own packages and desktop.
- Build a custom image for a fixed device: Buildroot and Yocto are tools for making small, purpose-built Linux systems.
- Learn by doing: Linux From Scratch teaches you how each layer fits together.
The AI part doesn't need kernel changes. It usually works as a system service: a local model runs through a tool such as Ollama or llama.cpp, and a small assistant app or voice layer sits on the desktop and can call approved actions like opening files or running commands. Keep that assistant in a sandbox with limited permissions, because an AI with root access is a serious security risk.
Two practical notes. Linux's kernel is under the GPL, so if you distribute a modified kernel you must share its source, while your own programs can usually stay private depending on their licenses. And the name 'Linux' is a trademark, so a custom system should use its own name.
If you tell me your hardware and what the AI should do (write code, manage files, answer questions by voice), I can suggest a starting setup.
Also discussed here:
- What open source accounting systems software are available?
- Why do people think Spark snd Ark protocol on bitcoin are open and …
