An overview of incus features

Incus has been around for a while, but confuses most people at first. It offers support for both new and old concepts, as well as entirely different guest instances types. Once you understand the basic concepts, it really starts to shine as a modern swiss army knife. This article aims to provide an abstract overview of incus features and benefits; it will not include commands or configuration.

Unified instance management

Arguably the most important feature of incus is the unified management it offers, either on the terminal through the incus command or the builtin HTTP API for automation and scripting.

It groups similar features under the same subcommands and attempts to infer context from metadata and sane defaults instead of enforcing explicit arguments every time.

For a linux administrator, it is what the love child of the docker operator experience combined with a builtin virsh replacement might look like, plus convenience features are a bonus.

Guest instance types

Incus supports three different types of instances:


OCI containers

Think classic docker/podman containers. A single application, usually single process, packaged alongside its dependencies. Shares the host kernel. starts in milliseconds and has near zero overhead. Primarily intended for ephemeral applications writing state to bind mounts or external storage. Updating usually means throwing the old container away and starting a newer version with the same storage mounts.


System containers

The evolution of LXC containers. Imagine "how close to a vm can you take a container without using hardware virtualization?". The result are system containers: software-isolated environments sharing the host kernel and starting in milliseconds, but running a complete init system and separate OS dependencies inside. They are meant to be long-running and stateful instances, retaining their disk contents and getting updates at runtime. Think of them as lightweight virtual machines.

However, sharing the host kernel means they can only run linux inside, and cannot load kernel modules.


Virtual machines

Full KVM guests, robust isolation through hardware virtualization. Requires compatible hardware and comes with the slow starts and performance overhead you are likely already familiar with. Since they are fully isolated and run their own kernels, they can accommodate any guest operating system, and freely manage their kernel modules.


Although all guest instance types are different and incompatible with one another, they are managed with the same incus commands. Administrators use the same interface to interact with any of them, streamlining the process and saving time.

Images

Incus takes the image concept from OCI containers and extends it to system containers and virtual machines as well. Images are basically a blueprint for an instance, containing necessary files and metadata.

What exactly is contained in an incus image heavily depends on type. For containers, it is usually a snapshot of a filesystem and some programs, while for virtual machines it is a virtual disk image and some metadata, for example whether to enable secure boot.

Because of this difference, incus images are not compatible directly across instance types: A container image cannot be used to create a virtual machine or vice versa (although you can convert one to the other with a few extra steps).

Incus offers the most common linux distributions and popular variants like cloud-init enabled versions in its default registry, and allows automatically fetching and starting any one of them with a single command.

Converting a virtual disk image or customizing an existing incus image is also fairly simple.

Images can easily be shared between incus hosts directly or through a proxy host, without the need for an external registry. For air-gapped or distributed sharing, they can alternatively be exported and imported as well.

Incus agent

Container ecosystems have grown accustomed to easily transferring files between container and host, or launching programs inside containers without authentication. Incus allows installing the incus-agent program inside virtual machines to extend that ease of use to vms as well.

Across all instance types, you can push or pull files, mount filesystems or start a shell session without any further setup inside the guest instance or managing access credentials for every instance. The result is reduced operator friction and allows administrators to focus on administration, not access and logistics.

Profiles

Profiles are a set of default configurations for instances. They are a great way to predefine hardware limits and connected devices that can then be used with any incus image.

For example, you could define a profile with 1vcpu, 1GB of memory and 10GB of disk and start an opensuse instance with that profile, automatically inheriting those limits without any further manual action. Or create one with dual network interfaces and 5 virtual disks for storage, and launch several CEPH nodes from plain debian images instead.

Hardware limits and device passthrough have existed for a long time, but they had to be passed explicitly, so it was easy to forget about some or simply omit them under time pressure. Profiles allow you to spend time thinking about them once, then reuse across containers and vms indefinitely.

Projects

A project in incus is most close to namespaces in other orchestrators. A namespace will have its own network devices and storage pool, and instances cannot see or interact across namespaces at all. In multi-tenant deployments, every team could get their own self-managed namespace, complete with limits of total cpu/memory/disk resources allowed across all their instances and isolation from other team's deployments.

It can act as a self-service feature or be used to isolate deployments from one another.

Security

In terms of security, incus has learned from previous problems and decided to build solutions directly into the program. Containers and vms are entirely rootless by default, getting mapped to unprivileged system users when using bind mounts. Seccomp and apparmor profiles are shipped and enabled by default, projects allow isolation between instances and the builtin HTTP API uses mTLS encrypted connections and requires authentication by default.

Allowing image and instance transfer between incus hosts directly eliminates the need for separate registries in most cases, further reducing total attack surface.

Automation

Most introductions glance over this point, but it is invaluable the moment you use incus in production. Per-instance backup scheduling is available out of the box, as well as automatic self-updating containers when newer images become available.

The HTTP management API offers complete access to all incus features for advanced scripting or orchestration tasks,

Support for providing cloud-init metadata when launching new instances and infrastructure automation tools like OpenTofu/Terraform and Ansible complete the list.

Whether you need only local test vms or deploy across hundreds of machines, incus is flexible enough to grow with your needs as you go, without requiring a steep learning curve up front.

Clustering

Clustering is a first-class citizen in incus, built into it from the start and requiring no extra utilities or changes to commands. Pick a highly available storage pool option like CEPH, add nodes to the cluster and connect them over a management network (or wireguard across the internet), and you have a complete cluster.

A small warning is in order here: Incus clustering is not a replacement to orchestration platforms like kubernetes or openstack. It does offer high-availability and automatic evacutation / migration of instances, but lacks support for load balancers and automatic networking fallbacks, so an instance restarting on a new node does not automatically retain its IP and may not be externally reachable anymore.

More articles

Choosing the right self-hosted S3 object storage service

A comparison of open source options and their tradeoffs

Handling errors in C

From errno to platform-specific functions

Go unit testing essentials

Real world testing patterns and workarounds