Developing and running AI workloads under OpenSuSE Linux Kalpa Desktop

If you find any issues with the instructions then do not hesitate to create an issue here
This document is under European Union Public License 1.2 (EUPL-1.2).

Changelog

Date Description
2025-09-22 Recommendation to run any development activities (including AI) under a separate user account with no access to sensitive data. Added instructions on how to update software in distrobox. Additional editorial changes.
2025-09-21 Using OpenSuSE Kalpa Desktop with AMD ROCm for developing and running rootless large lanugage models (LLM) workloads

If you have already another operating system installed then please make a backup of all your data to a separate medium or different computer/host before changing the operating system. Otherwise you may loose all of it without recovery.

Introduction

Many of the personal computing hardware that we currently use has one additional powerful chip (Graphical Processing Unit (GPU)) that is used for various purposes:

There are nowadays also other chips to do this, but the for personal computers the typical available chip is a GPU.

We provide here some basic instructions on how to use a GPU in a secure way (rootless without administrative privileges).
Note: Although we provide them in a rootless way, GPUs are contrary to CPUs less protected, so always work with care.

We will also use here a immtuable Linux operating system as this is the future for deploying ML/AI workloads. They encompass the following features:
These characteristics make a deployment and development reproducible, so that you can bring your deployed applications easily from your laptop to your service.

We will use here the immutable Linux operating system Kalpa Desktop, which provides a desktop. The instructions should be working equally well on its server counterpart OpenSuSE MicroOS.

Before we start, as a last information that "artificial intelligence"/machine learning applications have a significant cost on society. Use them with care and when necessary.

Installation of Kalpa Desktop

Note: We assume you have an immutable Linux installed. If you do not have Linux installed then you can follow here the instructions. Do not forget to make a backup of all your data before installing an operating system.

The installation of Kalpa Desktop is rather simple and you can follow the instructions.
There is nothing specifically to consider here all hardware is detected and configured automatically. This also include latest GPUs.
You may want to configure the following aspects during installation (you can also do it afterwards, but it is simpler during installation):

Using Distrobox with AMD ROCm in rootless mode

Installation

We will use Distrobox to provide an isolated rootless environment to run our AI/ML workload on an AMD GPU using the AMD ROCm libraries. These libraries allow to run modern ML/AI/LLM frameworks, such as pytorch, ollama etc., on AMD GPUs.
The reason to use Distrobox is that is one of the recommended installation methods on Kalpa Desktop and it provides container-isolation. This is not as strong as hypervisor isolation (cf. Virtual Machines), but balances convenience for developing and isolation. You can change the isolation level if you deploy your solution on a server.

Keep in mind that by default everything running in distrobox has access to your home folder. Thus, it is recommended that you run any development activites (including neural networks) at least using a separate user account that has NOT access to your sensitive data (e.g. password databases, financial information, personal data). You can simply create another user in Linux (just go to Settings/User) and login using this user and then follow the instructions below.
Distrobox supports different runtimes to run containers. We need to install the crun runtime as the default installed runc runtime does not support integration of devices, such as GPU in the container, as it does not allow to maintain the groups (see here).
While adding a device to a container always exposes some risks (e.g. priviledge escalation due to defects in the driver), it is needed to run software on the GPU (as we pointed out before, this is due to the design of GPUs).

You need to run the following command to install the crun runtime:


Afterwards you should restart your PC to be able to use crun.

Then you need to add your user to the group "render" so that your user is allowed to use the GPU.


Now you can create an own distrobox, where you put all the libraries and software inside to run in isolation. This is very convenient as the libraries/software related to GPU tend to spread and mess up your system often very deeply. With Distrobox you can uninstall everything easily by simply deleting the distrobox. Also, when leaving the distrobox your system will look untouched and you can enter a distrobox at any time to have immediately all libraries/software available.
Let us now create a distrobox with GPU support in rootless mode:

Afterwards we need to enter our created distrobox to install all the necessary software.

Once entered you will see the first time the following.

Make always sure you entered your distrobox properly (you can see it in the prompt).
Now you can install the software. You need to add an additional repository that provides the ROCm software on OpenSuSE tumbleweed:

Now you can install the ROCm software

Sometimes, if you have a very recent device, you need to define your device via an environment variable "HSA_OVERRIDE_GFX_VERSION".

Check in the documentation the LLVM target version. For example, you have the "AMD Radeon RX 9070 XT", which has the LLVM target version "gfx1201". Then you need to set "HSA_OVERRIDE_GFX_VERSION" to "12.0.1".

You can now show some information about your GPU. Note: your system may have multiple GPUs (e.g. one "on-board" and a dedicated graphics card).
AMD-SMI allows you to manage and monitor your GPUs. You can show a list of your GPUs as follows:


ROCm-clinfo will show you ROCm related information to OpenCL.


ROCminfo provides information about the Heterogenous System Architecture (HSA) system attributes and agents. You can run it as follows:


There are plenty of other tools that can show you information about GPUs and if your installation was successful.

Update

You should regularly update your software to have security patches, performance improvements and new features. Kalpa Desktop can be updated using the commmand line or Discover.
You can enter on the command line the following if your are OUTSIDE a distrobox session:

This updates Kalpa Desktop, but you need also to update ADDITIONALLY your distrobox.
If you did not do it already then please enter the distrobox where you have installed AMD ROCm:

Now run the following command to update it.

Using Ollama

Ollama is a tool and a library to facilitate running of large language models (LLM) (see Supported GPUs). Current generations of LLMs depend often not only for training on GPUs, but also for inference.
We will show you here how to use it together with AMD ROCm in distrobox in rootless mode. See previous section for installation instructions of AMD ROCm.
If you did not do it already then please enter the distrobox where you have installed AMD ROCm:

Now install ollama

You can now run ollama serve, which runs a local service to query LLMs. Please be aware that you should provide in "HSA_OVERRIDE_GFX_VERSION" the right version matching your graphics card (see previous section)

Now the server starts

Now we want to actually query a LLM. Create a new terminal and enter the distrobox where you have installed AMD ROCm

We use in our example the Teuken LLM, but you can choose any model from the Ollama Model Registry. You can also run some of the models from Huggingface.
Important: Please always verify who published a model on Ollama and that it is genuine. Otherwise you may get malware on your computer, which is not good, even if you use isolation such as provided by distrobox/containers. You may also get a model that will provide harmful or fraudulent responses.
Run the a model like this

A chat window will appear that you can interact with the LLM

You should make sure that Ollama uses a GPU. Create a new terminal and enter the distrobox where you have installed AMD ROCm

List the currently running models as follows:

You should see a list of models and how much they use of the GPU (ideal is 100% GPU). If you do not use fully the GPU then try a quantified version of your model or another model, which is smaller. Usually smaller models are also fine, because a lot of things depend on how you designed your AI system and less on the model. They also consume less resources and are faster.

You can also install the Ollama python library. You should use a separate venv for each of your projects to avoid library conflicts.

Using Pytorch

You can also use Pytorch with ROCm (see also Pytorch installation documentation). We need to enter the distrobox where we installed AMD ROCm (see previous section)

Afterwards, we create a virtual environment "pytorchproject" using venv for the project.

Then we activate the environment

Afterwards we install pytorch. Please make sure that you install a version that is compatible with your ROCm version (see here).

Now you can use Pytorch with AMD ROCm on an AMD GPU in your Python program.