Blog

Mounting ZFS failed with error 6 during FreeBSD boot.

On a machine with FreeBSD 13.5, that was originally prepared with GELI encryption and ZFS file system by the FreeBSD 11 installer, the following error message appears during the automatic FreeBSD boot proces, before it stops at the moutroot prompt.

Mounting from zfs:zroot/ROOT/default failed with error 6.
Loader variables:
vfs.root.mountfrom=zfs:zroot/ROOT/default
Manual root filesystem specification:
mountroot>

Trying to mount gives label error?

mountroot> zfs: zroot/ROOT/default ?
Sun Solaris label '' ?

The problem occurred after a second, and never used before, SSD was added to the machine with the plan to provide extra storage to the existing ZFS storage pool. The secondary SSD was prepared with GELI and ZFS. The machine was then rebooted.

If the machine is booted from a FreeBSD USB memory stick, then GELI can attach each SSD, if ZFS imports the boot partition with the GELI encryption keyfile first.

POSIX error code 6? Missing device?

# cat loader.conf 
geli_ada0p4_keyfile0_load="YES"
geli_ada0p4_keyfile0_type="ada0p4:geli_keyfile0"
geli_ada0p4_keyfile0_name="/boot/encryption.key"
aesni_load="YES"
geom_eli_load="YES"
geom_eli_passphrase_prompt="YES"
vfs.root.mountfrom="zfs:zroot/ROOT/default"
kern.geom.label.gptid.enable="0"
zpool_cache_load="YES"
zpool_cache_type="/boot/zfs/zpool.cache"
zpool_cache_name="/boot/zfs/zpool.cache"
zfs_load="YES"
pf_load="YES"
pflog_load="YES"

Setting boot flag.

# kldload geom_eli
# kldload zfs
# zpool import -R /mnt bootpool
# geli attach -k /mnt/boot/encryption.key ada0p4
GEOM_ELI: Device ada0p4.eli created.
# geli attach ada1p1
# zpool export bootpool
GEOM_ELI: Device ada1p1.eli created.
# geli info ada0p4.eli
Flags: BOOT
# geli info ada1p1
Flags: AUTORESIZE
# geli configure -b ada1p1
# geli info ada1p1
Flags: BOOT, AUTORESIZE
# geli detach ..
# geli detach ..
# reboot

The solved the issue about the error 6. However, further action was needed, as it stopped at mounting filesystems.

ZFS cache.

Import the ZFS root storage pool. This ensures, that ZFS can see all the devices, and, that the pool is cleanly imported.

# kbdcontrol -l /usr/share/syscons/keymaps/danish.iso.kbd
# kldload geom_eli
# kldload zfs
# mdmfs -s 100m md0 /mnt
# zpool import -f -R /mnt bootpool
# geli attach -k /mnt/boot/encryption.key ada0p4
GEOM_ELI: Device ada0p4.eli created.
# geli attach ada1p1
GEOM_ELI: Device ada1p1.eli created.
# zpool import
  pool: zroot
 state: ONLINE
config:
  zroot         ONLINE
    ada0p4.eli  ONLINE
    ada1p1.eli  ONLINE
# zpool import -f -R /mnt zpool
# zpool status
  pool: zroot
 state: ONLINE
errors: No known data errors

Replace the symbolic link with a directory, that can be used for mounting the boot partition. Export and import the ZFS storage pools, but this time, so the root is complete with the boot partition, where the ZFS cache is stored. Create a backup of the old ZFS cache files and remove them. Generate a new ZFS cache file.

# file /mnt/boot
/mnt/boot: broken symbolic link to /bootpool/pool
# rm /mnt/boot
# mkdir -p /mnt/boot
# zpool export bootpool
# zpool export zroot
# zpool import -f -R /mnt zpool
# zpool import -f -R /mnt/boot bootpool
# find /mnt -type f -name 'zpool.cache'
# cp /mnt/boot/boot/zfs/zpool.cache /mnt/boot/boot/zfs/zpool.cache.bak
# rm /mnt/boot/boot/zfs/zpool.cache
# cp /mnt/etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache.bak
# rm /mnt/etc/zfs/zpool.cache
# zpool set cachefile=/mnt/boot/boot/zfs/zpool.cache zroot
# zpool set comment="Test" zroot
# find /mnt -type f -name 'zpool.cache'
# zpool export bootpool
# rmdir /mnt/boot
# ln -s /bootpool/boot /mnt/boot ?
# zpool export zroot
# geli detach ada1p1
# geli detach ada0p4
# reboot

This caused bootpool to not get imported. When imported, it mounted as root and bricked.

Boot not mounted.

If /boot is a symbolic link, then delete it.

# ls -ld /boot
/boot -> /bootpool/boot
# zfs get mountpoint bootpool
# zpool set cachefile=/boot/zfs/zpool.cache zroot
# find / -type f -name 'zpool.cache' -delete

X?

Could this be caused, if GELI on the second SSD was configured with same passphrase as primary SSD, but without the requirement for the keyfile component? Could this be caused by the ZFS cache?

Ideas?

  • Suspect that the GELI boot flag are required on secondary SSD for the boot loader to decrypt it. The boot flag might be required for the boot loader to decrypt partitions, that are required for mouting the root.
  • Suspect that ZFS cache causes the error. Refresh it? Delete it?
  • Suspect that incorrect or lack of ZFS labels causes the error. Clear or set?
  • Un-merge the secondary SSD from the ZFS storage pool and attempt to revert to previous working state of the machine.
  • Suspect that lack of GELI keyfile requirement for secondary SSD causes the error. Enable the use of the GELI keyfile for secondary SSD, so both SSDs can be attached with the same user passphrase and the same keyfile by the FreeBSD automatic boot proces.
  • Suspect that lack of GELI keyfile requirement for secondary SSD causes the error. Disable the use of the GELI keyfile for the primary SSD, so both SSDs can be attached with the same user passphrase by the boot loader.

Any ideas?

Conclusion.

Do not add extra storage devices to a FreeBSD machine, that use GELI and ZFS, by simply merging it into the existing pool. You will end up with a bricked system, that is full of catch 22 type critical errors. This is caused by, how the ZFS cache works, and, how the boot partition and root partitions are inter-connected through a symbolic link, that blocks mounting. The way to add extra storage devices is a fresh re-install of the machine. You get the current FreeBSD, which include file system related changes, such as GELI and ZFS changes. You also save yourself a lot of time.

References.

English Language and Danish Regional Setting in FreeBSD

This is the procedure for configuring localization (locale) for English language and Danish regional setting in FreeBSD. Tested on FreeBSD 14.2 with XFCE 4.20 on 2025-03-31.

Introduction.

I have tested the official FreeBSD handbook procedure. I have tested countless guides and advice from forums, discussions and website. They have all failed to meet my basic requirements: English language and Danish regional setting in system console, window manager and applications. See my test below. I wanted Danish ISO 8601 standard format for date, time and numbers. This is also known as YYYY-MM-DD HH:MM:SS. I did not want to see US imperial format for date, time and numbers. I suddently had an idea of hacking an existing English locale and simply linking to a Danish for regional setting. It turned out to work perfect.

What is a locale setting?

A locale setting is made up of the language, the regional setting and the character encoding standard. As an example, en_DK.UTF-8 locale would be for user, who prefer English language in a Danish regional setting with the UTF-8 character encoding standard.

Locale specific information can be proveded with the locale utility. It is important to know, that the utility respect the PATH_LOCALE environment variable. If this is set, then this is used instead of the default locale directory.

# locale -a | grep DK
da_DK.ISO8859-1
da_DK.ISO8859-15
da_DK.UTF-8

Create a new locale.

Create a new locale definition, that is based on English. This new locale definition will be English language with Danish regional setting and UTF-8 character encoding standard.

# cd /usr/share/locale/
# mkdir -p en_DK.UTF-8
# cp -r en_GB.UTF-8/* en_DK.UTF-8/

Link to Danish regional setting.

For time format, numeric format and monetary format, create symbolic links to the Danish locale definition.

# cd en_DK.UTF-8/
# ls
LC_COLLATE LC_CTYPE LC_MESSAGES LC_MONETARY LC_NUMERIC LC_TIME
# ln -sf ../da_DK.UTF-8/LC_TIME .
# ln -sf ../da_DK.UTF-8/LC_NUMERIC .
# ln -sf ../da_DK.UTF-8/LC_MONETARY .

Confirm new locale.

# locale -a | grep DK
da_DK.ISO8859-1
da_DK.ISO8859-15
da_DK.UTF-8
en_DK.UTF-8

Configure FreeBSD login class for English language and Danish regional setting locale.

Configure the FreeBSD login class for English language, Danish regional setting and UTF-8 character encoding standard. This will work for system console, window manager and applications without further configuration. Examples are XFCE and Thunderbird Mail.

# nano /etc/login.conf
default:\
:passwd_format=sha512:\
:copyright=/etc/COPYRIGHT:\
:welcome=/var/run/motd:\
:setenv=BLOCKSIZE=K:\
:mail=/var/mail/$:\
:path=/sbin /bin /usr/sbin /usr/bin /usr/local/sbin /usr/local/bin ~/bin:\
:nologin=/var/run/nologin:\
:cputime=unlimited:\
:datasize=unlimited:\
:stacksize=unlimited:\
:memorylocked=64K:\
:memoryuse=unlimited:\
:filesize=unlimited:\
:coredumpsize=unlimited:\
:openfiles=unlimited:\
:maxproc=unlimited:\
:sbsize=unlimited:\
:vmemoryuse=unlimited:\
:swapuse=unlimited:\
:pseudoterminals=unlimited:\
:kqueues=unlimited:\
:umtxp=unlimited:\
:pipebuf=unlimited:\
:priority=0:\
:ignoretime@:\
:umask=022:\
:charset=UTF-8:\
:lang=en_DK.UTF-8:

Apply the changes by running the capability database utility.

# cap_mkdb /etc/login.conf

Test English language and Danish regional settings.

You can now log out of all virtual terminals and system consoles and test, that the new locale works.

  • English language in FreeBSD system console.
  • Danish special characters input in FreeBSD system console.
  • Danish special characters output in FreeBSD system console.
  • Danish ISO 8601 date and time format in FreeBSD system console.
  • Danish number format in FreeBSD system console.
  • English language in XFCE window manager menu, panels and dialogs.
  • Danish ISO 8601 date and time format in XFCE window manager.
  • English language in Thunderbird Mail.
  • Danish ISO 8601 date and time format in Thunderbird Mail.
  • Danish number format in Thunderbird Mail.

Backup and restore the new locale.

Create a backup of the new locale. This ensures, that it can be restored, if it should get deleted. It can also be used to install on another host.

# tar -cvpzf ~/en_DK.UTF-8.tar.gz -C /usr/share/locale en_DK.UTF-8
a en_DK.UTF-8
a en_DK.UTF-8/LC_COLLATE
a en_DK.UTF-8/LC_TIME
a en_DK.UTF-8/LC_CTYPE
a en_DK.UTF-8/LC_MONETARY
a en_DK.UTF-8/LC_NUMERIC
a en_DK.UTF-8/LC_MESSAGES

The content can be confirmed.

# tar -tvf ~/en_DK.UTF-8.tar.gz

The locale setting can be restored.

# tar -xvpzf ~/en_DK.UTF-8.tar.gz -C /usr/share/locale
x en_DK.UTF-8/
x en_DK.UTF-8/LC_COLLATE
x en_DK.UTF-8/LC_TIME
x en_DK.UTF-8/LC_CTYPE
x en_DK.UTF-8/LC_MONETARY
x en_DK.UTF-8/LC_NUMERIC
x en_DK.UTF-8/LC_MESSAGES

Issue: Port misc/locale-en_DK not working.

I tested the official port and package misc/locale-en_DK. This failed Danish special characters input in FreeBSD system console. I even spent extra time testing custom environment variables.

# pkg install locale-en_DK
Message from locale-en_DK-0.1.1:
In order to set the en_DK.UTF-8 locale for the login shell of an single
user add the following configuration to ~/.login_conf:
me:\
:charset=UTF-8:\
:lang=en_DK.UTF-8:\
:setenv=PATH_LOCALE=/usr/local/share/locale:
More information about the process of configuring login class methods in
available in the handbook.
More information about the PATH_LOCALE environment variable is available in the
locale(1) manual page.

References.

How to disable XFCE power manager on FreeBSD

Blank screen and muted audio in XFCE.

The first issue, I noticed, when switching from GNOME to XFCE, was, how the HDMI output was cut-off after a few minutes in which there has been no user input, such as attending a video conference. Pressing a key would turn-on the screen, but the audio would never return again. The audio could not even be re-enabled with Pulse Audio controls.

Disabling power management from XFCE interface.

Using the settings dialog, I disabled any kind of power management. This includes any screen blank and suspend option. Despite of this, the video and audio is getting cut-off. I could not find any kind of screensaver, that should be causing this.

# ps aux | grep power
/usr/local/libexec/upowerd
xfce4-power-manager
# ps aux | grep saver
# ps aux | grep screen

Removing power manager from XFCE?

I assume, that this is related to power management, because a screen saver would probably not be so aggressive. Knowing, that I will not need power management, a simple approuch would be to remove it from the installation. However, XFCE went for a forced depedency on this one. What a piece of shit programming from XFCE. Power management should be optional – and not a plague!

Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 2 packages (of 0 packages in the universe):
Installed packages to be REMOVED:
xfce: 4.20
xfce4-power-manager: 4.20.0
Number of packages to be removed: 2
The operation will free 2 MiB.
Proceed with deinstalling packages? [y/N]: N

How does power managment in XFCE work?

A search for occurances indicates, that it could be started automatically as an unwanted service. It could even be started, if it was running in an earlier user session.

# find / -type f -name '*xfce4-power*'
/home/lightman/.config/xfce4/xfconf/xfce-perchannel-xml/xfce4-power-manager.xml
/usr/local/share/metainfo/xfce4-power-manager.appdata.xml
/usr/local/share/man/man1/xfce4-power-manager-settings.1.gz
/usr/local/share/man/man1/xfce4-power-manager.1.gz
/usr/local/share/applications/xfce4-power-manager-settings.desktop
/usr/local/etc/xdg/autostart/xfce4-power-manager.desktop
/usr/local/bin/xfce4-power-manager-settings
/usr/local/bin/xfce4-power-manager

Stopping power manager in XFCE.

To my surprice, XFCE did actually write a built-in help, which kindly lists a quit option. It also seemed to actually quit the unwanted service. However, this is probably just a shortlived pleasure. It might restart. Especially after a new session or a reboot.

$ xfce4-power-manager --help
Usage:
xfce4-power-manager [OPTION?]
Help Options:
-h, --help Show help options
--help-all Show all help options
--help-sm-client Show session management options
Application Options:
--daemon Daemonize
--debug Enable debugging
--dump Dump all information
--restart Restart the running instance of Xfce power manager
-c, --customize Show the configuration dialog
-q, --quit Quit any running xfce power manager
-V, --version Version information
$ xfce4-power-manager -q

Killing the XFCE power manager from cron as a work-around.

As it remains unclear, how the unwanted service is started, a work-around is to repeat killing it from cron, until I learn, how this service can be permanently disabled. This cron job kills it every minute, if running. I have tested this be working.

# pkg info -r xfce4-power-manager
xfce4-power-manager-4.20.0:
xfce-4.20
# pkg info -r upower
upower-1.90.7:
xfce4-settings-4.20.1
xfce4-power-manager-4.20.0
gnome-power-manager-3.32.0_3
gnome-control-center-43.2_4
mutter-42.4_3
gnome-settings-daemon-42.2_8
# nano /etc/crontab
* * * * * root pkill xfce4-power-manager
* * * * * root pkill upowerd
# service cron restart

Tips appreciated!

I appreciate any tips about this issue.

Managing PulseAudio on FreeBSD

Configuring PulseAudio on FreeBSD.

In this example, the PulseAudio is configured to output to HDMI by default. If USB headphones are attached, then audio output is automatically switched to these. I tested this to be working.

$ pactl list sinks
Sink #3
State: RUNNING
Name: oss_output.dsp3
Description: 3 - Intel (0x2818) (HDMI/DP 8ch)
Mute: no
# nano /usr/local/etc/pulse/default.pa
set-default-sink oss_output.dsp3
load-module module-switch-on-connect

Switching audio output with pactl on FreeBSD.

pactl is a control interface for a running PulseAudio sound server. In this example, the audio output is switched to a pair of USB headphones. The listings indicate, that the headphones are associated with sink 4 and name oss_output.dsp5. The output is switched to to this and it is un-muted. Warning: The volume is optionally set to 100%. The volume setting is the same as the one, that is set on the headphones themselves.

$ pactl list sinks
Sink #4
State: RUNNING
Name: oss_output.dsp5
Description: 5 - Corsair CORSAIR HS80 RGB Wireless Gaming Receiver
muted: yes
Sample Specification: s16le 2ch 44100Hz
$ pactl list sinks short
4 oss_output.dsp5 module-oss.c s16le 2ch 44100Hz RUNNING
$ pactl set-default-sink oss_output.dsp5
$ pactl set-sink-mute oss_output.dsp5 0
$ pactl set-sink-volume oss_output.dsp5 100%

References.

Poudriere build failure: net-im/signal-desktop

I noticed, that my Poudriere build machine failed to compile net-im/signal-desktop for FreeBSD 14 with latest ports branch, because it require devel/esbuild 0.25.0, while build environment has 0.24.0. Tested 2025-03-07 and 2025-03-10.

✘ [ERROR] Cannot start service: Host version "0.24.0" does not match binary version "0.25.0"
1 error
Error: The service was stopped: write EPIPE
at /wrkdirs/usr/ports/net-im/signal-desktop/work/Signal-Desktop-7.44.0/node_modules/esbuild/lib/main.js:968:34
at responseCallbacks.<computed> (/wrkdirs/usr/ports/net-im/signal-desktop/work/Signal-Desktop-7.44.0/node_modules/esbuild/lib/main.js:622:9)
at afterClose (/wrkdirs/usr/ports/net-im/signal-desktop/work/Signal-Desktop-7.44.0/node_modules/esbuild/lib/main.js:613:28)
at /wrkdirs/usr/ports/net-im/signal-desktop/work/Signal-Desktop-7.44.0/node_modules/esbuild/lib/main.js:1988:18
at onwriteError (node:internal/streams/writable:605:3)
at process.processTicksAndRejections (node:internal/process/task_queues:84:21)
ERROR: "build:esbuild" exited with 1.
*** Error code 1
Stop.
make: stopped in /usr/ports/net-im/signal-desktop
=>> Cleaning up wrkdir
===> Cleaning for signal-desktop-7.44.0_1
build of net-im/signal-desktop | signal-desktop-7.44.0_1 ended at Mon Mar 10 05:13:34 CET 2025
build time: 00:00:47
!!! build failure encountered !!!

I notified the FreeBSD maintainer. He kindly replied, that he has notified devel/esbuild and the issue will be fixed.

Earlier this year, I noticed, that a similar issue happened. The required version was 0.21, while build environment had 0.24. I notified the maintainer. It was fixed.

Prevent Signal from being removed during update.

As a consequence of the failed build, I locked Signal. This prevents it from being deinstalled, while it fails to build.

# pkg lock signal-desktop
signal-desktop-7.42.0: lock this package? [y/N]:

It can later be unlocked.

# pkg unlock signal-desktop

Work-around for “Shared object “libFLAC.so.12″ not found”.

The update, while Signal being locked, resulted in a problem with a link to a FLAC library.

$ signal-desktop &
$ ld-elf.so.1: Shared object "libFLAC.so.12" not found, required by "signal-desktop"

I fixed this with a temporary link to the correct FLAC library. This should automatically be fixed, when Signal is fixed upstream.

# pkg info -l flac | grep libFLAC
/usr/local/lib/libFLAC++.a
/usr/local/lib/libFLAC++.so
/usr/local/lib/libFLAC++.so.11
/usr/local/lib/libFLAC++.so.11.0.0
/usr/local/lib/libFLAC.a
/usr/local/lib/libFLAC.so
/usr/local/lib/libFLAC.so.14
/usr/local/lib/libFLAC.so.14.0.0
/usr/local/share/aclocal/libFLAC++.m4
/usr/local/share/aclocal/libFLAC.m4
# pkg which /usr/local/lib/libFLAC.so.12
/usr/local/lib/libFLAC.so.12 was not found in the database
# ln -s /usr/local/lib/libFLAC.so.14 /usr/local/lib/libFLAC.so.12
# ldconfig -R
$ signal-desktop &

It can later be removed.

# rm /usr/local/lib/libFLAC.so.12
# ldconfig -R

Fixed.

The port compiled again 2025-03-21 and the package could be unlocked and the symbolic link removed.

References.

Poudriere build failure: graphics/cmake

My Poudriere build machine suddently stopped at the following error, that seems to be caused by a problem with CMake or run-time dependency, which should be built automatically.

build started at Fri Mar  7 16:46:57 CET 2025
port directory: /usr/ports/graphics/evince
package name: evince-46.3.1
...
Poudriere version: poudriere-git-3.4.2
Host OSVERSION: 1402000
Jail OSVERSION: 1402000
...
Did not find CMake 'cmake'
Found CMake: NO
Run-time dependency gnome-desktop-3.0 found: NO (tried pkgconfig)
meson.build:237:20: ERROR: Dependency "gnome-desktop-3.0" not found, tried pkgconfig

Any tips for solutions are greatly appreciated.

References.