Guide to building a Bitcoin Super Node

Cyph3rp9nk

The node is virtualized under Proxmox and we will use Start9 (embassyOS) as operating system.

Proxmox uses KVM which is a type 1 hypervisor and has nothing to envy to solutions like VMware ESXi, and the best, it is open source, based on Debian and with a modified Ubuntu LTS kernel. Why virtualize the node? Because it has all the advantages of running it directly on the hardware.

Hardware isolation, in the case of embassyOS being based on PureOS, a version of Debian focused on security and only using open source drivers, hardware support is quite restricted, also embassyOS does not support UEFI and there are many new computers that come without the Legacy BIOS option.
Possibility of making a complete backup of the machine on network storage, or on a USB disk, and the possibility of restoring the backup on another Proxmox machine without worrying about the hardware underneath, in addition, backups can be scheduled.
Possibility to upgrade the disk in the future without having to reinstall everything or follow complicated procedures.
Possibility to use the host machine for more things, also proxmox supports Linux Containers (LXC) so we can set up many machines running services at a ridiculous cost of hardware resources.
Proxmox supports software RAID via ZFS and Btrfs; in our case we will use Btrfs because it consumes less hardware resources than ZFS. This is a great advantage over using raid by BIOS/UEFI because in hardware RAID you will need the same hardware to restore in case of hardware failure, with software RAID you isolate yourself from the hardware, the disks can be moved to different hardware.
Easy migration of the virtual machine to new hardware, we only need to make a backup and restore it.
Possibility to take snapshots of the machine, which allows us to perform, for example, a restore in case of a failed update.
Possibility of having a node in high availability if we set up a cluster with two or more servers.

Why Start9 (embassyOS) and not another platform like Umbrel or others? Simply because embassyOS is well built, the other platforms are not built with security and scalability in mind.

Modular architecture, right now they use Docker but it could be easily changed.
Integrated backup system
Migration and restoration functions
Encrypted volume
Application integration
Possibility to create your own services
Stability

Proxmox installation with Btrfs and Raid 1.

The recommended hardware is two ssd or mechanical disks, but preferably ssd and 8gb of ram memory minimum.

Download the Proxmox iso from:

https://www.proxmox.com/en/downloads/category/iso-images-pve

In this guide I won't go into the basic details of how to create a bootable USB and so on, this you should know and if not Google is your friend.

The first screen you will see will be this one:

On the second screen you accept the terms of the contract:

And now look at the next one, this is where we select Btrfs and Raid 1 in options. As you can see two disks appear, you do not have to touch any other option, just click Ok. I want to remember that we use Btrfs against ZFS because it consumes less resources and also in ZFS ECC memory is recommended to avoid data corruption. And why do we use Btrfs against ext4 or others? Because like ZFS, Btrfs has an integrated volume manager that allows us to perform software raids without the need for mdadm with excellent performance and Btrfs is a COW file system that also protects us against data corruption.

In the next screen we select the country and time zone:

Now we choose the root password and an email for notifications, you can make it up:

In the next screen you configure the machine's network:

Finally check your data and click on install:

The machine will reboot and you will be able to enter to manage the Proxmox environment through a web browser and the ip you have set.

I am not going to comment much about virtual environments again, I understand you are familiar with them, however, I will comment the main features that are useful for your node.

Add update repository without subscription

Proxmox has two modes of operation, subscription and non-subscription (free), it is exactly the same but in the subscription mode the updates are more tested, we could compare it to the model of Red Hat and Fedora.

The first thing to do is to go to the repositories and disable the enterprise repository and then enable the no-subscription repository.

Now we add the no-subscription repository:

After this in the updates section we refresh the repositories and proceed to update from the upgrade button.

Failure of a disk in Btrfs

In case a Raid 1 disk fails, the system will not boot, because Btrfs does not allow to boot a degraded raid unless you use the rootflags=degraded option in the grub, but I do not recommend it because you will know when a disk fails.

When a disk fails the system will enter the initramfs console and there you will have to run:

mount -o degraded /dev/sdXY /root

sdXY is the disk where your Proxmox root partition will be, in this case the disk that remains alive.

After executing the command we have to press Ctrl + D to continue with the boot.

Once our proxmox system is booted again with another good disk, you will proceed to copy the partitions from one disk to the other since the two disks have to be identical so that in case of failure of one you can boot from the other.

sfdisk -d /dev/sdb > parts

sfdisk /dev/sda < parts

These commands copy the partitions from the sdb disk to the sda disk.

Keep in mind after this to copy the contents of the second partition corresponding to the EFI grub boot, to do this you simply mount the sda2 partition in any directory and copy the contents of /boot/efi that corresponds to the mount point of sdb2.

Now it is time to recreate the btrfs partition corresponding to raid 1 using the following command:

btrfs replace start 1 /dev/sda3 /

Where the 1 after the start indicates the failed disk that in this case was almost the first one, this can be seen with the command "btrfs filesystem show" and /dev/sda3 corresponds to the Btrfs partition of the new disk, finally / the mount point.

With the following command we can see the progress of the reconstruction.

btrfs replace status /

Virtual machine backup

Proxmox allows us to make backup copies of the virtual machine which facilitates the task of migration to new hardware or restoration in case of disaster.

First you will have to add the desired type of storage for the copy, for them you have to go to the datacenter options on Storage:

If for example you want to add a USB disk you would only have to mount it first in Proxmox and then pass it the mount point and name and the content type in this case VZDump.

After that you can run backups or schedule them. I'm not going to extend much here better you try it, just keep in mind that Proxmox supports three types of backups, snapshots, suspend and stop. The suspend mode is used for compatibility reasons and I do not recommend using it. The snapshots mode is the fastest as the machine does not have to be stopped, but you run the risk of having some inconsistency especially in certain databases and the stop mode stops the machine to perform a backup but it is the most consistent. For the migration of a machine to a new hardware I recommend the stop mode.

Snapshots of the machine

I recommend taking snapshots of your node before an upgrade or any critical change, it will save you hours of work in case of a disaster. As easy as going to the machine in question and clicking on snapshot:

Installing embassyOS

The first thing is to download the ISO in embassyOS in the Proxmox storage. For them we go to our local-btrfs and in ISO Images we put the download address and click on Query URL to capture the name of the ISO or add the name that we left manually, after which we click on Download.

https://github.com/Start9Labs/embassy-os/releases/download/v0.3.3/embassyos_amd64.iso

Now we will create the virtual machine:

We enter the name of the machine:

Select the installation ISO:

In the next screen everything is default:

In this screen we choose the disk size:

We choose the number of CPUs:

The amount of RAM:

The network configuration is left as default:

Finally click on confirm:

We start the machine and in console we can see its screen:

We proceed to the installation of embassyOS.

Select the installation disk:

We delete the loaded ISO:

To do this go to Hardware, select the CD and check "Do not use any media":

We return to the console and restart.

In the next screen we click on "Start Fresh". The options of recover to be able to restore from a backup or to make a transfer of the data to another machine, but these functions are delegated to Proxmox, except for the backup of the services that we will see later.

Select the installation disk:

And enter the password:

The machine is initialized:

And now we can proceed with the Login:

I am not going to go into how to configure embassyOS as it is as simple as going to the Marketplace and installing the desired services and you have a very complete manual on their website:

https://docs.start9.com/latest/user-manual/

If I emphasize the importance of the backup of the services that you can perform on a USB device or network. The function is located in System:

The backup is encrypted with your embassyOS password and is necessary for example to safeguard LND onchain funds and channel funds. In case of disaster just mount a new embassyOS and restore from the backup, once the initial synchronization of the LND network is finished the channels will be closed and you will have all your balance in onchain.

Disk expansion in embassyOS

I am going to expand on this point because it is one of the reasons why we have installed embassyOS in a virtual environment. The reason is not to depend on the hardware and treat the virtual machine as a file, so we can take it wherever we want. In this case we will assume that we have set up a new Proxmox server with more storage and we are going to migrate the embassyOS machine by making a backup of it and importing it into the new environment as previously explained in the Backup section.

The first thing would be to expand the space of the virtual machine embassyOS for it we go to the Hardware options and edit the disk:

We tell you how much we want to increase it:

And we restart the machine.

We enter by ssh in the machine, for this we will first have to import in Embassy our public ssh key, I will not explain how to create a public/private key, I assume that you understand how ssh works. embassyOS for security does not allow the Login ssh with username and password, only through keys.

Once inside we will check that the virtual disk space has been increased with the command:

sudo lsblk /dev/sda

As we can see now the space is 64G compared to the previous 32.

The next step will be to increase the Embassy data partition which corresponds to sda3:

As we can see, it currently occupies 16G because the remaining 16G are reserved for the operating system.

The first thing to do is to destroy the partition and recreate it again with all the remaining disk space. Do not worry, there is no data loss, we only modify the segments that delimit the partition. For this we will use fdisk.

sudo fdisk /dev/sda

With the option "p" we see the information of the disk.

With "d" we delete the selected partition, which will be "3":

Now we recreate the partition with "n", the partition has to be primary and of type "Linux LVM", we will change this later and we keep the partition number, the option "First sector" and "Last sector" we leave them by default so that it does not take all the available space.

It is very important to keep the signature, otherwise we will break the LVM volume.

The partition type is changed with the "t" option and the type is 8e (Linux LVM):

Finally, type "w" to save the changes and check again with fdisk that the changes have been made:

The next step is to tell LVM that the disk has changed size by running:

sudo pvresize /dev/sda3

The next step is to increase the LVM volume, to do this we enter the volume manager with:

sudo lvm

Once inside with the "lvs" command we will see the volumes:

The volume we want to expand is the "package-data".

Now we will have to find out the path to expand, for this we execute the command lvdisplay:

And finally we extend the lv with the command:

"sudo lvextend -l +100%FREE /dev/EMBASSY_ZO5QLRUJOANYWETOOFKNZBLBCL33XK6C7MGG4FFN6K5HNMPPW3SQ/package-data"

Now we can see again that the volume has expanded:

Finally, the only thing left to do is to extend the file system.

First we find out the device mapping with "df -h" and run the following command:

"sudo resize2fs -f /dev/EMBASSY_ZO5QLRUJOANYWETOOFKNZBLBCL33XK6C7MGG4FFN6K5HNMPPW3SQ/package-data"

Now we can see that the file system has been extended:

And if we go to the embassyOs monitor we can also see it: