fbpx XOMedia

CloudOps

Ultimate openSUSE Leap Upgrade Guide

Quick Summary: In this playbook we demonstrate how to perform an enterprise-grade minor version upgrade of openSUSE.
  
Last updated: | Leave a comment

Overview

We’re big fans of SUSE Linux (SLES and openSUSE), it’s one of several operating systems we use here at XOMedia for development, testing and production purposes.

SUSE Linux is a popular choice for both home and business users. It is a powerful and versatile operating system that is easy to use and customize. It shares a code base with SUSE Linux Enterprise (SLES), which gives it a level of stability unmatched by other Linux distributions.

In this Ultimate guide we are going to detail how to perform an enterprise-grade minor version upgrade of openSUSE. The upgrade will be performed [live / online / in-place, e.g., applications running with active logged in users]. This approach will minimize system, user and application downtime.

Ultimate guide entails that if you purchase SUSE (or any vendor) professional services to perform your installations and upgrades, you end up with a certified implementation that you can guarantee. This is especially important if you’re building out systems that will support 25+ million customers. Many well-meaning enthusiast read and fumble through a home installation and then publish these results on a Blog. Don’t bet your business or risk your career on anything less.

This procedure has been tested with the following openSUSE versions:

15.3 > 15.4 > 15.5

NOTE: Transcripts in this article apply to 15.3 > 15.4 (but also apply to any 15.x point release).

The supported upgrade starting point is an openSUSE release with all current updates applied.

We have seen some people skip minor number upgrades (e.g., 15.2 to 15.5). We do not recommend this. If you do, know what you’re skipping by reviewing the release notes. In any event, this is why we have viable backups and conduct sanities.

For example, the following remarks appear conflicting:

https://en.opensuse.org/SDB:System_upgrade_to_Leap_15.5

‘Warning: Do not skip a release when upgrading! Example: do not upgrade from 15.3 to 15.5. Instead, from 15.3 upgrade to 15.4, and only then from 15.4 to upgrade to 15.5.’

https://en.opensuse.org/SDB:Offline_upgrade

‘Official Support: As far as I know, you can upgrade between two versions that are or were currently supported, like from 13.1 to 42.1. But this assertion needs an official statement I can not make.’

If this is a production system, it’s assumed you have already tested the upgrade in Dev > Test > Staging. Even if you do not have non-prod environments, you can still test your procedure using a virtual environment (e.g., VMWare / VirtualBox).

Log out active users, quiesce applications and the system as much as possible to minimize upgrade errors. Use good technical judgment to determine this in your case.

If uptime is paramount: You can minimize the outage window by executing a live upgrade ahead of the outage and simply reboot the system during a small outage to commit changes.

If uptime is not critical: You can execute an (offline) upgrade booted from an ISO image (DVD/USB). The advantages of this approach are that all applications, files (databases) and libraries are closed (not in use) during the upgrade process. This may result in a longer maintenance window, but may be much safer in your situation. Again, use good technical judgment.

Regardless of which approach is used, you should always have ‘validated’ backups. This backup should be retained indefinitely.

If this is a VM, take a snapshot at each major stage of the upgrade process:

1)	Snapshot #1: Adding an addition disk to increase the root volume size.
2)	Snapshot #2: The upgrade.
3)	Snapshot #3: Post upgrade patches.

If there was an error in judgment with the selected upgrade approach, you can always restore / rollback and reassess your upgrade approach.

Pre-checks

Review the Release_Notes for your target version, and intermediate versions if you’re jumping versions (e.g., 15.2 to 15.5).

Review the openSUSE:Most_annoying_bugs page for the target version and intermediate versions.

Also review the openSUSE upgrade documentation for any new requirements that may have been introduced in the upgrade process between minor versions.

For example, the new 4096 bit RSA signing key was introduced in releases 15.4 / 15.5:

https://en.opensuse.org/SDB:System_upgrade#0._New_4096_bit_RSA_signing_key

Performing the upgrade
0. New 4096 bit RSA signing key"

You can verify if you need to install keys with:

[root@devlinux:/root]# rpm -qa | grep gpg-pubkey
gpg-pubkey-3adca79d-550f2d7f
gpg-pubkey-45f2c3d5-5e81efb9
gpg-pubkey-3dbdc284-53674dd4
gpg-pubkey-ef5975ca-6421ce2b
gpg-pubkey-29b700a4-62b07e22
gpg-pubkey-65176565-61a0ee8f
gpg-pubkey-d38b4796-570c8cd3
gpg-pubkey-3fa1d6ce-63c9481c
gpg-pubkey-39db7c82-5f68629b
gpg-pubkey-7fac5991-4615767f
gpg-pubkey-25db7ae0-645bae34
gpg-pubkey-307e3d54-5aaa90a5
[root@devlinux:/root]#

Storage

Ensure you have adequate storage across all system partitions / file systems to accommodate the upgrade (in our situation, the root volume had 3.8GB free and still ran out of free space). Each scenario is different, use good judgment. If you’re not careful, you can lose your system.

You can remove unused packages and dependencies to free up space:

[root@devlinux:/root]# zypper clean -a
All repositories have been cleaned up.
[root@devlinux:/root]#

3rd Party REPOs

Disable any 3rd party REPOs before starting the upgrade, or lower the priority of these REPOs to make sure packages from the default system repositories take precedence.

You can re-enable them after the upgrade and edit their version string to match the version number of the upgraded system.

List all REPOs with zypper lr, note the number on the left of the REPO you want to disable. You can then disable it with (zypper mr -d 3):

Diagram 2

Baseline Checks

Take before and after baselines (using cfg2html and XOMedia’s sysinfo script).

Execute a baseline sanity reboot so you know you’re upgrading a viable system (example, a faulty /etc/fstab).

Once the upgrade completes, we will conduct system and application sanities before turning the environment back over to the customer(s).

OS:

[root@devlinux:/root]# cat /etc/os-release
NAME="openSUSE Leap"
VERSION="15.3"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.3"
PRETTY_NAME="openSUSE Leap 15.3"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.3"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"

Kernel:

[root@devlinux:/root]# uname -a
Linux devlinux 5.3.18-150300.59.106-default #1 SMP Mon Dec 12 13:16:24 UTC 2022 (774239c) x86_64 x86_64 x86_64 GNU/Linux

Refresh REPO, metadata update:

[root@devlinux:/root]# zypper refresh
Retrieving repository 'google-chrome' metadata ....................................................................................................................................................................................................................[done]
Building repository 'google-chrome' cache .........................................................................................................................................................................................................................[done]
Repository 'jenkins' is up to date.
Repository 'Online updates for openSUSE Leap:15.0 (standard)' is up to date.
Repository 'oss' is up to date.
Retrieving repository 'oss_update' metadata .......................................................................................................................................................................................................................[done]
Building repository 'oss_update' cache ............................................................................................................................................................................................................................[done]
Repository 'Update repository of openSUSE Backports' is up to date.
Repository 'Non-OSS Repository' is up to date.
Repository 'Main Repository' is up to date.
Repository 'Update repository with updates from SUSE Linux Enterprise 15' is up to date.
Repository 'Update Repository (Non-Oss)' is up to date.
All repositories have been refreshed.

Enter the following command to apply pending updates to the current release. Our starting point should always be a fully upgraded current release:

[root@devlinux:/root]# zypper update
Loading repository data...
Reading installed packages...

The following package update will NOT be installed:
  yum

The following 14 packages are going to be upgraded:
  google-chrome-stable jenkins libavcodec57 libavformat57 libavutil55 libsass-3_6_1-1 libswresample2 libswscale4 transmission-common transmission-daemon transmission-gtk transmission-gtk-lang xdg-utils yum-lang

The following 3 NEW packages are going to be installed:
  libwebp6 libwebpmux2 yast2-trans

The following 2 packages are going to be REMOVED:
  yast2-trans-en yast2-trans-en_US

14 packages to upgrade, 3 new, 2 to remove.
Overall download size: 191.6 MiB. Already cached: 0 B. After the operation, 5.0 MiB will be freed.
Continue? [y/n/v/...? shows all options] (y):
Retrieving package jenkins-2.401.3-1.2.noarch                                                                                                                                                                                      (1/17),  93.6 MiB ( 93.9 MiB unpacked)
Retrieving: jenkins-2.401.3-1.2.noarch.rpm ............................................................................................................................................................................................................[done (1.9 MiB/s)]
Retrieving package libavutil55-3.4.4-lp150.6.1.x86_64                                                                                                                                                                              (2/17), 213.2 KiB (432.9 KiB unpacked)
Retrieving: libavutil55-3.4.4-lp150.6.1.x86_64.rpm ....................................................................................................................................................................................................[done (5.2 KiB/s)]
Retrieving package libsass-3_6_1-1-3.6.1-lp150.2.3.1.x86_64                                                                                                                                                                        (3/17), 735.7 KiB (  2.7 MiB unpacked)
Retrieving: libsass-3_6_1-1-3.6.1-lp150.2.3.1.x86_64.rpm ................................................................................................................................................................................................[done (465 B/s)]
Retrieving package transmission-common-2.94-lp150.2.3.1.noarch                                                                                                                                                                     (4/17), 318.0 KiB (749.2 KiB unpacked)
Retrieving: transmission-common-2.94-lp150.2.3.1.noarch.rpm .............................................................................................................................................................................................[done (465 B/s)]
Retrieving package transmission-gtk-2.94-lp150.2.3.1.x86_64                                                                                                                                                                        (5/17), 408.9 KiB (918.1 KiB unpacked)
Retrieving: transmission-gtk-2.94-lp150.2.3.1.x86_64.rpm .............................................................................................................................................................................................[done (28.7 KiB/s)]
Retrieving package transmission-gtk-lang-2.94-lp150.2.3.1.noarch                                                                                                                                                                   (6/17), 399.3 KiB (  2.0 MiB unpacked)
Retrieving: transmission-gtk-lang-2.94-lp150.2.3.1.noarch.rpm ...........................................................................................................................................................................................[done (931 B/s)]
Retrieving package transmission-daemon-2.94-lp150.2.3.1.x86_64                                                                                                                                                                     (7/17), 321.5 KiB (653.3 KiB unpacked)
Retrieving: transmission-daemon-2.94-lp150.2.3.1.x86_64.rpm ...........................................................................................................................................................................................[done (1.2 KiB/s)]
Retrieving package xdg-utils-20170508-lp150.3.3.2.noarch                                                                                                                                                                           (8/17),  89.4 KiB (344.9 KiB unpacked)
Retrieving: xdg-utils-20170508-lp150.3.3.2.noarch.rpm ................................................................................................................................................................................................[done (86.7 KiB/s)]
Retrieving package yum-lang-3.4.3-lp150.4.4.1.noarch                                                                                                                                                                               (9/17), 201.1 KiB (  1.2 MiB unpacked)
Retrieving: yum-lang-3.4.3-lp150.4.4.1.noarch.rpm .......................................................................................................................................................................................................[done (931 B/s)]
Retrieving package libswscale4-3.4.4-lp150.6.1.x86_64                                                                                                                                                                             (10/17), 186.2 KiB (524.3 KiB unpacked)
Retrieving: libswscale4-3.4.4-lp150.6.1.x86_64.rpm ......................................................................................................................................................................................................[done (931 B/s)]
Retrieving package libswresample2-3.4.4-lp150.6.1.x86_64                                                                                                                                                                          (11/17),  98.4 KiB (136.3 KiB unpacked)
Retrieving: libswresample2-3.4.4-lp150.6.1.x86_64.rpm .................................................................................................................................................................................................[done (2.3 KiB/s)]
Retrieving package yast2-trans-84.87.20210502.7b34dbceae-1.1.noarch                                                                                                                                                               (12/17), 311.6 KiB (    0   B unpacked)
Retrieving: yast2-trans-84.87.20210502.7b34dbceae-1.1.noarch.rpm ........................................................................................................................................................................................[done (328 B/s)]
Retrieving package libwebp6-0.5.0-150000.3.7.1.x86_64                                                                                                                                                                             (13/17), 173.5 KiB (370.7 KiB unpacked)
Retrieving: libwebp6-0.5.0-150000.3.7.1.x86_64.rpm ................................................................................................................................................................................................................[done]
Retrieving package libwebpmux2-0.5.0-150000.3.7.1.x86_64                                                                                                                                                                          (14/17),  27.2 KiB ( 38.3 KiB unpacked)
Retrieving: libwebpmux2-0.5.0-150000.3.7.1.x86_64.rpm .................................................................................................................................................................................................[done (8.3 KiB/s)]
Retrieving package google-chrome-stable-115.0.5790.170-1.x86_64                                                                                                                                                                   (15/17),  92.3 MiB (301.9 MiB unpacked)
Retrieving: google-chrome-stable-115.0.5790.170-1.x86_64.rpm ..........................................................................................................................................................................................[done (6.2 MiB/s)]
Retrieving package libavcodec57-3.4.4-lp150.6.1.x86_64                                                                                                                                                                            (16/17),   1.4 MiB (  4.4 MiB unpacked)
Retrieving: libavcodec57-3.4.4-lp150.6.1.x86_64.rpm .................................................................................................................................................................................................[done (114.1 KiB/s)]
Retrieving package libavformat57-3.4.4-lp150.6.1.x86_64                                                                                                                                                                           (17/17), 876.3 KiB (  2.1 MiB unpacked)
Retrieving: libavformat57-3.4.4-lp150.6.1.x86_64.rpm .................................................................................................................................................................................................[done (90.0 KiB/s)]

Checking for file conflicts: ......................................................................................................................................................................................................................................[done]
( 1/17) Installing: jenkins-2.401.3-1.2.noarch ....................................................................................................................................................................................................................[done]
( 2/17) Installing: libavutil55-3.4.4-lp150.6.1.x86_64 ............................................................................................................................................................................................................[done]
( 3/17) Installing: libsass-3_6_1-1-3.6.1-lp150.2.3.1.x86_64 ......................................................................................................................................................................................................[done]
( 4/17) Installing: transmission-common-2.94-lp150.2.3.1.noarch ...................................................................................................................................................................................................[done]
( 5/17) Installing: transmission-gtk-2.94-lp150.2.3.1.x86_64 ......................................................................................................................................................................................................[done]
( 6/17) Installing: transmission-gtk-lang-2.94-lp150.2.3.1.noarch .................................................................................................................................................................................................[done]
( 7/17) Installing: transmission-daemon-2.94-lp150.2.3.1.x86_64 ...................................................................................................................................................................................................[done]
( 8/17) Installing: xdg-utils-20170508-lp150.3.3.2.noarch .........................................................................................................................................................................................................[done]
( 9/17) Installing: yum-lang-3.4.3-lp150.4.4.1.noarch .............................................................................................................................................................................................................[done]
(10/17) Installing: libswscale4-3.4.4-lp150.6.1.x86_64 ............................................................................................................................................................................................................[done]
(11/17) Installing: libswresample2-3.4.4-lp150.6.1.x86_64 .........................................................................................................................................................................................................[done]
(12/17) Installing: yast2-trans-84.87.20210502.7b34dbceae-1.1.noarch ..............................................................................................................................................................................................[done]
(13/17) Installing: libwebp6-0.5.0-150000.3.7.1.x86_64 ............................................................................................................................................................................................................[done]
(14/17) Installing: libwebpmux2-0.5.0-150000.3.7.1.x86_64 .........................................................................................................................................................................................................[done]
(15/17) Installing: google-chrome-stable-115.0.5790.170-1.x86_64 ..................................................................................................................................................................................................[done]
(16/17) Installing: libavcodec57-3.4.4-lp150.6.1.x86_64 ...........................................................................................................................................................................................................[done]
(17/17) Installing: libavformat57-3.4.4-lp150.6.1.x86_64 ..........................................................................................................................................................................................................[done]
There are running programs which still use files and libraries deleted or updated by recent upgrades. They should be restarted to benefit from the latest updates. Run 'zypper ps -s' to list these programs.

Reboot:

[root@devlinux:/root]# systemctl reboot
Connection to 192.168.1.30 closed by remote host.
Connection to 192.168.1.30 closed.

Apply post-install patches (if any are pending) and reboot once again.

After all updates and patching are applied, the tray update indicator should reflect that everything is indeed up to date:

Diagram 3

Once the system is back online, verify that the reboot actually took place:

[root@devlinux:/root]# uptime
 17:26:17  up   0:00,  1 user,  load average: 1.46, 0.35, 0.12

Did the kernel version change in this update?

[root@devlinux:/root]# uname -a
Linux devlinux 5.3.18-150300.59.106-default #1 SMP Mon Dec 12 13:16:24 UTC 2022 (774239c) x86_64 x86_64 x86_64 GNU/Linux

Once again, verify that there are no 3rd party REPOs enabled:

[root@devlinux:/root]# zypper repos --uri
Repository priorities are without effect. All enabled repositories share the same priority.

#  | Alias                       | Name                                                                                        | Enabled | GPG Check | Refresh | URI
---+-----------------------------+---------------------------------------------------------------------------------------------+---------+-----------+---------+------------------------------------------------------------------------
 1 | PlexRepo                    | PlexRepo                                                                                    | No      | ----      | ----    | https://downloads.plex.tv/repo/rpm/x86_64/
 2 | google-chrome               | google-chrome                                                                               | Yes     | (r ) Yes  | Yes     | https://dl.google.com/linux/chrome/rpm/stable/x86_64
 3 | jenkins                     | jenkins                                                                                     | Yes     | (r ) Yes  | Yes     | https://pkg.jenkins.io/opensuse-stable/
 4 | openSUSE-Leap-15.3-2        | openSUSE-Leap-15.3-2                                                                        | No      | ----      | ----    | cd:/?devices=/dev/disk/by-id/ata-VBOX_CD-ROM_VB2-01700376
 5 | openSUSE_Leap_15.0_Update   | Online updates for openSUSE Leap:15.0 (standard)                                            | Yes     | (r ) Yes  | No      | http://download.opensuse.org/update/leap/15.0/oss/
 6 | oss                         | oss                                                                                         | Yes     | (r ) Yes  | No      | https://download.opensuse.org/distribution/leap/15.3/repo/oss/
 7 | oss_update                  | oss_update                                                                                  | Yes     | (r ) Yes  | No      | https://download.opensuse.org/update/leap/15.3/oss/
 8 | repo-backports-debug-update | Update repository with updates for openSUSE Leap debuginfo packages from openSUSE Backports | No      | ----      | ----    | http://download.opensuse.org/update/leap/15.3/backports_debug/
 9 | repo-backports-update       | Update repository of openSUSE Backports                                                     | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/15.3/backports/
10 | repo-debug                  | Debug Repository                                                                            | No      | ----      | ----    | http://download.opensuse.org/debug/distribution/leap/15.3/repo/oss/
11 | repo-debug-non-oss          | Debug Repository (Non-OSS)                                                                  | No      | ----      | ----    | http://download.opensuse.org/debug/distribution/leap/15.3/repo/non-oss/
12 | repo-debug-update           | Update Repository (Debug)                                                                   | No      | ----      | ----    | http://download.opensuse.org/debug/update/leap/15.3/oss/
13 | repo-debug-update-non-oss   | Update Repository (Debug, Non-OSS)                                                          | No      | ----      | ----    | http://download.opensuse.org/debug/update/leap/15.3/non-oss/
14 | repo-non-oss                | Non-OSS Repository                                                                          | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/distribution/leap/15.3/repo/non-oss/
15 | repo-oss                    | Main Repository                                                                             | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/distribution/leap/15.3/repo/oss/
16 | repo-sle-debug-update       | Update repository with debuginfo for updates from SUSE Linux Enterprise 15                  | No      | ----      | ----    | http://download.opensuse.org/debug/update/leap/15.3/sle/
17 | repo-sle-update             | Update repository with updates from SUSE Linux Enterprise 15                                | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/15.3/sle/
18 | repo-source                 | Source Repository                                                                           | No      | ----      | ----    | http://download.opensuse.org/source/distribution/leap/15.3/repo/oss/
19 | repo-update-non-oss         | Update Repository (Non-Oss)                                                                 | Yes     | (r ) Yes  | Yes     | http://download.opensuse.org/update/leap/15.3/non-oss/
[root@devlinux:/root]#

Create a backup of your existing REPOS:

[root@devlinux:/root]# cd /etc/zypp
[root@devlinux:/etc/zypp]# cp -rp repos.d repos.d.bkp

Verify if any of the ‘baseurl’ parms are hard coded (2 were in our setup):

[root@devlinux:/root]# cd /etc/zypp/repos.d
[root@devlinux:/etc/zypp/repos.d]# grep baseurl /etc/zypp/repos.d/*.repo
/etc/zypp/repos.d/google-chrome.repo:baseurl=https://dl.google.com/linux/chrome/rpm/stable/x86_64
/etc/zypp/repos.d/jenkins.repo:baseurl=https://pkg.jenkins.io/opensuse-stable/
/etc/zypp/repos.d/openSUSE-Leap-15.3-2.repo:baseurl=cd:/?devices=/dev/disk/by-id/ata-VBOX_CD-ROM_VB2-01700376
/etc/zypp/repos.d/openSUSE_Leap_15.0_Update.repo:baseurl=http://download.opensuse.org/update/leap/15.0/oss/
/etc/zypp/repos.d/oss.repo:baseurl=https://download.opensuse.org/distribution/leap/15.4/repo/oss/
/etc/zypp/repos.d/oss_update.repo:baseurl=https://download.opensuse.org/update/leap/15.4/oss/
/etc/zypp/repos.d/plex.repo:baseurl=https://downloads.plex.tv/repo/rpm/$basearch/
/etc/zypp/repos.d/repo-backports-debug-update.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/backports_debug/15.4
/etc/zypp/repos.d/repo-backports-update.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/backports/
/etc/zypp/repos.d/repo-debug-non-oss.repo:baseurl=http://download.opensuse.org/debug/distribution/leap/$releasever/repo/non-oss/
/etc/zypp/repos.d/repo-debug-update-non-oss.repo:baseurl=http://download.opensuse.org/debug/update/leap/$releasever/non-oss/
/etc/zypp/repos.d/repo-debug-update.repo:baseurl=http://download.opensuse.org/debug/update/leap/$releasever/oss/
/etc/zypp/repos.d/repo-debug.repo:baseurl=http://download.opensuse.org/debug/distribution/leap/$releasever/repo/oss/
/etc/zypp/repos.d/repo-non-oss.repo:baseurl=http://download.opensuse.org/distribution/leap/$releasever/repo/non-oss/
/etc/zypp/repos.d/repo-oss.repo:baseurl=http://download.opensuse.org/distribution/leap/$releasever/repo/oss/
/etc/zypp/repos.d/repo-sle-debug-update.repo:baseurl=http://download.opensuse.org/debug/update/leap/$releasever/sle/
/etc/zypp/repos.d/repo-sle-update.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/sle/
/etc/zypp/repos.d/repo-source.repo:baseurl=http://download.opensuse.org/source/distribution/leap/$releasever/repo/oss/
/etc/zypp/repos.d/repo-update-non-oss.repo:baseurl=http://download.opensuse.org/update/leap/$releasever/non-oss/
[root@devlinux:/etc/zypp/repos.d]# 

If so, fix with the following:

sed -i 's/15.3/${releasever}/g' /etc/zypp/repos.d/*.repo

Verify REPOs again with ‘grep baseurl /etc/zypp/repos.d/*.repo’

List REPOs with ‘zypper –releasever=15.4 lr -u’, all ‘baseurl’ values should now reflect 15.4.

Verify the URLs in the URI column in a web browser to see if they exist. If they do not, remove (or disable) the REPO(s) (the column ‘Enable’ needs to have a value of ‘No’).

Also, verify that the minor version you want to upgrade to has actually been released:

Diagram 4

https://en.wikipedia.org/wiki/OpenSUSE

And:

https://www.opensuse.org/

https://www.opensuse.org/#Leap

https://get.opensuse.org/leap/15.5/

Begin Upgrade

Update the REPO metadata for 15.4:

[root@devlinux:/root]# zypper --releasever=15.4 refresh
Warning: Enforced setting: $releasever=15.4
Repository 'google-chrome' is up to date.
Repository 'jenkins' is up to date.
Repository 'Online updates for openSUSE Leap:15.0 (standard)' is up to date.
Repository 'oss' is up to date.
Repository 'oss_update' is up to date.
Retrieving repository 'Update repository of openSUSE Backports' metadata ..........................................................................................................................................................................................[done]
Building repository 'Update repository of openSUSE Backports' cache ...............................................................................................................................................................................................[done]
Retrieving repository 'Non-OSS Repository' metadata ...............................................................................................................................................................................................................[done]
Building repository 'Non-OSS Repository' cache ....................................................................................................................................................................................................................[done]
Retrieving repository 'Main Repository' metadata ..................................................................................................................................................................................................................[done]
Building repository 'Main Repository' cache .......................................................................................................................................................................................................................[done]
Retrieving repository 'Update repository with updates from SUSE Linux Enterprise 15' metadata .....................................................................................................................................................................[done]
Building repository 'Update repository with updates from SUSE Linux Enterprise 15' cache ..........................................................................................................................................................................[done]
Retrieving repository 'Update Repository (Non-Oss)' metadata ......................................................................................................................................................................................................[done]
Building repository 'Update Repository (Non-Oss)' cache ...........................................................................................................................................................................................................[done]
All repositories have been refreshed.
[root@devlinux:/root]#

We’ll be using the --download-in-advance option here. Ideally, in a large corporate setting you would have a shared REPO or use something like ‘Red Hat Satellite Servers’ or a ‘Capsule Server’ that caches content on the other side of the corporate web proxy in your organization.

We were once engaged in a situation where a team was upgrading 30+ servers via a saturated web proxy. This led to a hideous error that did not appear to be related to a network connection issue. Tickets were opened to the vendor with everyone going in circles for half a year.

Over-the-wire upgrades should be avoided, especially in corporate environments. This update approach can be vulnerable to disruptions caused by unreliable network connections, leading to unexpected and subtle issues during the update process.

Other caveats: If the upgrade process is interrupted due to (e.g., power outage, network disruption etc), you ‘could’ be left with a broken system. It is recommended to upgrade via the ISO image or download all required files prior to executing a disconnected (offline) upgrade.

Start the distribution update (abbreviated transcript):

[root@devlinux:/root]# zypper --releasever=15.4 dup --download-in-advance
Warning: Enforced setting: $releasever=15.4
Loading repository data...
Warning: Repository 'Online updates for openSUSE Leap:15.0 (standard)' appears to be outdated. Consider using a different mirror or server.
Reading installed packages...
Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.
Computing distribution upgrade...

The following 1190 packages are going to be upgraded:
  ImageMagick ImageMagick-config-7-SUSE Mesa Mesa-dri Mesa-gallium Mesa-libEGL1 Mesa-libGL1 Mesa-libglapi0 Mesa-libva ModemManager ModemManager-bash-completion ModemManager-lang MozillaFirefox MozillaFirefox-branding-openSUSE MozillaThunderbird NetworkManager
  NetworkManager-applet NetworkManager-applet-lang NetworkManager-branding-openSUSE NetworkManager-connection-editor NetworkManager-lang PackageKit PackageKit-backend-zypp PackageKit-branding-openSUSE PackageKit-gstreamer-plugin PackageKit-gtk3-module
[...]
The following 19 patterns are going to be upgraded:
  apparmor base basesystem documentation enhanced_base fonts fonts_opt imaging minimal_base multimedia sw_management x11 x11_enhanced x11_yast xfce xfce_basis xfce_office yast2_basis yast2_desktop

The following product is going to be upgraded:
  openSUSE Leap 15.3  15.3-2 -> 15.4-1

The following 42 packages are going to be downgraded:
  cogl-lang gimp gimp-lang gimp-plugins-python gstreamer-plugins-cogl icewm-theme-branding libcogl20 libebl-plugins libgimp-2_0-0 libgimpui-2_0-0 libhogweed4 libnettle6 libply-boot-client5 libply-splash-core5 libply-splash-graphics5 libply5 librsvg-2-2 librsvg-lang
  libsgutils2-1_43-2 libvncclient0 libyui-ncurses15 libyui-qt-graph15 libyui-qt15 libyui15 plymouth plymouth-dracut plymouth-plugin-label plymouth-plugin-label-ft plymouth-plugin-two-step plymouth-scripts plymouth-theme-bgrt plymouth-theme-spinner python2-cairo
  python2-cffi python2-dbus-python python2-pycparser python2-rpm python2-six python3-rpm rpm util-linux util-linux-lang

The following 2 packages are going to change architecture:
  ModemManager-bash-completion  x86_64 -> noarch
  xorg-x11-libX11-ccache        noarch -> x86_64

The following 144 NEW packages are going to be installed:
  abseil-cpp bash-sh crypto-policies crypto-policies-scripts ctags distribution-logos-openSUSE-icons dracut-mkinitrd-deprecated fontconfig-lang gcr-ssh-agent geoclue2 google-opensans-fonts google-poppins-fonts grub2-i386-pc-extras ibus ibus-gtk ibus-gtk3 ibus-lang
  irqbalance-ui kernel-default-5.14.21-150400.24.74.1 kernel-default-devel-5.14.21-150400.24.74.1 kernel-default-extra-5.14.21-150400.24.74.1 kernel-default-optional-5.14.21-150400.24.74.1 kernel-devel-5.3.18-57.3 kernel-devel-5.14.21-150400.24.74.1
[...]

The following 10 packages are going to be REMOVED:
  libfreebl3-hmac libgupnp-1_2-0 libsoftokn3-hmac libyui-ncurses-pkg15 libyui-qt-pkg15 nma-data perl-TermReadLine-Gnu python3-cairo python3-smbc systemd-icon-branding-openSUSE

The following 2 packages require a system reboot:
  kernel-default-5.14.21-150400.24.74.1 kernel-preempt

1190 packages to upgrade, 42 to downgrade, 144 new, 10 to remove, 2 to change arch.
Overall download size: 0 B. Already cached: 1.69 GiB. After the operation, additional 1.3 GiB will be used.

    Note: System reboot required.
Continue? [y/n/v/...? shows all options] (y): y
In cache MozillaFirefox-branding-openSUSE-68-lp154.5.1.x86_64.rpm                                                                                                                                                                (1/1376),  14.6 KiB (  8.4 KiB unpacked)
In cache branding-openSUSE-15.4.20220322-lp154.2.3.noarch.rpm                                                                                                                                                                    (2/1376),  11.6 KiB (  1.6 KiB unpacked)
In cache btrfsprogs-udev-rules-5.14-150400.3.6.noarch.rpm                                                                                                                                                                        (3/1376),  65.0 KiB (  387   B unpacked)
[...]
In cache patterns-xfce-xfce_office-20210209-lp154.1.2.x86_64.rpm                                                                                                                                                              (1376/1376),  11.1 KiB (   57   B unpacked)

Checking for file conflicts: .....................................................................................................................................................................................................................................[error]
Detected 1 file conflict:

File /usr/share/bash-completion/completions/lsblk
  from install of
     util-linux-2.36.2-2.29.x86_64 (oss)
  conflicts with file from install of
     util-linux-systemd-2.37.2-150400.8.17.1.x86_64 (Update repository with updates from SUSE Linux Enterprise 15)

File conflicts happen when two packages attempt to install files with the same name but different contents. If you continue, conflicting files will be replaced losing the previous content.
Continue? [yes/no] (no): yes

(   1/1376) Installing: MozillaFirefox-branding-openSUSE-68-lp154.5.1.x86_64 ......................................................................................................................................................................................[done]
(   2/1376) Installing: branding-openSUSE-15.4.20220322-lp154.2.3.noarch ..........................................................................................................................................................................................[done]
(   3/1376) Installing: btrfsprogs-udev-rules-5.14-150400.3.6.noarch ..............................................................................................................................................................................................[done]
[...]
Restarting apache (all instances)
Executing %posttrans scripts ......................................................................................................................................................................................................................................[done]
Update notifications were received from the following packages:
mariadb-10.6.14-150400.3.26.1.x86_64 (/var/adm/update-messages/mariadb-10.6.14-150400.3.26.1-something)
View the notifications now? [y/n] (n): y
There are running programs which still use files and libraries deleted or updated by recent upgrades. They should be restarted to benefit from the latest updates. Run 'zypper ps -s' to list these programs.

Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.

Unless there’s an issue that needs to be addressed, acknowledge any conflicts that may arise (using good judgment) and resume the upgrade.

For example, in our case, the upgrade was interrupted 3 times:

Retrieving package patterns-xfce-xfce_office-20210209-lp154.1.2.x86_64                                                                                                                                                           (1376/1376),  11.1 KiB (   57   B unpack
Retrieving: patterns-xfce-xfce_office-20210209-lp154.1.2.x86_64.rpm ..................................................................................................................................................................................................[do

Checking for file conflicts: ........................................................................................................................................................................................................................................[err
Detected 1 file conflict:

File /usr/share/bash-completion/completions/lsblk
  from install of
     util-linux-2.36.2-2.29.x86_64 (oss)
  conflicts with file from install of
     util-linux-systemd-2.37.2-150400.8.17.1.x86_64 (Update repository with updates from SUSE Linux Enterprise 15)

File conflicts happen when two packages attempt to install files with the same name but different contents. If you continue, conflicting files will be replaced losing the previous content.
Continue? [yes/no] (no):

Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
History:
 - ABORT request:

Please see the above error message for a hint.
[root@devlinux:/root]#

Handling the above interruption:

If MySQL/MariaDB is installed, you may be prompted by the following notice during the upgrade. Make sure you have DB Exports and a Cold DB backup. A safe procedure for backing up your database has been provided in: Appendix > ‘Database Backup’. Post upgrade, be sure to conduct Application and DB sanities.

Enter ‘q’ to resume upgrade process.

Use the Enter or Space key to scroll the text by lines or pages.)

Message from package mariadb:


WARNING: You are upgrading from different stable version of MySQL!

Your database will be migrated automatically during next restart of MySQL.
Before you do that make sure you have up to date backup of your data. It
should be mainly in /var/lib/mysql directory.


-----------------------------------------------------------------------------


(Press 'q' to exit the pager.)
/var/tmp/TmpFile.qYfUaQ lines 1-16/16 (END)

You can resume fully interrupted upgrades simply by re-executing the upgrade command:

[root@devlinux:/root]# zypper --releasever=15.4 dup --download-in-advance
Warning: Enforced setting: $releasever=15.4
Building repository 'oss' cache ...................................................................................................................................................................................................................................[done]
Building repository 'Non-OSS Repository' cache ....................................................................................................................................................................................................................[done]
Building repository 'Main Repository' cache .......................................................................................................................................................................................................................[done]
Loading repository data...
Warning: Repository 'Online updates for openSUSE Leap:15.0 (standard)' metadata expired since 2020-02-15 11:04:07 -05.

    Warning: Repository metadata expired: Check if 'autorefresh' is turned on (zypper lr), otherwise
    manualy refresh the repository (zypper ref). If this does not solve the issue, it could be that
    you are using a broken mirror or the server has actually discontinued to support the repository.

Reading installed packages...
Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.
Computing distribution upgrade...

The following 7 packages are going to be upgraded:
  ibus ibus-gtk ibus-gtk3 ibus-lang libibus-1_0-5 yum zypper-aptitude

The following package is going to be downgraded:
  python2-yum

The following NEW package is going to be installed:
  ibus-dict-emoji

7 packages to upgrade, 1 to downgrade, 1 new.
Overall download size: 8.7 MiB. Already cached: 0 B. After the operation, additional 35.5 MiB will be used.
Continue? [y/n/v/...? shows all options] (y): y
Retrieving: python2-yum-3.4.3-lp150.4.4.1.x86_64 (Online updates for openSUSE Leap:15.0 (standard))                                                                                                                                                  (1/9), 429.5 KiB
Retrieving: python2-yum-3.4.3-lp150.4.4.1.x86_64.rpm .....................................................................................................................................................................................................[done (77 B/s)]
Retrieving: yum-3.4.3-lp150.4.4.1.x86_64 (Online updates for openSUSE Leap:15.0 (standard))                                                                                                                                                          (2/9), 174.1 KiB
Retrieving: yum-3.4.3-lp150.4.4.1.x86_64.rpm ............................................................................................................................................................................................................[done (465 B/s)]
Retrieving: ibus-dict-emoji-1.5.25-150400.1.13.noarch (Main Repository)                                                                                                                                                                              (3/9),   7.0 MiB
[...]
(8/9) Installing: ibus-lang-1.5.25-150400.1.13.noarch .............................................................................................................................................................................................................[done]
(9/9) Installing: zypper-aptitude-1.14.61-150400.3.24.1.noarch ....................................................................................................................................................................................................[done]
Executing %posttrans scripts ......................................................................................................................................................................................................................................[done]
There are running programs which still use files and libraries deleted or updated by recent upgrades. They should be restarted to benefit from the latest updates. Run ‘zypper ps -s’ to list these programs.

Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
[root@devlinux:/root]#

What zypper ps -s looks like, but we’re rebooting:

[root@devlinux:/var/lib/mysql]# zypper ps -s
The following running processes use deleted files:

PID   | PPID | UID  | User       | Command                       | Service
------+------+------+------------+-------------------------------+--------------------
931   | 1    | 499  | messagebus | dbus-daemon (deleted)         | dbus
938   | 1    | 493  | polkitd    | polkitd                       | polkit
941   | 1    | 0    | root       | VBoxDRMClient                 | vboxclient
1009  | 1    | 0    | root       | systemd-logind (deleted)      | systemd-logind
1033  | 1    | 461  | jenkins    | java (deleted)                | jenkins
1451  | 1    | 0    | root       | VBoxService                   | vboxadd-service
1477  | 1346 | 459  | plex       | Plex Script Host              | plexmediaserver
1497  | 1    | 1001 | syncthing  | syncthing (deleted)           | syncthing@syncthing
1504  | 1497 | 1001 | syncthing  | syncthing (deleted)           | syncthing@syncthing
1720  | 1    | 0    | root       | agetty (deleted)              | getty@tty1
1763  | 1    | 0    | root       | lightdm (deleted)             | display-manager
1769  | 1763 | 0    | root       | Xorg (deleted)                | display-manager
1780  | 1763 | 0    | root       | lightdm (deleted)             |
1794  | 1    | 466  | lightdm    | systemd (deleted)             |
1795  | 1794 | 466  | lightdm    | systemd (deleted)             |
1802  | 1780 | 466  | lightdm    | lightdm-gtk-greeter (deleted) |
1807  | 1    | 0    | root       | systemd (deleted)             |
1808  | 1807 | 0    | root       | systemd (deleted)             |
1835  | 1794 | 466  | lightdm    | dbus-daemon (deleted)         |
1839  | 1794 | 466  | lightdm    | at-spi-bus-launcher (deleted) |
1851  | 1839 | 466  | lightdm    | dbus-daemon (deleted)         |
1854  | 1794 | 466  | lightdm    | gvfsd (deleted)               |
1862  | 1794 | 466  | lightdm    | gvfsd-fuse (deleted)          |
1874  | 1763 | 0    | root       | lightdm (deleted)             | display-manager
1877  | 1794 | 466  | lightdm    | at-spi2-registryd (deleted)   |
3618  | 1    | 0    | root       | sshd (deleted)                |
3620  | 3618 | 0    | root       | mksh                          |
3859  | 1    | 0    | root       | sshd (deleted)                |
3861  | 3859 | 0    | root       | mksh                          |
3931  | 1    | 0    | root       | sshd (deleted)                |
3933  | 3931 | 0    | root       | mksh                          |
7331  | 1    | 0    | root       | sshd (deleted)                |
7333  | 7331 | 0    | root       | mksh                          |
11359 | 1    | 468  | nscd       | nscd                          | nscd
11909 | 1807 | 0    | root       | gvfsd (deleted)               |
11914 | 1807 | 0    | root       | gvfsd-fuse (deleted)          |
13387 | 1    | 474  | chrony     | chronyd                       | chronyd

You may wish to restart these processes.
See 'man zypper' for information about the meaning of values in the above table.

Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
[root@devlinux:/var/lib/mysql]#

Reboot:

[root@devlinux:/var/lib/mysql]# shutdown -r now
Connection to 192.168.1.30 closed by remote host.
Connection to 192.168.1.30 closed.

Validations

We’re upgraded from 15.3 to 15.4:

[root@devlinux:/var/lib/mysql]# cat /etc/os-release
NAME="openSUSE Leap"
VERSION="15.4"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.4"
PRETTY_NAME="openSUSE Leap 15.4"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.4"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"
DOCUMENTATION_URL="https://en.opensuse.org/Portal:Leap"
LOGO="distributor-logo-Leap"
[root@devlinux:/var/lib/mysql]#
[root@devlinux:/var/lib/mysql]# hostnamectl
 Static hostname: devlinux
       Icon name: computer-vm
         Chassis: vm
      Machine ID: 590ae5aab6fe4d05b2b21d2f99bc8ec1
         Boot ID: f894d53583674698a411152d7607b6b4
  Virtualization: oracle
Operating System: openSUSE Leap 15.4
     CPE OS Name: cpe:/o:opensuse:leap:15.4
          Kernel: Linux 5.3.18-150300.59.106-default
    Architecture: x86-64
[root@devlinux:/var/lib/mysql]#

The kernel was upgraded from…

[root@devlinux:/var/lib/mysql]# uname -a
Linux devlinux 5.3.18-150300.59.106-default #1 SMP Mon Dec 12 13:16:24 UTC 2022 (774239c) x86_64 x86_64 x86_64 GNU/Linux

…to:

[root@devlinux:/root]# uname -a
Linux devlinux 5.14.21-150400.24.74-default #1 SMP PREEMPT_DYNAMIC Thu Jul 27 15:04:26 UTC 2023 (28a2488) x86_64 x86_64 x86_64 GNU/Linux

Use the following commands to further validate:

Manually look over the system log:

less /var/log/messages

Execute egrep -i "error|warning" /var/log/messages, looking for (googling) anything of concern. Depending on your terminal program, output should be color coded. Some errors will be benign.

dmesg | more journalctl -xe

Our DB process looks good:

You should conduct a complete Application and Database sanity.

[root@devlinux:/root]# ps -ef | grep mysql
mysql      991     1  0 01:57 ?        00:00:00 /usr/sbin/mysqld --defaults-file=/etc/my.cnf --user=mysql
root     13197 13156  0 02:00 pts/0    00:00:00 grep --color=auto mysql
[root@devlinux:/root]# systemctl status mysql
● mariadb.service - MariaDB database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
     Active: active (running) since Thu 2023-08-10 01:57:19 -05; 2min 51s ago
       Docs: man:mysqld(8)
             https://mariadb.com/kb/en/library/systemd/
    Process: 951 ExecStartPre=/usr/lib/mysql/mysql-systemd-helper install (code=exited, status=0/SUCCESS)
    Process: 981 ExecStartPre=/usr/lib/mysql/mysql-systemd-helper upgrade (code=exited, status=0/SUCCESS)
   Main PID: 991 (mysqld)
     Status: "Taking your SQL requests now..."
      Tasks: 8 (limit: 4915)
     CGroup: /system.slice/mariadb.service
             └─ 991 /usr/sbin/mysqld --defaults-file=/etc/my.cnf --user=mysql

Aug 10 01:57:19 devlinux systemd[1]: Starting MariaDB database server...
Aug 10 01:57:19 devlinux systemd[1]: Started MariaDB database server.
[root@devlinux:/root]#
[root@devlinux:/root]# ps -ef | egrep -i "jenkins|syncthing|VBoxService"
jenkins    872     1  1 01:57 ?        00:00:10 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
syncthi+   874     1  0 01:57 ?        00:00:00 /usr/bin/syncthing serve --no-browser --no-restart --logflags=0
syncthi+   959   874  0 01:57 ?        00:00:01 /usr/bin/syncthing serve --no-browser --no-restart --logflags=0
root      1286     1  0 01:57 ?        00:00:00 /usr/sbin/VBoxService --pidfile /var/run/vboxadd-service.sh
root     14833 13156  0 02:14 pts/0    00:00:00 grep -E --color=auto -i jenkins|syncthing|VBoxService
[root@devlinux:/root]#

All is clean here:

[root@devlinux:/root]# zypper ps -s
No processes using deleted files found.

No core libraries or services have been updated since the last system boot.
Reboot is probably not necessary.
[root@devlinux:/root]#

If all is good, clear the cache:

[root@devlinux:/root]# zypper clean -a
All repositories have been cleaned up.
[root@devlinux:/root]#

Appendix

Database Backup

Here’s an example of a safe, Cold DB backup.

Stop the database:

[root@devlinux:/root]# systemctl list-units  | grep mari
  mariadb.service                                                                          loaded active running   MariaDB database server
[root@devlinux:/root]# systemctl stop mariadb

Verify:

[root@devlinux:/root]# systemctl status mariadb
○ mariadb.service - MariaDB database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
     Active: inactive (dead) since Thu 2023-08-10 01:31:53 -05; 35s ago
       Docs: man:mysqld(8)
             https://mariadb.com/kb/en/library/systemd/
    Process: 12590 ExecStart=/usr/lib/mysql/mysql-systemd-helper start (code=exited, status=0/SUCCESS)
   Main PID: 12590 (code=exited, status=0/SUCCESS)
     Status: "MariaDB server is down"

Aug 10 00:51:14 devlinux mysql-systemd-helper[12547]: wikidatabase.watchlist_expiry                      OK
Aug 10 00:51:14 devlinux mysql-systemd-helper[12547]: Phase 7/7: Running 'FLUSH PRIVILEGES'
Aug 10 00:51:14 devlinux mysql-systemd-helper[12547]: OK
Aug 10 00:51:14 devlinux mysql-systemd-helper[12514]: Everything upgraded successfully
Aug 10 00:51:14 devlinux mysql-systemd-helper[12514]: Shutting down protected MySQL
Aug 10 00:51:15 devlinux mysql-systemd-helper[12514]: Final cleanup
Aug 10 00:51:15 devlinux systemd[1]: Started MariaDB database server.
Aug 10 01:31:53 devlinux systemd[1]: Stopping MariaDB database server...
Aug 10 01:31:53 devlinux systemd[1]: mariadb.service: Deactivated successfully.
Aug 10 01:31:53 devlinux systemd[1]: Stopped MariaDB database server.
[root@devlinux:/var/lib/mysql]#

Execute a Cold backup:

[root@devlinux:/root]# cd /var/lib
[root@devlinux:/var/lib]# cp -rp mysql mysql.cold
[root@devlinux:/var/lib]#

Verify:

[root@devlinux:/var/lib]# du -hs mysql*
221M    mysql
0       mysql-files
221M    mysql.cold
[root@devlinux:/var/lib]#

Make sure you test this Cold backup either on a different system (aka, standby server) that has the same database application version installed (Ex. MySQL or MariaDB) or moving the current DB directory / files aside and moving the Cold backup you executed in place and starting up the database (conducting a quick Application and Database sanity).

Conclusion

Congratulations, you have a fully upgraded system! We hope this guide was helpful. You can share, upvote, discuss or let us know if you have any further questions in the comment section!


Thanks for reading!

If you enjoyed this post, please share:  

XOMedia is a full-service IT solutions provider. Learn how your business can benefit from XOMedia's 30+ years of experience with our Consulting and Partnership services - all work is backed by our 100% guaranteed.



Back to Blog Home

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments
To top