Unix | Linux Storage Planning

Introduction

This post was inspired by an incident with a client that had sub-contracted Oracle to create Oracle Linux images in support of a 30+ million subscriber application. The Oracle engineers created (and defended) building out systems with a default storage layout of everything in slash (/).

For a desktop or workstation scenarios, everything in slash (/) is fine. But, I’d still argue in favor of a well thought out portioning layout.

For corporate images such as DEV / Test / Training / PROD ENVs: Always use sound partitioning for administrative control and system stability — as we’ll explain in the following sections.

Although the example that follows is demonstrated using a Linux system, the concepts explained are agnostic – and can be used regardless of OS, volume manager, file system type or underlying hardware. The aforementioned application images that Oracle built, were created on a laptop using Oracle VM VirtualBox and deployed to production.

OS

Use xfs and ext4 for maximum performance. ext4 has an advantage over ext3 like multi-block, delayed allocation etc.

File systems mount points should be owned by root, with a sub directory created for (and owned by) the project.

Keep system disk size at a sound minimum with room to grow for upgrades and patches.

slash (/): ‘If’ you planned your image build correctly, the disk group, number of partitions, partition / file system, sizes should be static. The less you touch prod, the better.

/var: Keep logging partition separate, you don’t want slash (/) at 100% in production.

/usr, /tmp, /home: Also on separate partitions. Again, we don’t want to take the system, app or dB down, cause corruption.

/opt: Separate, for vendor. (3rd party). If the application writes logs, either configure ti to point to /var or a separate logging partition.

This is a storage layout example for a corporate image for RHEL (7.x, 8.x, 9.x) flavors using LVM / EXT:

~]# vgs
  VG    #PV #LV #SN Attr   VSize     VFree
  vg00    1   9   0 wz--n- <49.00g   <4.00g
  vgapp01 1   5   0 wz--n- <99.00g   <49.00g

~]# lsblk
NAME                    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                       8:0    0   50G  0 disk
+-sda1                    8:1    0    1G  0 part /boot
+-sda2                    8:2    0   49G  0 part
  +-vg00-root           253:0    0    2G  0 lvm  /
  +-vg00-swap           253:1    0    4G  0 lvm  [SWAP]
  +-vg00-usr            253:2    0    6G  0 lvm  /usr
  +-vg00-var            253:3    0    6G  0 lvm  /var
  +-vg00-opt            253:4    0    6G  0 lvm  /opt
  +-vgapp01-opt-hds     253:5    0    8G  0 lvm  /opt/hds
  +-vgapp01-opt-intel   253:6    0   30G  0 lvm  /opt/intel
  +-vgapp01-var-log-hds 253:7    0    2G  0 lvm  /var/log/hds
  +-vgapp01-rpms        253:8    0   10G  0 lvm  /rpms
  +-vgapp01-var-opt-OV  253:9    0    1G  0 lvm  /var/opt/OV

~]# df -k
Filesystem                     1K-blocks       Used  Available Use% Mounted on
devtmpfs                        16342752          0   16342752   0% /dev
tmpfs                           16360596          0   16360596   0% /dev/shm
tmpfs                           16360596      33592   16327004   1% /run
tmpfs                           16360596          0   16360596   0% /sys/fs/cgroup
/dev/mapper/vg00-root            1992552     686604    1184708  37% /
/dev/mapper/vg00-usr             6102624    3255396    2516272  57% /usr
tmpfs                            4194304       7144    4187160   1% /tmp
/dev/sda1                         996780     220968     707000  24% /boot
/dev/mapper/vg00-opt             6119008    3112652    2717348  54% /opt
/dev/mapper/vgapp01-rpms        10218772        472    9677628   1% /rpms
/dev/mapper/vg00-var             6110816     785540    5015296  14% /var
/dev/mapper/vg00-var-log-hds     1992552         24    1871288   1% /var/log/hds
/dev/mapper/vgapp01-opt-hds      8154588         24    7718752   1% /opt/hds
/dev/mapper/vgapp01-opt-intel   30787492   23853820    5344424  82% /opt/intel
/dev/mapper/vgapp01-var-opt-OV    996780     166752     761216  18% /var/opt/OV

Applications

For large 3rd party applications like WebSphere or Intel oneAPI, install on separate partitions.

For example. the Intel compiler product is far too big to mix with other /opt APPs. Create a logical volume and file system separate from /opt:

VG: vgapp01
LV: intel
FS: /opt/intel

Or, if the developer team requires 10 versions of Java (for dev/test), create a /usr/java, or /java and sym link to /usr/java

Databases

Keep database partitions and filesystems separate from your system disk and other applications.

An example layout for Oracle:

File system sizing:

• Oracle home:
Oracle home, directory where Oracle products are installed: /ora01 ~15GB
• Oracle datafiles:
For our particular application, file system sizing typically depends on the number of environments: /ora[##] We use a formula 0.8 GB per runtime environment.
• Archive and Redo LOGs: /ora-redo*:
• Oracle tablespaces: 5GB for System, Temp, Undo.
• Oracle export files: /ora97: 10GB (or ideally, about 1/3 of the data files sizes).
• Hot/Cold backups: /ora98: 10GB
• Oracle archive files: /ora99: 10GB

File system mount options:

atime/noatime

When working with EXT file systems, set the noatime mount option for Oracle and Hadoop systems.

Linux records information about when files were created, last modified and when it was last accessed. There is a cost associated with recording the last access time. The EXT file system has an attribute that allows you to disable marking individual files such that their last access time is not recorded. This can lead to significant performance improvements on often accessed frequently changing files such as the contents of the /var/spool/news directory.

The below filesystem mount options will also affect performance. noatime will prevent excessive writes for file (and directory) by disabling access times- which are not required for certain applications (Oracle/Hadoop).

/etc/fstab

These file systems do require noatime:

/dev/vg0-ora01/ora02   /oravl02   ext4   defaults,noatime 1 2
/dev/vg0-ora01/ora03   /oravl03   ext4   defaults,noatime 1 2
/dev/vg0-ora01/ora04   /oravl04   ext4   defaults,noatime 1 2

These file systems do not require noatime:

/dev/vg0-ora01/ora01            /oravl01      ext4   defaults 1 2
/dev/vg0-ora01/oravl97          /oravl97      ext4   defaults 1 2
/dev/vg0-ora01/ora98            /ora98        ext4   defaults 1 2
/dev/vg0-ora01/ora99            /ora99        ext4   defaults 1 2
/dev/vg0-ora01/ora-redoSTB_gA   /redoSTB_gA   ext4   defaults 1 2
/dev/vg0-ora01/ora-redoSTB_gB   /redoSTB_gB   ext4   defaults 1 2
/dev/vg0-ora01/ora-redoSTB_gC   /redoSTB_gC   ext4   defaults 1 2
/dev/vg0-ora01/ora-redoSTB_gD   /redoSTB_gD   ext4   defaults 1 2
/dev/vg0-ora01/ora-redoSTB_gE   /redoSTB_gE   ext4   defaults 1 2

Summary

In this short post we explained how to properly plan out storage for Unix and Unix-like operating systems for administrative control and stability. Be sure to reference OS, APP and DB vendor documentation for file system sizing and tuning guidelines.