4. Installation Guide

4.1. Requirements

Smart Automation Manager for Hitachi has the following requirements:

  • Linux Server with 2 cores and 16GB Memory

  • 50 GB of disk space

  • Ruby (v2.4 or higher)

  • PostgreSQL (v9.6 or later)

  • Hitachi CCI

  • Hitachi Raidcom

  • Command device or IP command device for manageable storage arrays. (Non-IP command devices are recommended)

4.2. Installation

4.2.1. User

Create a separate user (e.g. sam4h) and execute all following commands under this user.

4.2.2. PostgreSQL

In order to run SAM4H, you will need a working PostgreSQL-Installation (v9.6 or newer).

It is recommended to use the PostgreSQL packages, which are delivered by your Linux distribution. You can also visit the PostgreSQL Manual. Please make sure, that you install all development packages, too. These are used to build the native PostgreSQL-drivers for Ruby.

4.2.3. Install SAM4H

4.2.3.1. Unpack SAM4H installation package

It is recommended to install SAM4H under /opt. But any other path is also supported.

mkdir -p /opt/sam4h/tmp/pids
cd /opt/sam4h
tar xvfz {path}/sam4h-{version}.tar.gz

4.2.3.2. Install Smart Automation Manager with the Install Script

SAM4H needs Ruby 2.4 or newer. In order to install all necessary packages run the following script.

scripts/install_ruby.sh

It will install all needed system packages and afterwards install rbenv and with it ruby in the currently needed version. Afterwards all needed gems will be installed with bundler.

At the end of the installation, all environment variables will be set accordingly.

Please refer to the configuration chapter to complete the initial installation.

4.2.4. Upgrade SAM4H

4.2.4.1. Manual Upgrade steps

To upgrade SAM4H, you need to execute the following steps:

/etc/init.d/sam4h stop
cd /opt/sam4h
tar xvfz /path/to/new_sam4h.tar.gz
source script/init_rbenv.sh
bundle --local
bundle exec rake db:migrate
bundle exec rake assets:precompile
/etc/init.d/sam4h start

4.2.4.2. Automatic Upgrade through Script

There exists a script which does the upgrade steps automatically:

/opt/sam4h/scripts/upgrade_sam4h.sh

4.2.4.3. Update the config.yml file

In order to guarantee a proper function after the upgrade, you need to update missing parameters in the config/config.yml file. All available parameters with its default values are either found in the section configuration or in the file config/config.yml.sample.

4.2.5. Export and Import Configuration

There exists scripts for exporting and importing the configuration. These could be used either to dump and store a certein configuration ord to create an active / passive cluster, where the secondary cluster imports the configuration on a daily base. You could run the script script/find_missing_config_params.sh to determine all missing parameters.

4.2.5.1. Export Script

The script creates a tar file in the application directory with the name ‘sam4h_config_dump.tar.gz’. It contains the following things:

  • Database dump

  • Configuration files
    • script/sam4h.rc

    • config/config.yml

    • config/port_pairs.yml

    • config/unicorn.rb

    • config/initializers/wicked_pdf.r

Usage:

/opt/sam4h/script/export_config.sh

Result file:

/opt/sam4h/sam4h_config_dump.tar.gz

4.2.5.2. Import Script

The script imports the config and loads the DB-dump, which the export script created. It will firest execute the export-script, to save the last configuration. ATTENTION: The database and the configuration files will be overwritten.

Usage:

/opt/sam4h/script/import_config.sh [/path/to/sam4h_config_dump.tar.gz]

Note: If you are using elastic search, you need to reindex elastic search:

/etc/init.d/sam4h rake create_es_indizes

4.3. Configuration

Smart Automation Manager for Hitachi has two places to make customer- and system-dependent configurations:

  • Config File (/opt/sam4h/config/config.yml)

  • Database

This section describes the configuration in the config.yml file. These are typically config options, which do not have to be changed a lot. Every change inside this file needs a restart of the application.

4.3.1. First time preparation

In order not to overwrite existing files during an upgrade, all configuration-files provided in the installation package have the ending ‘.sample’ To start a new installation from scratch, these sample-files must be copied to their real name.

cd /opt/sam4h/config
cp config.yml.sample config.yml
cp database.yml.sample database.yml
cp unicorn.rb.sample unciron.rb

4.3.2. Config.yml

This file stores all important lowlevel application settings. It has multiple sections: a common-section and a section for each environment. The common section is shared for all environments whereas every environment can have its own configuration parameters.

NOTICE: Every change in this file needs a restart of the application in order to activate the changes.

common:
  ##
  # HORCM-Settings:

  # HORCM-Base-Instance. Each Storage Array gets its own HORCM-Instance, based on this base-Instance: HORCM-NR = horcm_base_instance + storage_array.id
  horcm_base_instance: 2000

  # Raidcom User and Password
  default_raidcom_user: 'maintenance'
  default_raidcom_password: ''

  # Mail sender for Mails sent by SAM4H
  sender_email: 'sam4h@peaq.ch'

  # Default values for creating new DPV-Config Templates
  default_cu_range_begin: 0
  default_cu_range_end: 127
  default_snap_cu_range_begin: 128
  default_snap_cu_range_end: 255

  # Host for URLs in Mails
  mailer_url_default_host: 'my_server'
  # Mail Delivery Method: 'smtp' or 'sendmail'
  mail_delivery_method: 'sendmail'
  smtp_server: ''
  smtp_port: 587
  smtp_user: ''
  smtp_password: ''

  # Gad Reports
  gad_monitoring_email_addresses: []

  # Name for cost_center
  cost_center_name: 'TGAD-ID'
  # CostCenter mandatory when creating Clusters?
  cost_center_optional: true
  # Display Task-IDs in the Timeline
  show_task_ids: true

  # How to split Hostgroups, to determine its cluster name? (Used for GAD clusters, where each node needs an own HostGroup with its preferred path)
  cluster_hg_split_char: '@'
  # Position of the cluster name (can be 0 or 1)
  cluster_hg_split_pos: 0

  # LDAP Configuration
  ldap_authentication: false
  ldap_host: 'svx-ipap01.linux.balgroupit.com'
  ldap_tls: false
  ldap_base_tree: 'DC=users,DC=linux,DC=balgroupit,DC=com'
  authorized_ldap_groups: {}
  enable_remember_user: true
  user_session_timeout: 1.hour

  # Page limit defines, how many elements per page are displayed
  page_limit: 15
  # Set to true, if you do not want to be warned when the queue dispatcher is not running
  disable_qd_warning: true

  # Enable Pool-Classes
  enable_pool_classes: true

  # Support Link on the About-Page
  enable_support_link: true

  # Separate WWNs per port when creating HostGroups
  wwns_per_port: true

  # Set Lun-Nicknames to the name of the cluster, when creating LUNs
  label_luns: true
  # Schema: %h: HostGroup-name, %l: CuUa
  label_schema: '%h_%l'
  # Set Lun-Nicknames to the name of the cluster, when unmapping LUNscreating LUNs
  unmap_label_luns: true
  # Schema: %h: HostGroup-name, %l: CuUa
  unmap_label_schema: '%h_%l'

  # Set simulation mode. No command will be executed, if simulation mode is enabled
  simulation_mode: true

  # Enable / Disable ALUA. Only enable it, for CCI Version > 13.05
  alua_enabled: true

  # Enable Config Sync between 2 databases
  enable_config_sync: true

  # Disable GAD Replication Restore
  skip_label_gad_luns: false

  # Custom tier relocation policies
  custom_tier_relocation_policies: {}

  # Elastic Search for searchable Tasklist History
  enable_elastic_search: true
  elastic_search_url: 'http://localhost:9200'

  # Enable Deduplication & Compression
  capacity_saving: true

  # Columns for Cluster LUN Report
  cluster_lun_report_columns: {lun_nr: true, nicknames: true, volume: true, virtual_volume: false, pool: true, size: true, used_capa: false}

production:

development:

test:
skip_label_gad_luns

Disable labeling of GAD-Luns. If enabled, GAD-Luns gets labeled with prefix ‘_GADP_’, if it is the primary LUN and ‘_GADS_’, if it is the secondary LUN. If GAD-Lun_Labeling is disabled, the GAD-Replication-Restore under Administration cannot be used.

Default: false

horcm_base_instance

Set the base number for the horcm instances. If set to 1000, the first horcm- instance will have the number 1000, the next 1001 and so on.

Default: 1000

default_raidcom_user

The raidcom user which is used to access the storage arrays.

Default: ‘maintenance’

default_raidcom_password

The raidcom password which is used to access the storage arrays.

sender_email

The email address which is used to send mails.

Default: ‘sam4h@peaq.ch

default_cu_range_begin

Default value for CU-Range-Begin in the DPV-Config-Template form.

Default: 0

default_cu_range_end

Default value for CU-Range-End in the DPV-Config-Template form.

Default: 127

default_snap_cu_range_begin

Default value for Snapshot CU-Range-Begin in the DPV-Config-Template form.

Default: 0

default_snap_cu_range_end

Default value for Snapshot CU-Range-End in the DPV-Config-Template form.

Default: 127

mailer_url_default_host

The host-name which is used for sending mails

mail_delivery_method

Can be either ‘sendmail’ or ‘smtp’. If set to ‘smtp’, additional parameters specify the SMTP-settings.

Default: ‘sendmail’

smtp_server

The SMTP-server which is used to send mails.

smtp_port

The port for of the SMTP-server which is used to send mails.

Default: 587

smtp_user

The user for the SMTP-server which is used to send mails.

smtp_password

The password for the SMTP-server whcih is used to send mails.

gad_monitoring_email_addresses

Email addresses where the report of the GAD replication monitoring is sent to. A report is only sent in case of an error.

cost_center_name

The name of the cost center for the Cost Center Chargeback report.

cost_center_optional

Specify, if the cost center is mandatory or not when creating new host groups.

show_task_ids

If this parameter is set to ‘true’, the task IDs are displayed in the timeline.

cluster_lun_report_columns

Let you define, which columns are displayed in the Cluster-LUN-Report.

Default: {lun_nr: true, nicknames: true, volume: true, virtual_volume: false, pool: true, size: true, used_capa: false}

enable_elastic_search

If enabled, all tasks in the tasklist are sent to an Elasticsearch server. This set up allows full-text-searches over all SAM4H tasks.

Default: false

elastic_search_url

Specify the URL for the Elasticsearch server.

Default: ‘http://localhost:9200

disable_cluster_autodetection

If set to true, the autodetection of clusters will be disabled.

4.3.3. Cron Jobs

There are some cron jobs which must be configured. This section describes all available cron-jobs and how they must be configured.

4.3.3.1. Reload Config Job

The reload config Job must be executed once a day and is responsible for reloading all storage arrays. A config reload could also be triggered manually and is always triggered after a job execution. The daily reload ensures that configuration changes are in the SAM4H DB at least once a day.

0 0 * * * /etc/init.d/sam4h rake reload_config > /opt/sam4h/cron_reload_config.log 2>&1

4.3.3.2. Statistic Update Job

This job updates the time series charts.

0 1 * * * /etc/init.d/sam4h rake update_statistics > /opt/sam4h/cron_update_statistics.log 2>&1

4.3.3.3. Resync scheduled Snapshots

Responsible for resyncing scheduled snapshots. Snapshots resyncs could be configured in the ‘administration/clusters’ view.

0 2 * * * /etc/init.d/sam4h rake do_scheduled_snap_resyncs > /opt/sam4h/cron_do_scheduled_snap_resyncs.log 2>&1

4.3.3.4. GAD Monitoring

This job checks for failed GAD Replications and sends an email if there are unpaired GAD Replications.

05 1 * * * /etc/init.d/sam4h rake monitor_gad_replications > /opt/sam4h/cron_monitor_gad_replications.log 2>&1

4.4. Docker

4.4.1. Introduction

The aim of the dockerized version is to simplify the setup procedure of SAM4H. All you have to do is to run the installer, which will deploy two docker images, i.e. postgresql and sam4h.

If you prefer to use the SAM4H with a seperate PostgreSQL you can still do so as described in Customizing the dockerized SAM4H.

4.4.2. Prerequisites

The following requirements must be fulfilled in order to run sam4h:

  • 2 Cores

  • Memory: 16 GB

  • Disk: 20 GB for the docker images (usually /var/lib/docker)

  • Disk: 30 GB for data under /opt/sam4h

  • docker installed (typical package name: docker)

  • docker-compose installed (typeical package name: docker-compose)

  • HDS CCI Installation which matches with the VSP firmware

  • Root-Access for the installation

Note

On CentOs install docker-compose as described here.

Note

If you want to use a different location for the docker images procede as follows:

  • stop the docker service

  • move the folder /var/lib/docker to /my/new/location

  • create a bind mount in your fstab. Symllinks will not work!

/my/new/location/docker    /var/lib/docker   none bind
  • mount the bind mount

  • start the docker service

4.4.3. Setup

All you need for the installation are:

  • sam4hinstaller.tgz

  • Path to your sam4h.lic file

Here is the console output of a typical installation:

tar -xvzf sam4hinstaller.tgz
cd sam4hinstaller
./installer.sh

Welcome to the SAM4H docker image installation

The installation will be under /opt/sam4h.
Enter TCP port samh4 should listen on: 8443
Enter full path to your sam4h.lic file: /path/to/sam4h.lic
How do you like to communicate with the Hitachi Storage Array?
[1] using IP
[2] using the command devices

Please choose: [1/2] 1

Note

You can use <tab> for automatic path completion.

Once you’ve made your choice, the docker layers will be installed, the database created, the schema lodaed and the initial seeding will be done.

If you choose option 2 i.e. using the command devices, the installer will try to map the needed device-files into the docker container.

After the installation has succeeded you’ll have to setup the cron jobs. Here we’ll describe how the setup is done with docker. Please read Cron Jobs section for further details.

Reload Config Job

0 0 * * * /opt/sam4h/bin/sam4h.sh rake-cron reload_config > /opt/sam4h/log/cron_reload_config.log 2>&1

Statistic Update Job

0 1 * * * /opt/sam4h/bin/sam4h.sh rake-cron update_statistics > /opt/sam4h/log/cron_update_statistics.log 2>&1

Resync scheduled Snapshots

0 2 * * * /opt/sam4h/bin/sam4h rake-cron do_scheduled_snap_resyncs > /opt/sam4h/log/cron_do_scheduled_snap_resyncs.log 2>&1

GAD Monitoring

05 1 * * * /opt/sam4h/bin/sam4h rake-cron monitor_gad_replications > /opt/sam4h/log/cron_monitor_gad_replications.log 2>&1

Dump DB

50 1 * * * /opt/sam4h/bin/sam4h.sh dump_db-cron > /opt/sam4h/log/cron_dump_db.log 2>&1

4.4.4. Running SAM4H

A systemd service is installed during the installation. This ensures that it is started when the server is booted. You can start/stop SAM4H with systemctl.

systemctl start sam4h.service

If you want to controll SAM4H on your own use /opt/sam4h/bin/sam4h.sh.

/opt/sam4h/bin/sam4h.sh --help
sam4h.sh

usage: sam4h.sh [OPTION] {start|start-daemon|stop|restart|status|rake|exec}
       -h, --help       print this usage
       -v, --verbose    verbose output

4.4.5. Customizing the dockerized SAM4H

4.4.5.1. PostgreSQL

If you don’t want to use the dockerized PostgreSQL version that comes with SAM4H, you can easily use your own database server. The whole magic happens in /opt/sam4h/config/docker-compose.yml. All you need to do is set the database name, database user, database password and database server ip in the configuration file docker-compose.yml via the environment section:

version: '3'
services:
  sam4h:
    image: sam4h:1.6
    environment:
      DB_NAME: name_of_the_db
      DB_USER: db_user
      DB_PASSWORD: db_password
      DB_HOST: ip_or_name_to_the_db_server
    tty: true
    ports:
      - "3003:3003"
    volumes:
      - /opt/sam4h/export:/opt/sam4h/export:z
      - /path/to/sam4h.lic:/opt/sam4h/sam4h.lic:z
      - /HORCM:/HORCM:z
      - /opt/sam4h/log:/opt/sam4h/log:z
      - /opt/sam4h/config/config.yml:/opt/sam4h/config/config.yml:z
      - /opt/sam4h/config/port_pairs.yml:/opt/sam4h/config/port_pairs.yml:z

4.4.5.2. Command Device

If you plan to switch from the ip to the command device, all you have to do, is to map the device-files into the SAM4H container. This is done with the devices section in /opt/sam4h/config/docker-compose.yml. Paste the output of the following script into the devices section.

for i in $(ls /dev/disk/by-id/); do
    while read -r line; do
        if [[ ${line} = *'HITACHI'* ]]; then
            echo "      - /dev/disk/by-id/${i}:/dev/disk/by-id/${i}"
        fi
    done < <(sg_inq -H /dev/disk/by-id/${i} 2>/dev/null | sed -n '1,4p'|cut -c61-);
done
version: '3'
services:
  db:
    image: postgres
    volumes:
      - /opt/sam4h/postgresql/data:/var/lib/postgresql/data:z
  sam4h:
    image: sam4h:1.6
    devices:
      # *** BEGIN ***
      # PASTE THE OUTPUT FROM THE PREVIOUS SCRIPT BETWEEN THE BEGIN AND END MARKER
      # *** END ***

    tty: true
    ports:
      - "3003:3003"
    depends_on:
      - db
    volumes:
      - /path/to/sam4h.lic:/opt/sam4h/sam4h.lic
      - /HORCM:/HORCM
      - /opt/sam4h/log:/opt/sam4h/log
      - /opt/sam4h/config/config.yml:/opt/sam4h/config/config.yml
      - /opt/sam4h/config/port_pairs.yml:/opt/sam4h/config/port_pairs.yml

4.4.5.3. HTTPS Configuration

In order to use an own valid certificate, you have to make it visible to the SAM4H docker container. To achieve this just put your server.key and server.crt in your /opt/sam4h/config directory and restart sam4h.

cp /path/to/server.key /opt/sam4h/config
cp /path/to/server.crt /opt/sam4h/config
/opt/sam4h/bin/sam4h.sh restart

If you want to change the port SAM4H is listening to you can do so by editing the ports section. Let’s assume you want SAnM4H to listen to port 8443. Then your /opt/sam4h/config/docker-compose.yml would look like:

version: '3'
services:
  db:
    image: postgres
    volumes:
      - /opt/sam4h/postgresql/data:/var/lib/postgresql/data:z
  sam4h:
    image: sam4h:1.6
    tty: true
    ports:
      - "8443:3003"
    depends_on:
      - db
    volumes:
      - /opt/sam4h/export:/opt/sam4h/export:z
      - /path/to/sam4h.lic:/opt/sam4h/sam4h.lic:z
      - /HORCM:/HORCM:z
      - /opt/sam4h/log:/opt/sam4h/log:z
      - /opt/sam4h/config/config.yml:/opt/sam4h/config/config.yml:z
      - /opt/sam4h/config/port_pairs.yml:/opt/sam4h/config/port_pairs.yml:z
      - /opt/sam4h/config/server.key:/opt/sam4h/config/server.key:z
      - /opt/sam4h/config/server.crt:/opt/sam4h/config/server.crt:z

4.4.6. Upgrade SAM4H

The upgrade procedure is straight forward. You do not have to stop the running SAM4H installation. Just start the installer as described under Setup. Once a previous SAM4H version is detected, you will be asked for a backup directory, where a copy of the current configuration and database will be stored.

Attention

All installed docker containers and images will be removed during the installation and replaced with the new ones.

Here is the console output of a typical installation:

tar -xvzf sam4hinstaller1.6.tgz
sudo ./installer.sh
Welcome to the SAM4H docker image installation

The installation will be under /opt/sam4h.
A previous found version of SAM4H was found under /opt/sam4h.
Do you wan't to update SAM4H? [y/n] : y
Stopping config_sam4h_1 ... done
Enter directory where the SAM4H backup should be stored: /path/to/sam4h/backup
Stopping config_db_1 ... done
WARNING! This will remove all stopped containers.
Are you sure you want to continue? [y/N] y

If you have other container running on the machine, please contact support@peaq.ch for support with SAM4H upgrade.