NOTICE 2019-11-24: We are currently investigating reports that the Unattended method is failing to work for some customers. We advise that you, instead, use the normal method and employ tmux to give you persistence of the SSH session, allowing you to disconnect after starting the Cluster Install Wizard and reconnect later to check on its completion. Please remember that EITHER METHOD can take between 30 and 90 minutes PER NODE depending on the instance types, region (some regions are more resource starved than others), and EBS provisioning you select.
NOTE: This guide is intended to be an addendum to our HA Cluster Deployment Guide. This allows for an Unattended method of simultaneously launching and deploying the EC2 Instance nodes of a Cluster instead of using the interactive Cluster Install Wizard via SSH. This can be handy if you need to replace a failed or detached cluster node, eliminating the need to maintain and monitor an SSH console while a new node is created and attached to the cluster.
In order to proceed with an Unattended node setup, you must have ALREADY completed the following steps from the normal HA Cluster Deployment Guide:
AWS Access Key ID and Secret Key created
Elastic IP allocated
Cluster Services Security Group created
RDS Instance created
EFS Instance created
If you are converting an EXISTING Production server into a cluster, you will still have to run the interactive Cluster Install Wizard on that Primary node. This Unattended process can only be used on instances as they are being launched fresh. You can use this to launch the Backup node once the Production server has been converted to the Primary node via the interactive Cluster Install Wizard.
If you are launching a brand new cluster from scratch, you can launch both the Primary and Backup nodes using this Unattended method. Then you may proceed to setup your FreePBX installation normally.
IMPORTANT: An Unattended node setup will take just as long as setting up a node using the interactive Install Wizard. Keep this in mind if you are launching a clean cluster with both nodes via Unattended setup. You MUST allow the Primary node to fully setup (30+ minutes) BEFORE you start the Backup node. You will be able to connect to the instance via SSH and run the following command to monitor the overall progress:
tail -f -n25 /var/log/cluster-install-unattended.log(CTRL+C will exit the tail utility) There are two scenarios here:
If you already have a Production instance (and it is running v3.0 AND in an EFS-enabled region), then you only need to launch one new instance per the instructions below. You will set all Instance launch parameters to be identical to what you have set for your current Production instance with ONE EXCEPTION:
You will want to select a DIFFERENT Availability Zone for the new Backup node than the current instance. You can identify which AZ your current instance is in via the EC2 Console. When you launch a new instance, the AZ is selected by choosing a specific Subnet during Step 3 of the Launch Instance Wizard. We will point this out again below, when appropriate.
If you are creating a brand new Production environment from the start, you will follow these steps TWICE while choosing a different Subnet for each one.
DO NOT CHANGE Number of Instances to more than 1 in Step 3 as a shortcut; this will create both in the same AZ, which will defeat one of the primary purposes of the HA Cluster...recovery from a complete failure of an AWS datacenter location. Different Availability Zones = Different Physical Datacenters in the Same Local Region = Better Protection from Downtime!
STEP 1:
Click one of the links below based on which AWSRegion you are building your Cluster in. This will open a new window/tab directly into the Launch Instance Wizard:
STEP 2:
Choose an instance size EITHER identical to your current Production server or appropriate for your new environment. You may review the EC2 Console Instances section for the size of your current Production server instance. If you are unsure of what size to choose for your new environment, refer to this page for suggestions: EC2 Deployment Guide
STEP 3:
You will need to change two options on this page from their defaults and there is a third option to consider if you chose a t2/t3 type in Step 2:
Subnet:
Again, you want to change this to a DIFFERENT subnet for each of the 2 instances you launch (or from the Primary server already in Production). Some AWS Regions may have more or less subnets than listed in the image below.
Enable Termination Protection:
CHECKED! - This will ensure that you can't accidentally terminate your instances from the EC2 Console.
T2/T3 Unlimited (OPTIONAL):
This option is for t2/t3 types only and allows an instance to burst well beyond its size limit for as long as it needs to (for an extra charge based on resource usage). You MAY want to enable this option if you are using a lower size and want to avoid random performance issues on occasional busy days.
NOTE: WE DO NOT ADVISE USING T2/T3 INSTANCE TYPES FOR CLUSTERS!!!
You will want to expand the Advanced Details section at the bottom of the page to expose the User Data field. This is where we will place the kickstart data for the Unattended node setup. First, you'll need to copy the following code into a notepad so you can edit it to include your Cluster parameters:
#!/bin/bash
#HA Cluster Install Kickstart (using cloud-init from launch)
#2019-02-11--0200
# Once you have added your information below, you must copy and paste this ENTIRE FILE (including the lines above) into the UserData field on Step 3 of a new instance launch.
########## BEGIN USERDATA VARIABLES - DO NOT EDIT ABOVE THIS LINE ##########
# First, you need to specify the Role. This is CaSe SeNsItIvE and can only be one of these 3 options:
# "Primary"
# "Backup"
# "Monitor"
Role=""
# If you are kickstarting a Backup or Monitor node, you only need to add data to these variables:
ClusterRDSendpoint=""
ClusterRDSusername=""
ClusterRDSpassword=""
# If you are kickstarting a Primary node, you must also add data to these variables:
ClusterEFSendpoint=""
ElasticIP=""
ElasticAllocationID=""
AWSaccessKeyID=""
AWSsecretAccessKey=""
AWSregion=""
# NOTE: For ClusterName, A-Z,a-z,0-9,-,_ ONLY, NO SPACES!
ClusterName="UnattendedTestCluster"
# Mail parameters are optional, but STONGLY recommended so you receive email alerts on cluster status changes:
MailHost=""
MailUser=""
MailPass=""
MailFrom=""
MailTo=""
########## END USERDATA VARIABLES - DO NOT EDIT BELOW THIS LINE ##########
# You must also include the lines below when you copy/paste into the UserData field on Step 3 of the instance launch.
wget -qP /var/tmp/awsfpbx/ https://files.thewebmachine.net/awsfpbx/cluster-install-unattended.sh
chmod +x /var/tmp/awsfpbx/cluster-install-unattended.sh
. /var/tmp/awsfpbx/cluster-install-unattended.sh
# END OF SCRIPT!All of your parameters must be properly encased in the provided double-quotes. You must ALWAYS specify Role=" ". If you are creating the Primary node for a clean cluster, ALL of the fields are necessary (Mail parameters are optional, but strongly recommended!). If you are merely adding a Backup node to the cluster, you only need to specify the ClusterRDSendpoint=" ", ClusterRDSusername=" ", and ClusterRDSpassword=" " parameters. Once you have added your parameters, copy and paste ALL of the code (from #!/bin/bash to # END OF SCRIPT!) and paste it into the User Data field of Step 3.
STEP 4:
You may specify a custom size for your Volumes if you desire. The Volume sizes of BOTH server Instances must be IDENTICAL! If you are launching a small to medium cluster or plan to use features like our S3 Sync for Call Recordings, IMAP Storage for VoiceMail, and/or Auto File Deletion for Call Recordings, you can most likely leave all settings here at their defaults.
STEP 5:
You will want to add a new tag with a Key of "Name" and a Value with something to identify your cluster instances (ex "Main Office Cluster"). On each of the two instances, one should specify "Primary" and one should specify "Backup" in the Value. This will ensure you can identify each of the nodes individually on the EC2 Console.
STEP 6:
You will want to accept the default Security Group provided by us on this page UNLESS you have customized your own copy of this SG for your existing Production instance. In this case, change Assign a Security Group to "Select an Existing Security Group" and choose your existing Production SG. If this is a new environment, use our provided SG for the first instance you launch, then "Select an Existing Security Group" and choose the newly created SG when launching the second instance. Both instances need to be using the same Security Group for ease of management.
STEP 7:
Review and then Launch the new Instance. Repeat, if necessary, to create a second Instance.
STEP 8:
As you launch each instance, you need to go assign the Cluster Services Security Group to each of the servers as an additional SG. From the EC2 Console Instances section, select each Cluster Instance, choose Actions > Networking > Change Security Groups. In the dialog, leave the "AWS FreePBX" SG checked and also check the "Cluster Services" SG. Then click Assign Security Groups.
IMPORTANT:
An Unattended node setup will take just as long as setting up a node using the interactive Install Wizard. Keep this in mind if you are launching a clean cluster with both nodes via Unattended setup. You MUST allow the Primary node to fully setup (30-90 minutes) BEFORE you start the Backup node. You will be able to connect to the instance via SSH and run the following command to monitor the overall progress:
tail -f -n25 /var/log/cluster-install-unattended.log(CTRL+C will exit the tail utility)
You will see one of the following messages in the log when a node is finished deploying:
Primary:
This concludes the unattended setup. The Primary node is Running. You may now proceed to deploy the Backup instance.Backup:
This concludes the unattended setup. The Backup node is in Standby. The Cluster should now be running normally.
