Finally we are able to get to our main setup script file app-setup.sh. This bash script could be used to configure AWS Elastic Beanstalk environment running an application using Tomcat, Ruby, Python or PHP.
After having done all the hard work so far we are now going to be breezing through various configuration and setup tasks and I hope that you would enjoy the pacey cruise.
Just to recap, whenever an application is deployed, AWS Elastic Beanstalk executes all the files present in “.ebextensions” directory whose extension is “.config”. We have just one file with “.config” extension and that file is “elastic-beanstalk.config” and is present in the “main” directory in our directory structure which is shown below.
|
|
The only job we try to do using elastic-beanstalk.config is to execute app-setup.sh, our main setup script file which is also present in the “main” directory.
First thing the script app-setup.sh does is in case if the files are edited under Windows environment, it fixes the line endings of all the files present anywhere under “.ebextensions” directory using dos2unix command.
Next the bootstrapping happens by including the contents of the file “.elastic-beanstalk-app” using source command. That’s why the first three lines of the file “.elastic-beanstalk-app” are extremely important as they provide the values for the variables ELASTICBEANSTALK_APP_NAME, ELASTICBEANSTALK_APP_PRIVATE_S3_BUCKET and ELASTICBEANSTALK_APP_DEPLOY_DIR.
Then the script sets permissions for few files and directories and creates a tmp directory. We do not use the global tmp directory and instead have our own.
A directory by name app-setup-log is created under tmp directory. The next line in the script is very interesting where the script redirects its current stdout and stderr to an output log file using exec command. Thus all the output of the current script and the output of all the called scripts get redirected to this output file. No more running around trying to locate the log file to check for the errors and output of our commands. We have our own log file for our setup scripts and we do not have to bother about any of the log files used by the AWS Elastic Beanstalk.
If it is a new EC2 instance and our script is running for the very first time then we set the required sudoers setting so that we could freely use the sudo command. Whenever we ssh into an EC2 instance we get logged on as ec2-user and the very first thing we always do is change to a super user. Hence we decided to alter .bashrc file of ec2-user so that we get changed to root user automatically.
Next we install all the required packages (package-install.sh), backup the original configuration files (config-files-backup.sh) and copy and overwrite any existing files (copy-to-slash.sh) with our versions.
Then aws-credentials-setup.sh gets called to set up AWS credentials in all the required files and also to download additional security credentials from the private bucket of Amazon S3.
Next the script deletes the file env-result.sh if it exists and re-creates it by executing source ~/.bash_profile. In our system, the file “.bash_profile” executes source ~/.bashrc which due to our modifications executes source env.sh. This computes all the environment variables, exports them into current bash environment and also causes env-result.sh file to be re-created with newly computed values for environment variables. Now the script attains its full power with the availability of all the useful and required environment variables.
Then the file include.sh is included so that the script can make use of all the implemented utility functions.
mount-all-instance-storage.sh gets executed if it is a new EC2 instance which mounts all the available local instance / ephemeral storage, resizes the root file system and also creates a swap file and reboots the system if the swap file is newly created.
The reboot causes AWS Elastic Beanstalk to deploy its current source bundle again which causes this script to be executed again. All the scripts are written in such a manner that executing them multiple times is harmless and they finish quickly if their tasks have been already executed.
The script continues with setting up of configuration files (config-files-setup.sh), logging facility (logs-setup.sh) and setting up of routine jobs (crontab-setup.sh) to be run at their scheduled interval.
Then the files needed for tomcat secondary instance are setup (tomcat-setup-secondary.sh). I modified that script file so that it does nothing if it is not a tomcat application and instead it is a Python, PHP or Ruby application.
Next app-custom-setup.sh is called to carry out the tasks specific to a particular application.
Finally if the EC2 instance is the leader of the auto scaling group, Route53 DNS records get updated (route53-update-dns.sh). If you do not use Route53 then the script should do nothing.
Apache web server is reloaded if it is running and the script exits after printing the ending date time so that the duration of the script execution could be calculated.
Finally the script always exits with a success status code and AWS Elastic Beanstalk is not notified of any error in our script so that it can continue completing what it has to as part of the deployment process. We could look at our output log file and rectify if something went wrong.
That’s it folks about the complete setup and configuration of AWS Elastic Beanstalk environment. Don’t you think the script went about doing its job in an elegant manner? Please leave your opinion in the comments section.
In our directory structure the only remaining directory to be introduced is “deployment”. It does not contain too many files but those files are quite important. From next post let us start breathing life into that folder.
Here is the source code of the main setup script file app-setup.sh.
#!/bin/bash
# Set DEBUG to 1 to debug this script. 2 for debugging scripts called by this script and so on.
# Execute "export DEBUG=1" to debug this script.
# Set value to 2 to debug this script and the scripts called within this script.
# Set value to 3,4,5 and so on to increase the nesting level of the scripts to be debugged.
[[ $DEBUG -gt 0 ]] && set -x; export DEBUG=$(($DEBUG - 1))
# Check if this is the very first time that this script is running
if ([ ! -f /root/.not-a-new-instance.txt ]) then
newEC2Instance=true
fi
# Get the directory of 'this' script
dirCurScript=$(dirname "${BASH_SOURCE[0]}")
# Fix the line endings of all files
find $dirCurScript/../../ -type f | xargs dos2unix -q -k
# Get the app configuration environment variables
source $dirCurScript/../../copy-to-slash/root/.elastic-beanstalk-app
export ELASTICBEANSTALK_APP_DIR="/$ELASTICBEANSTALK_APP_NAME"
appName="$ELASTICBEANSTALK_APP_NAME"
dirApp="$ELASTICBEANSTALK_APP_DIR"
dirAppExt="$ELASTICBEANSTALK_APP_DIR/.ebextensions"
dirAppTmp="$ELASTICBEANSTALK_APP_DIR/tmp"
dirAppData="$dirAppExt/data"
dirAppScript="$dirAppExt/scripts"
# Create tmp directory
mkdir -p $dirApp/tmp
# Set permissions
chmod 777 $dirApp
chmod 777 $dirApp/tmp
chmod -R 700 $dirAppScript
# Set permissions to run package installation script which needs some files in this folder
chmod -R 644 $ELASTICBEANSTALK_APP_DIR/.ebextensions/copy-to-slash/etc
# Redirect stdout and stderr and append it to our file
curDateTime=$(date "+%Y%m%d%H%M%S")
mkdir -p $dirAppTmp/app-setup-log
exec &>> $dirAppTmp/app-setup-log/app-setup-log-$curDateTime.txt
echo $(date)
if ([ $newEC2Instance ]) then
# Allow sudo command to be used as part of beanstalk ebextensions scripts without a terminal
grep -q 'Defaults:root !requiretty' /etc/sudoers.d/$appName || echo -e 'Defaults:root !requiretty\n' > /etc/sudoers.d/$appName
chmod 440 /etc/sudoers.d/$appName
# Add sudo command if not already present to .bashrc of ec2-user so that we are logged on as root when we use ssh
grep -q "sudo -s" /home/ec2-user/.bashrc || echo -e "\nsudo -s\n" >> /home/ec2-user/.bashrc
fi
# Install all required packages and command line utilities
if ([ $newEC2Instance ]) then
$dirAppScript/setup/package-install.sh
fi
# Backup original files which we are going to overwrite
$dirAppScript/setup/config-files-backup.sh
# Copy the directory structure which could overwrite some existing files with our version
$dirAppScript/setup/copy-to-slash.sh
# Setup AWS credentials so that all AWS command line utilties can work
$dirAppScript/setup/aws-credentials-setup.sh
# Remove env-result.sh so that it gets created newly
if ([ -f ~/env-result.sh ]) then
rm -rf ~/env-result.sh
fi
# Ensuring all the required environment settings after all the above setup
if ([ -f ~/.bash_profile ]) then
source ~/.bash_profile
fi
# include all the utility scripts
source $dirAppScript/include/include.sh
# Mount all local hard disks present, create swap file if not already present and reboot if swap file is newly created.
if ([ $newEC2Instance ]) then
$dirAppScript/util/mount-all-instance-storage.sh
fi
# Setup config files
$dirAppScript/setup/config-files-setup.sh
# Setup logging facility
$dirAppScript/setup/logs-setup.sh
# Setup crontab to run the routine tasks
$dirAppScript/setup/crontab-setup.sh
# Setup tomcat secondary instance configuration
$dirAppScript/setup/tomcat-setup-secondary.sh
# Setup apache web directories
if ([ $newEC2Instance ]) then
# Create apache web directory and set permissions
if ([ ! -z "$APACHE_DIR" ]) then
mkdir -p $APACHE_DIR
chmod -R 755 $APACHE_DIR
fi
fi
# Do application specific custom setup
$dirAppScript/setup/app-custom-setup.sh
# Commands to be executed only by the leader of AutoScaling group
if ([ $ELASTICBEANSTALK_CMD_LEADER ]) then
# Update DNS records of current instance
$dirAppScript/setup/route53-update-dns.sh
fi
# If apache is running then reload it.
$(isProcessRunning "httpd") && sudo service httpd reload
# If new instance, now it is not new anymore
if ([ $newEC2Instance ]) then
echo -n "" > /root/.not-a-new-instance.txt
fi
# Print the finish time of this script
echo $(date)
# Always successful exit so that beanstalk does not stop creating the environment
exit 0