Continuous Delivery For Magento With Fleet

This tutorial is primarily aimed at developers who want to adopt a continuous delivery system for Magento. For a non-technical overview of Fleet please watch this video here.

Overview

This is a practical top to bottom tutorial. We supply you with the tooling and a demo Magento store so all you need to do is literally follow the steps provided on this page.

A quick overview of this practical tutorial is a simple five (5) step process consisting of:

  1. Provide Anchor with your public SSH key so we can grant you access to our system Fleet.

  2. Enable Fleet on your local machine using the alias provided. (you don't have to download any software)

  3. Clone our demo Magento store locally onto your machine.

  4. Make a simple change/commit to the store and push to the changes via the special fleet-deploy branch.

  5. Load and activate your newly committed release into the production environment.

    1. Alternatively you can look at creating a staging environment and load a release so you test prior to pushing changes out into production.

    2. Also review the process of rolling back a release in the event that you've found a bug.

The whole process looks like so:

fleet-overview.png

Prereqs

A list of prerequisites (./) before continuing are:

  • You're fairly comfortable using the command line interface (CLI) (./)

  • Have a basic concept of what a revision control system is such as GIT (./)

  • Have a basic understanding of what Github or Bitbucket is used for. (./)

  • You're a web developer who wants to use a continuously delivery platform for Magento. (./)

  • Have a very basic understanding of what SSH is and how it is used. (./)

Learning

What you will get out this article once you've finished?

  • A practical tutorial on how to use continuous delivery system (Fleet) for Magento to manage your store. {OK}

  • A procedure for creating/destroying production ready environments to load new releases for your store for testing. {OK}

  • A proper roll out/back procedure for deploying code for your store. {OK}

Before You Start

{*} Before we get started in order for you to use Fleet will need you to send us your public SSH key so we can grant you access to our system. If you're having troubles trying to find your public SSH Key please read on or else click here to skip this section.

Public SSH Key

We need your public SSH key so you are able use Fleet locally from your machine. You can find this by running the following command:

$  cat ~/.ssh/id_rsa.pub

Simply copy the output which would look like something like this and give this to Anchor - this is your public key so there are no security risks here.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQffddfdfddfdfddfrS1OSRJEvnhaIccUXDImvK/+PweGw44nXGCF/FbbbWF02bjNWNPjI2aspb8nY/KcnDeyEVUAK+ahQkC6K7jKd5df3242432432rEQwbwOUQnUkacHWmut8e6NELlylvO9qij9DHyp2J/6HMZBoj2jLNrkdGWqsRY+hMilRxf1GH6qLWuy2fsfdfssdfsfdfdsA3DgZSKuQqGly6TJpTIjLIB1Oer/pMqGKv7HkzZV97JNYTA/4t3r4AaEN2dxvEB4cRz6areSy0uPEnq0/BouT3zPMlvVZThI+oRkthojoNtvvxAg65HoveE5JdfddfdgXrIA+ZgRP9M3ekiP username@computer.com.au

Or else if you haven't created a public key before you will see the following output saying the file or directory doesn't exist.

cat: /Users/username/.ssh/id_rsa.pub: No such file or directory

You can create an SSH key simply by running the following commands.

$ ssh-keygen -t rsa -b 2048 # just hit return on 'Enter file in which to save the key' but make sure you add a passphrase which is just a password
$ ssh-add # load your SSH key and enter the passphrase you added

Congrats you have created a SSH key! Simply run the first cat command again and capture the output.

Please email your SSH key to Anchor sales@anchor.com.au along with your full name, company name and contact number so we can grant you access to our system. Once we give you the OK then you can use Fleet, it only takes us a minute or two to process this request.

IRC Chat

Please note at any point during this tutorial feel free to reach out to the Anchor team directly via IRC for support. Simply {create a nick name} and enter the anchor channel then simply type in your message at the bottom of the page.

nickname: {create a nick name}
channel: anchor

message: Can someone please help me with on boarding onto Fleet?

Enable Fleet

OK, so you've given us your public SSH key and we've configured this onto our system. You're ready to use Fleet!!

Simply logon to a terminal and add the following alias to use Fleet - you don't have to download a program simply add this alias to remotely interact with our systems.

$ alias fleet='ssh -t deploy@aux.ancora.f.nchr.io --'

To test that Fleet is working invoke the program by simply typing the following command with the help argument (-h).

$ fleet -h

You should then see the following output which are the available commands for Fleet.

usage: fleet <command> [<args>]

Manage Fleet components

Available commands are:
  cert        manage SSL certificates
  database    manage databases
  env         manage environments
  key         manage SSH keys
  release     manage code releases
  snapshot    manage database snapshots
  whitelist   manage IP whitelists
  config      manage Fleet configuration
  log         view logs

positional arguments:
  command     the subcommand to run

optional arguments:
  -h, --help  show this help message and exit
  --version   show program's version number and exit
Connection to aux.ancora.f.nchr.io closed.

Congrats! If you see the output above then you've successfully configured Fleet. What this means is you're able to run commands locally from your machine to interact with the remote Fleet server.


/!\ You can append the -h (help) argument to any of the available commands in fleet to list available options. For instance to view what options there are for releases simply type:

$ fleet releases -h
usage: fleet release [-h] {list,destroy} ...

Manage releases

optional arguments:
  -h, --help      show this help message and exit

subcommands:
  {list,destroy}
Connection to aux.ancora.f.nchr.io closed.

As you can see you can list or destroy releases.

$ fleet release list
$ fleet release destroy

Local Repository

Before we start using Fleet we will first need to clone the demo Anchor Magento store, make a change/commit to the store and then push the changes to the remote repository via a branch called fleet-deploy in order to activate a new release in Fleet.

Clone

To clone the remote repository please run the following command. This will download the entire Anchor Magento shop locally onto your machine.

$ git clone git@bitbucket.org:ancora/demo-magento-store.git # Are you sure you want to continue connecting (yes/no)? yes

Now simply change into the working directory like so:

$ cd ./demo-magento-store/

As you can see you now have the whole demo Anchor Magento store locally on your machine by running the following command:

$ ls -al 

total 1360
drwxr-xr-x  32 username  group    1088 23 Mar 17:23 .
drwxr-xr-x   3 username  group     102 23 Mar 17:22 ..
drwxr-xr-x  13 username  group     442 23 Mar 17:23 .git
-rw-r--r--   1 username  group    1489 23 Mar 17:22 .gitignore
-rw-r--r--   1 username  group    5667 23 Mar 17:22 .htaccess
-rw-r--r--   1 username  group    4568 23 Mar 17:22 .htaccess.sample
-rw-r--r--   1 username  group   10679 23 Mar 17:22 LICENSE.html
-rw-r--r--   1 username  group   10410 23 Mar 17:22 LICENSE.txt
-rw-r--r--   1 username  group   10421 23 Mar 17:22 LICENSE_AFL.txt
-rw-r--r--   1 username  group     542 23 Mar 17:22 README.md
-rw-r--r--   1 username  group  585086 23 Mar 17:22 RELEASE_NOTES.txt
-rw-r--r--   1 username  group    2834 23 Mar 17:22 api.php
drwxr-xr-x   8 username  group     272 23 Mar 17:23 app
-rw-r--r--   1 username  group    2831 23 Mar 17:23 cron.php
-rw-r--r--   1 username  group     717 23 Mar 17:23 cron.sh
drwxr-xr-x  13 username  group     442 23 Mar 17:23 downloader
drwxr-xr-x  10 username  group     340 23 Mar 17:23 errors
-rw-r--r--   1 username  group    1150 23 Mar 17:23 favicon.ico
-rw-r--r--   1 username  group    5979 23 Mar 17:23 get.php
drwxr-xr-x   4 username  group     136 23 Mar 17:23 includes
-rw-r--r--   1 username  group    2642 23 Mar 17:23 index.php
-rw-r--r--   1 username  group    2366 23 Mar 17:23 index.php.sample
-rw-r--r--   1 username  group    6441 23 Mar 17:23 install.php
drwxr-xr-x  15 username  group     510 23 Mar 17:23 js
drwxr-xr-x  15 username  group     510 23 Mar 17:23 lib
-rw-r--r--   1 username  group    1319 23 Mar 17:23 mage
drwxr-xr-x   7 username  group     238 23 Mar 17:23 media
-rw-r--r--   1 username  group     886 23 Mar 17:23 php.ini.sample
drwxr-xr-x   5 username  group     170 23 Mar 17:23 pkginfo
drwxr-xr-x   6 username  group     204 23 Mar 17:23 shell
drwxr-xr-x   5 username  group     170 23 Mar 17:23 skin
drwxr-xr-x   4 username  group     136 23 Mar 17:23 var

Excellent you now have the Anchor Magento store locally on your server. Time to make some changes and use Fleet!

Commit

Lets make a change to the store and commit the change to the remote repository. For the sake of simplicity will edit the main index.php file using the command line text editor vim. Simply run the following command.

$ vim index.php 

Click on the letter i on your keyboard to enter insert mode so you can edit the file and update the code on line 3 to say release 1.0.1 then save it and quit the program by exiting insert mode by clicking on the escape button and then typing :wq on your keyboard.

1 <?php
2
3 echo "This a new release 1.0.1";
4
5 /**
6 * Magento
7 *
8 * NOTICE OF LICENSE

Check the change has been put onto the staging area on git by running the following command:

$ git status

On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   index.php

no changes added to commit (use "git add" and/or "git commit -a")

Now commit the change to GIT using the following command:

$ git commit -m 'Pushing out release 1.0.1' .

[master aa6b560] Pushing out release 1.0.1
 1 file changed, 1 insertion(+), 1 deletion(-)

You are now ready to push this new release out onto Fleet.

Push

In order to load new releases into Fleet you simply checkout into the special fleet-deploy branch and push out your changes to the remote repository.

$ git checkout fleet-deploy
$ git merge master 
$ git push

pushing-commits.png

You have now successfully created a new release onto Fleet by pushing your changes to the remote repository via the 'fleet-deploy' branch.

Using Fleet

Fleet is a command line tool that allows us to provide you with a continuous delivery system for Magento. When you run a commit via the fleet-deploy branch we bake code changes into machine images (releases) after which you can load releases into environments.

Releases

Releases are different versions of your Magento store that are available to be loaded into environments such as production or staging.

Create

Releases are automatically created every time you push out changes from your local working repository via the fleet-deploy branch. You have already done this in the previous step, if you went through the same process again you will create another release in Fleet.

View

To view the release you already committed simply run the following command:

$ fleet release list

name          status     modified                   message
-------       ---------  -------------------------  -------------------------
{COMMIT_SHA}  CREATING   2015-03-26 08:16:32+00:00  Pushing out release 1.0.1
037205f       AVAILABLE  2015-03-26 07:07:29+00:00  Pushing out release 1.0.0
Connection to aux.ancora.f.nchr.io closed.

{COMMIT_SHA} will be substituted with your own commit SHA under the 'name' column.

It takes ~10 minutes to create a new release after which the status will switch to AVAILABLE. The reason for the wait is simply because we are baking your release code into an Amazon Machine Image. By doing this it will allow you to easily load releases of your store into environments e.g(production, staging, dev) that you've created.

To check the status on your release simply run the 'fleet release list' command again, when the status changes to 'AVAILABLE' you can continue with on with the tutorial.

Environments

Releases are loaded into environments and then activated. Your environment contains all the AWS infrastructure you need to run a Magento store. With the exception of the default prod (production) environment we allow you to create/destroy them. For example this allows you to easily spin up a staging environment for testing a new feature for your store then you can destroy it when you're done and roll out the changes into production.

We need to load your new release into an environment and activate it.

View

To view a list of environments list run the following command. As you see we have an existing prod (production) environment running.

$ fleet env list

name    status    release    certificate    created                    updated
------  --------  ---------  -------------  -------------------------  -------------------------
prod    RUNNING   037205f    prod           2015-03-24 23:06:21+00:00  2015-03-26 07:53:15+00:00
Connection to aux.ancora.f.nchr.io closed.

Describe

To see further details about an environment you can describe it which will show you a list of releases running or active for that particular environment as well as the URL endpoints attached.

$ fleet env describe prod

----------------  -------------------------
name              prod
status            RUNNING
whitelist         allow-all
ssl certificate   prod
created           2015-03-24 23:06:21+00:00
updated           2015-03-26 07:53:15+00:00
health check      ON
tracked branches  fleet-deploy
----------------  -------------------------

Releases:
name     status      loaded                     updated
-------  ----------  -------------------------  -------------------------
037205f  * ACTIVE *  2015-03-26 07:28:21+00:00  2015-03-26 07:48:59+00:00

Endpoints:
--------  ------------------------------
admin     admin.prod.ancora.f.nchr.io
adminssh  adminssh.prod.ancora.f.nchr.io
www       www.prod.ancora.f.nchr.io
--------  ------------------------------
Connection to aux.ancora.f.nchr.io closed.

If you type in the www endpoint http://www.prod.ancora.f.nchr.io you can see the production environment for the demo Magento store we are about load your new release into.

Load Release

To load your new release into the production environment simply run the following command replacing your own {COMMIT_SHA} - just run the command 'fleet release list' to view your release again.

$ fleet env load prod {COMMIT_SHA}

Release {COMMIT_SHA} is now being loaded into environment prod
Connection to aux.ancora.f.nchr.io closed.

loading-release-prod.png

You've now successfully loaded your new release into the prod (production) environment. To check that release is being created simply run the describe command again and you'll see it on the release list. Once the release status changes from CREATING to RUNNING we can then activate the release.

Loading your release into production will take ~10-20 minutes as we are not simply updating a record here but baking your code into a machine image so it can be used to build front-end clusters on demand. This ensures there is no downtime when you need to 'activate' the new release into production.

Activate Release

Before you activate your new release you need to ensure that it's RUNNING and not still CREATING by running the describe command again. Only then can you activate the release by simply running the command.

$ fleet env activate prod {COMMIT_SHA}

Release {COMMIT_SHA} is now being activated for environment prod
Connection to aux.ancora.f.nchr.io closed.

You will need to wait ~2 minutes for the load balancer to start directing traffic away from the old to the new release you've just activated.

Congrats! If you have successfully activated the new release into the pro (production) demo Magento store you should now see version 1.0.1 listed on top of the home page on the URL http://www.prod.ancora.f.nchr.io/


In the event that you've found a bug in your new release don't worry. Given the previous release is still running within the production environment you can simply activate that release to roll back to the previous version of your store.

/!\ But wait isn't loading a new release straight into production before any user acceptance testing has been done bad practice? Wouldn't it be great if you could temporarily create a staging environment to test the new feature of your store then destroy it when you're done? Well no problem with Fleet you can do this please read on...

Create

Lets create a new environment but before we do will create a snapshot of your production database that will use to load into your new staging environment like so.

$ fleet snapshot create prod

Snapshot prod-1427413496 is now being created
Connection to aux.ancora.f.nchr.io closed.

Now you'll see the snapshot CREATING after which the status will change to AVAILABLE then it's ready to be loaded into your new staging environment - it will generally take ~5 minutes.

$ fleet snapshot list

name           environment    status     size    type       created
-------------  -------------  ---------  ------  ---------  -------------------------
bqaekpdas333q  prod           AVAILABLE  100GB   MANUAL     2015-03-25 06:45:04+00:00
1427238000     prod           AVAILABLE  100GB   AUTOMATED  2015-03-24 23:22:25+00:00
bqaekpm1a2pq   prod           AVAILABLE  100GB   MANUAL     2015-03-24 06:45:04+00:00
1w9f6180mk695  prod           AVAILABLE  100GB   MANUAL     2015-03-23 03:58:13+00:00

Now create a new staging environment and load the production database {SNAPSHOT-ID} into as well as the configurations for production.

$ fleet env create staging --snapshot {SNAPSHOT-ID} --source-environment prod  

Environment staging is now being created
Connection to aux.ancora.f.nchr.io closed.

$ fleet env list
name     status    release      certificate    created                    updated
-------  --------  ---------    -------------  -------------------------  -------------------------
staging  CREATING               prod           2015-03-26 00:37:49+00:00  2015-03-26 00:38:20+00:00
prod     RUNNING   {COMMIT_SHA} prod           2015-03-24 23:06:21+00:00  2015-03-25 01:12:38+00:00

new-environment.png

Environments will take ~10 minutes to create as we are using the same blue print that was used for production. This staging environment is 'exactly the same as production' consisting of all the same servers so we can guarantee any test here will behave exactly the same as in production.


Now as we've done before in the previous steps simply view available releases, load your latest release {COMMIT_SHA} into the staging environment, activate it and then run a describe on the environment to fetch the URL end point.

$ fleet release list
$ fleet env load staging {COMMIT_SHA} # waiting ~10 minutes to load until status changes from CREATING to RUNNING then activate release
$ fleet env activate staging {COMMIT_SHA}
$ fleet env describe staging

Now you have successfully created a staging environment to preview new releases of your store via the following URL http://www.staging.ancora.f.nchr.io

Destroy

Now that you've tested changes in your staging environment and loaded the release into production you can now destroy the staging environment which would typically avoid you having to pay any additional hosting infrastructure costs. You will be prompted to add the environment name you want to destroy which in this case is staging.

$ fleet env destroy staging 

This will PERMANENTLY DESTROY environment staging!
Please review the details below:
----------------  -------------------------
name              staging
status            RUNNING
whitelist         allow-all
ssl certificate   prod
created           2015-03-26 00:37:49+00:00
updated           2015-03-26 02:28:32+00:00
health check      ON
tracked branches  fleet-deploy
----------------  -------------------------

Releases:
name     status      loaded                     updated
-------  ----------  -------------------------  -------------------------
aa6b560  * ACTIVE *  2015-03-26 01:52:16+00:00  2015-03-26 02:13:20+00:00

Endpoints:
--------  ---------------------------------
admin     admin.staging.ancora.f.nchr.io
adminssh  adminssh.staging.ancora.f.nchr.io
www       www.staging.ancora.f.nchr.io
--------  ---------------------------------
Enter the environment's name to proceed or anything else to abort

> staging
Environment staging is now being destroyed
Connection to aux.ancora.f.nchr.io closed.

Congrats you have finished the Fleet on boarding process. There are many more features available in Fleet that you can explore so please do so. Simply run this command to view your options.

Authors

A list of people contributed to this article include Ben Leykam, Phillip Pace and Jez Kabanoff.