Tuesday, May 27, 2014

Quick Setup of OpenStack Icehouse Development Env on Ubuntu14

OpenStack - there are quite a lot of vendors (including large enterprises) gravitating around the OpenStack ecosystem. It is a good thing that there is ample competition, which means more options for customers and hopefully better business models driving the cloud enterprise market. I look forward to research more from a technical standpoint.

In my view, a good starting point to learn more about the different technical features in OpenStack, is to experiment with them on a  small scale local environment. My test bed is a Ubuntu 14.4 64-bit Linux system on a 8x Intel Xeon CPU machine, with 8G memory and two physical network interface cards. The goal is to basically setup an all-in-one configuration, where all the services, including compute services, are installed on the same node. A controller node is where most of the OpenStack services are configured, and will be installed on my Ubuntu system.

Here, I will discuss couple of quick OpenStack development environment setup options:
    a) Using stable Git Icehouse repository
                OR
    b) Using Vagrant Box

Option-A:  Deploy OpenStack IceHouse using Git repo

1. Create work directory for OpenStack project, say $ICE_STACK_DIR
 
    # mkdir /scratch/<user>/icehouse  

2. Clone stable/icehouse git repository

   - used Netbeans ide to clone the Git repository branch to workdir location; alternately, run the following command from $ICE_STACK_DIR
 
    # git clone -stable/icehouse https://github.com/openstack-dev/devstack.git
   
3. Modify Devstack configuration file to override default settings as needed

  - localrc is a user-maintained settings file used to configure DevStack. It is deprecated and has been replaced by local.conf. More details here

  Sample local.conf: 
# Sample ``local.conf`` for user-configurable variables in ``stack.sh``
# NOTE: Copy this file to the root ``devstack`` directory for it to
# work properly.
# ``local.conf`` is a user-maintained setings file that is sourced from ``stackrc``.
# This gives it the ability to override any variables set in ``stackrc``.
# Also, most of the settings in ``stack.sh`` are written to only be set if no
# value has already been set; this lets ``local.conf`` effectively override the
# default values.
# This is a collection of some of the settings we have found to be useful
# in our DevStack development environments. Additional settings are described
# in http://devstack.org/local.conf.html
# These should be considered as samples and are unsupported DevStack code.
# The ``localrc`` section replaces the old ``localrc`` configuration file.
# Note that if ``localrc`` is present it will be used in favor of this section.
[[local|localrc]]
# Minimal Contents
# ----------------
ADMIN_PASSWORD=welcome1
MYSQL_PASSWORD=$ADMIN_PASSWORD
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORD
# ``HOST_IP`` configuration; e.g. eth1 ip address
HOST_IP=192.168.1.6
# Logging
# -------
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2
SCREEN_LOGDIR=/opt/stack/logs
VERBOSE=True
# Nova logs will be colorized if ``SYSLOG`` is not set; turn this off by setting
# ``LOG_COLOR`` false.
#LOG_COLOR=False
# Enable Services
# ---------------
ENABLED_SERVICES=rabbit,mysql,key
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch,n-novnc,n-cauth
ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-lbaas
ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account
ENABLED_SERVICES+=,g-api,g-reg
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bak
ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng
ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond
ENABLED_SERVICES+=,horizon
# Using stable-icehouse branches
# ---------------------------------
CINDER_BRANCH=stable/icehouse
GLANCE_BRANCH=stable/icehouse
HORIZON_BRANCH=stable/icehouse
KEYSTONE_BRANCH=stable/icehouse
NOVA_BRANCH=stable/icehouse
NEUTRON_BRANCH=stable/icehouse
SWIFT_BRANCH=stable/icehouse
HEAT_BRANCH=stable/icehouse
TROVE_BRANCH=stable/icehouse
# Images - to create test instances
# ---------------------------------
# CirrOS - a minimal Linux distribution
IMAGE_URLS+=",http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img"
# Swift
# -----
# Swift is now used as the back-end for the S3-like object store. If Nova's
# objectstore (``n-obj`` in ``ENABLED_SERVICES``) is enabled, it will NOT
# run if Swift is enabled. Setting the hash value is required and you will
# be prompted for it if Swift is enabled so just set it to something already:
SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
# For development purposes the default of 3 replicas is usually not required.
# Set this to 1 to save some resources:
SWIFT_REPLICAS=1
# The data for Swift is stored by default in (``$DEST/data/swift``),
# or (``$DATA_DIR/swift``) if ``DATA_DIR`` has been set, and can be
# moved by setting ``SWIFT_DATA_DIR``. The directory will be created
# if it does not exist.
SWIFT_DATA_DIR=$DEST/data
# Glance
# ------
# required to resolve errors like "Failed to upload glance image (HTTP 500)
# Devstack Bug#1279384
[[post-config|$GLANCE_API_CONF]]
[DEFAULT]
default_store=file



4. Install DevStack as a non-root user

   # cd $ICE_STACK_DIR/devstack
   # ./stack.sh

  - Read more about the stack.sh script in the official documentation

 - The default services configured by DevStack are Identity (Keystone), Object Storage (Swift), Image Storage (Glance), Block Storage (Cinder), Compute (Nova), Network (Neutron), Dashboard (Horizon)

  - During install run, hit errors like below:
        cp: cannot create regular file '/etc/nova/policy.json': Permission denied
     
    the resolution was to basically edit the file work-dir/devstack/lib/nova and change to 'sudo cp' for the failing file-access occurrences; for e.g. the following changes were made

        sudo cp -p $NOVA_DIR/etc/nova/policy.json $NOVA_CONF_DIR

        # Get the sample configuration file in place
        sudo cp -p $NOVA_DIR/etc/nova/api-paste.ini $NOVA_CONF_DIR
        sudo chown $STACK_USER $NOVA_CONF_DIR

   - To give an estimate, deploying DevStack in  my environment took between 5-6 minutes (after prior multiple failed attempts)

5. Perform basic sanity tests

   - Run the test scripts
         # cd $ICE_STACK_DIR/devstack/tests
         # ./functions.sh
         # ./test_config.sh
   
   - Run the exercise scripts
        # cd $ICE_STACK_DIR/devstack/exercises
   
       # ./horizon.sh 
   
        -- expect to see something like the following message printed on the console if everything goes well with the deployment
        .............
        + set +o xtrace
        *********************************************************************
        SUCCESS: End DevStack Exercise: ./horizon.sh
        *********************************************************************

6. Launch OpenStack Horizon Dashboard

   - Go to URL http://my.eth1.ipv4.address
   - Logon as default user 'demo' or 'admin' and password $ADMIN_PASSWORD set in local.conf

   Here is a screenshot of the dashboard System Info panel



7. Try creating instances from the Dashboard

   - Refer OpenStack Admin Guide for more details on managing the resources and services using the Horizon dashboard    

8.  Stopping and Restarting DevStack

   - To stop all processes that were started by stack.sh
        # cd $ICE_STACK_DIR/devstack
        # ./unstack.sh
   
   - To restart DevStack
        # cd $ICE_STACK_DIR/devstack
        # ./rejoin-stack.sh

Option-B. Deploy DevStack Icehouse using Vagrant

I found this blog article 'OpenStack Cloud Computing Cookbook' by Kevin very helpful in setting up my local development environment virtual machines.

Installed the following:

1. VirtualBox 4.3.10
2. Vagrant 1.4.3
3. Vagrant Cachier plugin    
     
and then followed the instructions as-is, they just work as documented.

Hoping to share my experiments with OpenStack as I learn more ....