Install cloud-init on Ubuntu and use locally… NoCloud

Posted on Tuesday, April 12, 2016


  
What is cloud-init and why do you want it?

From their own document site @ https://cloudinit.readthedocs.org/en/latest/  [1]

Cloud-init is the defacto multi-distribution package that handles early initialization of a cloud instance.




So what does that mean?


Think of it as a nice scriptable way to define how you want a machine built.    You can define what users you want added, what packages you want installed, what drives to format, setting timezones, etc…

It gives you the ability to script your server build!  This gives you repeatability and a history (assuming you put your script in a git repo). 



How do you typically use it?


I use Ubuntu, and most Ubuntu cloud images already have cloud-init installed.   With that in mind when I kick off a new server in the cloud I also hand it a cloud-init file to run on start up.

As the server starts up it runs the cloud-init file on the first start up and sets the machine up according to my script.   (There are some ways to run the script on every start up, but for me I only run it on the first start up and that is it)



Why install it?


Well usually you would not need to, assuming you are using some cloud service like Amazon Web Services or Openstack, chances are your VMs have cloud-init installed.

But, what if you are not in the cloud?  

I currently have plans to build out an Openstack set up at home, but that keeps getting delayed.   In the meantime I have an ESXi box I use to run my test VMs.  I've gotten so used to using cloud-init in Openstack or Amazon that it drives me a little crazy when I have to build a machine locally in VMWare and manually install packages and set the box up.  I would rather do it the cloud-init way.

So, can I take a base Ubuntu 14.04.4 image install cloud-init and get it to run a cloud-init file locally?  I am sure I can I just have never bothered to try. J




First attempt (FAILED)


I have created a base Ubuntu 14.04.04 machine image in VMware Workstation.   The main user is named ubuntu, which follows the typical Ubuntu Cloud image setup.   I set up all my ssh key stuff so I can log in as the Ubuntu user via ssh keys and no one can login via passwords.




First step Install Cloud-Init


Cloud-init is installed by default on Ubuntu Cloud image https://help.ubuntu.com/community/CloudInit [2], but it is not installed in a typical non-cloud installation.

I may not get this right the first time, but here goes.  I found some good instructions on how to install cloud-init on Arch linux at https://wiki.archlinux.org/index.php/Cloud-init [3]

First let me prove I don't have cloud-init installed.



   > ls /etc/cloud





Does not exist.  Perfect J




Install it.


   > sudo apt-get -y install cloud-init





Confirm it's installed by opening /etc/cloud/cloud.cfg


   > sudo vi /etc/cloud/cloud.cfg


Looking over this file it looks good for my purposes one thing to note is in the system_info section you can see the default_user is the ubuntu user.




OK now what… 

I am used to passing in a file via an AWS or Openstack command that then is used when a new VM starts up.  But, in this case, I have an image already running.  So what is the best way to run a my cloud-init script?


I created a simple cloud-config file


#cloud-config
#
####################

##################################################
#
# Create groups
#
##################################################
groups:
 - patman

##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: pbailey
   passwd: $6$Actual-passwordfrom/etc/shadow
   ssh-authorized-keys:
     - ssh-rsa AAAActual-ssh-public-key
     - ssh-rsa AAAnother-public-key

##################################################
#
# runcmd:
#    List of commands to execute
#    If a command has a verbose setting, it's a
#    good idea to use it.  Just in case you
#    Need to debug later.
#    Output is logged to
#      /var/log/cloud-init-output.log
#
##################################################
runcmd:

##################################################
#
# passwd <username> -u
#   When a user is created and their password
# set... their password is locked in /etc/shadow
# (it has a '!' in front of their encrypted
# password, hard to see and a pain)
# This command unlocks it
#
##################################################
 - 'passwd patman -u'


Of course put your actual ssh public keys and encrypted password from /etc/shadow
Then attempt to run it manually.


   > sudo cloud-init -f cloud-init-test init --local







Now I have files in cloud-init



   > tree /var/lib/cloud/




Looks pretty empty


Trying  without the --local


   > sudo cloud-init -f cloud-init-test init




That is doing more.   But failed accessing user data




Failed calling a URL for meta-data




Giving up




Keys





Lot's more files now





Now users made though…

I think this is a bust.  Going back to an older  VM snapshot.






Second attempt (failed… but closer)


I think I finally found a good lead on what to do from this site http://jeffgran.com/2011/08/19/ubuntu-cloud-init-rackspace-fog-ruby/ [5]

Between that site and a few others I have been looking at it seems cloud-init needs to files fed to it to work.  One is the user-data file which is the cloud-init file I am used to feeding into an EC2 or Openstack instance.  The other file it needs is the meta-data file.  This is a different beast I am not used to.

With meta-data you can set things like the hostname and network-interface customization.  EC2 handles this by having a local url that can be hit from within a machine.  See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html [6]

You can use these yourself to retrieve data about the machine.  Ex.


   > curl http://169.254.169.254/latest/meta-data/
   > curl http://169.254.169.254/latest/user-data


From my reading I could set up a URL to point to in the cloud.cfg file.  But I don't want to do that just yet.  I want to just simply place a meta-data and a user-data file somewhere and have them run locally.



Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



instance-id: my-test-od
hostname:my-hostname





Make a very simple user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data




#cloud-config

runcmd:
 - 'echo ============ Hello World ================'





Fire it up


   > sudo cloud-init -d init





Failed loading yaml

Maybe just reboot ?

First remove the instance data that was made


   > sudo rm -rf /var/lib/cloud/instance/*




Then reboot it


   > sudo reboot now




Hey it did something.

… But I can't get back in.  It seems to have overwritten the default password and ssh key for my cloud-init user named ubuntu.







Third attempt (failed bad meta-data)


Hmmm…

The user-data seemed to load, but I am not sure what is going on with fiddling with the Ubuntu user.   My best bet now is to make a better user-data script that creates a new user I can login with.

Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



instance-id: my-test-id
hostname:my-hostname





Now for a more complex user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data




#cloud-config
#
####################

##################################################
#
# Create groups
#
##################################################
groups:
 - patman

##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: pbailey
   passwd: $6$Actual-passwordfrom/etc/shadow
   ssh-authorized-keys:
     - ssh-rsa AAAActual-ssh-public-key
     - ssh-rsa AAAnother-public-key

##################################################
#
# runcmd:
#    List of commands to execute
#    If a command has a verbose setting, it's a
#    good idea to use it.  Just in case you
#    Need to debug later.
#    Output is logged to
#      /var/log/cloud-init-output.log
#
##################################################
runcmd:

##################################################
#
# passwd <username> -u
#   When a user is created and their password
# set... their password is locked in /etc/shadow
# (it has a '!' in front of their encrypted
# password, hard to see and a pain)
# This command unlocks it
#
##################################################
 - 'passwd patman -u'



Of course put in your actual password from /etc/shadow and your actual .pub ssh key.



Then reboot it


   > sudo reboot now




OK now it is stuck on something…

A reboot got it back up, but something is not correct…



Looking at the logs


   > vi /var/log/cloud-init.log




I can see that it is reading in the user-data and meta-data I provided…



But it is failing to load it as a yaml blob


I found my mistake




Had an extra space where it should not be… well should actually add a space on the next line.








Fourth attempt (Small Success)


Looking around I found this http://askubuntu.com/questions/153486/how-do-i-boot-ubuntu-cloud-images-in-vmware [7]  The approved answer did things a little more simple than what I was attempting to do.  So I am going to follow the example and make it simple!

Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: localhost


This is very simple and should not screw up my system in such a way as to mess up my ability to ssh to the box (I've had that result in a few half attempts)
Now for a more complex user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data




#cloud-config
#
####################

password: mypassword


This will set the password for the ubuntu user to mypassword.  But it does require you to change it on first login.

Then reboot it


   > sudo reboot now






Yep it uses the new password to login and forces you to reset it on first login.

I can still login with my ssh key


   > ssh -i ~/.esxi/pats-keypair.pem ubuntu@192.168.0.13




So it did not wipe out the .ssh folder on ubuntu.   Good.




It did change the local-hostname to localhost (it was ubuntu before)








Fifth attempt (More Success)


OK, let me attempt to increment this just a little


Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: my-servername


Let me see if I can tweak this and not break it.


Now for a more complex user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data



#cloud-config
#
####################

##############################
#
# Generic cloud user (ubuntu) settings
#
##############################
password: mypassword

##################################################
#
# Install packages
#
##################################################
packages:
 - htop
 - tree

##################################################
#
# Reboot the machine when done.
# This is done to make sure the mount script is
# working
#
##################################################
power_state:
 delay: "now"
 mode: reboot
 message: First Reboot!
 timeout: 30




Installs a few packages and forces a reboot.


Then reboot it


   > sudo reboot now



Wahoo it did reboot! After 30+ seconds… It did have to install some packages first I suppose



htop and tree were installed!









Sixth attempt (Success!!)


OK, let me see if I can add a user now.

Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: my-servername


Let me see if I can tweak this and not break it.


Now for a more complex user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data



#cloud-config
#
####################

##############################
#
# Generic cloud user (ubuntu) settings
#
##############################
password: mypassword

##################################################
#
# Install packages
#
##################################################
packages:
 - htop
 - tree

##################################################
#
# Reboot the machine when done.
# This is done to make sure the mount script is
# working
#
##################################################
power_state:
 delay: "now"
 mode: reboot
 message: First Reboot!
 timeout: 30

##################################################
#
# Create groups
#
##################################################
groups:
 - patman


##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: patman
   passwd: $6$Ns43f3HWNOT-ACTUAL
   ssh-authorized-keys:
     - ssh-rsa AAAAB3NzaC-NOTACTUAL


Added a user


Then reboot it


   > sudo reboot now




Hey a user!  Let me see if I can login since it has my public ssh key


   > ssh patman@192.168.0.13




Hmm denied… Oh wait I think I know why!



If you open your /etc/shadow file you can see the issue


   > sudo vi /etc/shadow




See the ! in front of my password.  That says this user, even though they exist, are not allowed to login.

This can be fixed with a simple command.  (well I guess you could manualy edit the /etc/shadow file if you wanted)


   > sudo passwd patman -u




Worked but I also got that annoying host sudo error…


Let me try to login again


   > ssh patman@192.168.0.13


And also just use sudo to confirm my password works!




Yep worked!










Seventh attempt (Super Success!!)


I am getting pretty close to what I want.  Or at least able to exercise a lot of the tools in cloud-init I use.

Next I want try runcmd to fix my user lock and also to fix the sudo issue resolve host warning.

Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: my-servername


Same as last time.


Now for a more complex user-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data



#cloud-config
#
####################

##############################
#
# Generic cloud user (ubuntu) settings
#
##############################
password: mypassword

##################################################
#
# Install packages
#
##################################################
packages:
 - htop
 - tree

##################################################
#
# Reboot the machine when done.
# This is done to make sure the mount script is
# working
#
##################################################
power_state:
 delay: "now"
 mode: reboot
 message: First Reboot!
 timeout: 30

##################################################
#
# Create groups
#
##################################################
groups:
 - patman


##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: patman
   passwd: $6$C5Hp-NOTACTUAL
   ssh-authorized-keys:
     - ssh-rsa AAAAB3Nza-NotACTUAL


##################################################
#
# runcmd:
#    List of commands to execute
#    If a command has a verbose setting, it's a
#    good idea to use it.  Just in case you
#    Need to debug later.
#    Output is logged to
#      /var/log/cloud-init-output.log
#
##################################################
runcmd:

##################################################
#
# passwd <username> -u
#   When a user is created and their password
# set... their password is locked in /etc/shadow
# (it has a '!' in front of their encrypted
# password, hard to see and a pain)
# This command unlocks it
#
##################################################
 - 'passwd patman -u'


##################################################
#
# This fixes the hostname in /etc/hosts
# which removes annoying messages when you
# are trying to run commands using sudo
#
##################################################
 - 'echo 127.0.0.1 `hostname` >> /etc/hosts'



Now it should fix the /etc/hosts file so no more sudo warnings.  Also should unlock my user.


Then reboot it


   > sudo reboot now





Let me see if I can login now.


   > ssh patman@192.168.0.13





It worked!  And now I don't see the sudo warning.  Looking real quick at /etc/hosts


   > cat /etc/hosts





That gets me most of the stuff I want for now.  Well… There are two more things I would like to try.

  1. Set a static IP
  2. Pass in a .pub key to the Ubuntu user (AWS /Openstack style?) 










Eighth attempt (Partial Success)


Now let me try to set a static IP address on the new box and to add a public ssh key to the ubuntu (cloud user)

Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: my-servername
network-interfaces: |
  auto lo
  iface lo inet loopback
  #
  #
  auto eth0
  iface eth0 inet static
        address 192.168.0.75
        netmask 255.255.255.0
        broadcast 192.168.1.255
        gateway 192.168.0.1
        dns-nameservers 8.8.8.8


Hopefully this will set my /etc/network/interfaces and give me a static IP address.


Add ssh key to the ubuntu (cloud user)


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data



#cloud-config
#
####################

##############################
#
# Generic cloud user (ubuntu) settings
#
##############################
password: mypassword
ssh_authorized_keys:
   - ssh-rsa AAAAB3NzaC1-NOTREALKEY

##################################################
#
# Install packages
#
##################################################
packages:
 - htop
 - tree

##################################################
#
# Reboot the machine when done.
# This is done to make sure the mount script is
# working
#
##################################################
power_state:
 delay: "now"
 mode: reboot
 message: First Reboot!
 timeout: 30

##################################################
#
# Create groups
#
##################################################
groups:
 - patman


##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: patman
   passwd: $6$C5Hpbc-NOTREALKEY
   ssh-authorized-keys:
     - ssh-rsa AAAAB3NzaC-NOTREALKEY


##################################################
#
# runcmd:
#    List of commands to execute
#    If a command has a verbose setting, it's a
#    good idea to use it.  Just in case you
#    Need to debug later.
#    Output is logged to
#      /var/log/cloud-init-output.log
#
##################################################
runcmd:

##################################################
#
# passwd <username> -u
#   When a user is created and their password
# set... their password is locked in /etc/shadow
# (it has a '!' in front of their encrypted
# password, hard to see and a pain)
# This command unlocks it
#
##################################################
 - 'passwd patman -u'


##################################################
#
# This fixes the hostname in /etc/hosts
# which removes annoying messages when you
# are trying to run commands using sudo
#
##################################################
 - 'echo 127.0.0.1 `hostname` >> /etc/hosts'


Should allow me to login with using the .pem key I have


Then reboot it


   > sudo reboot now



Now I should be able to ssh as the ubuntu user and not use my .pem key.  Oh and the IP address should be different.


   > ssh ubuntu@192.168.0.75





OK looks like IP address did not change, let me try the old one.


   > ssh ubuntu@192.168.0.13





Hey that worked!  But my IP address did not get updated like I had hoped.

Looking at /etc/network/interfaces




That is not what I wanted.








Ninth attempt (Got it done!!!)


I tried several different ways of doing this and all failed.

I found a good example here http://cloudinit.readthedocs.org/en/latest/topics/datasources.html?highlight=network-interfaces#no-cloud   [9]  I think I am close to correct.



This did not work either.

Maybe it works in a new version of Ubuntu.  Pretty soon I will be switching over to Ubuntu 16.04 LTS and maybe that will fix my problem.

But for now I am using Ubuntu 14.04 so I think maybe I can update the /etc/network/interfaces file in the cloud-config file.


Starting all over again…

Create the /var/lib/cloud/seed/nocloud-net/ folder


   > sudo apt-get install cloud-init
   > sudo mkdir -p /var/lib/cloud/seed/nocloud-net


Make a very simple meta-data file


   > sudo vi  /var/lib/cloud/seed/nocloud-net/meta-data



local-hostname: my-servername





Use write_files to create a new /etc/network/interfaces file.


   > sudo vi  /var/lib/cloud/seed/nocloud-net/user-data



#cloud-config
#
####################

##############################
#
# Generic cloud user (ubuntu) settings
#
##############################
password: mypassword
ssh_authorized_keys:
   - ssh-rsa AAAAB3NzaC1-NOTREALKEY

##################################################
#
# Install packages
#
##################################################
packages:
 - htop
 - tree

##################################################
#
# Reboot the machine when done.
# This is done to make sure the mount script is
# working
#
##################################################
power_state:
 delay: "now"
 mode: reboot
 message: First Reboot!
 timeout: 30

##################################################
#
# Create groups
#
##################################################
groups:
 - patman


##################################################
#
# Create users.
#    * set their prefered shell
#    * Set their group membership
#    * give them admin permissions
#    * Set their password (copied from ecnrypted version @ /etc/shadow)
#    * Input Public SSH keys to allow access
#
##################################################
users:
 - default
 - name: patman
   shell: /bin/bash
   sudo: [ "ALL=(ALL) ALL" ]
   primary-group: patman
   passwd: $6$C5Hpbc-NOTREALKEY
   ssh-authorized-keys:
     - ssh-rsa AAAAB3NzaC-NOTREALKEY


##################################################
#
# runcmd:
#    List of commands to execute
#    If a command has a verbose setting, it's a
#    good idea to use it.  Just in case you
#    Need to debug later.
#    Output is logged to
#      /var/log/cloud-init-output.log
#
##################################################
runcmd:

##################################################
#
# passwd <username> -u
#   When a user is created and their password
# set... their password is locked in /etc/shadow
# (it has a '!' in front of their encrypted
# password, hard to see and a pain)
# This command unlocks it
#
##################################################
 - 'passwd patman -u'


##################################################
#
# This fixes the hostname in /etc/hosts
# which removes annoying messages when you
# are trying to run commands using sudo
#
##################################################
 - 'echo 127.0.0.1 `hostname` >> /etc/hosts'

##################################################
#
# write_files:
#    Files to write out.
#
##################################################
write_files:


##################################################
#
# Set static IP address (would rather do this in meta-data but it is no
# working at the moment)
#   @ /etc/network/interfaces
#
##################################################
 - path: /etc/network/interfaces
   permissions: '0644'
   owner: root:root
   content: |
      auto lo
      iface lo inet loopback
      #
      #
      auto eth0
      iface eth0 inet static
            address 192.168.0.75
            netmask 255.255.255.0
            broadcast 192.168.1.255
            gateway 192.168.0.1
            dns-nameservers 8.8.8.8


Should allow me to login with using the .pem key I have


Then reboot it


   > sudo reboot now





Now I should be able to ssh as the ubuntu user and not use my .pem key.  Oh and the IP address should be different.


   > ssh ubuntu@192.168.0.75




Yes it worked!

Also if I try to login with my password for the ubuntu user I need to use the password set in the cloud-init file.  It is mypassword and requires you to reset it after login.

 


I don't recommend doing it this way and I will probably remove this from an eventual cloud-init file I will use.
I should also be able to login as patman


   > ssh patman@192.168.0.75




And tree and htop should be installed.




Working!

Hey it's all working!!   Cool I got what I wanted.

I still need to work out how I am going to save config files in a git repo and deploy them to a server… but at least I got a basic setup working and I can expand from there.








References


[1]        cloud-init documentation
                Accessed 03/2016
[2]        CloudInit Summary Page
                Accessed 03/2016
[3]        Arch Linux install Cloud-Init
                Accessed 03/2016
[4]        Automating Openstack with cloud init run a script on VM's first boot
                Accessed 03/2016
[5]        Bootstrapping an Ubuntu Server on Rackspace Using Cloud-Init and Fog
http://jeffgran.com/2011/08/19/ubuntu-cloud-init-rackspace-fog-ruby/
Accessed 04/2016
[6]        Instance Metadata and User Data
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
Accessed 04/2016
[7]        How do I boot Ubuntu Cloud images in vmware?
http://askubuntu.com/questions/153486/how-do-i-boot-ubuntu-cloud-images-in-vmware
Accessed 04/2016
[8]        Getting started with cloud-init
[9]        NoCloud config

5 comments: