27Jul

HOWTO: AWS AMI Creation with Puppet

You are here: Home » AWS » HOWTO: AWS AMI Creation with Puppet
GitHub
Posted by in AWS, Devops, Operating Systems, Puppet, Virtualization
0

Here is a short HOWTO which results in a custom Amazon AWS EC2 AMI image with the Puppet Enterprise agent installed and setup to connect to the Puppet master server. This method uses a 3rd party tool called Packer.io which greatly simplifies and automates the AMI build process in a scriptable and pragmatic way. This is a very easy way to build hypervisor OS template images with the Puppet agent pre-installed for AWS, Docker, Google Cloud, OpenStack, Parallels, QEMU, Virtualbox, or VMWare. 

  1. Install packer.io.
  2. Start by picking the existing image AMI you want to use as your source. Note the AMI ID for the region you want.
    1. Example: CentOS 6 with Updates
    2. Example: CentOS 7 with Updates
  3. Before you begin you will need your AWS account access key and secret.
    1. Add them to the packer.io JSON at the top.
  4. Edit the JSON setting source_ami to point to the AMI source you want to use from step 2. Also ensure the region is set correctly for the AMI you chose.
  5. Edit the example cloud.cfg below to your taste and upload it to a web server. You can use an S3 bucket as a source if you want. The cloud.cfg and cloud-init tools are important because they automatically expand the filesystem on the EBS volume on first boot.
  6. [OPTIONAL] Validate the packer.io JSON file.
    1. ./packer validate “jsonfile“
  7. Run packer to spin up a fresh instance and run your provisioning script. This is where the Puppet agent is installed and OS updates are ran. You can also pre-install packages to further reduce application deployment times.
    1. ./packer build “jsonfile“
  8. If you run into OS template creation problems you can run packer in debug mode. This gives you verbose output to help identify the problem.
    1. PACKER_LOG=1 ./packer -debug build “jsonfile“




    disable_root: 0
    ssh_pwauth: 1
    manage_etc_hosts: false
    locale_configfile: /etc/sysconfig/i18n
    mount_default_fields: [~, ~, ‘auto’, ‘defaults,nofail’, ‘0’, ‘2’]
    resize_rootfs_tmp: /dev
    ssh_deletekeys: 0
    ssh_genkeytypes: ~
    cloud_init_modules:
    – migrator
    – write-files
    – growpart
    – resizefs
    cloud_config_modules:
    – mounts
    – locale



    {
    “variables”: {
    “access_key”: “AAjdifg84hjghjdf”,
    “secret_key”: “WefwasGJDShish4seoghswSRJGOwt”,
    “cloudcfg_url”: “https://raw.githubusercontent.com/bandrews/ami-automation/master/cloud.cfg”,
    “puppet_url”: “https://puppet.example.com:8140/packages/current/install.bash”
    },
    “builders”: [{
    “type”: “amazon-ebs”,
    “access_key”: “{{user `access_key`}}”,
    “secret_key”: “{{user `secret_key`}}”,
    “region”: “us-west-2”,
    “source_ami”: “ami-81d092b1”,
    “instance_type”: “t1.micro”,
    “associate_public_ip_address”: “true”,
    “ssh_username”: “root”,
    “ami_name”: “centos6-hvm-puppet {{timestamp}}”
    }],
    “provisioners”: [{
    “type”: “shell”,
    “inline”: [
    “sleep 20”,
    “/etc/init.d/rsyslog stop”,
    “shred -f /var/log/messages /var/log/secure /var/log/maillog /var/log/cron /var/log/spooler /var/log/boot.log /var/log/lastlog /var/log/dmesg* /var/log/dracut.log /var/log/audit/audit.log”,
    “curl -k {{user `puppet_url`}} | bash”,
    “service pe-puppet stop”,
    “sed -i ‘/^certname/d’ /etc/puppetlabs/puppet/puppet.conf”,
    “yum -y install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm”,
    “sed -i ‘s/mirrorlist=https/mirrorlist=http/’ /etc/yum.repos.d/epel.repo”,
    “yum -y install vim nano cloud-init”,
    “curl -k {{user `cloudcfg_url`}} > /etc/cloud/cloud.cfg”,
    “yum -y update”,
    “shred -f /root/.bash_history /etc/ssh/ssh_host_* /tmp/pip-build-root /etc/puppetlabs/puppet/ssl/*”,
    “shred -f /root/.ssh/authorized_keys; touch /root/.ssh/authorized_keys; chmod 600 /root/.ssh/authorized_keys”,
    “chkconfig getssh on”,
    “yum clean all”
    ]
    }]
    }


    {
    “variables”: {
    “access_key”: “AAjdifg84hjghjdf”,
    “secret_key”: “WefwasGJDShish4seoghswSRJGOwt”,
    “cloudcfg_url”: “https://raw.githubusercontent.com/bandrews/ami-automation/master/cloud.cfg”,
    “puppet_url”: “https://puppet.example.com:8140/packages/current/install.bash”
    },
    “builders”: [{
    “type”: “amazon-ebs”,
    “access_key”: “{{user `access_key`}}”,
    “secret_key”: “{{user `secret_key`}}”,
    “region”: “us-west-2”,
    “source_ami”: “ami-c7d092f7”,
    “instance_type”: “t2.micro”,
    “associate_public_ip_address”: “true”,
    “ssh_username”: “centos”,
    “ami_name”: “centos7-puppet {{timestamp}}”
    }],
    “provisioners”: [{
    “type”: “shell”,
    “inline”: [
    “sleep 10”,
    “sudo systemctl stop rsyslog”,
    “sudo shred -fu /var/log/messages /var/log/secure /var/log/maillog /var/log/cron /var/log/spooler /var/log/boot.log /var/log/lastlog /var/log/dmesg* /var/log/audit/audit.log”,
    “sudo curl -qk {{user `puppet_url`}} > /tmp/peinstall.sh”,
    “sudo /bin/bash /tmp/peinstall.sh”,
    “sudo systemctl stop pe-puppet”,
    “sudo sed -i ‘/^certname/d’ /etc/puppetlabs/puppet/puppet.conf”,
    “sudo yum -y install https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm”,
    “sudo mkdir -p /etc/cloud”,
    “sudo curl -qk {{user `cloudcfg_url`}} > /etc/cloud/cloud.cfg”,
    “sudo yum -y install vim nano cloud-init”,
    “sudo yum -y update”,
    “sudo shred -fu /tmp/peinstall.sh /etc/ssh/ssh_host_*”,
    “sudo shred -fu /root/.ssh/authorized_keys”,
    “shred -fu /tmp/script.sh”,
    “sudo yum clean all”
    ]
    }]
    }


     

    More

    Catagories