Amazon AWS cloud matt mccullough

Managing AWS EC2 Instances with Python

Today I will show you how I personally manage AWS EC2 Instances with Python. This post will show you a quick way to view what servers are running on your Amazon AWS Console. This tool will provide the ability to pause or continue your existing servers. I’ll only go into the basics of this script, however, this will give you the basic framework to continue building on top of.

For this tutorial you will need:

  • AWS Account
  • IAM Security Group
  • Boto3 for Python

Setting up your security group

Assuming that you have set up your AWS Account, I am going to go through the process of creating an IAM Security Group to connect to the API.

Follow this link to go to the IAM home page. *Note that this link will bring you to the “us-west-2” region, which I happen to be using. Changing this region is as simple as clicking in the top right and selecting which region you’d like to use.

Once you’ve landed on the IAM Homepage, continue to the ‘Users’ tab and create a new user by selecting ‘Add User’. Give this user a username. We’ll call it “cloudmanager” to indicate what we intend to use this user for. Now, make sure you give this user programatic permissions.

Amazon AWS Console Matt McCullough

Now that you’ve established your user, continue to permissions. If you have not set up a group, click “Create Group”. Give it the name “awsadmin”. Below is an image with the appropriate policies you should attach to this group.

AWS Policies Matt McCullough

Following this, review the user and group. Finally, continue ahead and confirm the created user.

Installing boto3 for Python

Assuming you have Python pip already installed on your computer, we simply have to type the following command in the terminal. We also will need to install the AWS CLI for command line interaction with AWS.

On UNIX Systems

sudo pip install boto3
sudo pip install awscli

If this asks for your password, go ahead and enter it in.

On Windows Systems

pip install boto3
pip install awscli

Getting your AWS Access ID and Secret

Now that you have boto3 installed, go back to your user account on AWS. You’ll want to click on the ‘Users’ tab again within the IAM homepage. Once there, direct yourself to the security credentials section.

Amazon AWS Security Credentials Matt McCullough

Click on Create Access Key

Follow the steps they give you. Plop the Access ID and Access Secret keys into a notepad for user later. You’ll need this information to be able to access your server information from python.

Setting up your AWS CLI config

Open up your terminal.

Type in:

aws configure

Now you’ll be prompted with a few things:

AWS Access Key ID [None]: <enter your access key id here>
AWS Secret Access Key [None]: <enter your secret access key here>
Default region name [None]: "us-west-2" (or whatever region you want)
Default output format [None]: <just press enter here>

Now that you’ve set this up, you may freely use boto3 to access your instances within the environment.

Managing AWS EC2 Instances with Python

In this section, I’ll show you a brief overview on what the program is doing. We will need to make a “manager.py” file.

Now here is the code I’ve written up for reading from your servers, stopping all of the active ones, and starting them again at another time. You should edit this code to your liking to add features like checking server statuses, stopping and starting servers at specific times, and logging.

import boto3.ec2
import time

class CloudManager:
    def __init__(self):
        self.active_ec2_servers = []
        self.active_ec2_instance_ids = []
        self.stopped_instance_ids = []
        self.empty = False

        #connect to boto client
        self.client = boto3.client("ec2")

    def get_active_ec2(self):
        """Populate list with current active AWS instances in EC2"""
        self.active_ec2_servers = []
        self.active_ec2_instance_ids = []
        reservationlist = self.client.describe_instances()["Reservations"]
        try:
            for reservations in reservationlist:
                instances = reservations["Instances"][0]

                if "stopped" not in str(instances["InstanceId"]):
                    self.active_ec2_servers.append(instances)
                    self.active_ec2_instance_ids.append(instances["InstanceId"])
            return self.active_ec2_servers
        except TypeError as e:
            print("Improper Type when pulling get_active_ec2: %s"%e.message)
        except IndexError as e:
            print("Improper use of index when pulling get_active_ec2: %s"%e.message)

   
    def stop_servers(self, instanceids=None):
        """Stop all active servers and remember which ones we stopped"""
        if instanceids is not None:
            print("Stopping Instances at: %s"%str(instanceids))
            self.client.stop_instances(InstanceIds=instanceids)
        else:
            print("Stopping Instances at: %s"%str(self.active_ec2_instance_ids))
            self.stopped_instance_ids = self.active_ec2_instance_ids
            self.client.stop_instances(InstanceIds=self.active_ec2_instance_ids)
            self.active_ec2_instance_ids = []
            return self.stopped_instance_ids

    def start_servers(self, instanceids=None):
        """Default start servers you've already stopped"""
        if instanceids is not None:
            print("Starting Instances in %s"%str(instanceids))
            self.client.start_instances(InstanceIds=instanceids)
        else:
            print("Starting Instances in %s"%str(self.stopped_instance_ids))
            self.active_ec2_instance_ids = self.stopped_instance_ids
            self.client.start_instances(InstanceIds=self.stopped_instance_ids)
            self.stopped_instance_ids = []
            return self.active_ec2_instance_ids


dashboard = CloudManager()

#Test that we are recieving data
for i in dashboard.get_active_ec2():
    print i

print("Stopping Servers")
dashboard.stop_servers()
time.sleep(120) #Wait 2 minutes to make sure they have all stopped

#You could include a section on waiting until the statuses officiall are off here.
print("Starting Servers:")
dashboard.start_servers()

#start the servers again

time.sleep(120) #wait 2 more minutes

for i in dashboard.get_active_ec2():
    print i

If you have any questions, feel free to comment below or email me directly at help@mattmccullo.com. Also, if you want more tutorials on Python Programming, feel free to sign up for my newsletter

Leave a Reply

Your email address will not be published. Required fields are marked *