Access Internet from AWS EC2 instance in private subnet

When you deploy your web application in AWS VPC subnet, most probably you will follow best practices and separate your app in at least 2 layers – web/application layer and database layer. It is recommended to have a database EC2 instance in private subnet. EC2 instances in private subnets don’t have public IP addresses, thus they cannot be accessed from Internet. But there are a lot of reasons why you would like to access Internet from this instance – for example, to download the latest security updates for your OS and services. So what to do? How to benefit from an EC2 intance being in a private subnet, on the other hand still be able to access Internet?

NAT Gateway

There is a special AWS-managed device you can create in your VPC – NAT Gateway. It allows EC2 instances from private subnets access Internet and it blocks all connections being initiated outside of your VPC. So if a connection is initiated from your EC2 instance (for example, by software update command), NAT Gateway will pass the traffic to Internet and all return traffic from Internet to your EC2 instance will be allowed to pass as well.

There is another way to achieve the same – to use so called NAT instance. It is a usual EC2 instance running in a public subnet and having a special software installed on it. Basically it provides the same functionality as NAT Gateway, but in NAT instance case you are managing it. So you are responsible for patching it, monitoring its health, etc. Right now NAT instance is deprecated and is not recommended to be used by AWS. So in this guide I will focus only on NAT Gateway.

Private subnet creation

Let’s start by creating a new private subnet within the VPC we will work with. Navigate to VPC tab inside your AWS Management console and select “Subnets”. Click on “Create subnet” and dialog window will open.

Select a VPC inside which you would like your private subnet to reside in, you can choose the particular Availability Zone for your subnet (it is optional) and IPv4 CIDR block. If you are not sure what it is and what to enter, just follow the example on screenshot above. Once done, press “Create”. By default, subnets do not assign public IPv4 addresses to EC2 instances. We are more than fine with this.

Next, let’s go to “Route Tables”, by clicking on the corresponding link in the left navigation panel. Then click “Create route table”. In the opened dialog window select a VPC in which we have just created a new subnet. Specify name tag property for easier this route table identification in future and press “Create”.

Now select this newly created route table and click on “Routes” tab. You should see only the default route which allows to communicate within your entire VPC.

The last thing we should do – is to associate this route table with our subnet. As by default each newly created subnet is associated with a default (Main) route table. Let’s go back to “Subnets”, select our new subnet and botton tabs we need to select “Route Table”. Make sure that the subnet is associated with our new route table. If not, click “Edit route table association”. A new dialog window will open. Select our route table and click “Save”.

Initial testing without NAT Gateway

Let’s now create a new EC2 instance in the subnet we have just created and try to access Internet resources and see what happens. The performance of the instance does not matter for this manual, so I recommend to select t2.nano instance type to reduced associated costs.

What is important to keep in mind during instance creation – you should select the subnet we created in the previous step and use in “Auto-assign Public IP” use subnet settings as per screenshot below.

While creating or choosing a security group for your newly created EC2, please keep in mind that this security group should allow SSH and HTTP traffic (22 and 80 ports respectively).

As you see, there is no public IP assigned to this instance, so we even cannot connect to it using usual methods. Still, if we have another EC2 instance in the same VPC and it is in a public subnet, we can still connect to a EC2 instance in a private subnet using ssh relay or key forwarding. In the next section, let’s quickly discuss how you can configure it. On the screenshot below, you can see that I have connected to my EC2 using ssh relay and tried to update OS. As you can see, I got “Network unreachable” error, since EC2 cannot connect to any Internet resource.

SSH relay or key forwarding in Putty

SSH relay allows you to use ssh keys which are stored locally on your laptop to be cached and used for connecting from one chained machine to another, where this key is not actually available. For example, you want from your laptop A connect to EC2 instance B and from EC2 instance B you want to connect to EC2 instance C. But the problem is that for B->C connection you need an ssh key, which does not present on machine B. However, this ssh key is available on laptop A. By using ssh relay, you can connect from B to C using an ssh key, which is not physically located on B. This is a huge security improvement, that you don’t need to store ssh keys on each EC2 instance. We will take in more details about security aspects of ssh relay in one of the upcoming posts dedicated to bastion host. So the prerequisite to connect to an EC2 instance in a private subnet is to have an EC2 instance in a public subnet. Then, using this public instance, we can connect to an instance in a private subnet.

Assuming you are using Putty and Windows 10, let’s see how we can setup ssh key forwarding.

Locate an ssh key, which should be used to authenticate ourselves on an EC2 instance. Double-click on it. Putty will add it as an available ssh key for authentication. Then, open Putty, enter our EC2 instance from public subnet details (IP, login), under Connection ->SSH->Auth check the option “Allow agent forwading”. That’s it!.

Now you should be able to connect to an EC2 instance and through it to our EC2 instance in a private subnet.

Providing access to Internet to private subnets

Now let’s go back to VPC service in AWS Management console and select “NAT Gateways”. In the opened tab, click “Create NAT Gateway”. A new dialog window will open.

Assign a name to our gateway, as a subnet choose any public subnet you have in your VPC. Gateway requires Elastic IP address to be able to work. If you don’t have any reserved yet, just click “Allocate new Elastic IP”. Once it is done, click “Create NAT Gateway”.

Now let’s go to “Route tables” and add a new route, so our EC2 instance from a private subnet could communicate with the outside world. In “Route tables” select the route table we previously created and on the bottom tab under “Routes” click “Edit routes”.

A new dialog window will open and here we are going to specify that for any destination which is outside of VPC, we should use our created NAT Gateway.

As destination we should put “0.0.0.0/0” – meaning any route which do not match any other previously defined routes (in our case – any route outside VPC, since we have only 172.31.0.0/16 route defined, which is route for routing inside VPC). As target we select “NAT Gateway” and our recently created NAT Gateway should be offered as an option. Select it and save by clicking “Save routes”.

The changes we made into a route table, should take effect immediately. So let’s try to update OS once again and see what happens now.

So as you can see, now we are able to initiate connections to Internet! We are able to download any file we want from Internet, do security patching, etc. Still, no incoming Internet connections are allowed, unless this is a response traffic to our initiated request. You can go back to EC2 service and confirm that our EC2 still doesn’t have a public IP address, but it is able to communicate with outside world!

On 30.10.2020 there will be a first AWS related free webinar, where we will go through some basic VPC security related things. You can register using this link. Hurry up, the number of the available seats is limited! A record will be also available on YouTube if for some reasons you miss this webinar.

Stay turned, more exiting AWS related content to follow!