Capture client IP address on the Web server logs which is behind Load Balancer

By default, the Apache Web server Captures the Load balancer IP in access logs

In this Blog, We are going to Learn about Capturing Client IP addresses in Apache web server logs . 

What is an Application Load balancer?

The application load balancer in AWS helps to distribute the traffic across multiple instances which are attached to the Target groups. When we create the Load balancer, By default DNS endpoint is getting  created. Endpoints are like HTTP URLs which we can browse on any browser ex: Chrome, Firefox

AWS is responsible for the infrastructure availability of the load balancer. Load Balancer DNS endpoints have dynamic IP addresses which will be taken care of by the AWS. However, we can store the ALB Access logs in the s3 by enabling Access logs in the attribute section of the load balancer. Hence s3 grants access to the load balancer to store the access logs. 

Disadvantages of storing ALB logs in s3

• Access logs will store in the form of zip files in the s3 bucket. That is in the compressed format 

• Compressed format files cant be read directly from the s3 console.

• We should download the zip file and then need to extract the file. An extracted log file will be in the unstructured format. 

• Difficult to read the unstructured format for a human. Hence AWS recommends us to use the Athena service to read the unstructured log file in tabular format in athena by using SQL queries. That will incur Some charges

• We can't see a live generation of logs and client IPs on s3.

By default, the Application web server captures the Load balancer IPs in the Application 

Solution: Capturing live Client IPs on application server logs helps to understand the traffic generation of users

Now, you have understood the Application load balancer use case and Access logs storing methods, which is useful based on the Application mechanism and requirements.

Going through the below steps We can achieve live client IP addresses capturing on the Apache web server logs

Step -1 Create Ec2 instance

Here I have created One Demo Instance

Take ssh access to the instance using the below command

• SSH -i “pemfilename.pem” username@Public_IP

Step 2 - Install the Apache Web server

Step 3 - Start the Apache service 

Step -4 Create the target group

Step- 5 Create an Application load balancer with the listening rule protocol 80

You can check the apache web page by using the DNS endpoint of the Load balancer

Now the Web server access Log files capturing the Load balancer IPs

Step - 6

Now our agenda is to capture the client IP address in these access logs. To achieve this we need to add the %{X-Forwarded-For}i line in the Log format section of the  apache configuration file which is located on the path /etc/httpd/conf/httpd.conf

Then save the file, Check the syntax using “httpd -t” command. Then restart the apache service

When I browse the application from the DNS endpoint 

Now you can see my client IP address “106.76.231.153” on the Access logs

conclusion

Now, you have understood the Capturing Client ip on web server logs which is behind load balancer. The main Step is to add X-Forwarded-For in Web server configuration file and ensure that the X-Forwarded-For header attribute is Appended in Load Balancer. This is very useful to analyse live generation of logs.

By Deepak Koppal, Cloud Engineer (Cloud.in)