by Stanislas Nanchen

Hosting a legacy Python Web Application on AWS

One of our customers has a website powered by a legacy Django web application. While the web application is built with old technologies (Python 2.7, Django 1.7) that have been obsoleted, it is stable and gets rarely updated.

We were approached to take over this web application and we decided, for the moment, to change only its hosting: from a manually managed linux server to an reproducible AWS installation. The initial challenge was to find a fit with all the services offered by AWS so that only minimal code changes were necessary. The following diagram illustrates the installation.

At the center of the installation is the Elastic Container Service, or ECS ①. To use ECS, we needed first to dockerize the web application and to create a reproducible build for it. The original installation had a manually managed virtual environment on a linux server. We refitted the requirements.txt file with pipenv and used an official “buster” Python 2 base docker image. After these transformations, we achieved a reproducible build and were able to create docker images for the web application. These Django images are stored into an ECR Docker repository (not shown on the diagram above).

The web application uses the django-filer plugin to manage images and, in the original installation, these images were stored on disk. Ideally, we would store such images in an S3 bucket and, indeed, it is possible to configure the django-filer to store images on an S3 bucket. Unfortunately, we use an older version of the django-filer that does not have this capabilility and updating the django-filer plugin would have required too many changes in the Python dependencies. So, instead of using a S3 bucket, we use an EFS ② file system that is mounted inside our docker container via the rexray/efs docker volume driver¹.

To use a EFS file system inside a container on ECS via CloudFormation, one has to add it as a volume in the the Task Definition resource, as illustrated in the following Cloudformation code snippet².

1
2
3
4
5
6
7
8
9
10
11
12
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
  Family: [...]
  ExecutionRoleArn: [...]
  ContainerDefinitions: [...]
  Volumes:
    - Name: !Sub "media-${Env}"
      DockerVolumeConfiguration:
        Autoprovision: false
        Driver: rexray/efs
        Scope: shared

Typically, a productive Django server does not serve images stored via the django filer. To serve such assets, we need a second docker container running a nginx server where the EFS file system is also mounted. The nginx container serves also as proxy for the django webserver.

Together, the Django container and the nginx container are configured in an ECS task and instantiated via an ECS service to make them run continuously. To actually run the ECS task, we need compute resources grouped in an ECS cluster. A simple EC2 t2.micro instance is sufficient for our purposes and it is instantiated via an Auto Scaling Group ③ to make sure it gets recreated should a failure occur. We use the official Amazon Linux 2 AMI for ECS and it does not come with rexray installed. To do that, we configure the launching configuration for the autoscaling group with a shell script in its User Data as shown in the following CloudFormation code snippet³.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
  IamInstanceProfile: [...]
  ImageId: !Ref ECSImageId
  InstanceType: t2.micro
  SecurityGroups: [...]
  UserData:
    Fn::Base64:
      Fn::Sub: |
        #!/bin/bash
        echo ECS_CLUSTER=${AWS::StackName} >> /etc/ecs/ecs.config
        echo ECS_UPDATES_ENABLED=true >> /etc/ecs/ecs.config
        echo ECS_ENABLE_TASK_IAM_ROLE=true >> /etc/ecs/ecs.config
        yum install -y aws-cfn-bootstrap amazon-efs-utils

        docker plugin install rexray/efs REXRAY_PREEMPT=true EFS_REGION=${AWS::Region} EFS_SECURITYGROUPS=${MediaFileSystemSecurityGroup} LINUX_VOLUME_FILEMODE=0755 EFS_TAG=website --grant-all-permissions

        /opt/aws/bin/cfn-signal -e 0 --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}

Also interesting in the previous code is the reference to the parameter ECSImageId on line 5. The parameter represents the Id for the Amazon Linux 2 AMI for ECS. AWS maintains the Id of that AMI in a global SSM parameter /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id and so the parameter ECSImageId is declared as follows and passed the SSM parameter name as argument on stack creation.

1
2
ECSImageId:
  Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'

One of the advantages of using a public cloud service like AWS is the choice of managed databases that are offered. Managed databases alleviate developers from the maintainance the database software and provide backup and point-in-time recovery out of the box. The original hosting used a locally running PostgreSQL database and we replaced it with a fully managed RDS PostgreSQL database ④.

The web application is accessed with SSL and we wanted to use ACM public SSL certificates. Thanks to DNS validation, the renewal of the certificates is done automatically. Therefore, an Application Load Balancer ⑤ terminates SSL thanks to the ACM certificates and forwards requests to the web application running inside the ECS task.

Finally, building the Django web application produces static assets that we can store in an S3 Bucket ⑥. The whole web application is served via a CloudFront distribution ⑦. Thanks to the CloudFront distribution, we can easily cache static assets from the S3 bucket and cache images served by nginx from the django-filer.

The installation is scripted in 3 CloudFormation templates making it easy to reconstitute the productive environment and to create and tear down test environments as they are needed.