Preparing for a Machine Learning Engineer Position? This Article is for You!

If you’re gearing up for a Machine Learning Engineer position, you’re likely aware that the journey involves not just understanding algorithms and data but also deploying models into production. In this article, we will build upon a Flask prototype and create a fully functional and scalable service. Specifically, we will set up a Deep Learning application served by uWSGI and Nginx. We will explore everything step by step, from starting with a simple Flask application to wiring up uWSGI as a full web server and hiding it behind Nginx as a reverse proxy. This will provide a more robust connection handling for our deep learning project, which performs semantic segmentation on images using a custom Unet model and TensorFlow.

Recap of Our Journey So Far

Up until this moment in our series, we have taken a Colab notebook and converted it into a highly optimized project with unit tests, performance enhancements, trained the model in Google Cloud, and developed a Flask prototype to serve it to users. Now, we will enhance this Flask prototype and build a fully functional and scalable service.

What is uWSGI?

According to the official website, uWSGI is an application server that aims to provide a full stack for developing and deploying web applications and services. While it is language-agnostic, its most popular use is for serving Python applications. uWSGI is built on top of the WSGI spec and communicates with other servers over a low-level protocol called uwsgi.

Clarifying the Terminology

To understand uWSGI better, let’s clarify some terms:

• WSGI (Web Server Gateway Interface): This is an interface specification that defines the communication between a web server and a web application. It standardizes how requests and responses are passed back and forth between the server and the application.

• uWSGI: This application server communicates with applications based on the WSGI spec and with other web servers over various protocols (like HTTP). It often acts as middleware, translating requests from a conventional web server into a format that the application understands.

• uwsgi: This is a low-level binary protocol that enables communication between web servers. It defines the format of the data sent between servers and instances.

In summary, our software is exposed to a web app using Flask, served by the uWSGI server, which communicates with it using the WSGI spec. Meanwhile, the uWSGI server is hidden behind another web server (Nginx) that communicates using the uwsgi protocol.

Why Do We Need uWSGI? Isn’t Flask Adequate?

While Flask can act as an HTTP web server, it was not developed and optimized for security, scalability, and efficiency. It is primarily a framework for building web applications. In contrast, uWSGI was created as a fully functional web server, solving many problems that Flask does not address out of the box.

Benefits of uWSGI

• Process Management: Handles the creation and maintenance of multiple processes for concurrent applications.
• Clustering: Can be used in a cluster of instances.
• Load Balancing: Balances the load of requests to different processes.
• Monitoring: Provides built-in functionality to monitor performance and resource utilization.
• Resource Limiting: Configurable to limit CPU and memory usage.
• Configuration: Offers a wide variety of configurable options for full control over execution.

What is Nginx and Why Do We Need It?

Nginx is a high-performance, highly scalable web server that acts as a load balancer, reverse proxy, and caching mechanism. It can serve static files, provide security and encryption, and handle thousands of simultaneous connections.

Why Use Nginx with uWSGI?

Using Nginx in front of uWSGI allows us to leverage the strengths of both. While uWSGI provides Python-specific features, Nginx adds functionalities like:

• Load Balancing: Distributes traffic evenly across multiple uWSGI instances.
• Security: Prevents attacks and encrypts communications.
• Caching: Improves performance by caching content and responses.

In this setup, Nginx acts as a reverse proxy, forwarding requests from the web to our uWSGI server and back, serving as a single point of communication with the outside world.

Setting Up a uWSGI Server with Flask

In our previous article, we developed a Flask application that receives an image as a request, predicts its segmentation mask using the TensorFlow Unet model, and returns it to the client. Here’s a quick recap of the Flask code:

import os
import traceback
from flask import Flask, jsonify, request
from executor.unet_inferrer import UnetInferrer

app = Flask(__name__)
APP_ROOT = os.getenv('APP_ROOT', '/infer')
HOST = "0.0.0.0"
PORT_NUMBER = int(os.getenv('PORT_NUMBER', 8080))
u_net = UnetInferrer()

@app.route(APP_ROOT, methods=["POST"])
def infer():
    data = request.json
    image = data['image']
    return u_net.infer(image)

@app.errorhandler(Exception)
def handle_exception(e):
    return jsonify(stackTrace=traceback.format_exc())

if __name__ == '__main__':
    app.run(host=HOST, port=PORT_NUMBER)

Running uWSGI

After installing uWSGI with pip:

pip install uwsgi

You can spin up an instance with the following command:

uwsgi --http 0.0.0.0:8080 --wsgi-file service.py --callable app

This command tells uWSGI to run a server on 0.0.0.0 and port 8080, using the application located in service.py.

Using a Configuration File

Instead of passing all parameters via the command line, you can create a config file (e.g., app.ini):

[uwsgi]
http = 0.0.0.0:8080
module = app.service
callable = app
die-on-term = true
chdir = /home/aisummer/src/soft_eng_for_dl/
virtualenv = /home/aisummer/miniconda3/envs/Deep-Learning-Production-Course/
processes = 1
master = false
vacuum = true

To execute the server, run:

uwsgi app.ini

Wire Up Nginx as a Reverse Proxy

Next, we need to set up Nginx. First, install it:

sudo apt-get install nginx

Creating the Nginx Configuration File

Create a configuration file in the /etc/nginx/sites-available directory:

sudo nano /etc/nginx/sites-available/service.conf

Add the following minimal configuration:

server {
    listen 80;
    server_name 0.0.0.0;

    location {
        include uwsgi_params;
        uwsgi_pass unix:/home/aisummer/src/soft_eng_for_dl/app/service.sock;
    }
}

Linking and Starting Nginx

Link the configuration file to the sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/service /etc/nginx/sites-enabled

Test the Nginx configuration:

sudo nginx -t

If everything is set up correctly, you should be able to access your application via localhost.

Conclusion

In this article, we successfully set up a uWSGI server for our Flask application and hid it behind an Nginx reverse proxy. This configuration allows our Deep Learning application to scale to millions of users seamlessly. The next step is to deploy our application in the cloud using Docker Containers and Kubernetes, which we will cover in future articles.

As a side note, I highly recommend the TensorFlow: Advanced Techniques Specialization course by deeplearning.ai on Coursera for a foundational understanding of TensorFlow.

Resources

Deep Learning in Production Book 📖

Learn how to build, train, deploy, scale, and maintain deep learning models. Understand ML infrastructure and MLOps using hands-on examples. Learn more.

Disclosure: Please note that some of the links above might be affiliate links, and at no additional cost to you, we will earn a commission if you decide to make a purchase after clicking through.

By following this guide, you will not only enhance your technical skills but also prepare yourself for the challenges of a Machine Learning Engineer position. Happy coding!