Monitoring Laravel-Apache using Datadog APM

Description:

Set up PHP Laravel app on Apache web server using Docker Compose and monitoring it to Datadog. This project guide you setting up Laravel, setting up Datadog agent on Docker, and setting up APM on PHP with Apache server. It helps you gain experience with Datadog APM, Laravel, Apache web server, and docker compose.

Tech Stack:


- Datadog
- Datadog APM
- PHP
- Laravel
- Apache
- Docker

Features:

Laravel application deployment using Apache on Docker Compose

Datadog agent deployment on Docker

Datadog APM service catalog

Table of Contents 1: System Architecture 2: Setup API Key on Datadog 3: Run Datadog Agent on Docker Host 4: Set Up Laravel using Docker 4.1: Dockerfile using PHP Apache 4.2 Docker Compose 4.3 Up and Running Troubleshoot

1: System Architecture

Laravel is a PHP framework for building web application. It is integrate well with database engine such as MySQL using MVC to migrate well the model to database.

Datadog itself is a cloud-based monitoring and analytics platform. From it we can observe infrastructure, applications, and logs in real-time.

Datadog have a feature called Datadog APM that provides end-to-end visibility into application performance. It gather application in forms of traces, metrics, and logs so application developer can identify bottlenecks and issues on running applications.

To collects metrics, logs, and traces and forward them to Datadog cloud platform, we need a lightweight software installed on servers and that is the job of Datadog Agent.

With that all components, take a look at following architecture we’re going to demonstrate.

2: Setup API Key on Datadog

First we need to set up credentials that will be used by Datadog Agent so they can push data into Datadog.

On Datadog platform:

Navigate to Organization settings, then click the API Keys tab.

Click the New Key button

Enter a name for your key or token

Click Create API Key

Copy the API Key and save it, we’re going to use it on next step when running Datadog Agent

3: Run Datadog Agent on Docker Host

Before we’re move to use APM feature, let’s run agent on our Docker host, use the API Key we created, and see the infrastructure observability data is able to enter the Datadog.

Get the latest Datadog Agent docker image.


docker pull datadog/agent:latest

Create network first


docker network create backend-network

Run using this command:


docker run -d \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \     # Access Docker metrics
  -v /proc/:/host/proc/:ro \                            # Access host metrics
  -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \           # Access cgroup metrics
  -p 0.0.0.0:8126:8126/tcp \                            # Expose APM port
  -e DD_API_KEY=<your_api_key> \                        # Your Datadog API key
  -e DD_SITE="datadoghq.com" \                          # Datadog region (US or EU)
  -e DD_APM_ENABLED=true \                              # Enable APM
  -e DD_FORWARDER_TIMEOUT=20 \                          # Set forwarder timeout
  --name datadog-agent \                                # Name the container
  --network backend-network \                           # Optional network
  datadog/agent:latest                                  # Datadog Agent image

4: Set Up Laravel using Docker

With Docker, we can board our applications without worried about dependencies and setup initial configurations. Laravel itself has already provide boilerplate code available on https://github.com/laravel/laravel.git . Let’s clone it and set to the development environment.


git clone https://github.com/laravel/laravel.git
cd laravel
git checkout -b dev-env

This boilerplate is initial Laravel framework project structure that can be setup using composer. Compose is a dependency manager for PHP that allows us downloads and installs the dependencies automatically.

From this boilerplate, we’re going to add docker-compose.yml and Dockerfile into the root of it. The idea is we’ll building the image as well as running docker compose commands from the main application folder, while run folder contains necessary config and local database for development.

We’re going to mount docker volumes so we can keep the source, the vendor dependencies and local development database in our host, while all the runtime like Apache and PHP are kept and managed by the container.

4.1: Dockerfile using PHP Apache

We’re going to use php:7.2-apache image from the php Dockerhub that is configureable and with functional Apache webserver. Additionally we’re going to install extensions as well as the Datadog extension to enable us run Laravel application and sent data to Datadog.


# Dockerfile
FROM php:7.2-apache

# 1. Install development packages and clean up apt cache.
RUN apt-get update && apt-get install -y \
    curl \
    g++ \
    git \
    libbz2-dev \
    libfreetype6-dev \
    libicu-dev \
    libjpeg-dev \
    libmcrypt-dev \
    libpng-dev \
    libreadline-dev \
    sudo \
    unzip \
    zip \
 && rm -rf /var/lib/apt/lists/*

# 2. Apache configs + document root.
ENV APACHE_DOCUMENT_ROOT=/var/www/html/public
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf

# Insert Datadog environment variables into the VirtualHost configuration
RUN sed -i '/<VirtualHost \*:80>/a \\n \
    SetEnv DD_SERVICE "laravel-apache-docker"\n \
    SetEnv DD_ENV "dev"\n \
    SetEnv DD_VERSION "1.0.0"\n \
    SetEnv DD_PROFILING_ENABLED "true"\n \
    SetEnv DD_TRACE_ENABLED "true"\n \
    SetEnv DD_AGENT_HOST "datadog-agent"\n \
    SetEnv DD_TRACE_AGENT_PORT "8126"\n' /etc/apache2/sites-available/000-default.conf

# 3. mod_rewrite for URL rewrite and mod_headers for .htaccess extra headers like Access-Control-Allow-Origin-
RUN a2enmod rewrite headers

# 4. Start with base PHP config, then add extensions.
RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini"

RUN docker-php-ext-install \
    bcmath \
    bz2 \
    calendar \
    iconv \
    intl \
    mbstring \
    opcache \
    pdo_mysql \
    zip

# Install this extension for setting up Datadog
COPY datadog-setup.php .

RUN php datadog-setup.php --php-bin=all --enable-profiling

# 5. Composer.
COPY --from=composer:latest /usr/bin/composer /usr/bin/composer

# 6. We need a user with the same UID/GID as the host user
# so when we execute CLI commands, all the host file's permissions and ownership remain intact.
# Otherwise commands from inside the container would create root-owned files and directories.
ARG uid
RUN useradd -G www-data,root -u $uid -d /home/devuser devuser
RUN mkdir -p /home/devuser/.composer && \
    chown -R devuser:devuser /home/devuser

Let’s take a look at it. The php-apache image by default set document root to /var/www/html . However, since Laravel index.php is inside /var/www/htmp/public , we need to edit the apache config as well as sites-available.


# Dockerfile
...
ENV APACHE_DOCUMENT_ROOT=/var/www/html/public
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
...

Then for our PHP that run on Apache application can be identified as APM Server, we need to add following parameters into Apache configuration.


# Insert Datadog environment variables into the VirtualHost configuration
RUN sed -i '/<VirtualHost \*:80>/a \\n \
    SetEnv DD_SERVICE "laravel-apache-docker"\n \
    SetEnv DD_ENV "dev"\n \
    SetEnv DD_VERSION "1.0.0"\n \
    SetEnv DD_PROFILING_ENABLED "true"\n \
    SetEnv DD_TRACE_ENABLED "true"\n \
    SetEnv DD_AGENT_HOST "datadog-agent"\n \
    SetEnv DD_TRACE_AGENT_PORT "8126"\n' /etc/apache2/sites-available/000-default.conf

Basically what this command does is to add the essential properties for Datadog agent can identifed Laravel application. It should be installed into /etc/apache2/sites-available/*.conf file with following as end result:


<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        SetEnv DD_SERVICE 'laravel-apache-docker'
        SetEnv DD_ENV 'dev'
        SetEnv DD_VERSION '1.0.0'
        SetEnv DD_PROFILING_ENABLED 'true'
        SetEnv DD_TRACE_ENABLED 'true'
        SetEnv DD_AGENT_HOST 'datadog-agent'
        SetEnv DD_TRACE_AGENT_PORT '8126'
        SetEnv DD_TRACE_DEBUG=true

        ServerAdmin webmaster@localhost
        DocumentRoot ${APACHE_DOCUMENT_ROOT}

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

For Datadog PHP extensions, what we’re doing here is to copy the datadog-setup.php to container directory and running it with flag:


# Dockerfile
...
# Install this extension for setting up Datadog
COPY datadog-setup.php .

RUN php datadog-setup.php --php-bin=all --enable-profiling
...

For Laravel points, what we’re doing is fetching the composer binary located at /usr/bin/composer/ from composer:latest docker image.


# Dockerfile
...
COPY --from=composer:latest /usr/bin/composer /usr/bin/composer
...

Lastly, since we’re going to mount the application source code from host into the container for development, any command run from within the container CLI shouldn’t affect host files/folder ownership.


# Dockerfile
...
ARG uid
RUN useradd -G www-data,root -u $uid -d /home/devuser devuser
RUN mkdir -p /home/devuser/.composer && \
    chown -R devuser:devuser /home/devuser

4.2 Docker Compose

The container for our webserver is set. Now we need a way to run it alongside with the database.


# docker-compose.yml
services:
  laravel-app:
    build:
      context: '.'
      args:
        uid: ${UID}
    container_name: laravel-app
    environment:
      - APACHE_RUN_USER=#${UID}
      - APACHE_RUN_GROUP=#${UID}
    volumes:
      - .:/var/www/html
    ports:
      - 8000:80
    networks:
      backend:
        aliases:
          - laravel-app

  mysql-db:
    image: mysql:5.7
    container_name: mysql-db
    volumes:
      - ./run/var:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=securerootpassword
      - MYSQL_DATABASE=db
      - MYSQL_USER=dbuser
      - MYSQL_PASSWORD=secret
    networks:
      backend:
        aliases:
          - db

networks:
  backend:
    external:
      name: backend-network

4.3 Up and Running

Now let’s build and up it. First, let’s add the datadog-setup.php first into our directory:

Check whether the Datadog agent extension already properly installed into our application:


php -m | grep ddtrace
php -m | grep ddprof

Troubleshoot

If you change the Apache configuration and install PHP extension later after app running, you can restart the Apache afterward:


service apache2 [restart|status]

Check up logs to check whether it receive the data from APM and send to datadoghq.com successfully:


docker logs datadog-agent

If the Datadog agent properly setup to receive APM data from application, it will get this logs after we hit the application:


2024-11-14 07:41:43 UTC | TRACE | INFO | (pkg/trace/info/stats.go:98 in LogAndResetStats) | [lang:php lang_version:7.2.34 interpreter:apache2handler tracer_version:1.4.2 endpoint_version:v0.4 service:laravel-apache-docker] -> traces received: 8, traces filtered: 0, traces amount: 48339 bytes, events extracted: 0, events sampled: 0

And you should see this logs to validate that the Datadog agent successfully sends data to Datadog platform:


2024-11-14 07:42:43 UTC | PROCESS | INFO | (comp/forwarder/defaultforwarder/transaction/transaction.go:455 in internalProcess) | Successfully posted payload to "https://process.datadoghq.com/api/v1/container" (202 Accepted)