How to Add Elastic APM Integration in Python Without Using a Supported Framework
Elastic APM (Application Performance Monitoring) is a powerful tool for monitoring and observing the performance of your application. By using APM, you can get insights into transaction times, errors, and request tracing, among other things. While Elastic APM offers integrations for various frameworks like Django, Flask, and FastAPI, you can also use it without any framework.
In this post, we'll walk through how to integrate Elastic APM into a Python application that doesn't use any of the supported frameworks. By doing this, you’ll gain insights into your Python application's performance with minimal setup.
Prerequisites
Before you begin, ensure that you have the following:
- Elastic APM Server installed and running.
- Elastic APM Python agent installed.
- Python 3.x environment.
You can install the APM agent using pip:
pip install elastic-apm
Step 1: Set Up Your Elastic APM Configuration
The first step is configuring the Elastic APM agent. You'll need to connect your application to an APM server, so the agent can send performance data.
Create a file named
apm_config.py
to store your APM configuration:from elasticapm import Client # APM server URL APM_SERVER_URL = 'http://<APM Server URL>:8200' # Change this to your APM server URL # Service name and environment (optional) SERVICE_NAME = 'my-python-app' ENVIRONMENT = 'developmet' # Create an APM client client = Client({ 'SERVICE_NAME': SERVICE_NAME, 'SERVER_URL': APM_SERVER_URL, 'ENVIRONMENT': ENVIRONMENT, })
Step 2: Start Tracking Transactions and Errors
In Elastic APM, transactions represent units of work, such as HTTP requests, database queries, or any other operation you'd like to monitor. You can manually create transactions and capture performance data within your application.
For example, in your main application file (
app.py
), start a transaction for a function or block of code that you want to monitor:Example 1: Wrapping a Specific Function
import time from elasticapm import capture_span def slow_function(): """Simulate a slow function.""" time.sleep(2) print("Function completed.") def app_logic(): """Main application logic.""" with capture_span('slow-function'): slow_function() if __name__ == '__main__': app_logic()
In this example,
capture_span
is used to track the duration of the slow_function
execution. You can use this to track custom spans, such as database queries, external API calls, or other performance-intensive operations.Example 2: Wrapping the Entire Application in a Transaction
If you want to wrap the entire application in a single transaction, you can start a transaction that encompasses all of your code, making it easier to monitor the overall performance of the application.
import time from elasticapm import Client, capture_span # APM Configuration APM_SERVER_URL = 'http://localhost:8200' SERVICE_NAME = 'my-python-app' ENVIRONMENT = 'production' client = Client({ 'SERVICE_NAME': SERVICE_NAME, 'SERVER_URL': APM_SERVER_URL, 'ENVIRONMENT': ENVIRONMENT, }) def slow_function(): """Simulate a slow function.""" time.sleep(2) print("Function completed.") def app_logic(): """Main application logic.""" slow_function() if __name__ == '__main__': with client.begin_transaction('app_transaction'): app_logic()
In this second example, we use
begin_transaction
to start a transaction that wraps the entire execution of the app_logic
function. This is especially useful if you want to monitor the performance of the application as a whole, including all sub-functions and processes.Step 3: Capture Errors and Exceptions
To capture errors and exceptions, simply wrap your code in a try-except block. When an error occurs, the APM agent will automatically capture the exception and send it to the server.
try: # Simulate an error 1 / 0 except ZeroDivisionError as e: client.capture_exception() print("Error captured by APM:", e)
In this case, the division by zero will trigger a
ZeroDivisionError
, which will be captured by Elastic APM and sent to your APM server.Step 4: Manually Capture Custom Metrics
Elastic APM allows you to track custom metrics in your application. For example, if you want to track the number of successful database queries, you can capture these metrics like so:
from elasticapm import capture_metrics def track_custom_metric(): # Track a custom metric (e.g., successful database queries) capture_metrics('db.successful_queries', value=1) track_custom_metric()
This will capture the custom metric
db.successful_queries
with a value of 1
each time it's invoked. You can visualize this metric in Kibana alongside your other APM data.Step 5: Sending Performance Data to the APM Server
Finally, ensure that your APM agent is sending performance data to your APM server. This can be done by simply starting the application. The data will automatically be sent to the APM server based on the configurations you provided in
apm_config.py
.Example of a Full Python Application with APM
Below is an example of a simple Python application that integrates Elastic APM for performance tracking and error monitoring:
import time from elasticapm import Client, capture_span, capture_exception # APM Configuration APM_SERVER_URL = 'http://localhost:8200' SERVICE_NAME = 'my-python-app' ENVIRONMENT = 'production' client = Client({ 'SERVICE_NAME': SERVICE_NAME, 'SERVER_URL': APM_SERVER_URL, 'ENVIRONMENT': ENVIRONMENT, }) def slow_function(): """Simulate a slow function.""" time.sleep(2) print("Function completed.") def app_logic(): """Main application logic.""" slow_function() def simulate_error(): """Simulate an error.""" try: 1 / 0 except ZeroDivisionError as e: client.capture_exception() print("Error captured by APM:", e) if __name__ == '__main__': # Wrap the entire application logic in a transaction with client.begin_transaction('app_transaction'): app_logic() simulate_error()
Step 6: Visualize in Kibana
Once you have the application running, you can log into Kibana and navigate to the APM tab to visualize your application's performance metrics. You'll be able to see your transactions, errors, and custom metrics in real time.
Conclusion
Integrating Elastic APM into a Python application without using any supported framework is a straightforward process. By following the steps outlined above, you can easily add performance monitoring to your application, track errors, and capture custom metrics.
Elastic APM provides a detailed view of your application's behavior, which is crucial for improving performance and identifying issues before they impact your users.