How to set the log level in a python application running in k8s

To set the log level in a Python application running in Kubernetes, you can use the built-in Python logging module. You can set the log level in your Python code by calling the “basicConfig()” method of the logging module and setting the level parameter to one of the following predefined levels:

  • DEBUG -INFO -WARNING -ERROR -CRITICAL

Here is an example of how you can set the log level to DEBUG:

import logging

logging.basicConfig(level=logging.DEBUG)

Another way is to set the log level through the environment variable and use it in the application. you can use a ConfigMap or Secret to store the environment variable and then mount it to the pod as an environment variable. You also can use a sidecar container to watch for configmap updates and propagate the new environment variable to the application without having to restart the pods.

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
data:
  LOG_LEVEL: "DEBUG"
---
apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  - name: mycontainer
    image: myimage
    env:
    - name: LOG_LEVEL
      valueFrom:
        configMapKeyRef:
          name: my-config
          key: LOG_LEVEL

then in your application, you can use os.environ.get('LOG_LEVEL') it to get the value of the environment variable.

import os
log_level = os.environ.get('LOG_LEVEL', 'INFO')
logging.basicConfig(level=log_level)

How to Use a sidecar to check for configMap updates automatically

To use a sidecar container to watch for ConfigMap updates and propagate them to the application without having to restart the pods, you can use a tool such as Envy.

Envy is an open-source tool that can run as a sidecar container and watch for changes to ConfigMap or Secrets and propagate them to the environment variables of a specified container.

Here is an example of how you can use Envy as a sidecar container in your Kubernetes deployment:

Create a ConfigMap that contains the log-level environment variable.

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
data:
  LOG_LEVEL: "DEBUG"

Create a Kubernetes deployment that includes both your application container and the Envy sidecar container.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-image
        envFrom:
        - configMapRef:
            name: my-config
      - name: envy
        image: envy
        args:
        - --configmap=my-config
        - --container=my-app

Envy container will watch for updates to the ConfigMap and propagate them to the environment variables of the specified container, in this case, the “my-app” container.

With this approach, when you update the ConfigMap, the Envy sidecar container will detect the change and update the environment variables of the application container, allowing you to update the log level without having to restart the pods.

Please note that Envy is just one example of a tool that you can use for this purpose, there are other tools like confd, envconsul, etc.

Will envy sidecar slow things down?

Envy, or any other sidecar container that watches for ConfigMap updates, may add some overhead to your application’s performance. The amount of overhead will depend on factors such as the frequency with which the ConfigMap is updated and the performance of the tool you are using.

However, the overhead is generally considered minimal and the benefits of being able to update the configuration without having to restart the pods usually outweigh the potential performance impact.

Additionally, you can optimize the performance of Envy or any other tool by configuring the polling interval, which is the frequency at which the tool checks for updates to the ConfigMap. A longer polling interval will result in less overhead but may also result in a longer delay before the new configuration takes effect.

It’s also worth noting that, Envy has a lightweight design and its performance impact will be negligible.

You can also monitor your application and resource usage, to evaluate if any additional optimization steps are required.

Conclusion

In conclusion, it’s important to keep the log level of your application in k8s configmap and use a sidecar to monitor the changes automatically.

Atiqur Rahman

I am MD. Atiqur Rahman graduated from BUET and is an AWS-certified solutions architect. I have successfully achieved 6 certifications from AWS including Cloud Practitioner, Solutions Architect, SysOps Administrator, and Developer Associate. I have more than 8 years of working experience as a DevOps engineer designing complex SAAS applications.

Leave a Reply