Multiprocessing and PID 1 Issue in Docker

When running a multiprocessing application in a Docker container, you might encounter strange “zombie processes” (defunct) that never disappear. This is often caused by the fact that Docker containers do not run a traditional init process (PID 1) by default. Let’s break down why this happens, and how to fix it.

PID 1 in Linux

In a regular Linux system, PID 1 is the init (or systemd) process. Besides bootstrapping the system and starting services, it has an important job:

Reaping zombie processes — processes that have exited, but whose parent has not yet collected their exit status (using wait() or poll() function).

If the original parent process exits, its children become orphans and are adopted by PID 1. When these adopted processes eventually exit, they first enter the zombie state until PID 1 calls wait() to clean them up.

What happens in Docker

In a Docker container, the process you start with CMD or ENTRYPOINT becomes PID 1.

There is no init process unless you explicitly run one. And most application processes do not implement zombie reaping.

Here’s a real-world example I encountered:

Docker container:
|→ python web_gui.py  (PID 1 - container [CMD] process)
   |→ python main.py        # main process
      |→ workers...         # multiprocessing workers processes
      |→ resource_tracker   # multiprocessing resource manager

• The container starts web_gui.py as PID 1 to provide a UI.
• When a user triggers a task, web_gui.py launches main.py.
• main.py spawns several workers doing actual tasks, and Python multiprocessing auto-creates a resource_tracker process for shared resources (e.g., semaphores, shared memory, quque)

The zombie process problem

1. If a worker process crashes, main.py can catch the error and exit.
2. The resource_tracker process is managed by Python internally. When it sees that all its tracked processes are gone, it also exits.

Here’s the problem:

When main.py exits, the resource_tracker’s parent process becomes PID 1 — in this case, web_gui.py.
But web_gui.py is not written to reap child processes.
So resource_tracker becomes a zombie (defunct) and stays in the process table forever.

Fix: Use tini as PID 1

The solution is to run a minimal init process inside your container. tini is a popular, tiny init system for containers. It:

• Spawns your actual application process
• Reaps orphaned zombie processes automatically

Example Dockerfile snippet:

FROM python:3.11-slim

# Install tini
RUN apt-get update && apt-get install -y tini && apt-get clean

ENTRYPOINT ["/usr/bin/tini", "--"]

CMD ["python", "web_gui.py"]

Now PID 1 is tini, which will clean up any zombies left behind.

Zombie vs Orphan processes

Zombie process:

• The process has exited (no CPU or memory usage except for a small process table entry).
• Its parent has not yet called wait() to collect the exit status.
• Solution: Make the parent process call wait() or poll().

Orphan process:

• The parent has exited, but the process is still running.
• It gets adopted by PID 1.
• It will keep running until it finishes by itself (or is killed).
• Solution: In child processes, periodically check if the parent is alive and exit if not.

Takeaway

If you run a multi-process application in Docker without an init process, PID 1 will be your application, and it probably won’t reap zombies.

Always consider running tini or another init system as PID 1 to keep your container clean and avoid process table leaks.