Don't Kill the Process: Migrating Long-Running Jobs to tmux in Real Time

The server’s been syncing for 4 hours. Your phone rings. “The protest is coming down your street. Go home now.” You look at your terminal with horror. The VPN won’t last. If you lose this sync…

The Sysadmin’s Nightmare

It’s 3:47 PM on a Friday. You’ve been running a Greenbone Vulnerability Manager (GVM) feed sync for the past 2.5 hours. The progress bar crawls forward at an agonizing pace: 73%. Your boss pings you:

[URGENT] Need you in the conference room. Now. CEO is asking questions about the pentest.

You glance at your terminal. The sync is running directly in your shell, not in tmux, not in screen. The classic rookie mistake. You’ve been meaning to fix your dotfiles to auto-start tmux, but… well, here we are.

The meeting will require you to log out. Your dwm window manager needs a restart to apply urgent security patches. The protests downtown (“maandamano”) are intensifying, and the power grid is already flickering.

If you lose this sync, you’ll be starting from scratch. On Monday.

This is a story about salvaging the unsalvageable – rescuing processes from the jaws of death with surgical precision. No reboots. No tears. Just pure sysadmin wizardry.

The Problem: Real-World Process Interruption

You’re familiar with the scenario:

• A critical, long-running process (GVM sync, database migration, ML model training)
• Running in a bare terminal, not in tmux/screen (we all make mistakes)
• External pressure forcing you to close your session (meetings, power issues, system updates)
• The crushing weight of knowing you’ll lose hours of work if you kill the process

The standard advice – “always run important jobs in tmux” – doesn’t help when you’re already 2 hours deep into a bare-terminal process. What you need is a rescue operation.

The Lifeline: reptyr + tmux

Enter reptyr, the unsung hero of process management. This magical tool allows you to grab a running process by its PID and transplant it into a new terminal – specifically, into the safety of a tmux session.

The Step-by-Step Rescue Plan

1. Identify Your Process

First, you need to find the exact PID of your critical process:

ps aux | grep -i "feed\|sync\|rsync\|gvm"

Look for the actual worker process, not just the shell. For example:

user     12345  3.2  1.8 458796 151024 pts/3  S+   13:20   2:41 rsync -ltvrP gvm-feeds.example.com:/feeds /var/lib/gvm/

Here, you want PID 12345.

2. Install the Rescue Tools

In another terminal (don’t touch the one with your critical process!), install the necessary packages:

sudo apt install reptyr tmux procps

On Arch/Manjaro:

sudo pacman -S reptyr tmux procps-ng

3. Create a Safe Haven with tmux

Start a new tmux session that will become the new home for your process:

tmux new -s rescue

The -s rescue flag names your session, making it easier to find later if you have multiple sessions.

4. Prepare the System for Process Migration (Optional)

Some modern Linux kernels have security measures that might prevent process reparenting. If you encounter permission issues with reptyr, you might need to temporarily disable this protection:

echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

Important: This lowers security slightly. Remember to reset it after the rescue by running echo 1 | sudo tee /proc/sys/kernel/yama/ptrace_scope

5. Perform the Rescue

From within your new tmux session, run:

sudo reptyr 12345  # Replace with your actual PID

You might see some screen garbage – don’t panic! That’s just the process redrawing its output in the new terminal. After a moment, your long-running process should appear in the tmux session, continuing right where it left off.

6. Verify and Detach

Take a breath. Check that your process is running correctly in tmux. Now, the magic moment – detach from tmux with:

# Press Ctrl+b, then d

Your process is now safely running in a detached tmux session. You can log out, restart your window manager, deal with the power outage, or handle whatever emergency beckoned you away.

7. Return When You’re Ready

Hours later, when the crisis has passed and stability returns, simply:

tmux attach -t rescue

And there’s your process, still chugging along, blissfully unaware of the chaos you just navigated.

Real-Life Rescue Scenarios

The Unexpected Building Evacuation

Just as you begin a critical kernel compilation, the fire alarm blares. Evacuation is mandatory, and you have 3 minutes to secure everything. Your compilation is 30 minutes in, with an hour to go.

1. Quickly open a second terminal
2. ps aux | grep gcc
3. sudo apt install -y reptyr tmux (if not already installed)
4. tmux new -s kernelbuild
5. echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
6. sudo reptyr [PID]
7. Detach with Ctrl+b, d
8. Grab your laptop and evacuate

When you return and reconnect to your VPN, your compilation is waiting for you in tmux.

The DWM Patch Dilemma

You’ve been applying patches to your dwm build – alpha transparency and uselessgaps – when you realize you need to restart the window manager to test them. But your GVM vulnerability scan is running directly in a terminal.

1. Open a terminal in another workspace
2. ps aux | grep gvm
3. tmux new -s gvmscan
4. sudo reptyr [PID]
5. Detach from tmux
6. Restart dwm with confidence: Mod+Shift+Q
7. Log back in, tmux attach -t gvmscan

The “Maandamano” Power Crisis

The protests downtown have reached your area. The power is fluctuating, and your UPS is beeping warnings. Your database migration has been running for 90 minutes, and you’re only halfway done.

1. Quickly connect to your VPN from a mobile hotspot
2. SSH into the server
3. ps aux | grep -i migrate
4. tmux new -s dbmigration
5. sudo reptyr [PID]
6. Detach from tmux
7. If power fails, the server’s UPS keeps it running, and your SSH session may drop, but the process survives in tmux
8. Reconnect when possible and tmux attach -t dbmigration

Why This Works: The Technical Explanation

reptyr performs process reparenting by manipulating the process’s terminal connection. It:

1. Attaches to the target process using ptrace
2. Redirects the process’s stdin, stdout, and stderr to the new terminal
3. Detaches from the old terminal completely
4. Resumes execution in the new environment

This is like performing heart surgery on a running process – disconnecting its circulatory system from one body and connecting it to another, all while keeping it alive.

Pro Tips for the Process Surgeon

Prepare Your Environment in Advance

Add these to your .bashrc or .zshrc:

# Alias for quick process adoption
alias adopt='sudo reptyr'

# Function to quickly create a rescue session
rescue() {
  tmux new-session -d -s rescue
  echo "Rescue session created. Use 'tmux a -t rescue' to attach."
  echo "Then use 'adopt PID' to rescue a process."
}

Create a “Rescue Kit” Script

Save this as ~/bin/rescue-process.sh:

#!/bin/bash
# Process Rescue Kit - Interactive Version
# By Victor Nthuli

# Colors for better visibility
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
RED='\033[0;31m'
NC='\033[0m' # No Color

# Print banner
echo -e "${BLUE}╔════════════════════════════════════════╗${NC}"
echo -e "${BLUE}║${YELLOW}  PROCESS RESCUE OPERATION - INTERACTIVE  ${BLUE}║${NC}"
echo -e "${BLUE}╚════════════════════════════════════════╝${NC}"

# Check if PID is provided
if [ -z "$1" ]; then
  echo -e "${YELLOW}No PID specified. Let's find your process...${NC}"

  # Help user find their process
  read -p "Enter a keyword to search for your process: " KEYWORD
  echo -e "${BLUE}Searching for processes matching '${KEYWORD}'...${NC}"

  ps aux | grep -i "$KEYWORD" | grep -v grep

  read -p "Enter the PID you want to rescue: " PID

  if [ -z "$PID" ]; then
    echo -e "${RED}No PID entered. Exiting.${NC}"
    exit 1
  fi
else
  PID=$1
fi

# Verify the PID exists
if ! ps -p $PID > /dev/null; then
  echo -e "${RED}Error: Process $PID does not exist!${NC}"
  exit 1
fi

# Get process info for confirmation
PROCESS_INFO=$(ps -p $PID -o pid,ppid,user,cmd | tail -n 1)
echo -e "${YELLOW}About to rescue:${NC} $PROCESS_INFO"
echo

# Ask for session name or generate one
read -p "Enter a name for the tmux rescue session [rescue_$PID]: " SESSION
SESSION=${SESSION:-rescue_$PID}

# Check if tmux and reptyr are installed
echo -e "${BLUE}Checking required tools...${NC}"
MISSING_TOOLS=""

for cmd in tmux reptyr; do
  if ! command -v $cmd &> /dev/null; then
    MISSING_TOOLS="$MISSING_TOOLS $cmd"
  else
    echo -e "✓ ${GREEN}$cmd is installed${NC}"
  fi
done

# Install missing tools if any
if [ ! -z "$MISSING_TOOLS" ]; then
  echo -e "${YELLOW}Need to install:${MISSING_TOOLS}${NC}"
  read -p "Install missing tools? [Y/n] " INSTALL
  INSTALL=${INSTALL:-Y}

  if [[ $INSTALL =~ ^[Yy]$ ]]; then
    echo -e "${BLUE}Installing missing tools...${NC}"
    if command -v apt &> /dev/null; then
      sudo apt install -y $MISSING_TOOLS
    elif command -v pacman &> /dev/null; then
      sudo pacman -S --noconfirm $MISSING_TOOLS
    else
      echo -e "${RED}Couldn't determine package manager. Please install manually:${MISSING_TOOLS}${NC}"
      exit 1
    fi
  else
    echo -e "${RED}Cannot continue without required tools. Exiting.${NC}"
    exit 1
  fi
fi

# Check if ptrace needs to be enabled
PTRACE_SCOPE=$(cat /proc/sys/kernel/yama/ptrace_scope)
MODIFIED_PTRACE=0

if [ "$PTRACE_SCOPE" -ne "0" ]; then
  echo -e "${YELLOW}Current ptrace_scope is $PTRACE_SCOPE${NC}"
  echo -e "${YELLOW}You might need to temporarily allow ptrace for reptyr to work.${NC}"
  read -p "Temporarily modify ptrace_scope? [y/N] " MODIFY_PTRACE

  if [[ $MODIFY_PTRACE =~ ^[Yy]$ ]]; then
    echo -e "${BLUE}Temporarily allowing ptrace...${NC}"
    echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope > /dev/null
    MODIFIED_PTRACE=1
    echo -e "${GREEN}✓ ptrace temporarily allowed${NC}"
  else
    echo -e "${YELLOW}Continuing with current ptrace settings...${NC}"
  fi
fi

# Create tmux session
echo -e "${BLUE}Creating tmux session '$SESSION'...${NC}"
tmux new-session -d -s "$SESSION"
echo -e "${GREEN}✓ tmux session created${NC}"

# Show command to be run
echo -e "${BLUE}Will execute in tmux:${NC} sudo reptyr $PID"

# Confirm before proceeding
read -p "Ready to perform the rescue. Continue? [Y/n] " CONTINUE
CONTINUE=${CONTINUE:-Y}

if [[ ! $CONTINUE =~ ^[Yy]$ ]]; then
  echo -e "${RED}Operation cancelled.${NC}"
  tmux kill-session -t "$SESSION"

  # Restore ptrace if modified
  if [ $MODIFIED_PTRACE -eq 1 ]; then
    echo -e "${BLUE}Restoring ptrace security...${NC}"
    echo $PTRACE_SCOPE | sudo tee /proc/sys/kernel/yama/ptrace_scope > /dev/null
  fi

  exit 1
fi

# Perform the rescue
echo -e "${YELLOW}Attempting to rescue process $PID into tmux session '$SESSION'...${NC}"
tmux send-keys -t "$SESSION" "sudo reptyr $PID" C-m

echo -e "${BLUE}Waiting for reptyr to complete...${NC}"
sleep 3

# Check if process is still running
if ps -p $PID > /dev/null; then
  echo -e "${GREEN}✓ Process $PID is still running${NC}"
  echo -e "${GREEN}✓ Rescue operation likely successful!${NC}"
else
  echo -e "${RED}⚠ Process $PID is no longer running.${NC}"
  echo -e "${YELLOW}This could mean either:${NC}"
  echo -e "  - The rescue was successful and the process completed normally"
  echo -e "  - There was an issue with the rescue operation"
fi

# Restore ptrace setting if modified
if [ $MODIFIED_PTRACE -eq 1 ]; then
  echo -e "${BLUE}Restoring ptrace security...${NC}"
  echo $PTRACE_SCOPE | sudo tee /proc/sys/kernel/yama/ptrace_scope > /dev/null
  echo -e "${GREEN}✓ ptrace security restored${NC}"
fi

echo
echo -e "${GREEN}═════════════════════════════════════════${NC}"
echo -e "${GREEN}   NEXT STEPS:${NC}"
echo -e "${BLUE}   Attach to your session:${NC}"
echo -e "   tmux attach-session -t $SESSION"
echo
echo -e "${BLUE}   If you need to find it later:${NC}"
echo -e "   tmux list-sessions"
echo -e "${GREEN}═════════════════════════════════════════${NC}"

Make it executable:

chmod +x ~/bin/rescue-process.sh

Now you can rescue a process with a single command, with helpful interactive prompts along the way:

~/bin/rescue-process.sh 12345

Or even search for your process interactively:

~/bin/rescue-process.sh

The Ultimate Defensive Setup

For the truly paranoid (or those who’ve been burned too many times):

1. Auto-tmux: Configure your terminal to always start in tmux
2. Session persistence: Use tmux-resurrect to save sessions across reboots
3. Command history: Integrate atuin to track and sync shell history
4. Directory jumping: Add zoxide for quick navigation after reconnecting

This combination creates an almost indestructible workflow.

When Traditional Methods Fail

Sometimes, reptyr might not work, especially with heavily interactive applications. In these cases, try:

1. nohup: If the process can be paused (Ctrl+Z), you can try: bash bg # Background the process disown -h %1 # Detach it from the shell

2. cryopid: A more aggressive tool for freezing and thawing processes

3. checkpoint/restore: For critical systems, consider CRIU (Checkpoint/Restore In Userspace)

Conclusion: Never Lose a Process Again

            BEFORE REPTYR
            ──────────────
┌────────────┐
│  dwm tty1  │
└────┬───────┘
     │
     ▼
┌────────────┐
│ gvm-setup  │  PID: 9511 (running rsync)
└────────────┘


        ⛔ Can't restart dwm!
      Process is tied to tty1


            AFTER REPTYR
            ─────────────
┌────────────┐
│  tmux      │
│  session:  │
│  gvm       │
└────┬───────┘
     │
     ▼
┌────────────┐
│ gvm-setup  │  PID: 9511 (rsync)
└────────────┘


        ✅ Safe to restart dwm!
   Process now lives inside tmux 🎉

In the unpredictable world of system administration, where protests can cut power, CEOs demand immediate meetings, and window managers need urgent security patches, the ability to rescue running processes is a superpower.

With reptyr and tmux in your toolkit, you transform from a helpless observer of process death into a surgical interventionist, capable of rescuing hours of work from the very edge of oblivion.

The next time you’re staring at a terminal, watching a critical process run, and your phone rings with an emergency – smile. You now have the skills to save your work, handle the crisis, and return to find everything exactly as you left it.

Because in the end, the true measure of a sysadmin isn’t preventing all problems – it’s having the tools and knowledge to perform miracles when problems inevitably arise.

About the Author: Victor Nthuli is a battle-scarred sysadmin who has lost one too many processes to unexpected interruptions, and has finally found peace through the power of reptyr+tmux.