WaitLock is a portable UNIX/POSIX command-line tool that provides mutex and semaphore functionality for shell scripts. It enables synchronized access to resources across multiple processes with automatic cleanup when processes die.
Features
- Mutex Mode: Single lock holder (default)
- Semaphore Mode: Multiple concurrent lock holders
- Automatic Cleanup: Locks released when process dies
- Signal-based Release: Clean lock release with
--doneflag - CPU-aware Locking: Can scale locks to CPU count
- Lock Inspection: List and check active locks
- Multiple Output Formats: Human, CSV, and null-separated
- Command Execution: Run commands while holding locks
- UNIX Integration: Environment variables, stdin, syslog
- Portable C Implementation: Runs on any POSIX system
Quick Start
# Install dependencies (Ubuntu/Debian) sudo apt-get install build-essential autoconf # Build and install ./configure make sudo make install # Basic usage - acquire exclusive lock (RECOMMENDED) waitlock myapp || { echo "Another instance is already running" exit 1 } # ... do exclusive work ... # Lock automatically released when script exits # Execute command with lock (BEST PRACTICE) waitlock database_backup --exec "/usr/local/bin/backup.sh --daily" # List active locks waitlock --list
Table of Contents
- Installation
- Usage
- Examples
- Command Reference
- Environment Variables
- Exit Codes
- Advanced Usage
- Contributing
- License
Installation
From Source
Prerequisites
- C compiler (gcc, clang, or compatible)
- GNU Make
- autoconf (for building from git)
Build Instructions
# Clone the repository git clone https://github.com/user/waitlock.git cd waitlock # Generate configure script (if building from git) autoreconf -fi # Configure and build ./configure make # Run tests make check # Install system-wide sudo make install # Or install to custom prefix ./configure --prefix=/usr/local make install
Build Options
# Debug build ./configure CFLAGS="-g -O0 -DDEBUG" # Release build with optimizations ./configure CFLAGS="-O2 -DNDEBUG" # Cross-compilation example ./configure --host=arm-linux-gnueabihf
Package Installation
# Ubuntu/Debian (when available) sudo apt-get install waitlock # CentOS/RHEL (when available) sudo yum install waitlock # macOS with Homebrew (when available) brew install waitlock
Usage
Basic Syntax
waitlock [options] <descriptor> waitlock --list [--format=<fmt>] [--all|--stale-only] waitlock --check <descriptor> echo <descriptor> | waitlock [options]
Simple Examples
# Acquire mutex lock (RECOMMENDED APPROACH) waitlock myapp || { echo "Another instance is already running" exit 1 } # ... do exclusive work ... # Lock automatically released when script exits # Check if lock is available if waitlock --check myapp; then echo "Lock is available" else echo "Lock is held by another process" fi # Execute command while holding lock (BEST PRACTICE) waitlock backup_job --exec rsync -av /source /destination # Use with timeout waitlock --timeout 30 critical_resource || echo "Timeout!"
⚠️ Important: Background Usage Warning
DO NOT use background execution (&) for script coordination!
# ❌ WRONG - Don't do this for script coordination: waitlock myapp & # This returns immediately, whether lock was acquired or not # Both scripts may think they got the lock # Requires complex PID management and cleanup # ✅ CORRECT - Use foreground execution: waitlock myapp || { echo "Another instance is already running" exit 1 } # This blocks until lock is acquired or fails # Clear success/failure indication # Automatic cleanup when script exits
Why foreground is better:
- ✅ Reliable - Clear success/failure indication
- ✅ Simple - No PID management or manual cleanup needed
- ✅ Safe - No race conditions
- ✅ Automatic - Lock released when process exits
When to use background (&):
⚠️ Only for testing - When you need to verify lock behavior⚠️ Never for production - Use--execor foreground instead
Examples
1. Basic Mutex (Exclusive Access)
#!/bin/bash # Ensure only one backup process runs at a time waitlock database_backup || { echo "Another backup is already running" exit 1 } # Perform backup mysqldump --all-databases > backup.sql gzip backup.sql # Lock automatically released when script exits
2. Semaphore (Multiple Concurrent Access)
#!/bin/bash # Allow up to 4 concurrent download processes waitlock --allowMultiple 4 download_pool || { echo "Too many downloads already running" exit 1 } # Perform download wget "https://example.com/file.tar.gz" # Lock automatically released when script exits
3. CPU-Based Semaphore
#!/bin/bash # Use one lock per CPU core, reserving 2 cores for system waitlock --onePerCPU --excludeCPUs 2 cpu_intensive_task || { echo "All CPU slots are busy" exit 1 } # Run CPU-intensive task ./compute_job.sh
4. Command Execution Mode
#!/bin/bash # Execute command while holding lock (recommended approach) waitlock database_backup --exec bash -c " mysqldump --all-databases > backup.sql gzip backup.sql echo 'Backup completed' "
5. Lock Monitoring and Management
#!/bin/bash # Monitor active locks # List all locks in human-readable format waitlock --list # List in CSV format for parsing waitlock --list --format csv # Show only stale locks waitlock --list --stale-only # Count active locks waitlock --list --format csv | tail -n +2 | wc -l
6. Pipeline and Batch Processing
#!/bin/bash # Process files with controlled parallelism find /data -name "*.csv" | while read file; do basename "$file" | waitlock --allowMultiple 3 --exec process_file "$file" done # Or with xargs for better performance find /data -name "*.csv" | \ xargs -P 10 -I {} sh -c 'waitlock -m 3 batch_processor --exec "process_file {}"'
7. Using with Environment Variables
#!/bin/bash # Configure via environment variables export WAITLOCK_TIMEOUT=60 export WAITLOCK_DIR="/var/lock/myapp" export WAITLOCK_DEBUG=1 waitlock myapp_task --syslog --syslog-facility local0
8. Error Handling
#!/bin/bash # Robust error handling waitlock --timeout 30 critical_resource case $? in 0) echo "Lock acquired successfully" ;; 1) echo "Lock is busy" >&2; exit 1 ;; 2) echo "Timeout expired" >&2; exit 1 ;; 3) echo "Usage error" >&2; exit 1 ;; *) echo "Unexpected error" >&2; exit 1 ;; esac # Your critical section here perform_critical_operation
9. Signal-based Lock Release
#!/bin/bash # Clean lock release using --done flag (for testing/debugging) # ⚠️ Note: This is mainly for testing - use --exec for production # Start long-running process with lock (background for demonstration) waitlock long_running_task & LOCK_PID=$! # Simulate some work sleep 2 # Later, signal the process to release the lock cleanly waitlock --done long_running_task # Wait for the process to exit gracefully wait $LOCK_PID echo "Process exited with code: $?" # ✅ BETTER APPROACH: Use --exec for production # waitlock long_running_task --exec "./long_running_script.sh"
10. Resource Pool Management
#!/bin/bash # Manage GPU resources (RECOMMENDED APPROACH) # Acquire semaphore slot and run computation waitlock --allowMultiple 4 gpu_pool || { echo "All GPU slots are busy" exit 1 } # Use WAITLOCK_SLOT environment variable for GPU selection export CUDA_VISIBLE_DEVICES=$WAITLOCK_SLOT ./gpu_computation.py # Lock automatically released when script exits # ✅ ALTERNATIVE: Use --exec for cleaner approach # waitlock --allowMultiple 4 gpu_pool --exec bash -c ' # export CUDA_VISIBLE_DEVICES=$WAITLOCK_SLOT # ./gpu_computation.py # '
11. Distributed Locking (NFS)
#!/bin/bash # Coordinate across multiple machines using NFS export WAITLOCK_DIR="/mnt/shared/locks" waitlock cluster_job --timeout 300 --exec bash -c " echo 'Running on $(hostname)' ./distributed_task.sh "
Command Reference
Core Options
| Option | Description |
|---|---|
-m, --allowMultiple N |
Allow N concurrent holders (semaphore mode) |
-c, --onePerCPU |
Allow one lock per CPU core |
-x, --excludeCPUs N |
Reserve N CPUs (reduce available locks by N) |
-t, --timeout SECS |
Maximum wait time before giving up |
--check |
Test if lock is available without acquiring |
--done |
Signal lock holder to release lock (sends SIGTERM) |
-e, --exec CMD |
Execute command while holding lock |
Output Options
| Option | Description |
|---|---|
-q, --quiet |
Suppress all non-error output |
-v, --verbose |
Verbose output for debugging |
-f, --format FMT |
Output format: human, csv, null |
--syslog |
Log operations to syslog |
--syslog-facility FAC |
Syslog facility (daemon|local0-7) |
Management Options
| Option | Description |
|---|---|
-l, --list |
List active locks and exit |
-a, --all |
Include stale locks in list |
--stale-only |
Show only stale locks |
Configuration Options
| Option | Description |
|---|---|
-d, --lock-dir DIR |
Directory for lock files |
-h, --help |
Show usage information |
-V, --version |
Show version information |
Environment Variables
| Variable | Description | Default |
|---|---|---|
WAITLOCK_DIR |
Lock directory path | auto-detect |
WAITLOCK_TIMEOUT |
Default timeout in seconds | infinite |
WAITLOCK_DEBUG |
Enable debug output | disabled |
WAITLOCK_SLOT |
Preferred semaphore slot | auto |
Environment Variable Examples
# Set default timeout export WAITLOCK_TIMEOUT=300 # Use custom lock directory export WAITLOCK_DIR="/var/lock/myapp" # Enable debug output export WAITLOCK_DEBUG=1 # Prefer specific semaphore slot export WAITLOCK_SLOT=2
Exit Codes
| Code | Meaning |
|---|---|
| 0 | Success |
| 1 | Lock is busy |
| 2 | Timeout expired |
| 3 | Usage error |
| 4 | System error |
| 5 | Permission denied |
| 6 | Lock directory not accessible |
| 75 | Temporary failure |
| 126 | Command not executable |
| 127 | Command not found |
Advanced Usage
Syslog Integration
# Log all operations to syslog waitlock --syslog --syslog-facility local0 myapp # Monitor syslog for lock operations tail -f /var/log/syslog | grep waitlock
Lock File Format
WaitLock uses binary lock files with the following structure:
- Magic number (0x57414C4B = "WALK")
- Process metadata (PID, PPID, UID)
- Lock information (type, slot, max holders)
- Timestamps and command line
- CRC32 checksum for integrity
Platform Support
WaitLock is tested on:
- Linux (glibc, musl)
- FreeBSD
- OpenBSD
- NetBSD
- macOS
Performance Considerations
- Lock files are stored in
/var/lock/waitlock(system) or/tmp/waitlock(user) - Directory scanning is O(n) where n = number of lock files
- Use hierarchical descriptors for namespace separation
- Consider tmpfs for high-frequency locking
Troubleshooting
Common Issues
-
Permission Denied
# Check directory permissions ls -la /var/lock/waitlock # Use user-specific directory export WAITLOCK_DIR="$HOME/.waitlock"
-
Stale Locks
# List stale locks waitlock --list --stale-only # Clean up automatically (locks are cleaned on next access) waitlock --check any_descriptor
-
High Contention
# Monitor lock contention waitlock --verbose --timeout 1 busy_resource # Use exponential backoff (built-in) waitlock --timeout 60 busy_resource
Debug Mode
# Enable debug output export WAITLOCK_DEBUG=1 waitlock --verbose myapp # Or use command line waitlock --verbose myapp
Contributing
Development Setup
# Clone repository git clone https://github.com/user/waitlock.git cd waitlock # Install development dependencies sudo apt-get install autoconf automake libtool # Generate build files autoreconf -fi # Configure for development ./configure --enable-debug CFLAGS="-g -O0" # Build and test make make check
Running Tests
# Run internal test suite ./src/waitlock --test # Run shell-based tests ./test_basic.sh ./test_semaphore.sh ./test_timeout.sh
Code Style
- Follow POSIX C89/C90 standards
- Use 4-space indentation
- Include comprehensive error handling
- Add tests for new features
Submitting Changes
- Fork the repository
- Create a feature branch
- Make changes with tests
- Submit a pull request
License
WaitLock is released under the MIT License. See LICENSE for details.
Support
- Documentation: See
man waitlockafter installation - Issues: Report bugs on GitHub Issues
- Discussions: Join discussions on GitHub Discussions
Acknowledgments
WaitLock was designed following UNIX philosophy principles and inspired by tools like flock(1), lockfile(1), and sem(1). Special thanks to the POSIX standards committee for providing a solid foundation for portable system programming.