commit 0dd5cdb1665ebbdd7631727a1cf8420657bf0c68 Author: Leyrasoft Date: Sun May 31 18:27:53 2026 -0400 first commit diff --git a/README.md b/README.md new file mode 100644 index 0000000..b417df3 --- /dev/null +++ b/README.md @@ -0,0 +1,593 @@ +# m3u proxy + +![logo](./static/favicon.png) + +A high-performance HTTP proxy server for IPTV content with **true live proxying**, per-client connection management, and seamless failover support. Built with FastAPI and optimized for efficiency. + +### Questions/issues/suggestions + +Feel free to [open an issue](https://github.com/m3ue/m3u-proxy/issues/new?template=bug_report.md) on this repo, or hit us up on [Discord](https://discord.gg/rS3abJ5dz7) + +### Join us on Discord + +Join our [Discord](https://discord.gg/rS3abJ5dz7) server to ask questions and get help, help others, suggest new ideas, and offer suggestions for improvements! You can also try out and help test new features! πŸŽ‰ + +## Features + +### Core Streaming +- πŸš€ **Pure HTTP Proxy**: Zero transcoding, direct byte-for-byte streaming +- πŸ”— **Connection Sharing**: Multiple viewers share one upstream provider connection (HLS, Transcoded, and live Direct/TS streams) +- ⚑️ **Truly Ephemeral**: Provider connections open only when clients are consuming; close when the last client disconnects +- πŸ“Ί **HLS Support**: Optimized playlist and segment handling (.m3u8) +- πŸ“‘ **Continuous Streams**: Primary/subscriber broadcast for .ts, .mp4, .mkv, .webm, .avi live streams +- πŸ”„ **Real-time URL Rewriting**: Automatic playlist modification for proxied content +- πŸ“± **Full VOD Support**: Per-client connections with byte-range requests and seeking +- 🎬 **Strict Live TS Mode**: Enhanced stability for live MPEG-TS with pre-buffering & circuit breaker + +### Performance & Reliability +- ⚑ **uvloop Integration**: 2-4x faster async I/O operations +- πŸ”„ **Seamless Failover**: <100ms transparent URL switching per client +- 🎯 **Immediate Cleanup**: Connections close instantly when client stops + +### Transcoding Available with Hardware Acceleration +- 🎬 **FFmpeg Integration**: Built-in hardware-accelerated video processing +- πŸš€ **GPU Acceleration**: Automatic detection of NVIDIA, Intel, and AMD GPUs +- ⚑ **VAAPI Support**: Intel/AMD hardware encoding (3-8x faster than CPU) +- 🎯 **NVENC Support**: NVIDIA hardware encoding (10-20x faster than CPU) +- πŸ”§ **Auto-Configuration**: Zero-config hardware acceleration setup +- πŸ“Š **Multiple Codecs**: H.264, H.265/HEVC, VP8, VP9, AV1 support + +### Management & Monitoring +- πŸ‘₯ **Client Tracking**: Individual client sessions and bandwidth monitoring +- πŸ“Š **Real-time Statistics**: Live metrics on streams, clients, and data usage +- πŸ”Ž **Stream Type Detection**: Automatic HLS/VOD/Live detection +- 🧹 **Automatic Cleanup**: Inactive streams and clients auto-removed +- πŸ“£ **Event System**: Real-time events and webhook notifications +- 🩺 **Health Checks**: Built-in health endpoints for monitoring +- 🏷️ **Custom Metadata**: Attach arbitrary key/value pairs to streams for identification + +## Quick Start + +#### 🐳 Docker compose example + +Use the below example to run using the precompiled Dockerhub image. +You can also replace `latest` with `dev` or `experimental` to try another branch. + +```yaml +services: + m3u-proxy: + image: sparkison/m3u-proxy:latest + container_name: m3u-proxy + ports: + - "8085:8085" + # Hardware acceleration (optional) + devices: + - /dev/dri:/dev/dri # Intel/AMD GPU support + # For NVIDIA GPUs, use this instead: + # deploy: + # resources: + # reservations: + # devices: + # - driver: nvidia + # count: all + # capabilities: [gpu] + environment: + # Server Configuration + - M3U_PROXY_HOST=0.0.0.0 + - M3U_PROXY_PORT=8085 + - LOG_LEVEL=INFO + + # Base path (default: /m3u-proxy for m3u-editor integration) + # Set to empty string if not using reverse proxy: ROOT_PATH= + - ROOT_PATH=/m3u-proxy + + # Hardware acceleration (optional) + - LIBVA_DRIVER_NAME=i965 # For older Intel GPUs + # - LIBVA_DRIVER_NAME=iHD # For newer Intel GPUs + + # Timeouts (optional) + - CLIENT_TIMEOUT=300 + - CLEANUP_INTERVAL=60 + restart: unless-stopped + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8085/health"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 10s +``` + +## Building from source + +### 1. Clone Repo & Install Dependencies + +#### Prerequisits + +- `python` installed on your system: `>=3.10` +- `pip` installed on your system: `>=23` + +```bash +git clone https://github.com/sparkison/m3u-proxy.git && cd m3u-proxy +pip install -r requirements.txt +``` + +### 2. Start the Server + +```bash +python main.py --debug +``` + +Server will start on `http://localhost:8085` + +### 3. Create a Stream + +```bash +# HLS stream with custom user agent +curl -X POST "http://localhost:8085/streams" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://your-stream.m3u8", "user_agent": "MyApp/1.0"}' + +# Direct IPTV stream with failover +curl -X POST "http://localhost:8085/streams" \ + -H "Content-Type: application/json" \ + -d '{ + "url": "http://server.com/stream.ts", + "failover_urls": ["http://backup.com/stream.ts"], + "user_agent": "VLC/3.0.18" + }' + +# Using the CLI client +python m3u_client.py create "https://your-stream.m3u8" --user-agent "MyApp/1.0" +python m3u_client.py create "http://server.com/movie.mkv" --failover "http://backup.com/movie.mkv" +``` + +## API Documentation + +### Stream Management + +#### Create Stream +```bash +POST /streams +Content-Type: application/json + +{ + "url": "stream_url", + "failover_urls": ["backup_url1", "backup_url2"], + "user_agent": "Custom User Agent String" +} +``` + +#### List Streams +```bash +GET /streams +``` + +#### Stream Information +```bash +GET /streams/{stream_id} +``` + +#### Delete Stream +```bash +DELETE /streams/{stream_id} +``` + +#### Trigger Failover +```bash +POST /streams/{stream_id}/failover +``` + +### Statistics & Monitoring + +#### Comprehensive Stats +```bash +GET /stats +``` + +#### Health Check +```bash +GET /health +``` + +#### Client Information +```bash +GET /clients +GET /clients/{client_id} +``` + +## CLI Client Usage + +The included CLI client (`m3u_client.py`) provides easy access to all proxy features: + +```bash +# Create a stream with failover +python m3u_client.py create "https://primary.m3u8" --failover "https://backup1.m3u8" "https://backup2.m3u8" + +# List all active streams +python m3u_client.py list + +# View comprehensive statistics +python m3u_client.py stats + +# Monitor in real-time (updates every 5 seconds) +python m3u_client.py monitor + +# Check health status +python m3u_client.py health + +# Get detailed stream information +python m3u_client.py info + +# Trigger manual failover +python m3u_client.py failover + +# Delete a stream +python m3u_client.py delete +``` +## Configuration + +### Environment Variables + +```bash +# Server configuration +M3U_PROXY_HOST=0.0.0.0 +M3U_PROXY_PORT=8085 + +# Base path for API routes (useful for reverse proxy integration) +# Default: /m3u-proxy (optimized for m3u-editor integration) +# Set to empty string for root path +ROOT_PATH=/m3u-proxy + +# API Authentication (optional) +# Set API_TOKEN to require authentication for management endpoints +# Leave unset or empty to disable authentication +API_TOKEN=your_secret_token_here + +# Client timeout (seconds) +CLIENT_TIMEOUT=300 + +# Cleanup interval (seconds) +CLEANUP_INTERVAL=60 + +# Stream Retry Configuration (improves reliability for unstable connections) +# Number of retry attempts before failover or giving up +STREAM_RETRY_ATTEMPTS=3 +# Delay between retries (seconds) +STREAM_RETRY_DELAY=1.0 +# Total timeout across all retries (seconds, 0 to disable) +STREAM_TOTAL_TIMEOUT=60.0 +# Use exponential backoff for retry delays (false/true) +STREAM_RETRY_EXPONENTIAL_BACKOFF=false +# Timeout for receiving data chunks (seconds) +LIVE_CHUNK_TIMEOUT_SECONDS=15.0 + +# Sticky Session Handler (prevents playback loops with load-balanced providers) +# Locks to specific backend after redirect to maintain playlist consistency +USE_STICKY_SESSION=false +``` + +### API Authentication + +When `API_TOKEN` is set in the environment, all management endpoints require authentication via the `X-API-Token` header. This includes: + +- `/` - Root endpoint +- `/streams` - Create, list, get, delete streams +- `/stats/*` - All statistics endpoints +- `/clients` - Client management +- `/health` - Health check endpoint +- `/webhooks` - Webhook management +- `/streams/{stream_id}/failover` - Failover control +- `/hls/{stream_id}/clients/{client_id}` - Client disconnect + +**Stream endpoints (the actual streaming URLs) do NOT require authentication** since they are accessed by media players that identify streams via `stream_id`. + +Example with authentication: + +```bash +# Set your API token +export API_TOKEN="my_secret_token" + +# Method 1: Using header (recommended for API calls) +curl -X POST "http://localhost:8085/streams" \ + -H "Content-Type: application/json" \ + -H "X-API-Token: my_secret_token" \ + -d '{"url": "https://your-stream.m3u8"}' + +# Method 2: Using query parameter (useful for browser access) +curl -X POST "http://localhost:8085/streams?api_token=my_secret_token" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://your-stream.m3u8"}' + +# Browser access example +# Visit: http://localhost:8085/stats?api_token=my_secret_token + +# Without token - will get 401 error +curl -X POST "http://localhost:8085/streams" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://your-stream.m3u8"}' +``` + +To disable authentication, simply leave `API_TOKEN` unset or set it to an empty string. + +## Hardware Acceleration + +m3u-proxy includes comprehensive hardware acceleration support for video transcoding operations using FFmpeg with GPU acceleration. + +### Supported Hardware + +- **πŸ”₯ NVIDIA GPUs**: CUDA, NVENC, NVDEC (10-20x faster than CPU) +- **⚑ Intel GPUs**: VAAPI, QuickSync (QSV) (3-8x faster than CPU) +- **πŸš€ AMD GPUs**: VAAPI acceleration (3-5x faster than CPU) +- **πŸ’» CPU Fallback**: Software encoding when no GPU available + +### Automatic Detection + +The container automatically detects available hardware on startup: + +``` +πŸ” Running hardware acceleration check... +βœ… Device /dev/dri/renderD128 is accessible. +πŸ”° Intel GPU: Intel GPU (Device ID: 0x041e) +βœ… FFmpeg VAAPI acceleration: AVAILABLE +πŸ’‘ For older Intel GPUs, try: LIBVA_DRIVER_NAME=i965 +``` + +### Docker Setup Examples + +#### Intel/AMD GPU Support +```yaml +services: + m3u-proxy: + image: sparkison/m3u-proxy:latest + devices: + - /dev/dri:/dev/dri + environment: + - LIBVA_DRIVER_NAME=i965 # For older Intel GPUs + # - LIBVA_DRIVER_NAME=iHD # For newer Intel GPUs +``` + +#### NVIDIA GPU Support +```yaml +services: + m3u-proxy: + image: sparkison/m3u-proxy:latest + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] +``` + +#### Docker Run Commands +```bash +# Intel/AMD GPU +docker run -d --name m3u-proxy \ + --device /dev/dri:/dev/dri \ + -e LIBVA_DRIVER_NAME=i965 \ + -p 8085:8085 \ + sparkison/m3u-proxy:latest + +# NVIDIA GPU +docker run -d --name m3u-proxy \ + --gpus all \ + -p 8085:8085 \ + sparkison/m3u-proxy:latest +``` + +### Programming Interface + +The hardware acceleration is available through Python APIs: + +```python +from hwaccel import get_ffmpeg_hwaccel_args, is_hwaccel_available + +# Check if hardware acceleration is available +if is_hwaccel_available(): + # Get optimized FFmpeg arguments + hwaccel_args = get_ffmpeg_hwaccel_args("h264") + + # Example: Hardware-accelerated transcoding + cmd = ["ffmpeg"] + hwaccel_args + [ + "-i", "input_stream.m3u8", + "-c:v", "h264_vaapi", # Hardware encoder + "-preset", "fast", + "-b:v", "2M", + "output_stream.mp4" + ] +``` + +### Performance Benefits + +| Hardware | Encoding Speed | Concurrent Streams | CPU Usage Reduction | +|----------|---------------|-------------------|-------------------| +| NVIDIA GPU | 10-20x faster | 4-8 streams | 95%+ | +| Intel GPU | 3-8x faster | 2-4 streams | 90%+ | +| AMD GPU | 3-5x faster | 2-3 streams | 85%+ | +| CPU Only | Baseline | 1 stream | N/A | + +### Supported Codecs + +- **H.264/AVC**: High compatibility, universal support +- **H.265/HEVC**: Better compression, 4K/8K content +- **VP8/VP9**: WebM containers, streaming optimized +- **AV1**: Next-gen codec, best compression +- **MJPEG**: Low latency, surveillance applications + +For detailed hardware acceleration setup and troubleshooting, see [Hardware Acceleration Guide](docs/HARDWARE_ACCELERATION.md). + +### Server Startup Options + +```python +# Main server with all features +python main.py + +# With custom options +python main.py --port 8002 --debug --reload +``` + +## Troubleshooting + +### Common Issues + +1. **Stream Won't Load** + - Check original URL accessibility + - Verify CORS headers if accessing from browser + - Check server logs for detailed errors + +2. **High Memory Usage** + - Reduce `CLIENT_TIMEOUT` for faster cleanup + - Monitor client connections and cleanup inactive ones + - Consider horizontal scaling for high loads + +3. **Failover Not Working** + - Verify failover URLs are accessible + - Check failover trigger conditions in logs + - Test manual failover via API + +### Debug Mode + +```bash +# Enable detailed logging +export LOG_LEVEL=DEBUG +python main.py --debug +``` + +## Integration Examples + +### HTML5 Video Player +```html + +``` + +### FFmpeg +```bash +ffplay "http://localhost:8085/hls/{stream_id}/playlist.m3u8" +``` + +### VLC +```bash +vlc "http://localhost:8085/hls/{stream_id}/playlist.m3u8" +``` + +## πŸ“‘ Event System & Webhooks + +The proxy includes a comprehensive event system for monitoring and integration: + +### Webhook Configuration +```bash +# Add webhook to receive events +curl -X POST "http://localhost:8085/webhooks" \ + -H "Content-Type: application/json" \ + -d '{ + "url": "https://your-server.com/webhook", + "events": ["stream_started", "client_connected", "failover_triggered"], + "timeout": 10, + "retry_attempts": 3 + }' +``` + +### Available Events +- `stream_started` - New stream created +- `stream_stopped` - Stream ended +- `client_connected` - Client joined stream +- `client_disconnected` - Client left stream +- `failover_triggered` - Switched to backup URL + +### Webhook Payload Example +```json +{ + "event_id": "uuid", + "event_type": "stream_started", + "stream_id": "abc123", + "timestamp": "2025-09-25T22:38:34.392830", + "data": { + "primary_url": "http://example.com/stream.m3u8", + "user_agent": "MyApp/1.0" + } +} +``` + +### Demo Events +```bash +# Try the event system demo +python demo_events.py +``` + +πŸ“– **Full Documentation**: See [EVENT_SYSTEM.md](docs/EVENT_SYSTEM.md) for complete webhook integration guide. + +## Additional Documentation + +- **[Documentation Index](docs/README.md)** - Complete, always-current list of all documentation + +## Development + +### Project Structure +``` +β”œβ”€β”€ docker/ # Container and deployment assets +β”œβ”€β”€ docs/ # Full documentation set +β”‚ β”œβ”€β”€ README.md # Canonical docs index +β”‚ └── *.md # Architecture, failover, retry, auth, etc. +β”œβ”€β”€ logs/ # Runtime logs (local/dev) +β”œβ”€β”€ src/ +β”‚ β”œβ”€β”€ __init__.py # Package marker +β”‚ β”œβ”€β”€ api.py # FastAPI server application +β”‚ β”œβ”€β”€ broadcast_manager.py # Client broadcast coordination +β”‚ β”œβ”€β”€ config.py # Configuration management +β”‚ β”œβ”€β”€ events.py # Event system with webhooks +β”‚ β”œβ”€β”€ hwaccel.py # Hardware acceleration detection/helpers +β”‚ β”œβ”€β”€ models.py # Data models and schemas +β”‚ β”œβ”€β”€ pooled_stream_manager.py # Shared/pooling stream orchestration +β”‚ β”œβ”€β”€ redis_config.py # Redis settings +β”‚ β”œβ”€β”€ redis_manager.py # Redis coordination layer +β”‚ β”œβ”€β”€ stream_manager.py # Live proxy core (broadcast model for direct/TS streams) +β”‚ └── transcoding.py # FFmpeg transcoding pipeline +β”œβ”€β”€ static/ # Static assets (icons, images) +β”œβ”€β”€ tests/ # Test suite +β”‚ β”œβ”€β”€ integration/ # Integration tests +β”‚ └── test_*.py # Unit tests +β”œβ”€β”€ tools/ # Utility scripts and tools +β”‚ β”œβ”€β”€ performance_test.py # Performance testing +β”‚ β”œβ”€β”€ m3u_client.py # CLI client +β”‚ β”œβ”€β”€ demo_events.py # Event system demo +β”‚ └── run_tests.py # Enhanced test runner +β”œβ”€β”€ docker-compose.yml # Default compose stack +β”œβ”€β”€ Dockerfile # Container build definition +β”œβ”€β”€ main.py # Server entry point (uvloop support) +β”œβ”€β”€ requirements.txt # Python dependencies +β”œβ”€β”€ pytest.ini # Test configuration +└── README.md # This file +``` +--- + +## 🀝 Want to Contribute? + +> Whether it’s writing docs, squashing bugs, or building new features, your contribution matters! ❀️ + +We welcome **PRs, issues, ideas, and suggestions**!\ +Here’s how you can join the party: + +- Follow our coding style and best practices. +- Be respectful, helpful, and open-minded. +- Respect the **CC BY-NC-SA license**. + +1. Fork the repository +2. Create a feature branch +3. Add tests for new functionality +4. Submit a pull request + +--- + +## βš–οΈ License + +> m3u editor is licensed under **CC BY-NC-SA 4.0**: + +- **BY**: Give credit where credit’s due. +- **NC**: No commercial use. +- **SA**: Share alike if you remix. + +For full license details, see [LICENSE](https://creativecommons.org/licenses/by-nc-sa/4.0/). +