first commit

This commit is contained in:
2026-05-31 18:27:53 -04:00
commit 0dd5cdb166

593
README.md Normal file
View File

@@ -0,0 +1,593 @@
# m3u proxy
![logo](./static/favicon.png)
A high-performance HTTP proxy server for IPTV content with **true live proxying**, per-client connection management, and seamless failover support. Built with FastAPI and optimized for efficiency.
### Questions/issues/suggestions
Feel free to [open an issue](https://github.com/m3ue/m3u-proxy/issues/new?template=bug_report.md) on this repo, or hit us up on [Discord](https://discord.gg/rS3abJ5dz7)
### Join us on Discord
Join our [Discord](https://discord.gg/rS3abJ5dz7) server to ask questions and get help, help others, suggest new ideas, and offer suggestions for improvements! You can also try out and help test new features! 🎉
## Features
### Core Streaming
- 🚀 **Pure HTTP Proxy**: Zero transcoding, direct byte-for-byte streaming
- 🔗 **Connection Sharing**: Multiple viewers share one upstream provider connection (HLS, Transcoded, and live Direct/TS streams)
- ⚡️ **Truly Ephemeral**: Provider connections open only when clients are consuming; close when the last client disconnects
- 📺 **HLS Support**: Optimized playlist and segment handling (.m3u8)
- 📡 **Continuous Streams**: Primary/subscriber broadcast for .ts, .mp4, .mkv, .webm, .avi live streams
- 🔄 **Real-time URL Rewriting**: Automatic playlist modification for proxied content
- 📱 **Full VOD Support**: Per-client connections with byte-range requests and seeking
- 🎬 **Strict Live TS Mode**: Enhanced stability for live MPEG-TS with pre-buffering & circuit breaker
### Performance & Reliability
-**uvloop Integration**: 2-4x faster async I/O operations
- 🔄 **Seamless Failover**: <100ms transparent URL switching per client
- 🎯 **Immediate Cleanup**: Connections close instantly when client stops
### Transcoding Available with Hardware Acceleration
- 🎬 **FFmpeg Integration**: Built-in hardware-accelerated video processing
- 🚀 **GPU Acceleration**: Automatic detection of NVIDIA, Intel, and AMD GPUs
-**VAAPI Support**: Intel/AMD hardware encoding (3-8x faster than CPU)
- 🎯 **NVENC Support**: NVIDIA hardware encoding (10-20x faster than CPU)
- 🔧 **Auto-Configuration**: Zero-config hardware acceleration setup
- 📊 **Multiple Codecs**: H.264, H.265/HEVC, VP8, VP9, AV1 support
### Management & Monitoring
- 👥 **Client Tracking**: Individual client sessions and bandwidth monitoring
- 📊 **Real-time Statistics**: Live metrics on streams, clients, and data usage
- 🔎 **Stream Type Detection**: Automatic HLS/VOD/Live detection
- 🧹 **Automatic Cleanup**: Inactive streams and clients auto-removed
- 📣 **Event System**: Real-time events and webhook notifications
- 🩺 **Health Checks**: Built-in health endpoints for monitoring
- 🏷️ **Custom Metadata**: Attach arbitrary key/value pairs to streams for identification
## Quick Start
#### 🐳 Docker compose example
Use the below example to run using the precompiled Dockerhub image.
You can also replace `latest` with `dev` or `experimental` to try another branch.
```yaml
services:
m3u-proxy:
image: sparkison/m3u-proxy:latest
container_name: m3u-proxy
ports:
- "8085:8085"
# Hardware acceleration (optional)
devices:
- /dev/dri:/dev/dri # Intel/AMD GPU support
# For NVIDIA GPUs, use this instead:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: all
# capabilities: [gpu]
environment:
# Server Configuration
- M3U_PROXY_HOST=0.0.0.0
- M3U_PROXY_PORT=8085
- LOG_LEVEL=INFO
# Base path (default: /m3u-proxy for m3u-editor integration)
# Set to empty string if not using reverse proxy: ROOT_PATH=
- ROOT_PATH=/m3u-proxy
# Hardware acceleration (optional)
- LIBVA_DRIVER_NAME=i965 # For older Intel GPUs
# - LIBVA_DRIVER_NAME=iHD # For newer Intel GPUs
# Timeouts (optional)
- CLIENT_TIMEOUT=300
- CLEANUP_INTERVAL=60
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8085/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
```
## Building from source
### 1. Clone Repo & Install Dependencies
#### Prerequisits
- `python` installed on your system: `>=3.10`
- `pip` installed on your system: `>=23`
```bash
git clone https://github.com/sparkison/m3u-proxy.git && cd m3u-proxy
pip install -r requirements.txt
```
### 2. Start the Server
```bash
python main.py --debug
```
Server will start on `http://localhost:8085`
### 3. Create a Stream
```bash
# HLS stream with custom user agent
curl -X POST "http://localhost:8085/streams" \
-H "Content-Type: application/json" \
-d '{"url": "https://your-stream.m3u8", "user_agent": "MyApp/1.0"}'
# Direct IPTV stream with failover
curl -X POST "http://localhost:8085/streams" \
-H "Content-Type: application/json" \
-d '{
"url": "http://server.com/stream.ts",
"failover_urls": ["http://backup.com/stream.ts"],
"user_agent": "VLC/3.0.18"
}'
# Using the CLI client
python m3u_client.py create "https://your-stream.m3u8" --user-agent "MyApp/1.0"
python m3u_client.py create "http://server.com/movie.mkv" --failover "http://backup.com/movie.mkv"
```
## API Documentation
### Stream Management
#### Create Stream
```bash
POST /streams
Content-Type: application/json
{
"url": "stream_url",
"failover_urls": ["backup_url1", "backup_url2"],
"user_agent": "Custom User Agent String"
}
```
#### List Streams
```bash
GET /streams
```
#### Stream Information
```bash
GET /streams/{stream_id}
```
#### Delete Stream
```bash
DELETE /streams/{stream_id}
```
#### Trigger Failover
```bash
POST /streams/{stream_id}/failover
```
### Statistics & Monitoring
#### Comprehensive Stats
```bash
GET /stats
```
#### Health Check
```bash
GET /health
```
#### Client Information
```bash
GET /clients
GET /clients/{client_id}
```
## CLI Client Usage
The included CLI client (`m3u_client.py`) provides easy access to all proxy features:
```bash
# Create a stream with failover
python m3u_client.py create "https://primary.m3u8" --failover "https://backup1.m3u8" "https://backup2.m3u8"
# List all active streams
python m3u_client.py list
# View comprehensive statistics
python m3u_client.py stats
# Monitor in real-time (updates every 5 seconds)
python m3u_client.py monitor
# Check health status
python m3u_client.py health
# Get detailed stream information
python m3u_client.py info <stream_id>
# Trigger manual failover
python m3u_client.py failover <stream_id>
# Delete a stream
python m3u_client.py delete <stream_id>
```
## Configuration
### Environment Variables
```bash
# Server configuration
M3U_PROXY_HOST=0.0.0.0
M3U_PROXY_PORT=8085
# Base path for API routes (useful for reverse proxy integration)
# Default: /m3u-proxy (optimized for m3u-editor integration)
# Set to empty string for root path
ROOT_PATH=/m3u-proxy
# API Authentication (optional)
# Set API_TOKEN to require authentication for management endpoints
# Leave unset or empty to disable authentication
API_TOKEN=your_secret_token_here
# Client timeout (seconds)
CLIENT_TIMEOUT=300
# Cleanup interval (seconds)
CLEANUP_INTERVAL=60
# Stream Retry Configuration (improves reliability for unstable connections)
# Number of retry attempts before failover or giving up
STREAM_RETRY_ATTEMPTS=3
# Delay between retries (seconds)
STREAM_RETRY_DELAY=1.0
# Total timeout across all retries (seconds, 0 to disable)
STREAM_TOTAL_TIMEOUT=60.0
# Use exponential backoff for retry delays (false/true)
STREAM_RETRY_EXPONENTIAL_BACKOFF=false
# Timeout for receiving data chunks (seconds)
LIVE_CHUNK_TIMEOUT_SECONDS=15.0
# Sticky Session Handler (prevents playback loops with load-balanced providers)
# Locks to specific backend after redirect to maintain playlist consistency
USE_STICKY_SESSION=false
```
### API Authentication
When `API_TOKEN` is set in the environment, all management endpoints require authentication via the `X-API-Token` header. This includes:
- `/` - Root endpoint
- `/streams` - Create, list, get, delete streams
- `/stats/*` - All statistics endpoints
- `/clients` - Client management
- `/health` - Health check endpoint
- `/webhooks` - Webhook management
- `/streams/{stream_id}/failover` - Failover control
- `/hls/{stream_id}/clients/{client_id}` - Client disconnect
**Stream endpoints (the actual streaming URLs) do NOT require authentication** since they are accessed by media players that identify streams via `stream_id`.
Example with authentication:
```bash
# Set your API token
export API_TOKEN="my_secret_token"
# Method 1: Using header (recommended for API calls)
curl -X POST "http://localhost:8085/streams" \
-H "Content-Type: application/json" \
-H "X-API-Token: my_secret_token" \
-d '{"url": "https://your-stream.m3u8"}'
# Method 2: Using query parameter (useful for browser access)
curl -X POST "http://localhost:8085/streams?api_token=my_secret_token" \
-H "Content-Type: application/json" \
-d '{"url": "https://your-stream.m3u8"}'
# Browser access example
# Visit: http://localhost:8085/stats?api_token=my_secret_token
# Without token - will get 401 error
curl -X POST "http://localhost:8085/streams" \
-H "Content-Type: application/json" \
-d '{"url": "https://your-stream.m3u8"}'
```
To disable authentication, simply leave `API_TOKEN` unset or set it to an empty string.
## Hardware Acceleration
m3u-proxy includes comprehensive hardware acceleration support for video transcoding operations using FFmpeg with GPU acceleration.
### Supported Hardware
- **🔥 NVIDIA GPUs**: CUDA, NVENC, NVDEC (10-20x faster than CPU)
- **⚡ Intel GPUs**: VAAPI, QuickSync (QSV) (3-8x faster than CPU)
- **🚀 AMD GPUs**: VAAPI acceleration (3-5x faster than CPU)
- **💻 CPU Fallback**: Software encoding when no GPU available
### Automatic Detection
The container automatically detects available hardware on startup:
```
🔍 Running hardware acceleration check...
✅ Device /dev/dri/renderD128 is accessible.
🔰 Intel GPU: Intel GPU (Device ID: 0x041e)
✅ FFmpeg VAAPI acceleration: AVAILABLE
💡 For older Intel GPUs, try: LIBVA_DRIVER_NAME=i965
```
### Docker Setup Examples
#### Intel/AMD GPU Support
```yaml
services:
m3u-proxy:
image: sparkison/m3u-proxy:latest
devices:
- /dev/dri:/dev/dri
environment:
- LIBVA_DRIVER_NAME=i965 # For older Intel GPUs
# - LIBVA_DRIVER_NAME=iHD # For newer Intel GPUs
```
#### NVIDIA GPU Support
```yaml
services:
m3u-proxy:
image: sparkison/m3u-proxy:latest
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
```
#### Docker Run Commands
```bash
# Intel/AMD GPU
docker run -d --name m3u-proxy \
--device /dev/dri:/dev/dri \
-e LIBVA_DRIVER_NAME=i965 \
-p 8085:8085 \
sparkison/m3u-proxy:latest
# NVIDIA GPU
docker run -d --name m3u-proxy \
--gpus all \
-p 8085:8085 \
sparkison/m3u-proxy:latest
```
### Programming Interface
The hardware acceleration is available through Python APIs:
```python
from hwaccel import get_ffmpeg_hwaccel_args, is_hwaccel_available
# Check if hardware acceleration is available
if is_hwaccel_available():
# Get optimized FFmpeg arguments
hwaccel_args = get_ffmpeg_hwaccel_args("h264")
# Example: Hardware-accelerated transcoding
cmd = ["ffmpeg"] + hwaccel_args + [
"-i", "input_stream.m3u8",
"-c:v", "h264_vaapi", # Hardware encoder
"-preset", "fast",
"-b:v", "2M",
"output_stream.mp4"
]
```
### Performance Benefits
| Hardware | Encoding Speed | Concurrent Streams | CPU Usage Reduction |
|----------|---------------|-------------------|-------------------|
| NVIDIA GPU | 10-20x faster | 4-8 streams | 95%+ |
| Intel GPU | 3-8x faster | 2-4 streams | 90%+ |
| AMD GPU | 3-5x faster | 2-3 streams | 85%+ |
| CPU Only | Baseline | 1 stream | N/A |
### Supported Codecs
- **H.264/AVC**: High compatibility, universal support
- **H.265/HEVC**: Better compression, 4K/8K content
- **VP8/VP9**: WebM containers, streaming optimized
- **AV1**: Next-gen codec, best compression
- **MJPEG**: Low latency, surveillance applications
For detailed hardware acceleration setup and troubleshooting, see [Hardware Acceleration Guide](docs/HARDWARE_ACCELERATION.md).
### Server Startup Options
```python
# Main server with all features
python main.py
# With custom options
python main.py --port 8002 --debug --reload
```
## Troubleshooting
### Common Issues
1. **Stream Won't Load**
- Check original URL accessibility
- Verify CORS headers if accessing from browser
- Check server logs for detailed errors
2. **High Memory Usage**
- Reduce `CLIENT_TIMEOUT` for faster cleanup
- Monitor client connections and cleanup inactive ones
- Consider horizontal scaling for high loads
3. **Failover Not Working**
- Verify failover URLs are accessible
- Check failover trigger conditions in logs
- Test manual failover via API
### Debug Mode
```bash
# Enable detailed logging
export LOG_LEVEL=DEBUG
python main.py --debug
```
## Integration Examples
### HTML5 Video Player
```html
<video controls>
<source src="http://localhost:8085/hls/{stream_id}/playlist.m3u8" type="application/x-mpegURL">
</video>
```
### FFmpeg
```bash
ffplay "http://localhost:8085/hls/{stream_id}/playlist.m3u8"
```
### VLC
```bash
vlc "http://localhost:8085/hls/{stream_id}/playlist.m3u8"
```
## 📡 Event System & Webhooks
The proxy includes a comprehensive event system for monitoring and integration:
### Webhook Configuration
```bash
# Add webhook to receive events
curl -X POST "http://localhost:8085/webhooks" \
-H "Content-Type: application/json" \
-d '{
"url": "https://your-server.com/webhook",
"events": ["stream_started", "client_connected", "failover_triggered"],
"timeout": 10,
"retry_attempts": 3
}'
```
### Available Events
- `stream_started` - New stream created
- `stream_stopped` - Stream ended
- `client_connected` - Client joined stream
- `client_disconnected` - Client left stream
- `failover_triggered` - Switched to backup URL
### Webhook Payload Example
```json
{
"event_id": "uuid",
"event_type": "stream_started",
"stream_id": "abc123",
"timestamp": "2025-09-25T22:38:34.392830",
"data": {
"primary_url": "http://example.com/stream.m3u8",
"user_agent": "MyApp/1.0"
}
}
```
### Demo Events
```bash
# Try the event system demo
python demo_events.py
```
📖 **Full Documentation**: See [EVENT_SYSTEM.md](docs/EVENT_SYSTEM.md) for complete webhook integration guide.
## Additional Documentation
- **[Documentation Index](docs/README.md)** - Complete, always-current list of all documentation
## Development
### Project Structure
```
├── docker/ # Container and deployment assets
├── docs/ # Full documentation set
│ ├── README.md # Canonical docs index
│ └── *.md # Architecture, failover, retry, auth, etc.
├── logs/ # Runtime logs (local/dev)
├── src/
│ ├── __init__.py # Package marker
│ ├── api.py # FastAPI server application
│ ├── broadcast_manager.py # Client broadcast coordination
│ ├── config.py # Configuration management
│ ├── events.py # Event system with webhooks
│ ├── hwaccel.py # Hardware acceleration detection/helpers
│ ├── models.py # Data models and schemas
│ ├── pooled_stream_manager.py # Shared/pooling stream orchestration
│ ├── redis_config.py # Redis settings
│ ├── redis_manager.py # Redis coordination layer
│ ├── stream_manager.py # Live proxy core (broadcast model for direct/TS streams)
│ └── transcoding.py # FFmpeg transcoding pipeline
├── static/ # Static assets (icons, images)
├── tests/ # Test suite
│ ├── integration/ # Integration tests
│ └── test_*.py # Unit tests
├── tools/ # Utility scripts and tools
│ ├── performance_test.py # Performance testing
│ ├── m3u_client.py # CLI client
│ ├── demo_events.py # Event system demo
│ └── run_tests.py # Enhanced test runner
├── docker-compose.yml # Default compose stack
├── Dockerfile # Container build definition
├── main.py # Server entry point (uvloop support)
├── requirements.txt # Python dependencies
├── pytest.ini # Test configuration
└── README.md # This file
```
---
## 🤝 Want to Contribute?
> Whether its writing docs, squashing bugs, or building new features, your contribution matters! ❤️
We welcome **PRs, issues, ideas, and suggestions**!\
Heres how you can join the party:
- Follow our coding style and best practices.
- Be respectful, helpful, and open-minded.
- Respect the **CC BY-NC-SA license**.
1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Submit a pull request
---
## ⚖️ License
> m3u editor is licensed under **CC BY-NC-SA 4.0**:
- **BY**: Give credit where credits due.
- **NC**: No commercial use.
- **SA**: Share alike if you remix.
For full license details, see [LICENSE](https://creativecommons.org/licenses/by-nc-sa/4.0/).