How to Use Proxy Services with Large Language Models

Proxy Configuration for LLM API Calls

Understanding Proxy Requirements for LLM APIs
LLM service providers like OpenAI, Anthropic, and Hugging Face typically communicate via HTTP/HTTPS protocols, making standard proxy configurations straightforward. Key considerations include:

Proxy Protocol: HTTP/HTTPS proxies are sufficient for most API calls; SOCKS5 may be needed for advanced routing
Authentication: Basic auth (username/password), token-based auth, or IP whitelisting
Regional Proxies: To access region-specific LLM endpoints or mimic user locations

Python Implementation with OpenAI API

One of the most common use cases is connecting to OpenAI's API via a proxy. Here's a step-by-step implementation in

Python:
import openai
import requests
import os
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# Method 1: Configure proxy via environment variables (recommended for security)
os.environ["HTTP_PROXY"] = "http://your_proxy:port"
os.environ["HTTPS_PROXY"] = "http://your_proxy:port"

# If proxy requires authentication
os.environ["HTTP_PROXY"] = "http://username:password@your_proxy:port"
os.environ["HTTPS_PROXY"] = "http://username:password@your_proxy:port"

# Method 2: Directly configure proxy in OpenAI client (alternative approach)
openai.proxy = {
    "http": "http://your_proxy:port",
    "https": "http://your_proxy:port"
}

# Method 3: Use requests with custom proxy configuration (for advanced handling)
def create_proxied_session(proxy_url):
    session = requests.Session()
    session.proxies = {
        "http": proxy_url,
        "https": proxy_url
    }

    # Add retry logic to handle proxy connection failures
    retry = Retry(connect=3, backoff_factor=0.5)
    adapter = HTTPAdapter(max_retries=retry)
    session.mount("http://", adapter)
    session.mount("https://", adapter)
    return session

# Example usage with custom session
proxied_session = create_proxied_session("http://your_proxy:port")

# Make an API call to OpenAI
def call_llm(prompt):
    response = openai.Completion.create(
        engine="text-davinci-003",
        prompt=prompt,
        max_tokens=100
    )
    return response.choices[0].text

# Test the proxy connection
try:
    print(call_llm("What is the role of proxies in LLM applications?"))
except Exception as e:
    print(f"Proxy error: {e}")

Browser Proxy Setup for LLM Web Interfaces

Configuring Proxies for ChatGPT and Other Web-Based LLMs

When using browser-based LLM interfaces like ChatGPT, Bing Chat, or Character.AI, proxy configuration occurs at the browser level:

Google Chrome (and Chromium-based browsers)
1.Manual Proxy Setup:
Go to Settings > Privacy and security > Security > Proxy settings (Windows) or Settings > Network > Proxy (macOS)
Under "Manual proxy setup," enable HTTP proxy and enter your proxy address and port
Check "Also use this proxy for HTTPS" and add authentication if required
2.Using Proxy Extensions:
Install extensions like Proxy SwitchyOmega
Configure proxy profiles with different servers for flexibility

Mozilla Firefox
1.Built-in Proxy Settings:
Go to Settings > Network Settings
Under "Connection," click "Settings"
Select "Manual proxy configuration," enter proxy details, and enable "Use this proxy server for all protocols"
2.Extension Options:
Use extensions like FoxyProxy for advanced proxy management

Secure Proxy Practices for Browser-Based LLMs

Residential Proxies: Prefer residential over data center proxies to avoid detection (e.g., Thordata's residential proxy service)
Proxy Rotation: Use tools or services that rotate IP addresses to mimic human browsing patterns
HTTPS Proxy Verification: Ensure your proxy supports HTTPS to encrypt traffic between your browser and the LLM server

Choosing the Right Proxy Service for LLMs

Key Features to Look For

Residential IPs: Essential for avoiding detection (data center IPs are often blacklisted)
Global Coverage: Access to proxies in regions where LLM services are available
High Throughput: Sufficient bandwidth for handling large LLM responses
API Support: For programmatic proxy management and rotation
Reliable Uptime: Look for providers with 99%+ uptime guarantees
Recommended Proxy Provider: Thordata Residential Proxies

Constantine @heesungf5