Rotating Proxy Github
mattes/rotating-proxy: Rotating TOR proxy with Docker – GitHub
Docker Container
————————————-
<-> Polipo 1 <-> Tor Proxy 1
Client <----> HAproxy <-> Polipo 2 <-> Tor Proxy 2
<-> Polipo n <-> Tor Proxy n
Why: Lots of IP addresses. One single endpoint for your client.
Load-balancing by HAproxy.
Usage
# build docker container
docker build -t mattes/rotating-proxy:latest.
#… or pull docker container
docker pull mattes/rotating-proxy:latest
# start docker container
docker run -d -p 5566:5566 -p 4444:4444 –env tors=25 mattes/rotating-proxy
# test with…
curl –proxy 127. 0. 1:5566
# monitor
Further Readings
Tor Manual
Tor Control
HAProxy Manual
Polipo
Please note: Tor offers a SOCKS Proxy only. In order to allow communication
from HAproxy to Tor, Polipo is used to translate from HTTP proxy to SOCKS proxy.
HAproxy is able to talk to HTTP proxies only.
create your rotating proxy server with docker. – GitHub
Fully Optimized for Web Scraping Usage.
HTTP/HTTPS Support
socks5 with Authorization Proxy to HTTP(S) proxy convert compatible by Gost
You can use a VPN as an HTTP proxy.
Making it IP address based authentication makes it easier to use in your program. (selenium, puppeteer etc)
Docker Container
———————————-
Client <----> Squid <-> HTTP/HTTPS Rotate Proxies—\
|
—————|-> Gost <-> Socks5 Proxy — Internet
————–<-> VPN to HTTP Proxy <--------/
It can be used in two ways.
tomatically control the proxy and rotate each request -> use Squid
ntrol the proxy programmatically -> use Gost Port
Usage Example
Configuring IPs to allow the use of the Rotating Proxy
If you want to use it from outside, please specify the your IP address to
Example:
93. 184. 216. 34
108. 62. 57. 53
1. Create your (If HTTP/Socks is provided)
Search FreeProxy List or Paid/Subscribe ProxyService Provider.
example: Format
IPAddress:Port:Type(socks5 or or or quid):Username:Password
IPAddress:Port:Type(socks5 or or or quid)
IPAddress:Port
1. 1 Create Your OpenVPN Config(If HTTP/Socks is NOT provided)
see example
2. Generate
git clone cd docker-rotating-proxy && cd setup
docker run –rm -it -v “$(pwd):/app” composer install
cd..
# If you don’t want to set up OpenVPN, please remove it.
rm -rf. /openvpn/*
docker run –rm -it -v “$(pwd):/app/” php:7. 4-cli php /app/setup/
cat
docker-compose up -d
curl –proxy
How to it works?
Sometimes you may need the same IP address for a series of steps.
To deal with this problem, we have built a new relay server via gost.
Most open proxies will be unavailable in a few days.
Therefore, it is useless to build a server for every open proxy, so we use squid’s cache_peer to rotate a large number of open proxies.
Example1
127. 0. 1:1080:socks5:yourUsername:yourPassword
127. 1:44129:quid:mysquidproxy:mysquidpassword
127. 1:29128:quid:rotatingserviceUsername:password
169. 254. 1:1080:socks5:paidsocksUsername:paidsocksPassword
127. 1:80
172. 31. 22. 222:8080
Example2
Here are some practical examples.
using NordVPN, TorGuard, Luminati
173. 222. 146:1080:socks5:yourTorGuardUsername:Password
Generated example
version: ‘3. 3’
services:
squid:
ports:
– ‘3128:3128’
image: ‘b4tman/squid:latest’
volumes:
– ‘. /config:/etc/squid/conf. d:ro’
container_name: dockersquid_rotate
environment:
– ‘SQUID_CONFIG_FILE=/etc/squid/conf. d/’
hola1:
– ‘10021:8080’
image: ‘yarmak/hola-proxy:latest’
command: ‘-country us -proxy-type peer’
socks1:
– ‘10022:10022’
image: ‘ginuerzh/gost:latest’
command: ‘-L=:10022 -F=socks5vpn:unlimited@159. 89. 206. 161:2434’
proxy1:
– ‘49152:49152’
container_name: dockergost_1
command: ‘-L=:49152 -F=socks5vpn:unlimited@159. 161:2434’
proxy2:
– ‘49153:49153’
container_name: dockergost_2
command: ‘-L=:49153 -F=socks5vpn:unlimited@142. 93. 68. 63:2434’
proxy3:
– ‘49154:49154’
container_name: dockergost_3
command: ‘-L=:49154 -F=socks5vpn:unlimited@82. 196. 7. 200:2434’
vpn4:
– ‘49155:3128’
image: curve25519xsalsa20poly1305/openvpn
container_name: dockervpn_4
devices:
– ‘/dev/net/tun:/dev/net/tun’
cap_add:
– NET_ADMIN
– ‘. /openvpn/’
– OPENVPN_CONFIG=/vpn/
vpn5:
– ‘49156:3128’
container_name: dockervpn_5
Now try it out
port 3128 is rotation port.
sh-4. 2# curl –proxy {
“origin”: “209. 197. 26. 75”}
“origin”: “185. 155. 98. 168”}
“origin”: “173. 245. 217. 48”}
“origin”: “81. 171. 85. 49”}
sh-4. 2#
and.. try static ip gateway
# curl –proxy {
“origin”: “139. 99. 54. 109”}
“origin”: “159. 161”}
Example of using a large number of public proxies with real-time updates
see
0 * * * * /your_sh_path_here/
TODO
Username/Password Auth for Enterprise
rotating-proxy · GitHub Topics
Here are
19 public repositories
matching this topic…
Code
Issues
Pull requests
Rotating TOR proxy with Docker
Updated
Feb 28, 2021
Ruby
An AWS Lambda powered HTTP/SOCKS web proxy
Jun 19, 2020
Go
A fast, modern and intelligent proxy rotator perfect for crawling and scraping public data.
Mar 16, 2018
create your rotating proxy server with docker.
Sep 11, 2021
PHP
Discussions
Apr 3, 2021
Python
Rotating Proxy Manager module for
Dec 22, 2018
JavaScript
Learn about how to rotate proxies by using Python.
Jul 12, 2021
Dockerized load balanced Tor SOCKS proxy gateway
Nov 14, 2016
Shell
ProxyBot is a simple discord bot which scrapes proxies from the web!
Apr 7, 2021
Best Rotating Proxy & Scraping API Alternative
May 3, 2021
Best Rotating Proxy & Scraping API Alternative. C# Example.
C#
A Bot that uses a Rotating Proxy to simulate many clients making a request to a single server
Jan 16, 2019
a collection of all personal web crawler projects
Oct 6, 2021
This is small server built which extracts and keep free live / proxies across various free proxy sites and keeps track of their health. Server serves number of clients over tcp and provides them rotated proxy address on demand. Server responds over tcp using json.
Oct 28, 2018
Java
SDK to access ZenRows API directly from We handle proxies rotation, headless browsers and CAPTCHAs for you.
Sep 23, 2021
TypeScript
toraio – a pool of proxies, shifting on each request
Oct 19, 2021
Aug 31, 2020
A rotating tor proxy service that starts a configurable number of tor socks proxies and expose them under one reverse proxy
Jun 7, 2021
scraper using scrapy with rotating proxies
Dec 22, 2019
Improve this page
Add a description, image, and links to the
rotating-proxy
topic page so that developers can more easily learn about it.
Curate this topic
Add this topic to your repo
To associate your repository with the
topic, visit your repo’s landing page and select “manage topics. ”
Learn more
