StatProxy


perl -w statProxy.pl [-x n] [-q] [-C <CONNECT proxy> [-t <tests>] [-u] (-l <file>|<IPaddress:port>|transparent|http://your.url.com/path)

where
-l file specifies a file with a list of proxies to check,
-t <tests> specifies the tests to be done, where <tests> looks like range0:range1:..., and range may look like n or 0..15 or 0to15 or all or -n (to disable a test from a previously specified range). A common use is statProxy -t all:-18 proxy1.emirates.net.ae:8080 which tests all the emirates proxies, running all tests except the Socks proxy test (test 18, which is rather slower than the others and will fail anyway). '-t d' asks for just a connection test of all the proxies (very fast, useful for a quick look at a new list of doubtful proxies).
-u asks for safe mode to be off. In safe mode, statProxy will not test various unsafe looking proxy addresses and will not repeat a test on the same host wih a different port.
-x n sets the debug level to n
-q asks for the F or P result to be changed to X or Y if the proxy under test is not the one which gets the web pages for you (you're probably looking at an array of proxies). The proxy getting the web pages is mentioned in the data at the end of the test results (connBack).
-C <CONNECT capable proxy> causes all tests to be done through a CONNECT tunnel. This is handy if you want anonymity (choose an anonymous CONNECT proxy), or if you want to test proxies your firewall doesn't normally allow you to get to. This is slower than the normal method.

<IPaddress:port> may be numeric, or fqdn (which may resolve to multiple IP addresses to test).
'transparent' checks your transparent proxy (if you have one).
http://your.url.com/path asks for this url to be treated as a list of proxies to test. StatProxy can handle many formats, but is not as good as findProxy at this yet.

The result is a list of one-line test results for each proxy tested suitable for piping to mergeHosts for incorporation into the hosts or user config data structures.
The results are saved to a uniqely named file (the file name is announced with the results on the screen).
If you are testing a long list of proxies, a single control-c will wait for the current proxy/test to finish, print the results so far, and then continue testing.
If you want to abort, a double control-c will wait for the current proxy/test to finish, print the results so far, and exit. The results are not written to a file in this case.

In benchmark tests on the worst lists of old proxies (lots of timeouts and slow fails), statProxy was averaging something like 40 tests per minute. In practice that came to about 2 proxies per minute for the full range of (19 non-Socks) tests on each proxy. Test 0 is more important, and so the timeout is longer than for other tests. Test 13 (anonymity) is also slow, because SP must wait for the proxy to connect back.

perldoc statProxy.pl will show all of the documentation embedded in the Perl code, including the meaning of each of the tests.
For those who just want to get started, test 0 is printed first on the line. Test results are R (connection refused), T (connection timed out), P (pass), F (fail).
Test 0 is a normal proxy GET test.
Test 1 is a CONNECT panix.com:22 test.
Test 13 is the anonymity test.
Test 14 is the non-censoring test.
Test 18 is the Socks4 capability test.

Quick notes about statProxy:

People were sending me test results from their own ISP's proxies, but that's dropped to a trickle for some reason.
This was a good thing because
1) many of these are firewalled off from the rest of the Internet,
    so I can't test them. The only people who can test them iare inside that ISP. In practice this means you!.
2) with this data, I can keep a reasonably good database for the
    newbies who don't yet know how to run statProxy themselves,
    but still want to run localProxy.
3) I haven't yet decided how best to handle the procedures allowing
    localProxy to update the speeds and capabilities of these proxies
    itself.

To encourage this, and to make life easier for those doing these tests, I'll mention typical uses here:
1) statProxy will allow you to specify the fixed ISP proxy fqdn address to check. For example:
perl statProxy.pl -t all proxy.nesma.net.sa:80
will test all of the Nesma proxies currently up (212.71.32.95, 212.71.32.97, 212.71.32.98, 212.71.32.94).
'-t all:-18' will be qicker if you don't expect (or don't care) that any proxies are Socks capable.
This (or testing long lists of proxies) may take some time, but a single ctrl-c will display the results so far, and a double-ctrl-c will display the results and exit.

2) The results are most useful to yourself, of course, so you can save them into your own configuration (that way, you don't need to worry about losing your results next time you get my latest hosts.zip down). Please send to me too, though!
Do it like this:
perl statProxy.pl -t all proxy.nesma.net.sa:80 > results.out
(or forget the results.out and use the unique filename SP saves it's results to, then ...)
perl mergeHosts.pl results.out config-wayne.xml
(substitute your own proxy address to be tested and your own config, of course). The ISP config file in the distribution (config-KSA-nesma.xml) can be used, but you lose your results next time you get a new one from CVS, or a package. I suggest copying it to one with your own name first (e.g. config-john.xml).

3) Keep your own information (and again, mine, please!) up to date on the other proxies that localProxy will select for your use in each new build. This can be done easily too. When you have the localProxy GUI on the screen with a new build displayed, copy all the layer 0 proxies you can see (click different services, commStrats, but only layer 0) into a file (say, current.in), and do this:
perl statProxy.pl -t all -l current.in > current.out
perl mergeHosts.pl current.out config-john.xml
(substitute your own config again)
When you restart services with configuration 'john', LP will be using the new test results to select proxies for your build.

4) statProxy also has other modes of operation:
perl statProxy.pl -t all -l proxyListInAFile.txt
perl statProxy.pl -t all http://proxy.list.com/specialList.html
perl statProxy.pl -t all 194.170.1.66:8080
perl statProxy.pl 194.170.1.66:8080 (deprecated)
etc. etc. ...