Proxies used to be relatively straightforward. They used to sit in the IT server room, pretty much forgotten for the majority of time. Not because they weren’t used much, quite the opposite. Most proxies are among the busiest servers in an organisation and are rarely idle. This is because all web traffic both outbound and inbound would normally be routed through the proxy.
The proxy would facilitate all access to the web , plus protect return traffic often checking for viruses and malware heading for the desktop. That’s not all, one of the other primary roles of a proxy server is caching, making sure that any popular sites or pages are stored locally. This reduces impact on the network plus reduces outbound bandwidth bills. By storing these pages locally, access time is reduced significantly and overall traffic is reduced too.
So why where they forgotten? Well in truth it’s because proxy technology has always been pretty robust. Unlike a myriad of other flaky application servers, proxies in the corporate environment had simple but important tasks and generally did them well. IT staff usually spent the time to set them up properly because of their importance, and were normally on high specification hardware too.
Of course, if you don’t work in IT then the concept of proxy servers would probably mean something fairly different. They’re rarely used to cache directly and people use them more for privacy and anonymity rather than speed and security. Although it’s perfectly feasible to do so, most people don’t have a working proxy server on their home network for example.
Options on Proxy Services
Now there are thousands of companies who provide proxies and VPN services online. In fact the prices and options of these have been falling dramatically over the last few years. The best ones are still not cheap but the speed and reliability is significantly better than was available a decade or so ago.
Customer requirements have also changed particularly with regards the privacy aspect. The problem is that although a well configured proxy is difficult to detect, there is an aspect which is more difficult to conceal. Many proxy services are not detected because of some configuration problem or information leakage but simply because of the IP address that they use. Proxies will generally be assigned commercial IP address ranges allocated from datacenters across the world.
These addresses function perfectly well but they do make the connection more identifiable. An ordinary home user would never have and IP address like this and so it makes the connection identifiable as a proxy. The developments withing proxy services have been to combat this issue by using IP addresses which are not registered as ‘commercial’.
The solution is relatively simple but difficult to implement effectively. All these proxies need is to be assigned the same sort of IP addresses as home users are allocated. Normally referred to as residential addresses they work in exactly the same way but have no ‘commercial’ footprint. The problem is that they are also difficult and expensive to obtain. The methods involved to gain access to these addresses are outside the scope of this article but needless to say they involve a lot more effort and expense to the proxy service providers.
Which is why there are numerous methods designed to maximise the use of the servers. It’s important to keep down the price because some of the biggest demand for these proxies comes from internet marketers and social media management firms. If you’re managing lots of social media accounts then you’ll be looking to buying private proxies for Instagram or Facebook use which means lots of residential IP addresses.
For dedicated, private access to servers with these addresses is going to be extremely expensive simply because of the cost of obtaining the IPs. In order to reduce these costs then companies have introduced innovative new ways of distributing and sharing them. Mostly this involves taking control of the allocation of these addresses through large databases. Instead of having exclusive access to a range of IP address on a monthly basis you will have exclusive access for minutes. By rotating these addresses throughout lots of users you can still avoid concurrent connection and also significantly reduce the unit cost.
It’s a clever system and insures that anyone can have that huge block of Ninja IP addresses without having to pay thousands for accessing them exclusively. There are lots of names for these proxy services but are generally to be found under rotating or backconnect proxy services from the biggest suppliers.
It’s interesting to see if the battle for privacy will move back towards pure technology in the future however it’s difficult to imagine. Being able to detect and classify a connection based on it’s IP address is much easier to do. There are many large databases which provide classification of these addresses and they can be checked easily online.
