Rishav Anand (08010157) Ravi Tandon (08010140) Instructor : Dr. T. Venkatesh Course project report for CS 544
HTTP_Proxy Objective Authors Introduction Implementation Product features Problems Observed and their Solutions Experiments Results and Analysis Conclusion Source code
Objective To implement an HTTP-PROXY with caching capabilities. The report also report measurement results and analysis aimed at finding out the optimal parameter to consider while adopting an efficient cache eviction strategy.
Design of a proxy server Authors Ravi Tandon [
[email protected]] Rishav Anand [
[email protected]]
Introduction A proxy server is an intermediary between a client (seeking requests for resources) and servers (that provide those resources). We have designed an HTTP proxy-server that supports efficient caching of resources. Concurrent service to client requests, document caching, server side load-balancing, share distribution of bandwidth between the clients and the proxy server are some of the primary features of our proxy server. The efficiency has been measured and analysis of the observed measurements has been detailed in section[]. The product has been developed in C++ using C-Sockets. This report mentions the feature coded into a our proxy server’s design. We have also documented the problems we stumbled upon while development of the product. We also mention our strategies as the respective solutions to those problems. We have also studied the performance of the proxy based on variation in the parameter deciding the cache eviction policy used by the proxy server.
Implementation The code has been written in C++ using C- sockets with concurrency implemented using POSIX pthread library.
Product features ●
Concurrency in proxy server with thread safety In order to enhance the response time of each web browser, the client sets up parallel
●
●
●
●
●
connections with each of the clients and handles their requests parallely. Access log We also keep a track record of all the accesses made by the clients. The records are time-stamped. Also the URL, the size of the object and the client’s IP address are recorded in the log. The proxy server is a non-anonymous. Caching of objects based on different caching policies The most prominent feature of our proxy server is its ability to cache the different objects. We primarily tested out the performance of five different cache eviction policies. The results are mentioned in the experimentation and results section. Number of connections restricted to each client In order to reduce the workload on the proxy server and also to ensure fair distribution of proxy server’s resources to each of the requesting clients, the number of concurrent connections from each of the clients is restricted to 7. DNS based load balancing The proxy server can also balance the requests to the web server. We have employed a simple round robin technique. This leads to a reduction in the workload on the webserver. Client server can inform the proxy server as to which objects have to be cached Each of the request for an object from the client as well the response from the web server consists of a field in the HTTP header that can indicate whether the object has to cached. The request from the client side contains the Cache-Control field which can have the value “no-cache”. Also the response from the server side contains the “Pragma” field that can have the value “no-cache”. If either of the client or the server indicates to the client that the object is not to be cached then the object is stored flushed by the proxy server.
Problems Observed and their Solutions Issue of synchronizing the client (web browser) and the web server: The Problem : One of the issues we encountered while implementing the proxy server, was the issue of synchronizing the requests from the client and the appropriate response from the server. The size of the response from the web-server was not known initially. Thus, in our earlier implementation the proxy server did not know when to close the connection with the client. We observed that the client web browser kept on waiting for the proxy server to close the TCP connection and this lead to a poor response on the client side. The Solution: To solve this issue we had to monitor the response from the web-server. This was done using the standard UNIX “select” command. The file descriptor corresponding to the web-server connection was monitored. If the file descriptor being monitored is either zero or positive (that is no error state) then the buffer (on which the web server writes data) is read. On getting a non positive value on this we close the connection on the client side as well as the web server's side.
The 404 problem: When a requested object returned a 404 not found error code the browser instead of closing the connection kept on sending multiple request resulting in the connection never getting closed and the thread remained alive thereby choking the client if the client connection to the proxy server has been constrained. Hence in those cases the proxy willfully returns a server error to
force the browser to close the connection to the proxy server. The Experimentation Horror: To test out the performance gain with different cache eviction strategy we created an URL request sequence with Zipf-like distribution with locality constraints put in place, however since each HTTP request entails a number of object request as well hence taking measurements became a high time consuming event. We solved the problem by running the proxy once on this synthetic request sequence and then simulating the performance gain obtained by using different cache eviction policy.
Experiments Traces: We generated synthetic traces. As mentioned above in order to test the proxy performance we 1. Created a URL request database by crawling the IITG intranet network and using about 100 unique live links. 2. We then created a HTTP request sequence by creating a Zipf like distribution request pattern. 3. To ensure temporal locality of reference, we generated an appropriate access pattern. We took a window size of twice the size of the access frequency. The access request was then randomly distributed in this window. Proxy Simulation: As mentioned above in “Experimentation Horror” section above, we simulated the proxy server in order to test the performance of the proxy server with various eviction policies. We implemented different eviction policies. These were: 1. Least Recently Used The object which was accessed the earliest in time, out of all the objects, was evicted. 2. Least Frequently Used The object which was accessed the least number of times was evicted. 3. Lowest Latency First The object that had the lowest fetch latency was evicted. The “fetch latency” was defined as the total time it took the proxy to download the object from the web server. 4. Lowest Size First The object with the lowest size was evicted first. 5. Greedy Dual The greedy dual uses the size and the frequency of access. The eviction parameter is the product of size and access frequency. The smaller the product the higher is the probability of an object’s eviction. Observation parameters: We first of all fired the URL sequence request to the proxy server to obtain all the relevant eviction related performance parameters at one place. These parameters are for eg.
● file size ● file get latency ● web server connection setting delay The motivation behind this exercise was the realization that running proxy to test the performance under various eviction policy is highly computation expensive when the same job can be done by caching the parameters values obtained after the first run. ● Thereafter we calculated the total cache misses given the given URL sequence for a particular cache size of the proxy under each eviction policy.
Results and Analysis The parameters we considered for evaluation are: 1. Cache Misses These are the total number of misses that were encountered in the complete execution of the workload. 2. Cache Misses (Bytes) This parameter is the sum of the size of all the objects accessed from the web server. Essentially it includes the size of all the objects fetched from the web server on a cache miss (including the cold cache misses too). 3. Latency (Milliseconds) This parameter calculates the overall latency in fetching the documents from the webserver. It includes the latency in fetching all the objects on which a miss took place.
Some of the observations from the graphs were as follows: 1. Observation: The lowest latency first eviction policy varies closely with the SIZE eviction policy. Rationale: The latency in downloading a file depends upon two major factors. The first factor is the network delay to the server and the other factor is the size of the document. The processing delay at the server and the client side would be too small to have any visible effect. Since the experiment was on a LAN and almost all the requests would be served on similar paths the delays in the network due to the queuing delay, switching were same. Thus the lowest latency first reduced closely resembled the lowest size first eviction policy. Since the lowest size first does not consider the temporal locality of accesses we get poor performance in terms of the number of misses when using the lowest latency first.
2. Observation: Lowest latency first performs well as the size of the cache increases. Rationale: As the cache size increases the number of cache misses for every eviction policy starts decreasing. However, the decrement in the lowest latency first is rapid as compared to the LRU.
We could understand this through the following. Our model considers a strong temporal locality among the client accesses and each access of a particular page can be broken into a set of object accesses. A set of accesses that are related to each other can be considered to lie in a window of access (called the access window).In our model the sites are mostly accessed within a window size almost twice the size of the access frequency which is in a Zipf like distribution. The lowest latency first eviction evicts the object which has the smallest latency. The cache misses would go high if the current window is sufficiently large and has an object that is accessed with a sufficiently high frequency and has a very low latency. We call the object evicted by lowest latency first the lowest latency victim. When the cache size is low the probability of the lowest latency victim lying in the access window becomes high. Thus, the lowest latency victim is accessed frequently. This increases the cache misses. Although the latency of access still remains low. When the cache size becomes high the probability of the lowest latency victim lying in the access window becomes low. By the time the lowest latency victim lies in the access window, the window moves forward. Thus the number of cache misses are reduced. 3. Observation: The greedy dual size performs almost similar to LFU. Rationale: The greedy dual size takes into consideration the parameters frequency of access (AF) and size (S). Let the set of cached items be Scache. Scache = {AF_1*S_1, AF_2*S_2, …....}. The eviction parameter for the ith object in the cache can be defined as Pi, where Pi = AFi*Si. Let the object with the smallest size be S_small. We then normalize the size of each of the objects in the cache with the smallest object’s size. The normalized parameter then becomes NPi = AFi * (S_i/S_small) = AF_i * NS_i. NSi stands for Normalized Size of the ith object. We conjecture that the value of NSi is almost equivalent to 1. Therefore the normalized value of the eviction parameter (NPi) is almost equivalent to the eviction parameter for the least frequently used case. Other observations: The Least Recently Used scheme performs the best when the cache misses were being measured. This was because of the temporal locality of accesses that we modelled within our access traces. The Lowest Size Eviction policy performed the best when the cache misses were measured in terms of the size of the cache objects fetched. The Lowest Latency First performed the best when the overall latency was measured to complete the overall workload. The lowest latency has nothing to do with the cache misses that may happen in the caching model. At small sizes of the cache Eviction Policy
Cache Misses
Cache Misses Bytes
Latency
Least Recently Used
Low
High
High
Least Frequently Used
Moderate
High
Moderate
Lowest Size First
Moderate
Low
Moderate
Lowest Latency First
Moderate
Moderate
Low
Greedy Dual Size
Moderate
High
Moderate
Eviction Policy
Advantage
Disadvantage
Least Recently Used
Reduces the number of cache misses.
Does not the size or the latency of download of the victim object into comparison.
Least Frequently Used
Kees the highly accessed objects in cache.
The lowest frequency may be from the access window itself.
Lowest Size First
Fetch latency as well the cache miss in terms of size is low.
Does not take the temporal locality into consideration.
Lowest Latency First
Optimize upon the net latency to fetch the object from the server hence reduces response time visible to the client or web browser.
It does not optimizes the cache hits. So retention of popular content in the cache does not happens.
Greedy Dual Size
Is a mixture of LFU and SIZE algorithms. However goes very close to LFU
Same as LFU
Conclusion In this project we implemented an HTTP web proxy server with a couple of features like client connection control, basic load balancing schemes, etc.. We also compared five different eviction techniques and found that for different parameters different techniques work well.
Source code Hosted @ http://goo.gl/7gRVf
