GoogleGlobalCache-R720

Application Note and Technical Overview Release Dell R720xd

April 09, 2014

Google Global Cache [email protected]

Contents

1

Introduction

1

2

Overview 2.1 GGC Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 How GGC Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Determining where to serve users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 3 4

3

Equipment Provided by Google 3.1 Typical Server Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 5

4

Configuration Overview 4.1 The Peering Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 BGP Session Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 7 7

5

Network Connectivity 5.1 Access Control Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Proxies and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Configuration Options with Multiple Cache Nodes . . . . . . . . . . . . . . . . . . . . . . . . .

9 10 10 10

6

Maintenance and Support Overview

11

7

Frequently Asked Questions

13

8

Further Reading 8.1 Google Global Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Server Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 15 15

i

ii

CHAPTER 1

Introduction

The explosion of broadband access and rich multimedia content continually increases the demand on service provider networks. Google Global Cache (GGC) allows you to serve Google content, primarily video, from the edge of your own network. This eases congestion within your network, and reduces the amount of traffic on your peering and transit links. In short, GGC saves you money while improving the experience of your users. Note: The details in this document are subject to change. This agreement and associated activity is commercially confidential.

1

Application Note and Technical Overview, Release Dell R720xd

2

Chapter 1. Introduction

CHAPTER 2

Overview

2.1 GGC Features • Transparent to users Google transparently serves your users requests from caches inside your network. • Reduced traffic through your network Although cache hit rates can vary (based on the unique consumption patterns of end users on each operator’s network), typically between 70-90% of Google cacheable traffic can be served from GGC. • Robust : A GGC installation provides resilience through multiple levels of redundancy which are transparent to users. Google takes care to ensure that your users are always served by the best available node, even in the event of a temporary outage. If a GGC node is unavailable for any reason, user requests are automatically redirected to other nodes in your network. When no further nodes exist in your network, your users are automatically redirected to other Google serving locations (same behavior as having no GGC node). • Easy to set up : Google provides the server hardware, and takes care of the remote configuration of the cache once you have completed the initial setup. You only need to provide rack space, a keyboard and monitor, and a connection to your network.

2.2 How GGC Works When a user requests cacheable Google content, such as video or image data: • GGC application will determine whether the user requesting the content is authorized to access the GGC node. If not, the user is redirected to other Google serving locations. • The GGC application will then determine if the requested content should be served from a GGC node inside your network. If not, the user is redirected to other Google serving locations. If the GGC node already has the requested content in its local cache, it will serve the content directly to the end user, improving the user experience and saving transit costs. If the content is not stored on the GGC node, an algorithm will determine whether the content and the request meet the criteria for caching. Cacheable content is then fetched from the Google network and served to the user. Should the caching criteria not be met, the user’s request is redirected to other Google serving locations.

3


For more information on how the GGC application determines which users it should serve, please refer to ( BGP Session Overview) in the Configuration chapter.

2.3 Determining where to serve users Users are directed towards specific GGC nodes based on their Source IP and Resolver IP addresses. The GGC node can also determine if a user is permitted to request content from it, and redirect users that are not. For this process to work, the GGC node requires a list of User and Resolver prefixes that you would like served by the node. You must provide this information by advertising the prefix list to the node through a dedicated BGP session. Note that the BGP session is not used for traditional routing, it is only used to collect the list of prefixes you authorize to be served by the node. User mapping variations can occur in the following scenarios; • Another GGC location has a higher prefix specificity (most specific prefix wins) • Another GGC location has a Shorter AS path (shortest path wins) • Multiple GGC locations with equal criteria (location offering “currently best” user experience wins)

4

Chapter 2. Overview

CHAPTER 3

Equipment Provided by Google

A GGC node can comprise between 3 and 8 servers, which Google will provide. An installation can consist of many GGC nodes working together (please refer to section 4.6 for further details on multi-node configurations). Each GGC node will require space, power, and network connectivity (exactly how much will vary depending on the size of the configuration). Each server supports multiple (up to 4) bonded Gigabit Ethernet interfaces. Link Aggregation Control Protocol (LACP) is used between each GGC machine and your network switch to achieve the bonding. Google will work with you to determine the best configuration for your network. Table 3.1: Power Requirements # Servers 3 4 6 8

Rack Space 6RU 8RU 12RU 16RU

Power 900W 1200W 1800W 2400W

Peak Power 1200W 1600W 2400W 3200W

# GE ifs 12 16 24 32

IPv4 Subnet /26 /26 /26 /26

IPv6 Subnet /64 /64 /64 /64

Each machine has an A and B power feed. Each feed should ideally be plugged into an independent power source. The GGC node requires a dedicated subnet and broadcast domain. Each server has a single physical address and a number of virtual IP addresses (VIPs): • The physical address is used for monitoring and management of the machine itself. • All traffic is served from the VIPs VIPs can move between machines in the event of equipment failure or during maintenance. The GGC system is designed with multiple levels of redundancy. Content and user requests are spread across all VIPs, so if a server failure occurs, another server in the GGC node can immediately take over its VIPs and respond to the users’ requests. If a server is unable to respond to a request, it will redirect the users back to Google. External monitoring systems are used to ensure that no traffic is sent to an unavailable node.

3.1 Typical Server Specifications • 2U Rack-mountable chassis (28.46”/723mm D x 17.48”/444mm W x 3.44”/87.3mm H) • Server weight: 71.5lbs/32.5kg each • Input Power: 100/240 VAC (each server has redundant power with IEC 60320 C13 to C14 cords) • Heat dissipation: 2891 BTU/h • 4 x 1000Base-T copper Gigabit Ethernet interfaces

5


6

Chapter 3. Equipment Provided by Google

CHAPTER 4

Configuration Overview

4.1 The Peering Portal Google maintains a web portal at http://peering.google.com/ . Use this site to provide node configuration and shipping information. An initial user is invited to access the online portal by the Google Global Cache team. After this user accepts the GGC agreement, that person can invite additional users. In order to properly configure the GGC node, Google requires the following information: • Maximum bit rate of traffic you want to be sent from the GGC node towards your users. • IP subnet addresses (IPv4/IPv6) for the GGC node as described in chapter 3 • Peer ASN and IP address (IPv4/IPv6) for the BGP feed. • Technical point(s) of contact, in the event physical access is required. • Shipping instructions, contacts, and addresses.

4.2 BGP Session Overview GGC node requires a list of User and Resolver prefixes that you would like served by the node, and will establish a BGP session with your network to collect this information. Through this session, you will advertise the list of User and Resolver prefixes that you authorize to be served by the node. The BGP session is not used for traditional routing, it is only used to collect the list of prefixes you would like served by the node, so any transient BGP flapping is not considered an issue. Note: - The node does not learn routes via BGP, a default route is used for all outgoing traffic. - The node does not advertise any prefixes via BGP. - The BGP session is not used to determine if the node is available. - BGP multihop is supported. Detailed configuration information is provided in http://peering.google.com/static/downloads/GGCInstallation-R720.pdf

the

Installation

Guide

Note: To achieve maximum reliability, the GGC system does not interpret an interruption of the BGP session as a loss of the node. The node will continue to serve based on the most recent valid feed received until the session is restored.

7


8

Chapter 4. Configuration Overview

CHAPTER 5

Network Connectivity

The following is intended as an overview. Detailed configuration information can be found in the Installation Guide. Each server in the GGC node is connected to a single ISP-provided Ethernet switch via (up to) 4 x Gigabit Ethernet interfaces with LACP link aggregation. In addition VRRP is used to move VIPs between servers in the event of equipment failure See the illustration Network Connectivity Diagram.

Figure 5.1: Network Connectivity Diagram

9


5.1 Access Control Lists Access Control Lists (ACLs) are not recommended on network equipment serving the GGC node (an IP firewall runs on each server in the GGC node). If ACLs are used, the following ports must be allowed for the entire subnet: • Inbound and outbound HTTP and HTTPS (TCP/80, TCP/443) • Inbound and outbound ICMP • Inbound and outbound SSH (TCP/22) • Inbound and outbound insecure and secure QUIC (UDP/80, UDP/443) • Outbound DNS (UDP/53 and TCP/53) • Outbound NTP (UDP/123), with no restriction on source port • Outbound BGP (TCP/179) The node must be reachable from any IP on the Internet. Note: Future services may require additional ports.

5.2 Proxies and Filters No transparent proxies or filters may be placed in the path of communications between the GGC Node and Google’s back-end servers.

5.3 Configuration Options with Multiple Cache Nodes Some service provider networks require multiple cache nodes. There are two scenarios for multiple node installations, each of which is configured slightly differently. • Multiple caches nodes can be deployed in a single logical location or region for increased serving capacity: * The cache catalog is automatically spread across the nodes in that location * In the event that a node (or part of a node) becomes unreachable, requests can be served either from another logical location in your network (if you have multiple logical locations) or from caches on Google’s network • If your network is divided into multiple regions with limited inter-region bandwidth and/or high inter-region latency, cache nodes can be deployed in each region: * Each region’s nodes are configured as a separate logical location * In the event that a node (or part of a node) becomes unreachable, requests can be served from nodes in another region (or logical location) in your network or from caches on Google’s network Further information can be found in the ‘MultiNode Concepts Deployment Guide’ Further Reading.

10

Chapter 5. Network Connectivity

CHAPTER 6

Maintenance and Support Overview

Google’s GGC operations team remotely monitor the node for server, connectivity, and application health. Server and Application issues are handled by the GGC Operations team (including ordering of replacement hardware), with additional support from your (onsite) technical contacts in the event that physical server interaction is required. Connectivity issues within your network are reported by the GGC Operations team to your technical contacts for resolution. Specific details of Support and Maintenance responsibilities are provided in the Contract of Service.

11


12

Chapter 6. Maintenance and Support Overview

CHAPTER 7

Frequently Asked Questions

What is the expected hit rate of the cache? Cache hit rate will vary by network based on the number of users served by the cache, their usage patterns, the size and type of GGC node, and the number of nodes deployed in your network. We have typically seen hit rates of 70% to 90%. Will the users be required to make any changes to take advantage of GGCs? No, users do not have to make any changes to take advantage of the GGC node. Requests are routed to the node automatically by Google’s servers. If the GGC node is unavailable for any reason, user requests will be sent to another node or directly to Google. What changes are required in our network or DNS infrastructure to send traffic to the GGC node? You must ensure that the GGC node is reachable from the Internet, with no filters or proxies in the path. A BGP session is also required between the node and your BGP peer, so that you can advertise the list of prefixes that you would like served by the node. Further information can be found in the Installation Guide. We provide transit services for other ISPs. Will their end users use the GGC node? They certainly can, and if your network is their primary path to Google, then they probably should. If you would like this to happen, you must ensure that their User and DNS resolver IP ranges are included in the prefixes you advertise to the node. How will hardware and software installation be handled? Google will deliver the rack-mountable servers to your specified shipping address. You are then required to physically install these servers into your rack(s) and carry out the initial software installation. Google takes care of the remaining installation remotely. Further details are available in the Install Guide. Can I control the bandwidth used by the GGC node in my network? Yes, Google works with operators during activation to set the maximum bandwidth level for each GGC node deployed in an operator’s network. This process takes into account the node’s configuration and network capacity available to the GGC node. This is generally ramped up over several days. After activation, this maximum can be changed by contacting the GGC Operations team. How will Google send content to the GGC Node? Will there be content pre-loaded on the machines? The local cache is filled on a read-through basis when content is requested by the end user. If the GGC node already has the requested content in its local cache, it will serve the content via your network to the end user, improving the user experience and saving bandwidth. If the content is not stored on the GGC node, and the content is cache-eligible, the node will retrieve it from Google, serve it to the user, and store it for future requests. Otherwise, the request will be served from the nearest upstream node which has the content. No content is pre-loaded.

13


Who will own the GGC node? Google retains ownership of the hardware and software that makes up the node. Will sensitive ‘Personally Identifiable Information’ (PII) be stored on the GGC node? No, user privacy is of utmost importance to Google. No PII is ever stored on a GGC node. Will other ISPs’ customers be sent to the GGC node on our network? The node gracefully redirects away all requests from users outside of the prefixes you are advertising. For this reason, access control lists limiting IP ranges that can reach the cache are not permitted. If you provide service to downstream ASNs, you must ensure that their prefixes and resolvers are provided via the BGP feed.node. What Google services will be supported by the GGC node? Typically, a majority of the traffic routed through the GGC node is static content such as YouTube videos and Android Market downloads. Other Google web services, such as Google Search, may also be proxied and/or cached based on a number of factors, including legal requirements, available capacity and expected improvement in performance for end users. These services could include (but are not limited to): • YouTube • Google Search • Google Plus • Google Maps (including map tiles and street view) • Google Earth • Google Docs • Google Scholar • Google News • Android Market • Picasa Web Albums • DoubleClick by Google Note: Exact services cached and/or proxied by GGC will vary based on Google’s legal and commercial requirements, and Google’s discretion on possible performance impact of serving traffic through the node. Please understand that Google’s intent is to serve users from the best possible location for each service, which may or may not be the local GGC node. Can I talk about GGC with my customers and peers? Can I include references to GGC in my marketing collateral? In short, no. We ask your company to consider this agreement and associated activity to be commercially confidential.

14

Chapter 7. Frequently Asked Questions

CHAPTER 8

Further Reading

8.1 Google Global Cache • GGC Multi-Node concepts: – http://peering.google.com/static/downloads/GGCMultinodeDeployments.pdf • GGC Install Guide: – http://peering.google.com/static/downloads/GGCInstallation-R720.pdf • Section on caching in ‘Google Peering & Content Delivery’: – https://peering.google.com/about/ggc.html

8.2 Server Hardware • Dell PowerEdge R720 and R720xd Technical Guide: http://i.dell.com/sites/content/shared-content/datasheets/en/Documents/dell-poweredge-r720-r720xd-technical-guide.pdf • Dell PowerEdge R720 and R720xd Owner’s Manual: ftp://ftp.dell.com/Manuals/Common/poweredger720_Owner’s%20Manual_en-us.pdf

15

GoogleGlobalCache-R720

Recommend Documents