Dedicated server provider Softlayer started its own content delivery network. This will allow large web site to serve static files like images, pdf and other media files faster. Many large sites like youtube, yahoo use CDN network to serve content
Hadoop is like MogileFS, is a Open Source distributed file system. http://wiki.apache.org/hadoop/ Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a
MySQL proxy is created by author of lighttpd web server. http://forge.mysql.com/wiki/MySQL_Proxy MySQL Proxy is a simple program that sits between your client and MySQL server(s) that can monitor, analyze or transform their communication. Its flexibility allows for unlimited uses; common
Varnish is a HTTP accelerator, build from ground up with performance in mind. Varnish is targeted primarily at the FreeBSD 6/7 and Linux 2.6 platforms, and will take full advantage of the virtual memory system and advanced I/O features offered
Unison is a file-synchronization tool for Unix and Windows. It allows two copies of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to
MogileFS is our open source distributed filesystem. http://danga.com/mogilefs/ Some of the advantages are No Single point Failure Automatic file replication Better than RAID Shared-Nothing — MogileFS doesn’t depend on a pricey SAN with shared disks. Every machine maintains its own
WordPress.com use Amazon S3 for storing images, they do not serve images directly from Amazon, they use Varnish reverse proxy at front end, so not all images need to be served from S3, Varnish cache popular images and serve directly
Many large web sites use reverse proxy servers in front end. For example flicker use squid in reverse proxy mode, wordpress use varnish in front end. Varnish was written from the ground up to be a high performance caching reverse
Video Sharing site DailyMotion (Alexa Rank – 104) use Isilon for storing video. The Isilon IQ family of clustered storage systems was designed from the ground up to meet the needs of data-intensive enterprises and high-performance computing environments
