Category Archives: large sites

Softlayer started content delivery network

Posted on May 22, 2008 by flashweb

Dedicated server provider Softlayer started its own content delivery network. This will allow large web site to serve static files like images, pdf and other media files faster. Many large sites like youtube, yahoo use CDN network to serve content … Continue reading →

Posted in large sites | Tagged , | Leave a comment

Hadoop Distributed File System

Posted on May 13, 2008 by flashweb

Hadoop is like MogileFS, is a Open Source distributed file system. http://wiki.apache.org/hadoop/ Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a … Continue reading →

Posted in large sites | Tagged , , , , , , | Leave a comment

MySQL Proxy

Posted on May 2, 2008 by flashweb

MySQL proxy is created by author of lighttpd web server. http://forge.mysql.com/wiki/MySQL_Proxy MySQL Proxy is a simple program that sits between your client and MySQL server(s) that can monitor, analyze or transform their communication. Its flexibility allows for unlimited uses; common … Continue reading →

Posted in large sites | Tagged | Leave a comment

Scaling PHP applications with Varnish

Posted on April 24, 2008 by flashweb

Varnish is a HTTP accelerator, build from ground up with performance in mind. Varnish is targeted primarily at the FreeBSD 6/7 and Linux 2.6 platforms, and will take full advantage of the virtual memory system and advanced I/O features offered … Continue reading →

Posted in large sites | Tagged , | Leave a comment

Unison rsync like tool work on windows and linux

Posted on April 24, 2008 by flashweb

Unison is a file-synchronization tool for Unix and Windows. It allows two copies of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to … Continue reading →

Posted in large sites, Linux | Tagged , , , , | Leave a comment

MogileFS distributed filesystem

Posted on April 24, 2008 by flashweb

MogileFS is our open source distributed filesystem. http://danga.com/mogilefs/ Some of the advantages are No Single point Failure Automatic file replication Better than RAID Shared-Nothing — MogileFS doesn’t depend on a pricey SAN with shared disks. Every machine maintains its own … Continue reading →

Posted in large sites | Tagged , , , | Leave a comment

WordPress use Amazone S3 for storing images

Posted on April 21, 2008 by flashweb

WordPress.com use Amazon S3 for storing images, they do not serve images directly from Amazon, they use Varnish reverse proxy at front end, so not all images need to be served from S3, Varnish cache popular images and serve directly … Continue reading →

Posted in large sites | Tagged , , , | 1 Comment

varnish reverse proxy server

Posted on April 9, 2008 by flashweb

Many large web sites use reverse proxy servers in front end. For example flicker use squid in reverse proxy mode, wordpress use varnish in front end. Varnish was written from the ground up to be a high performance caching reverse … Continue reading →

Posted in large sites | Tagged , , | Leave a comment

Clustered Storage

Posted on March 24, 2007 by flashweb

Video Sharing site DailyMotion (Alexa Rank – 104) use Isilon for storing video. The Isilon IQ family of clustered storage systems was designed from the ground up to meet the needs of data-intensive enterprises and high-performance computing environments

Posted in large sites | Tagged , | Leave a comment