HostOnNet Blog

Category Archives: Large Sites

Softlayer started content delivery network

Dedicated server provider Softlayer started its own content delivery network. This will allow large web site to serve static files like images, pdf and other media files faster. Many large sites like youtube, yahoo use CDN network to serve content

Hadoop Distributed File System

Hadoop is like MogileFS, is a Open Source distributed file system. Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a

MySQL Proxy

MySQL proxy is created by author of lighttpd web server. MySQL Proxy is a simple program that sits between your client and MySQL server(s) that can monitor, analyze or transform their communication. Its flexibility allows for unlimited uses; common

Scaling PHP applications with Varnish

Varnish is a HTTP accelerator, build from ground up with performance in mind. Varnish is targeted primarily at the FreeBSD 6/7 and Linux 2.6 platforms, and will take full advantage of the virtual memory system and advanced I/O features offered

Unison rsync like tool work on windows and linux

Unison is a file-synchronization tool for Unix and Windows. It allows two copies of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to

MogileFS distributed filesystem

MogileFS is our open source distributed filesystem. Some of the advantages are No Single point Failure Automatic file replication Better than RAID Shared-Nothing — MogileFS doesn’t depend on a pricey SAN with shared disks. Every machine maintains its own

WordPress use Amazone S3 for storing images use Amazon S3 for storing images, they do not serve images directly from Amazon, they use Varnish reverse proxy at front end, so not all images need to be served from S3, Varnish cache popular images and serve directly

varnish reverse proxy server

Many large web sites use reverse proxy servers in front end. For example flicker use squid in reverse proxy mode, wordpress use varnish in front end. Varnish was written from the ground up to be a high performance caching reverse

Clustered Storage

Video Sharing site DailyMotion (Alexa Rank – 104) use Isilon for storing video. The Isilon IQ family of clustered storage systems was designed from the ground up to meet the needs of data-intensive enterprises and high-performance computing environments