Monday, April 30, 2012

Image Server Revisited

Since my last post I have made important changes to how images (mostly thumbnails) are serve on myVidster.  Stated in my last post I divided the image workload between the new image server and the master server.  I really did not like this setup because the master server should not be wasting disk I/O on image hosting.

My first attempt was to setup the image server to handle the entire load.  This solution would have work if I was running RAID on the server, but I was not.  It only took a few minutes before the server's hard disk was overwhelm.  When you run into the dreaded disk I/O bottleneck you have two options which are to upgrade your hardware (SSD or RAID) or memory caching.

If you have been keeping up with my blog, you will know that I try to use in memory caching whenever possible so I did some research and learn about using a reverse proxy for image caching.   For the reverse proxy I decided to give Varnish Cache a try, which was recommended by a developer/friend.

Setting up Varnish has straight forward the only catch was that I needed to disable cookies (remove req.http.Cookie) so that the images were always cached.  Technicality the image server is now a web application accelerator with the task of caching and serving images.

As for performance, currently I am getting a hit rate around 85% with 400-500 requests per second.  CPU load hovering around 10% which gives the server lots of room for handling more traffic when needed.  Overall I am very pleased with Varnish's caching performance and would recommend this solution to those who want to accelerate image serving and to reduce disk I/O.

For the curious here is a list of memory caching solutions that myVidster uses.
Alternative PHP Cache (APC) opcode caching for PHP
Memcached for user session caching

Take care everyone and keep collecting!

14 comments:

Anna university said...

OMG, but how did you get it, - any way thanks for sharing :) anna university results 2017

Edward Cross said...

It was a wonderful chance to visit this kind of site and I am happy to know. thank you so much for giving us a chance to have this opportunity! I will be back soon for updates.custom essay writers

Oliver Maurice said...

Take a look here for some information about phone gps locators and monitoring systems that allow its users to track someone's device location via the world. That's great!

ernestiriveraas said...

In the long run, after total transfer, your backup is total. Official Page If you have a Jailbroken apple iphone after that you could make use of Cydia App.

Michael Jones said...

Many students face the problem of stress in college. Whether it is because of social anxiety, or not being able to clear exam etc. Most of the times, students get stressed due to their piled up assignments. If it is with you also then you might seek for Java Assignment Help

Jonna Richard said...

Welcome to the Best writer Review, Here you can get the best All Assignment Help reviews sites. We strongly urge you to check our entire website once and we will assure you will find this review website very useful. Our hard work will be rewarded if students like you will appreciate our effort and spread the message about this site with your class-fellows and friends.

kepmari said...

TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TANGEDCO Latest Recruitment 2018

Anonymous said...

Thank you for posting such a great article! I found your website perfect for my needs
how to get windows pin

mahdi pedram said...

خرید بلیط هواپیما تیک بان، یکی از کامل ترین سایت های فروش بلیط هواپیما در ایران است که با تجمیع بلیط های موجود در کلیه ی مسیرها دست شما را در انتخابی مقرون به صرفه و ایده آل باز میگذارد.  بلیط هواپیما چارتر، بلیط هواپیما ماهان، خرید بلیط هواپیما آسمان، خرید بلیط هواپیما ایران ایر، بلیط هواپیما ماهان، بلیط هواپیما مشهد، بلیط هواپیما لحظه آخری، بلیط قطار و .... را میتوانید توسط سامانه رزرو بلیط هواپیما تیک بان تهیه نمایید.

mahdi pedram said...

خرید بلیط هواپیما تیک بان، یکی از کامل ترین سایت های فروش بلیط هواپیما در ایران است که با تجمیع بلیط های موجود در کلیه ی مسیرها دست شما را در انتخابی مقرون به صرفه و ایده آل باز میگذارد.  بلیط هواپیما چارتر، بلیط هواپیما ماهان، خرید بلیط هواپیما آسمان، خرید بلیط هواپیما ایران ایر، بلیط هواپیما ماهان، <a href="https://tikban.com/

Dave Leena said...

If you are in need for online writing assistance for an intricate thesis topic, then avail our assignment writing service in U.S. and save your time to relax and do your studies properly. Our assignment help online service in USA has earned huge popularity among both domestic and international students. There’s no better place in the USA than MyAssignmenthelp.com. Contact us now to buy assignments online in the USA Leave your tensions to us and enjoy your free time.

Aswin said...

Hello there. I found your web site by the use of Google at the same time as searching for a comparable subject, your site came up. It seems great. I have bookmarked it in my google bookmarks to come back then.

Acupressure Care ProductsPersonal Care ProductsPersonal Care Products Shops in ChennaiAcupressure Care ProductsPersonal Care ProductsPersonal Care Products Shops in ChennaiAcupressure Care Products in ChennaiAcupressure Care Products in IndiaBuy Acupressure Care Products OnlineAcupressure Care Products PricesAcupuncture Needle Shops in Chennai
Acupuncture Needle Prices in ChennaiBuy Acupuncture Needle Online in IndiaAcupuncture NeedlesAcupuncture Needle Products in Chennai
Acupuncture Ring in ChennaiBuy Acupuncture Ring Acupuncture Ring IndiaAcupuncture Ring ShopsAcupressure Foot Roller ShopsAcupressure Mat Shops in ChennaiAcupressure Mat Shops in IndiaSpine Massager in ChennaiBuy Spine Massager Online in India

Dave Leena said...

Struggling with your dissertation and searching for best dissertation help services in usa then contact with assignmenthelp.us. you can get best help with dissertation topics from our experts.

SheritaGreen456 said...

Interesting article thank you for sharing them, I hope you will continue to have similar posts to share with everyone. TV Series Black Hoodie Jacket