Monday, April 30, 2012

Image Server Revisited

Since my last post I have made important changes to how images (mostly thumbnails) are serve on myVidster.  Stated in my last post I divided the image workload between the new image server and the master server.  I really did not like this setup because the master server should not be wasting disk I/O on image hosting.

My first attempt was to setup the image server to handle the entire load.  This solution would have work if I was running RAID on the server, but I was not.  It only took a few minutes before the server's hard disk was overwhelm.  When you run into the dreaded disk I/O bottleneck you have two options which are to upgrade your hardware (SSD or RAID) or memory caching.

If you have been keeping up with my blog, you will know that I try to use in memory caching whenever possible so I did some research and learn about using a reverse proxy for image caching.   For the reverse proxy I decided to give Varnish Cache a try, which was recommended by a developer/friend.

Setting up Varnish has straight forward the only catch was that I needed to disable cookies (remove req.http.Cookie) so that the images were always cached.  Technicality the image server is now a web application accelerator with the task of caching and serving images.

As for performance, currently I am getting a hit rate around 85% with 400-500 requests per second.  CPU load hovering around 10% which gives the server lots of room for handling more traffic when needed.  Overall I am very pleased with Varnish's caching performance and would recommend this solution to those who want to accelerate image serving and to reduce disk I/O.

For the curious here is a list of memory caching solutions that myVidster uses.
Alternative PHP Cache (APC) opcode caching for PHP
Memcached for user session caching

Take care everyone and keep collecting!

241 comments:

«Oldest   ‹Older   201 – 241 of 241
Real Jackets said...

Does operating a well-established blog such as yours take a large amount of work?
I’m completely new to running a blog but I do write in my journal on a daily basis.
I’d like to start a blog so I can share my personal experience and views online.
Please let me know if you have any ideas or tips for brand
new aspiring blog owners. Thankyou!The North Face Brown Jacket

mudasir said...

The Big Pizza He.s Very GreatFull ANd Very Gorgeous Colour Combination

thebigpizza

ArizDigital said...

thank you PPC marketing dubai

Unknown said...

Wonderful submit, very informative. I wonder why the opposite experts of this sector don't notice this. You should continue your writing. I am sure, you have a great readers' base already!
TellPopeyes

مبلمان اداری said...

مبلمان اداری بینو
تولید کننده انواع مبلمان اداری
مبلمان اداری کلاسیک
مبلمان اداری مدولار
مبلمان اداری مدرن
با ما تماس بگیرید
https://bayno-co.com

Anonymous said...

He Is Very Great And Very Knowledge Full Artical And Very Help FUll Artical
Commercial Moving Decatur GA

Avail Assignment Help said...

Avail Assignment Help provides the best assignment help. Our experts are Ph.D. holders and know every subject. They provide the document before the deadline at pocket-friendly prices.
https://www.availassignmenthelp.com/make-my-assignment-help

Learn Hacking said...

Thanks for sharing, great post. Check AWS Solutions Architect Associate Dumps

Anonymous said...

You delivered such an impressive piece to read, giving every subject enlightenment for us to gain information. Thanks for sharing such information with us due to which my several concepts have been cleared. 사설토토

chrome hearts hoddie said...

Hello. Extremely decent site!! Man .. Incredible .. Superb .. I'll bookmark this site and take the feeds likewise… I am glad to find such a lot of supportive data here inside the article. A debt of gratitude is in order for sharing…

Sarah Josave said...

Hello. This is a fantastic website!! Man.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. Amazing.. I'm going to bookmark this site and subscribe to the RSS as well... I'm delighted to discover so many useful information within the article. For sharing, I owe you a debt of appreciation... Cruella Jacket

Emma williams said...

We fly to the most sought-after destinations in and around the US. Plan your trips at ease with us by our best destination guides. you can even grab a chance to find the best-suited deal for your next flight with us at a reasonable price. Explore seattle to houston flights for the best deals.

mohsen said...

خودتان است ،به این دلیل که قم قطب تولید مبلمان کشور است قیمت مبل در این شهر حداقل 30 درصد پایین تر از قیمت مبل در شهر خودتان است

یک مبل در شهر قم در تولید مبل با قیمت 16 میلیون ممکن است در نمایشگاه مبل شهر شما یا حتی در نمایشگاه مبل در قم ،از 18 تا 24 میلیون نیز فروخته میشود پس از لحاظ قیمت خرید مبل از تولید کننده بهتر است،و اکثرا”اطلاعاتی از این ندارند که مبل را از تولیدی خرید کنند.

2-کیفیت مبل راحتی

شما وقتی از تولیدی مبل راحتی سفارش مبل میدهید در جریان مواد استفاده شده در مبل قرار میگیرید و مطمعن هستید که نسب به قیمتی که پرداخت میکنید از مبل با کیفیتی برخوردار میشوید.
خرید مستقیم مبل راحتی

Real Jackets said...

I’m very happy to look at your publish. Thanks a lot. I appreciated your blog. Thank you for the information and sharing. Desirable activity is excellent for good fortune
black widow vest

Service Apartments Gurgaon said...

Wow, such an awesome blog you have written there and you and I get exactly what information I am looking for, in the third paragraph you put amazing effort to explain the theme of the content. As a content writer, I can understand efforts because when students ask me for bsbpmg511, I do the same.

sexy stories hindi said...

New best Hindi and English Sex Stories for free, sex stories hindi, Indian sexy stories daily, hot girls, bhabhi and aunties. फ्री सेक्स कहानी ka मजा लें.

Taniya Vashist said...

Thanks for sharing such captivating information with us. And I hope you will share some more information about. Please keep sharing!

Taniya Vashist said...

Thanks for sharing such captivating information with us. And I hope you will share some more information about. Python Training in Noida please keep sharing!

nightqueen said...

I am feeling much blessed to be here and reading your thought provoking blog. It has truly increased my knowledge. I will surely come again and again to overcome my thirst for knowledge.New best Hindi and English Sex Stories for free, sex stories hindi, Indian sexy stories daily, hot girls, bhabhi and aunties. फ्री सेक्स कहानी ka मजा लें.

hindi porn kahani

Marry Jones said...

This is marry, a professional blogger, Yet working with Avail Coupon Code, For more visit Cupshe Coupon Code

nightqueen said...


The concepts you are giving in the blogs are really worthless and priceless because these are paving the way to progress in the field of sciences. Heads off.
New best Hindi and English Sex Stories for free, sex stories hindi, Indian sexy stories daily, hot girls, bhabhi and aunties. फ्री सेक्स कहानी ka मजा लें.

freesexkahani

jhon said...

Thanks for your comments! We haven't thought about the gaze problem, so I appreciate your ideas. I personally don't think that a head-mounted display is required for this.Divide the Youth Hoodie

Anonymous said...

에볼루션접속 먹튀검증 안전노리터 go

Lisa Alix said...

vapordna coupon code

Lisa Alix said...

The deal grabbers is aiming to provide you tips for saving with our reliable vapordna coupon code which are updated recently in 2022.

سطحة شمال الرياض said...

شراء سيارات تشليح
تشليح سيارات If you have been keeping up with my blog, you will know that I try to use in memory caching whenever possible so I did some research and learn about using a reverse proxy for image caching. For the reverse proxy I decided to give Varnish Cache a try, which was recommended by a developer/friend.

Setting up Varnish has straight forward the only catch was that I needed to disable cookies (remove req.http.Cookie) so that the images were always cached. Technicality the image server is now a web application accelerator with the task of caching and serving images.

As for performance, currently I am getting a hit rate around 85% with 400-500 requests per second. CPU load hovering around 10% which gives the server lots of room for handling more traffic when needed. Overall I am very pleased with Varnish's caching performance and would recommend this solution to those who want to accelerate image serving and to reduce disk I/O.

SPORTSTOTOTOP said...

𝒢𝑅𝐸𝒜𝒯 𝒫𝒪𝒮𝒯! 𝐼 𝒟𝒪 𝐿𝐼𝒩𝒦 𝐵𝒰𝐼𝐿𝒟𝐼𝒩𝒢 𝐹𝒪𝑅 𝑀𝒴 𝐵𝒰𝒮𝐼𝒩𝐸𝒮𝒮 𝒫𝑅𝒪𝑀𝒪𝒯𝐼𝒪𝒩 𝒮𝒪 𝒯𝐻𝐼𝒮 𝒫𝒪𝒮𝒯 𝒱𝐸𝑅𝒴 𝒰𝒮𝐸𝐹𝒰𝐿 𝐹𝒪𝑅 𝑀𝐸, 𝐵𝐿𝒪𝒢 𝒞𝒪𝑀𝑀𝐸𝒩𝒯𝐼𝒩𝒢 𝐼𝒮 𝒜 𝒢𝑅𝐸𝒜𝒯 𝒮𝒦𝐼𝐿𝐿 𝒯𝐻𝒜𝒯 𝒟𝐸𝒫𝐸𝒩𝒟𝒮 𝒪𝒩 𝒜 𝒞𝒪𝒰𝒫𝐿𝐸 𝒪𝐹 𝐹𝒜𝒞𝒯𝒪𝑅𝒮 𝒯𝐻𝒜𝒯 𝒜𝑅𝐸 𝒱𝐸𝑅𝒴 𝒲𝐸𝐿𝐿 𝐸𝒳𝒫𝐿𝒜𝐼𝒩𝐸𝒟.
토토

PACHINKOSITE said...

G͆R͆E͆A͆T͆ P͆O͆S͆T͆! I͆ D͆O͆ L͆I͆N͆K͆ B͆U͆I͆L͆D͆I͆N͆G͆ F͆O͆R͆ M͆Y͆ B͆U͆S͆I͆N͆E͆S͆S͆ P͆R͆O͆M͆O͆T͆I͆O͆N͆ S͆O͆ T͆H͆I͆S͆ P͆O͆S͆T͆ V͆E͆R͆Y͆ U͆S͆E͆F͆U͆L͆ F͆O͆R͆ M͆E͆, B͆L͆O͆G͆ C͆O͆M͆M͆E͆N͆T͆I͆N͆G͆ I͆S͆ A͆ G͆R͆E͆A͆T͆ S͆K͆I͆L͆L͆ T͆H͆A͆T͆ D͆E͆P͆E͆N͆D͆S͆ O͆N͆ A͆ C͆O͆U͆P͆L͆E͆ O͆F͆ F͆A͆C͆T͆O͆R͆S͆ T͆H͆A͆T͆ A͆R͆E͆ V͆E͆R͆Y͆ W͆E͆L͆L͆ E͆X͆P͆L͆A͆I͆N͆E͆D͆.
슬롯머신

RACESITE said...

T͙H͙A͙N͙K͙ Y͙O͙U͙ F͙O͙R͙ S͙H͙A͙R͙I͙N͙G͙ T͙H͙E͙ B͙L͙O͙G͙ C͙O͙M͙M͙E͙N͙T͙I͙N͙G͙ S͙I͙T͙E͙S͙. T͙H͙E͙S͙E͙ W͙I͙L͙L͙ S͙U͙R͙E͙L͙Y͙ H͙E͙L͙P͙ O͙U͙R͙ W͙E͙B͙S͙I͙T͙E͙ T͙O͙ R͙A͙N͙K͙ O͙N͙ T͙H͙E͙ S͙E͙A͙R͙C͙H͙ E͙N͙G͙I͙N͙E͙S͙. K͙E͙E͙P͙ S͙H͙A͙R͙I͙N͙G͙.
온라인경마

BACARASITE said...

G͎R͎E͎A͎T͎ S͎I͎T͎E͎S͎ L͎I͎S͎T͎, T͎H͎A͎N͎K͎S͎ F͎O͎R͎ S͎H͎A͎R͎I͎N͎G͎ I͎ S͎U͎B͎M͎I͎T͎T͎E͎D͎ M͎Y͎ W͎E͎B͎S͎I͎T͎E͎ T͎O͎ A͎L͎L͎ T͎H͎E͎S͎E͎ S͎I͎T͎E͎S͎. I͎T͎’S͎ R͎E͎A͎L͎L͎Y͎ H͎E͎L͎P͎F͎U͎L͎ F͎O͎R͎ M͎E͎ T͎H͎A͎N͎K͎S͎ F͎O͎R͎ S͎H͎A͎R͎I͎N͎G͎ T͎H͎I͎S͎ B͎L͎O͎G͎ C͎O͎M͎M͎E͎N͎T͎I͎N͎G͎ S͎I͎T͎E͎S͎.
바카라

Zayn Khaled said...

The Singapore Prayer Time today's is sundown, or iftar, is 7:11 AM, and the time for dawn break, or the conclusion of Sehri, is 5:57 AM. You can read the Singapore Sehri and Iftar times for Fiqa Hanafi (sunni) and Fiqa Jafria in addition to the Singapore Prayer hours (shia).

ثبت نام وان ایکس said...

ثبت نام وان ایکس

Afterpaylogininfo said...

GOOD INFORMATION FOR YOU, CLICK THE LINK GIVEN BELOW

https://afterpay-login.info/

THANK YOU FOR VISIT, PLEASE VISIT AGAIN

Afterpay Login Info said...

afterpay login

hemanikashmira said...

GOOD INFORMATION
vystar login

Anonymous said...

[https://kinnsernet-login.info/]

[Kinnsernet Login]

Anonymous said...

[https://ncedcloud-login.info/]

[ncedcloud login]

neha said...

CentralReach Login

John Nicholas said...

Thanks for sharing valuable content. Feel free to check: Subfloor vapor barrier

Anonymous said...

I really appreciate this wonderful post that you have provided for us. I

http://mission-lane-login.info/



mission-lane-login.

pdfbama said...

دانلود کتاب آدمهای سمی pdf

«Oldest ‹Older   201 – 241 of 241   Newer› Newest»