Wednesday, May 30, 2012

myVidster Embraces NoSQL

If you are a hardcore web techie you have seen the word NoSQL toss around.  In a nutshell its an alternative database management system.  Most websites (including myVidster) are build with mySQL, a traditional relational database management system (RDBMS).  For the most part mySQL is one of the best database solutions out there and will meet most (if not all) of your data needs.

As myVidster user base grows so does its need for more and more data.  When your data gets into millions of stored rows and you are trying to insert 3000+ rows of data every 2 minutes, mySQL faults begin to show.

In my experience mySQL does everything well expect for inserting large amounts of data in a given batch job and your queries can take a performance hit when you are have to sort large amounts of user data.  Most developers will agree with my issue with inserts, but some will argue that proper use of indexes will solve your performance issues when sorting.  I beg to differ.

Lets say you want to sort all user notification by date and return the last 5.  You will put an index on both the user id and the post date.  If the users has 5000 notifications, mySQL will run a filesort on the post date index (all 5000).  Disk I/O starts to become an issue when you are auto refreshing user's notification every minute.  Now if mySQL provided sorted indexes, then fetching the last 5 rows by date would not require a filesort, but sadly mySQL does not provide this feature.

I knew I have reach the limits of what mySQL can do so I decided to look into MongoDB. MongoDB is a NoSQL document oriented storage database.  During my testing MongoDB could insert documents (there like rows in a table) at a rate of 1000+/sec.  That solved my first problem, what about querying large data sets?  MongoDB provides sorted indexes and as long as you define the sort order of your indexes correctly (ascending or descending) MongoDB will not need to perform an nscanned (its like mySQL filesort) on the user's data.  I am also using MongoDB for backing up and achieving data as well.

Am I planning on replacing mySQL with MongoDB?  For now my answer is no, MongoDB does not support table linking and the code rewrite would be massive and the performance gain might not be worth it.  I do see myself using MongoDB for new features that I have in the pipeline.

Well, I think that is enough geek talk for now, take care and keep collecting!

46 comments:

Jyemenai said...

...this post turned me on, lol.

Anna Smith said...

Embraces NoSQL.... LOL..
Madhyamam Epaper is the one of the most popular Malayam Language NEwspaper. Dinathanthi Epaper is the Tamil Daily NEwspaper. Searching for Tarun Bharat Nagpur Online.

Anna university said...

What you said is absolutely correct, thanks for sharing :) anna university results 2017

Haseeb Khan said...

this is very interesting information keep it up nice sharing... Windows 7 Loader Activator

downloadnetflixapk said...

Thanks for sharing. It's great

Saurav Arora said...

Thanks for your personal marvelous posting!

I quite enjoyed reading it, you happen to be a great author.

I will make sure to bookmark your blog and will often come back in the future.
I want to encourage that you continue your great posts , have a nice weekend!

JANA GANGA said...

Psychologist in Chennai
Psychologist Doctor in Chennai
Psychotherapy in Chennai
PERSONALITY DEVELOPMENT TRAINING IN CHENNAI
Life Coach in Chennai

crack download said...


Hi there to everybody, it’s my first visit of this blog; Wondershare Dr.Fone Crack
Hi there to everybody, it’s my first visit of this blog;
this webpage consists of remarkable and actually excellent material in favor of visitors.
this webpage consists of remarkable and actually excellent material in favor of visitors.

RIchard Single said...

great work Malwarebytes Anti-Malware 3.3.1

elizabith said...
This comment has been removed by the author.
360 Total Security 9 Key said...

It’s actually very complicated in this busy life to listen to news on Television, thus I
simply use internet for that reason, and obtain the most recent information. fl studio 12 crack
Freemake Video Downloader Key

Oliver Maurice said...
This comment has been removed by the author.
Imran Hashmi said...

Amazing some works here keep sharing with us.
Lucky Patcher
Freemake Video

maria hashmi said...

I like this blog for a sharing.Its a good blog.
adobe-after

maria hashmi said...

Awesome blog I really impressed.
Cyber Link Photodirector

Spotify Premium said...

Download Spotify Premium APK for android.

fifa world cup 2018 said...

fifa world cup 2018 Russia will be the quadrennial International Football Competition that will be going to be kicked off from 14th June and run till 15th July 2018 in Russia.

Latest Technoogy said...

Kohls Credit Card Login - Portable My Kohl's Charge. Manage your record ANYTIME, ANYWHERE! Check your change, pay your bill, and view late history in a rush. Snap here to sign in to My Kohl's Charge.

Champa Kali said...


KMFUSA is the full-form of Kia Motors Financial Service, KMFUSA is also known as the online login service for the customers of Kia Motors Finance that offers a big range of loans for all the auto finance along with providing leasing solution for their customer all over the world. KMFUSA Login

maria hashmi said...

Amazing sharing, keep it up.
Gridinsoft Anti-Malware

maria hashmi said...

Great post, thanks for sharing.
Total Uninstall
DVDVideoSoft
FL Studio
GridinSoft Anti-Malware

Jonna Richard said...
This comment has been removed by the author.
Michael Jones said...

All Assignment Help is a web portal where students get help in making assignments for all the subjects, with the help of our experts. You will get 100% plagiarism free assignment. Expert’s consultation is also available for students. If they have any query they can contact with our experts anytime.

kepmari said...

TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TNEB Latest Recruitment 2018
TANGEDCO Latest Recruitment 2018

dll files fixer crack said...

Thank you for providing information about it

Students Assignment Help said...

Students Assignment Help serves the best dissertation editors services to the college or university students at an affordable price. To score good marks in their college and university they can avail our excellent services. Our writer can write complicated homework assignments as they are well-versed with every academic topic. So, you can email us for any academic help at [email protected]

Zaib Hafeez said...

thank you dear.. informative site
Camtasia crack version with keygen

Aswin said...

Hello there. I found your web site by the use of Google at the same time as searching for a comparable subject, your site came up. It seems great. I have bookmarked it in my google bookmarks to come back then.

Acupressure Care ProductsPersonal Care ProductsPersonal Care Products Shops in ChennaiAcupressure Care ProductsPersonal Care ProductsPersonal Care Products Shops in ChennaiAcupressure Care Products in ChennaiAcupressure Care Products in IndiaBuy Acupressure Care Products OnlineAcupressure Care Products PricesAcupuncture Needle Shops in Chennai
Acupuncture Needle Prices in ChennaiBuy Acupuncture Needle Online in IndiaAcupuncture NeedlesAcupuncture Needle Products in Chennai
Acupuncture Ring in ChennaiBuy Acupuncture Ring Acupuncture Ring IndiaAcupuncture Ring ShopsAcupressure Foot Roller ShopsAcupressure Mat Shops in ChennaiAcupressure Mat Shops in IndiaSpine Massager in ChennaiBuy Spine Massager Online in India

Satta Market said...

This is really a nice and informative, containing all information and also has a great impact on the new technology. Thanks for sharing it. Satta Matka

กวิน ฮา said...

thank for good sharing,....

โกลเด้นสล็อต
goldenslot
golden slot
ทางเข้า goldenslot
goldenslot online

Ava Lee said...

You all may feel that you need a better assistance from a online ghostwriter UK in your academic so that you can boost it up.Our service has a pool of ghost writer in UK who are actually expertise in these fields. If you do not wish to risk that, it is always safe to place your bets on the most trusted academic.If you do not wish to risk that, it is always safe to place your bets on the most trusted academic writing service in the UK

MyAssignmentHelpAu said...

This is one of the finest post i have ever seen. The information is genuine and relatable and i am looking forward to avail the services of My Assignment Help . You can email us at [email protected] or Phone Number: +61-2-8005-8227

sharon smith said...

Being an academic writer from past 5 years providing assignment writing services to college and university students also associated with Myassignmenthelp platform. I am dedicated in providing best online academic writing services to the college students at the affordable rates.

Javed Hashmi said...

wow nice post thanks admin

Microsoft Office 2019 Crack
pro evolution soccer 2019 crack

Ryan Langlands said...

garageband for windows is an application which operates just less the recording tool but higher than that. It is an entire workstation of music that has to be incorporated in your device. facetime for windows is a Excellent program for Apple Users for video calling, while there are programs which could be either purchased or are all available at no cost. In fact, I do enjoy the Facebook Messenger and WhatsApp Messenger, but no-one can ever overcome Apple's iMessage on PC.

write my essay said...

If you are looking for a service provider that can deliver your essays in the least amount of time, then, MyAssignmenthelp.com is the best place for you. You will only have to ask us, “help me write my essay ”, and your order will be delivered to you on time.

Samuel Sam said...

Hire experienced dissertation experts with MyAssignmenthelp.com
We provide students with other dissertation services like:
dissertation editors online
Proposal writing service
Methodology writing service
Dissertation correction service
Our service charges are low, and we always complete the dissertations on time.

Nikki Heysen said...

StudentsAssignmentHelp.com experts present excellent Case Study With Solution For MBA for the students struggling with this complex assignment writings. We have top-class academic experts that compose all your academic paper from scratch and make sure you score best grades.

Australian assignment Help said...

Get best marketing assignment help in Australia provided by Australian Assignment Help. The marketing experts are highly talented and can provide help on any marketing related topics. Get finest quality marketing help in Australia
.

UKDissertationHelp said...

The above post has given reliable and genuine information about Assignment Help Australia. Looking forward to avail their eminent services.

hermilie johnson said...


Losing sleep over unfinished essays but can't afford the assistance of costly essay writers? Wondering whether the online service providers will write my essay cheap in UK? Facing a deadline for finishing the essay? Well, your anxieties end here. We have with us a team of essay writing services for UK students who can solve all your writing issues so that you score high grades in your essay assignment. They will deliver the completed essay to you well before the deadline. With decades of experience in helping students, this team of professional essay typers best in UK.

FirstAssignmentHelp said...

Thanks for sharing such a nice piece of information to us. This is very knowledgeable for me. I am John and i am offering Assignment Help to students of Australia and all over the world.

Aliya said...

Amazing Post I really enjoyed after reading.
Advanced SystemCare

MyAssignmentHelpSG said...

Blog Commenting: An unmatched and nonpareil post i have ever seen. The content is so appealing that it has created an impulse to avail Assignment Help Singapore services.

Eva Levin said...

You need thesis help or assistance from experts. hire our scholars and get rid of all your worries. if you are in the dilemma whether it’s right to avail professional help, then you should stop thinking too much.our service has a pool of ghostwriter in UK who are actually expertise in these fields. we have 4000+ writers who are very much flexible and also professionals. thus, they know every tricks and tips of accomplishing academic tasks.

ABHINAV GARG said...

Good information is very useful, you have given nice outlook to this topic. I definitely promote your content to other website so that lot of people get aware about this topic. It reveals how nicely you understand this subject.

Protect Your PC With Best Virus Scanner