I have been at Shutterfly for almost two and a half years now. I am very proud of the teams achievements in this timeframe. We brought a completely new technology into Shutterfly, and while we had some kinks, I am very proud of where we ended up. Shutterfly is one of the larger and perhaps more complex MongoDB production installations in existence. For some amount of time I will continue to help Shutterfly transition.

We created ObjectRocket with the idea that we can do something better than anyone else. Even the ‘big guys’. It’s time for me to focus on making customers happy with fantastic Engineering and Technology. Our initial concept has some legs, so I am going to pour my energies into making this idea a success. Frankly, I have never been so excited about something I have been doing in my life.

What is it we are doing? Well, we aren’t quite ready to announce that quite yet. But I can say that we plan to solve, what we believe are, serious problems in the MongoDB and NoSQL space. For now, it’s heads down on building something completely new and amazing. If you are curious, we will provide updates on Twitter so just follow us.

Mongoperf can be found in the bin directory of your 2.1+ MongoDB distribution. The utility is invoked with options to tell it how to perform a sample I/O run, and it gives output indicating the performance of your disk I/O subsystem. The utility generates random I/O over a single file. One great aspect of this utility is it accesses the I/O subsystem very much like MongoDB itself does. It uses the same memory mapped files interface just like MongoDB itself. Running the utility with –help outputs the various options for running. The utility takes a JSON document as input. Typically I like to store the various options in a .js file and emit that into stdin of the utility as such:

$>echo "{nThreads:2,w:true,r:false,mmf:true}" > perfsession1.js
$>
$>mongoperf < perfsession1.js

The most powerful usage of this tool is to bypass the Linux page cache, and perform Direct I/O. Running in this manner shows true gauge on how fast your storage subsystem is. This is done via the {mmf:false} assignment in the config document. When mmf is false, then Mongoperf performs direct I/O via the O_DIRECT flag. So no system memory is used to cache file buffers. That said, anything below the linux filesystem could be cached, including controller and drive caches. For example:

$>echo "{nThreads:2,w:true,r:false,mmf:false,fileSizeMB:10000}" > perfsession1.js
$>
$>mongoperf < perfsession1.js

The output of the utility shows the throughput as the threads ramp up to the nThreads limit you specify and then will just run at that level of concurrency forever. The output is in Disk Operations per Second. For example:

mongoperf < perfsession1.js
mongoperf
use -h for help
options:
{ nThreads: 2, w: true, r: false, mmf: true, fileSizeMB: 256 }
creating test file size:256MB ...
testing...
new thread, total running : 1
9922222 ops/sec 
11197085 ops/sec 
11130190 ops/sec 
11308340 ops/sec 
11068894 ops/sec 
11084711 ops/sec 
11447503 ops/sec 
11233556 ops/sec 
new thread, total running : 2
12925223 ops/sec 
12142161 ops/sec 
12290461 ops/sec 
12476609 ops/sec 
12392879 ops/sec 
11772720 ops/sec 
12308988 ops/sec

There are a couple things to note about this utility. For one, be sure to specify a reasonably large file for fileSizeMB. Maybe like 10000. This ensures you are wider than a single stripe if using RAID, this also ensures that you have enough data on disk that random I/O is truly random. Let the test run long enough that the system normalizes (caches get populated, etc). Try for 3-5 minutes or so as a general guideline. Also try different nThreads values to get a feel for the level of disk concurrency your system can tolerate. Also, another great test is to test the read and write options together, and separate to get a feel for how well your system does on each on of these types of workload. For instance, you can test the write cache on a controller quite effectively using this approach.

In the future I think having an option for creating multiple files to I/O against would be a great option and allow for a bit more realistic workload. Perhaps an option to output the results in BSON/JSON would be cool too.

If you really want to geek out, check out the deep dive series of video’s on the amazing drivetrain in the car.

I have a couple take-aways from this years conference:

• The new aggregation features of MongoDB are going to be great, I am excited to start playing with them.
• 10gen is listening to us about the locking behavior of MongoDB, and the roadmap is promising.
• Lots of people use AWS for MongoDB.  More than I would have thought considering EBS is horrid.  Most everyone thought AWS and EBS was painful, but the pain was outweighed by the perceived benefits.
• Everyone agrees schema design is key.  The money quote from the conference was:

Schema free != Design free –@nathenharvey @nathanharvey 

First of all, I am incredibly honored to be considered in such a group of high caliber folks. But beyond that I was struck by the level of commitment that 10gen has to making sure the roadmap is proper and that they are listening very closely to the community. I feel like with such a commitment the areas for improvement in MongoDB will get good attention and focus. I think this is a huge component for consideration when looking at MongoDB. The community is very very strong.  I made lots of good contacts and was able to connect with friends.

Thanks 10gen for putting on such a good event, and above all, listening to your customers.

*changed the video to the updated version from 10gen

1. Documents: Using MongoDB instead of a XML based system.

MongoDB is a document oriented data store. XML is a document language. By moving a traditional XML app to MongoDB one can experience a few key advantages. The typical pattern in XML is to fetch an entire document, work with it, and put it back to the server. This approach has many downsides including the amount of data transmitted over the wire, collision detection/resolution, data set size, and server side overhead. In the MongoDB model, documents can be updated atomically, fetched by index, and even partially fetched. Applications are simpler, faster, and more robust.

2. Metadata storage systems.

Any system that stores metadata can be a great use case in MongoDB. Such systems typically have a pattern of adding attributes about some type of entitiy, and then needing to query/sort/filter based on these items. The prototypical use case for such a system is the use of tags. The tag implementation is so superior in MongoDB that almost single handedly compels one to use MongoDB for any system needing tags. Simply put:

db.mymetadata.save({stuff:"some data here", thing:"/x/foo/bar.mpg", tags:['cats','beach','family']})
db.mymetadata.ensureIndex({"tags":-1})
db.mymetadata.find({tags:'cats'})
...
"indexBounds" : {
		"tags" : [
			[
				"cats",
				"cats"
			]
		]

In many metadata systems the schema may vary depending on the metadata itself. This allows for huge degrees of flexibility in the data modeling of applications that store metadata. Imagine a media storage service that can store video and image data in the same collection but with different attributes about each type of metadata. No joins needed on query, and the logical I/O path is minimized! MongoDB now supports sparse indexes, so indexes on attributes that are not in every document are kept at a minimum size.

3. Read intensive systems

Any system where the amount of change is low, and read is high is a nice sweet spot for MongoDB. MongoDB has a nice scaling property with both the replica sets functionality (setting SLAVE_OK), as well as using sharding. Combine this with the document model, and metadata storage capabilites one has an excellent system for say a gawker clone. Reads can come off any one of N sharded nodes by say, story_id, and reads can be geographically targeted to a slave for reads. Keep your data clustered by key for super fast I/O.

MongoDB Performance Tuning