Charity: Homes for our Troops

Today I joined the program to donate monthly to <a href=http://www.homesforourtroops.org/site/PageServer?pagename=homepage>Homes for our Troops</a>.

These guys really help our folks who need it the most and deserve our help and gratitude. I decided it was time I show some and will be donating monthly to help our Vetrans and their families in need. Join me!

[Read Charity: Homes for our Troops]


code-fest

[Read code-fest]


Sign up now for MongoDB Seattle happening on September 14 2012. I will be presenting on MongoDB Performance Tuning. I will be demonstrating some interesting ways of looking at MongoDB profiler data using the new Aggregation Framework as well as SSD disk usage, data locality, tools, read vs write tuning, and MongoDB 2.2 enhancements. Hope to see old friends and new!

[Read Upcoming performance tuning presentation at MongoDB Seattle]


Hello ObjectRocket

I am very excited to say that it’s time to say hello to ObjectRocket! A few weeks ago I made the move from a great place, to an even better opportunity. A couple of amazing guys and myself founded ObjectRocket Inc. It’s time to make ObjectRocket my full time job.

I have been at Shutterfly for almost two and a half years now. I am very proud of the teams achievements in this timeframe. We brought a completely new technology into Shutterfly, and while we had some kinks, I am very proud of where we ended up. Shutterfly is one of the larger and perhaps more complex MongoDB production installations in existence. For some amount of time I will continue to help Shutterfly transition.

We created ObjectRocket with the idea that we can do something better than anyone else. Even the ‘big guys’. It’s time for me to focus on making customers happy with fantastic Engineering and Technology. Our initial concept has some legs, so I am going to pour my energies into making this idea a success. Frankly, I have never been so excited about something I have been doing in my life.

What is it we are doing? Well, we aren’t quite ready to announce that quite yet. But I can say that we plan to solve, what we believe are, serious problems in the MongoDB and NoSQL space. For now, it’s heads down on building something completely new and amazing. If you are curious, we will provide updates on <a href=https://twitter.com/#!/objectrocket>Twitter</a> so just follow us.

[Read Hello ObjectRocket]


Mongoperf

Starting with version 2.1, MongoDB has a new utility called Mongoperf. Mongoperf is a great little utility for quickly testing the I/O performance of your system. I thought I would go over it a little bit.

[Read Mongoperf]


Chevrolet Volt

I have had my Volt for about a month now, and it’s a pretty darn amazing piece of engineering. If you don’t know much about the volt, here is a quickie intro.

If you really want to geek out, check out the <a href=http://www.youtube.com/watch?v=an-VyIau-FM>deep dive series</a> of video’s on the amazing drivetrain in the car.

[Read Chevrolet Volt]


SOPA and PIPA explained

This is a nice explanation of the implications of SOPA and PIPA.

[Read SOPA and PIPA explained]


MongoSV follow up

What a madhouse!  The 2011 MongoSV conference was a blast, it didn’t hurt there were over 1000 attendees.  I really enjoyed some excellent talks, and good discussions with colleges and 10gen employees.  Plus, the opportunities in the evenings for a couple cold beverages were fantastic.

I have a couple take-aways from this years conference:

  • The new aggregation features of MongoDB are going to be great, I am excited to start playing with them.
  • 10gen is listening to us about the locking behavior of MongoDB, and the roadmap is promising.
  • Lots of people use AWS for MongoDB.  More than I would have thought considering EBS is horrid.  Most everyone thought AWS and EBS was painful, but the pain was outweighed by the perceived benefits.
  • Everyone agrees schema design is key.  The money quote from the conference was:
Schema free != Design free –@nathenharvey @nathanharvey 

</p>
  • Some people are using MongoDB for what I would call, non-optimal use cases.  I think one has to be careful to really think hard about when to use MongoDB and when not to.  It’s an exciting product, but not a cure-all.  I listened to more than one talk where folks will have a hard time continuing to use MongoDB.
  • There is still a lot to be learned about how to operate MongoDB in a production environment.  Hrm, seems like a good subject for future talks.
  • Someone needs to do a talk on locking internals so folks can really understand that locks in MongoDB are not the same locks as in most RDBMS’s.
  • The SJC airport is awesome.  I had to fly out Friday night to LA, and it was seamless.  Sorry I missed the after-party.
Here is the video from my session:


Watch live video from mongodb on Justin.tv

[Read MongoSV follow up]


Masters of MongoDB

The day before this years MongoSV Conference a few folks, chosen by 10gen, and labelled the Masters of MongoDB gathered to discuss the MongoDB platform, swap war stories, share gripes, and overall give feedback on the direction and roadmap of MongoDB to the 10gen folks.

First of all, I am incredibly honored to be considered in such a group of high caliber folks. But beyond that I was struck by the level of commitment that 10gen has to making sure the roadmap is proper and that they are listening very closely to the community. I feel like with such a commitment the areas for improvement in MongoDB will get good attention and focus. I think this is a huge component for consideration when looking at MongoDB. The community is very very strong.  I made lots of good contacts and was able to connect with friends.

Thanks 10gen for putting on such a good event, and above all, listening to your customers.

[Read Masters of MongoDB]


MySQL/Hbase tech talk at FB

I was sorry to miss the event as some really good stuff was being discussed. Paul has some really great points about single server performance at the Q&A at the end.

Watch live streaming video from fbtechtalks at livestream.com

[Read MySQL/Hbase tech talk at FB]


I am speaking at the MongoSV 2011 conference

I had a great time presenting at MongoSV <a href=http://www.kennygorman.com/wordpress/?p=776>last year</a>.  This year I am presenting a talk on MongoDB performance tuning, similar to what I <a href=http://www.kennygorman.com/wordpress/?p=818>presented at MongoSF</a> but with some updated items around scalability, tuning, and hardware!  My college Luciano Resende is also speaking.  I hope to give some insight into some of the work we have done at Shutterfly around performance tuning and scalability.  I also hope to speak in more detail about our implementation of the <a href=https://github.com/facebook/flashcache/>Facebook Flashcache</a> technology.  If you haven’t signed up, you can register here for the conference. Here is a video 10gen put out for the conference:

*changed the video to the updated version from 10gen

[Read I am speaking at the MongoSV 2011 conference]


Stellar use cases for MongoDB

MongoDB has a nice wide sweet spot where it’s a very useful persistence platform, however, it’s not for everything. I thought I would quickly enumerate a couple great use cases that have come up in the last year and a half and why they are such a great fit for MongoDB.

  1. Documents: Using MongoDB instead of a XML based system.

MongoDB is a document oriented data store. XML is a document language. By moving a traditional XML app to MongoDB one can experience a few key advantages. The typical pattern in XML is to fetch an entire document, work with it, and put it back to the server. This approach has many downsides including the amount of data transmitted over the wire, collision detection/resolution, data set size, and server side overhead. In the MongoDB model, documents can be updated atomically, fetched by index, and even partially fetched. Applications are simpler, faster, and more robust.

  1. Metadata storage systems.

Any system that stores metadata can be a great use case in MongoDB. Such systems typically have a pattern of adding attributes about some type of entitiy, and then needing to query/sort/filter based on these items. The prototypical use case for such a system is the use of tags. The tag implementation is so superior in MongoDB that almost single handedly compels one to use MongoDB for any system needing tags. Simply put:

db.mymetadata.save({stuff:"some data here", thing:"/x/foo/bar.mpg", tags:['cats','beach','family']})
db.mymetadata.ensureIndex({"tags":-1})
db.mymetadata.find({tags:'cats'})
...
"indexBounds" : {
		"tags" : [
			[
				"cats",
				"cats"
			]
		]

In many metadata systems the schema may vary depending on the metadata itself. This allows for huge degrees of flexibility in the data modeling of applications that store metadata. Imagine a media storage service that can store video and image data in the same collection but with different attributes about each type of metadata. No joins needed on query, and the logical I/O path is minimized! MongoDB now supports sparse indexes, so indexes on attributes that are not in every document are kept at a minimum size.

  1. Read intensive systems

Any system where the amount of change is low, and read is high is a nice sweet spot for MongoDB. MongoDB has a nice scaling property with both the replica sets functionality (setting SLAVE_OK), as well as using sharding. Combine this with the document model, and metadata storage capabilites one has an excellent system for say a gawker clone. Reads can come off any one of N sharded nodes by say, story_id, and reads can be geographically targeted to a slave for reads. Keep your data clustered by key for super fast I/O.

[Read Stellar use cases for MongoDB]