Anyone Knowledgeable on Mongoose/MongoDB?

Atomic Pineapple

I am currently dabbling in the MEAN stack and trying to learn as much as possible about MongoDB in particular and using the NodeJS Module Mongoose to interact with it.

I have followed initial tutorials and followed some of the best practices they mentioned with regards to relationships that are similar to one to many relationships in a relational DB.

This worked for my basic queries but now I have a query whereby the way I have setup the Schema's for the models doesn't really suit and I am trying to figure out the best way to do what I need.

My app has a RollerCoaster and a CoasterReview model, each RollerCoaster can have many CoasterReviews, so for this to work the tutorials suggested that when a CoasterReview gets created the ID of the RollerCoaster associated to it gets added to the CoasterReview model.

This approach works when I need to get a list of the CoasterReviews and populate the RollerCoaster for each CoasterReview within the model.

However I also have a need to get a list of RollerCoasters and also have the array of CoasterReviews per RollerCoaster populated within the returned list of RollerCoaster models.

Here's the RollerCoaster Schema:

/**
 * Rollercoaster Schema
 */
var RollercoasterSchema = new Schema({
    name: {
        type: String,
        default: '',
        required: 'Please fill Rollercoaster name',
        trim: true
    },
    created: {
        type: Date,
        default: Date.now
    },
});

And here's the CoasterReview schema:

/**
 * Coasterreview Schema
 */
var CoasterreviewSchema = new Schema({
    review: {
        type: String,
        default: '',
        required: 'Please fill Coasterreview review',
        trim: true
    },
    created: {
        type: Date,
        default: Date.now
    },
    rollercoaster: {
        type: Schema.ObjectId,
        ref: 'Rollercoaster'
    }
});

And here is what I have so far with trying to get a list of RollerCoasters that each have the array of CoasterReviews associated with them:

Rollercoaster.find(query).sort('-created').populate('user', 'displayName').populate('themepark').exec(function(err, rollercoasters) {
        if (err) {
            return res.status(400).send({
                message: errorHandler.getErrorMessage(err)
            });
        } else {
            //list of ids
            var rcIds = rollercoasters.map(function(el){return el._id;});
            //select all reviews with 'rollercoaster' = any _id from list
            CoasterReview
                .find({'rollercoaster':{$in:rcIds}}).sort('-created').exec(function(err, reviews) {
                    //res.jsonp(rollercoasters);
                    res.jsonp(reviews);
                });
        }
    });

This gives me the list of RollerCoasters and the list of Reviews separately, I could from here loop through the results and manipulate the JSON myself to get what I need but it feels like there is probably a better way to either structure the Schema's or to use Mongoose in a more efficient way.

Maybe I need to look into MapReduce?

Here is a what I want the resulting JSON to look like:

[
  {
    "_id": "55c2106227f2a878292a564d",
    "__v": 0,
    "created": "2015-08-05T13:32:18.473Z",
    "name": "Jurassic Park : The Ride",
    coasterreviews: [
    {
      "_id": "55dcbd9662a5516c39d27946",
      "rollercoaster": "55c2106227f2a878292a564d",
      "__v": 0,
      "created": "2015-08-25T19:10:14.392Z",
      "review": "The theming is fantastic"
    },
    {
        "_id": "55d8bb292d03db3c1310d465",
        "rollercoaster": "55c2106227f2a878292a564d",
        "__v": 0,
        "created": "2015-08-22T18:10:49.201Z",
        "review": "Woah what a ride, really thrilling and a great experience."
      }
    ]
  },
  {
    "_id": "55c20f0f27f2a878292a564c",
    "__v": 0,
    "created": "2015-08-05T13:26:39.055Z",
    "name": "Dragon Challenge : Hungarian Horntail",
    coasterreviews: [
      {
      "_id": "55d8d21908eda5a018c22ed4",
      "rollercoaster": "55c20f0f27f2a878292a564c",
      "__v": 0,
      "created": "2015-08-22T19:48:41.854Z",
      "review": "Great ride"
      }
    ]
  },
]

Can anyone help point me in the right direction for achieving this?

Talisman

The more data you request from the database the slower the response of your application. Make the application fast by only requesting the data you require when you need it.

The View that displays the list of Rollercoasters does not need to know about the Coasterreviews in order to display the listing. Instead you add button/image/link for the Coasterreviews, e.g. "See Reviews". Clicking the item should fire a request to the server for the reviews of the selected Rollercoaster. Cache the response and update the View with the Coasterreview items.

This pattern will minimise the volume of data requested from the server at any time.

Atomic Pineapple

The app does need to know about the reviews at this stage, it is part of the information I plan to show on the client at this stage.

Talisman

Just to be clear - You want ALL of the Rollercoasters and ALL of their reviews?

Atomic Pineapple

Talisman wrote: »

Just to be clear - You want ALL of the Rollercoasters and ALL of their reviews?

No, it will be done on a search, so user will search for a keyword and rollercoasters that match that keyword will be returned in groups of 10. Then in this view I will need to know the number of reviews a rollercoaster has and the average from those reviews.

I am not so much worried about the speed at this stage as it is just a personal project I am developing to try to learn more about full stack development. I am more interested in getting the implementation correct at this stage and gaining an understanding of that correctness.

Talisman

You can update the schema so that you don't need the list of reviews for that View. Add two new fields to your Rollercoaster Schema: numberOfReviews, reviewScoreTotal.

Every time a review is added for a Rollercoaster, increment the numberOfReviews field and add the review score to the reviewScoreTotal field.

On the client side, the average review score is a simple calculation: reviewScoreTotal / numberOfReviews.

Atomic Pineapple

I had thought about that approach but how legitimate is that? Again I am from a mobile development background and I am up skilling in back end development as much as possible so please excuse my ignorance, it may be perfectly acceptable to do it this way but I want to make sure.

My main concern with doing it this way is that it means storing effectively the same data in two different places and increases the chances of one of the sets of data to get out of sync with the other.

Is this a valid concern or am I overthinking things?

Talisman

It's completely legitimate and the simplest solution, but there are other options.

Option 1:
You could have decided to implement your solution by imbedding the review documents within the rollercoaster. The benefit of this is that you would have an array of reviews within your rollercoaster so you will know how many reviews there are due to the size of the array. The down side to this approach is that disk use will bloat - MongoDB wants to write the document to disk as a single block of data, if it can't find a large enough space in the data store it will consume more disk space to append the data. The only way to recover the unused fragments of disk space in the store is to compress the database which means your application needs to die for a while until the task is complete.

Option 2:
If you want you can use MongoDB to perform a count and aggregate the average review score for each rollercoaster. There is no magic sauce with MongoDB, internally the data is stored as JSON in a binary format, it can read the documents from the disk and perform the calculation for you. It will be trivial initially but as the number of reviews grow the time taken will increase as disk reads are expensive.

The impact on the performance of the application might cause you to decide to cache the results of the calculation for a period of time to speed things up. This adds an additional layer of complexity. What happens if a new review is added? Do you delete the cached item and perform the calculations again? Or do you wait for the cache to expire?

At some point you might have a light bulb moment and come to the realisation that the problem could be solved by keeping a running total on the number of reviews and the total of the scores for each rollercoaster.