MongoDB, Oracle and Toad for Cloud Databases

We recently added support for MongoDB in Toad for Cloud Databases, so I took the opportunity of writing my first MongoDB Java program and taking the Toad functionality for a test drive.

MongoDB is a non-relational, document oriented database that is extremely popular with developers (see for instance this Hacker news poll).   Toad for cloud databases allows you to work with non-relational data using SQL by normalizing the data structures and converting SQL to the non-relational calls.

I wanted to get started by creating some MongoDB collections with familiar data.  So I wrote a Java program that takes data out of the Oracle sample schema, and loads it into Mongo as documents.  The program is here.

The key parts of the code are shown here:

   1: while (custRs.next()) { // For each customer

   2:     String custId = custRs.getString("CUST_ID");

   3:     String custFirstName = custRs.getString("CUST_FIRST_NAME");

   4:     String custLastName = custRs.getString("CUST_LAST_NAME");

   5:  

   6:     //Create the customer document 

   7:     BasicDBObject custDoc = new BasicDBObject();

   8:     custDoc.put("_id", custId);

   9:     custDoc.put("CustomerFirstName", custFirstName);

  10:     custDoc.put("CustomerLastName", custLastName);

  11:     // Create the product sales document 

  12:     BasicDBObject customerProducts = new BasicDBObject();

  13:     custSalesQry.setString(1, custId);

  14:     ResultSet prodRs = custSalesQry.executeQuery();

  15:     Integer prodCount = 0;

  16:     while (prodRs.next()) { //For each product sale 

  17:         String  timeId=prodRs.getString("TIME_ID"); 

  18:         Integer prodId = prodRs.getInt("PROD_ID");

  19:         String prodName = prodRs.getString("PROD_NAME");

  20:         Float Amount = prodRs.getFloat("AMOUNT_SOLD");

  21:         Float Quantity = prodRs.getFloat("QUANTITY_SOLD");

  22:         // Create the line item document 

  23:         BasicDBObject productItem = new BasicDBObject();            

  24:         productItem.put("prodId", prodId);

  25:         productItem.put("prodName", prodName);

  26:         productItem.put("Amount", Amount);

  27:         productItem.put("Quantity", Quantity);

  28:         // Put the line item in the salesforcustomer document 

  29:         customerProducts.put(timeId, productItem);

  30:         if (prodCount++ > 4) { // Just 5 for this demo

  31:             prodCount = 0;

  32:             break;

  33:         }

  34:     }

  35:     // put the salesforcustomer document in the customer document 

  36:     custDoc.put("SalesForCustomer", customerProducts);

  37:  

  38:     System.out.println(custDoc);

  39:     custColl.insert(custDoc);  //insert the customer 

  40:     custCount++;

  41:  

  42: }

Here’s how it works:

Lines	Description
1-4	We loop through each customer,   retrieving the key customer details
7-10	We create a basic MongoDB document that contains the customer details
12	We create another MongoDB document that will contain all the product sales for the customer
16-21	Fetching the data for an individual sale for that customer from Oracle
23-27	We create a document for that single sale
29	Add the sale to the document containing all the sales
36	Add all the sales to the customer
39	Add the customer document to the collection

The MongoDB API is very straight forward; much easier than similar APIs for HBase or Cassandra.

When we run the program, we create JSON documents in Mongo DB that look like this:

   1: { "_id" : "7" , "CustomerFirstName" : "Linette" , "CustomerLastName" : "Ingram" , 

   2:     "SalesForCustomer" : {

   3:         "2001-05-30 00:00:00" : { "prodId" : 28 , "prodName" : "Unix/Windows 1-user pack" , "Amount" : 205.76 , "Quantity" : 1.0} , 

   4:         "1998-04-18 00:00:00" : { "prodId" : 129 , "prodName" : "Model NM500X High Yield Toner Cartridge" , "Amount" : 205.48 , "Quantity" : 1.0}

   5:     }

   6: }

   7: { "_id" : "8" , "CustomerFirstName" : "Vida" , "CustomerLastName" : "Puleo" , 

   8:     "SalesForCustomer" : { 

   9:         "1999-01-27 00:00:00" : { "prodId" : 18 , "prodName" : "Envoy Ambassador" , "Amount" : 1726.83 , "Quantity" : 1.0} , 

  10:         "1999-01-28 00:00:00" : { "prodId" : 18 , "prodName" : "Envoy Ambassador" , "Amount" : 1726.83 , "Quantity" : 1.0} , 

  11:         "1998-04-26 00:00:00" : { "prodId" : 20 , "prodName" : "Home Theatre Package with DVD-Audio/Video Play" , "Amount" : 608.39 , "Quantity" : 1.0} ,

  12:         "1998-01-19 00:00:00" : { "prodId" : 28 , "prodName" : "Unix/Windows 1-user pack" , "Amount" : 216.99 , "Quantity" : 1.0} , 

  13:         "1998-03-19 00:00:00" : { "prodId" : 28 , "prodName" : "Unix/Windows 1-user pack" , "Amount" : 216.99 , "Quantity" : 1.0} 

  14:     }

  15: }

Toad for Cloud “renormalizes” the documents so that they resemble something that we might use in a more conventional database.  So in this case,   Toad creates two tables from the Mongo collection, one for customers, and one which contains the sales for a customer.   You can rename the auto-generated foreign keys and the sub-table name to make this a bit clearer, as in the example below:

We can more clearly see the relationships in the .NET client by using Toad’s visual query builder (or we could have used the database diagram tool):

MongoDB has a pretty rich query language, but it’s fairly mysterious to those of us are used to SQL, and it’s certainly not as rich as the SQL language.  Using Toad for Cloud, you can issue ANSI standard SQL against your MongoDB tables and quickly browse or perform complex queries.  Later this year,   this Mongo support will emerge in some of our commercial data management tools such as Toad for Data Analysts and our soon to be announced BI tools.