Next Generation
Databases: NoSQL,
NewSQL and Big Data

Buy at Amazon
Buy at Apress


Oracle Performance Survival Guide

Buy It
Read it on Safari
Scripts and Examples
Sample Chapter


 MySQL Stored procedure programming

Buy It
Read it on Safari
Scripts and Examples 


« Statistical analysis of Oracle performance data using R | Main | Correlation probability in Oracle »

Using Toad with Hive in Amazon Elastic Map Reduce

The Toad for Cloud Databases eclipse client has support for Hive queries which makes it really easy for me to run queries against our test hadoop clusters.  It also supports Hive running on top of Amazon Elastic Map Reduce (EMR), but you do need to be aware that in EMR the default ports are different from what we have come to expect.

Firstly, if you have started an EMR cluster with Hive 0.5 support, then the Hive server will be running on port 10001, not port 10000.  The second difference is that the JobTracker is running on port 9100, rather than 50030.  So when attaching to EMR, you would set up your hive connection something like this:

1-02-2011 5-56-03 PM

Once you’ve done that, the Hive connection will show all the Hive tables and you can enter HQL queries in the SQL editor.  You can drag table and column names into the editor as well:

1-02-2011 5-57-29 PM

One of the simple, but really useful things about the hive client is that you can jump to the jobtracker web page while the HQL is running to see how it is going:

1-02-2011 5-59-19 PM

Here’s the resulting JobTracker console.  We can see the job running and – if we scroll to the right or maximize the window – we can see how the Map and reduce phases of the Hive job are progressing:

1-02-2011 8-11-07 PM

References (2)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (3)

Nice post,
I am trying to use the toad for cloud, eclipse plugin in my own environment,

But I am facing a problem, once I got results for my query, I can't do anything with them,
I couldn't figure out, how to copy paste them to excel or export them to excel, or even an ordinary text file.

Maybe you know if I can?

February 23, 2011 | Unregistered CommenterGuy

Hi, Sorry for not noticing this comment immediately.

We are putting this feature into the product at the moment - you might have been in contact with the development team? Should have a patch for you in a few days


March 7, 2011 | Registered CommenterGuy Harrison

It really is very valuable information given to us, come the December article in the Amazon Elastic Map Reduce (EMR), Hive, and TOAD and it pays to be aware of your blog!

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>