the lost logbook » Technology

Hadoop – The Future of Large Data sets

Utkarsh — Wed, 24 Mar 2010 22:14:12 +0000

Every one is talking about “large data sets”, “scalability” and “handling huge data being generated in the web day by day”. Which is supposed to be around 500 billion GB presently, according to the Guardian! Google, Yahoo, Facebook, Twitter, you think of a big company and they run Hadoop! Here are some of the companies which use Hadoop. In that last link, if you’ll notice Yahoo has a cluster of 4000 nodes!

Looking forward to the Hadoop Meet at Yahoo today!

Interesting reads of Hadoop

bp-3.com

Facebook: Hive – A Petabyte Scale Data Warehouse Using HadoopSimilar Posts:

Amazing Video Wall by Blinkx.com

Utkarsh — Sun, 20 Dec 2009 12:43:16 +0000

Share on Facebook

You can get one of these Blinkx video wall by: 1. Hover the mouse to the top of the wall posted above; 2. Copy the embed code.Similar Posts:

3 Best Possible If-Else Construct Alternatives

Utkarsh — Sat, 12 Dec 2009 02:12:56 +0000

Share on Facebook

My current JAVA project requires a lot of check points, for example, 1. if the format of the ID passed is correct, then go ahead and 2. check if ID exists or not, if it does, 3. then check if ID has some kind of specific attributes or not and so on!

So, I was using the regular if-else for something like 6-7times!

if([condition]) { [do this] }else if([condition]){ [do this]}else if{[do this] }else if([condition]){ }else{ [do this]}

When I looked back at the code, it sucked! The deep nesting of the if-else construct made the code hard to understand and comprehend!

Comic Via

So, I searched a little for some logical if-else construct alternatives and I came up with these three useful techniques;

Ternary operator (?:) and
Logic Grid
Switch Statement (It is also a good alternative which makes the code look good. Its also pretty easy, so I won’t cover it here.)

Lets take a look at the 1st and a simpler process using ternary operator:

1. If-Else Construct Usage;

int opening_time;
if (day == WEEKEND)
opening_time = 12;
else {opening_time = 9;}

Usage of ternary operator in the above example;

int opening_time = (day == WEEKEND) ? 12 : 9;

[condition ? value if true : value if false]

Ok, so that was easy! Now lets take a look at LogicGrid technique which is very impressive and and an elegant way!

2. If-Else Construct:

if (a) {
if (b) {
result = 1;
} else {
result = 2;
}
} else if (c) {
if (b) {
result = 3;
} else {
result = 4;
}
} else if (!c) {
result = 5;
}

Using Logic Grid: (C++ code here changed to Java)

int getResult (boolean a, boolean b,boolean c)
{
int index = 0;
int[] results = { 5,4,5,3,2,2,1,1 };
if(a) index = index | 4;
if(b) index = index | 2;
if(c) index = index | 1;
return results[index];
}

The logic grid method is amazing!! Once you get hold of it, you will always use it, the code is much more organized and understandable now!

Hope it helps! Similar Posts:

Google Public DNS for Faster Browsing

Utkarsh — Fri, 04 Dec 2009 12:38:28 +0000

Share on Facebook

Today, Google launched their new service, Google Public DNS for “faster browsing experience”! (At the cost of vital technical details which Google will log for their analysis). Google claims they won’t save any personal data.

So, what is a Public DNS anyway?! To put it in simple words,

When you type in utkarshsengar.com; that text is converted to my server’s IP address by your default DNS and it routes the request to view my blog to the web server where the blog is actually hosted. The default DNS is generally provided by your ISP.

So, why use a 3rd party DNS provider rather than your default DNS?

Faster browsing (Not any other 3rd party providers, go for big ones!)
More secure
Web content filtering (not provided by Google Public DNS)

There is already a big player in the public and premium DNS business, OpenDNS.

If you are interested in using Google Public DNS, read this. Read the detailed blog post on Official Google Blog here

Here are some technical performance and security benefits.

Similar Posts:

Gartner Hype Cycle 2009 – Is Cloud Computing Over Hyped?

Utkarsh — Thu, 03 Dec 2009 08:34:37 +0000

Share on Facebook

I came across this interesting white paper – Hype Cycle of Emerging Technologies, 2009 published by Gartner Inc. This paper describes the emerging technology trends and how are these technologies performing in the year 2009. The hype cycle is divided into 5 phases based on their conception and market adoption.

So, what is so interesting about a Hype Cycle?! Here is how wikipedia describes it:

A hype cycle is a graphic representation of the maturity, adoption and business application of specific technologies. The term was coined by Gartner, an analyst/research house based in the United States that provides opinions, advice and data on the global information technology industry.

They come up with many Hype cycles on various markets every year (which are not free of course $$, that’s how they make money!).

The most interesting thing I came across is at the peak of the hype cycle is the most talked about topic these days, cloud computing. According to Garther, cloud computing is the most hyped technology of year 2009.

The above graph is divided into 5 phases;

1. Technology Trigger - Now, this is the phase when a new technology evolves and has a chance of becoming one of the main stream technologies of the future. Some of the technology triggers of 2009 according to Gartner are – 3D flat panel displays, Quantum computing and Human Augmentation. I personally think these stuff will take a very long time to develop, especially Human Augmentation!

2. Peak of inflated expectation: Now here comes the interesting part! The 2nd phase has those technologies which are over hyped and have unrealistic expectations! On top of this peak is Cloud Computing and E-book readers! I wonder what Amazon has to say about that, since they are one of the biggest players of the “cloud” and also of the e-book readers, the Kindle! Can’t help mentioning the “cloud” OS – Google Chrome OS and Jolicloud.

3. Trough of Disillusionment: This phase covers those technologies which failed to live up to their expectations! One of the biggest looser which I see here is RFID! But RFID still has a long way to go. Gartner think RFID is on a slow and steady growth and it might catch up other technologies in the next phase, the Slope of Enlightenment in 5-10 years.

4. Slope of Enlightenment: This phase covers one of the most promising technologies of tomorrow which have become a part of our lives but we haven’t actually started to tap its potential. Biggest example is Twitter! Some of the technologies in this phase are Web 2.0 (Its every where!), social network analysis (FaceBook plans to do that), over the air mobile phone system (Jack Dorsey is doing that). From where I see, these technologies have really gained some solid ground and have a promising future ahead!

5. Plateau of Productivity: Well, the final phase consists of those technologies which are already proven to be a breakthrough in technology and is broadly accepted by various markets. The year 2009 does not have any technologies in the 5th phase, though speech recognition is inching towards the 5th phase. Last year, web services was in this category and presently in the year 2009, virtually all the major websites run some kind of web services and the enterprise systems depend heavily on web services!

Similar Posts:

Get the new Google Homepage!

Utkarsh — Wed, 02 Dec 2009 13:20:08 +0000

Share on Facebook

Check out the new Google homepage and a permanent sidebar in the search result page. The Google guys are still testing it and you can enable this by the following steps:

1. Go to Google.com

2. Paste this code on the address bar in place of http://www.google.com/

javascript:void(document.cookie="PREF=ID=20b6e4c2f44943bb:U=4bf292d46faad806:TM=1249677602:LM=1257919388:S=odm0Ys-53ZueXfZG;path=/; domain=.google.com");

3. Refresh (F5)

AND YOU ARE DONE!

Note: If you want to revert back to the old Google.com, simply clear your browser’s cache/cookies.

Here is a screen shot of the homepage and the search result page.

New Homepage

New result page

Old HomePage

Old Result page

Similar Posts:

Sync Google Calendar with GNOME clock

Utkarsh — Thu, 26 Nov 2009 06:22:06 +0000

Share on Facebook

You can merge your GNOME desktop (Ubuntu 9.10) with Google Calendar via iCalendar Format. This a very nifty hack for people who use Google Calendar and want your Linux to in sync with Google Calendar.

What is iCalendar?

XML or iCalendar – Use these formats to view a read-only version of your calendar from other applications. You can use this address to access your calendar from feed readers (like Google Reader) as well as products that support the iCalendar format (like Apple’s iCal). via Google Support

1. Go to your Google Calendar and select the down arrow with calendar you want to sync with GNOME and select Calendar Setting

2. Copy the link of the Green button, iCal next to Private Address.

3. Now, open the linux Terminal and type the following:

/usr/lib/evolution-webcal/evolution-webcal URL_YOU_COPIED

That’s it! You will see an appointments option under the GNOME clock.

via Bryan Clark

Similar Posts:

Install Google Chrome OS on Virtual Box

Utkarsh — Fri, 20 Nov 2009 23:36:43 +0000

Share on Facebook

Google just released their latest pet, Google Chrome OS. This is not exactly what you think it is. The existing idea of an operating system is a software which is packed with load of goodies, games, system tools, a desktop, some kind of menu bar, task bar and blah blah blah!

But Google is trying to defy that concept of a traditional operating systems. They have a built this OS which primarily targets the Internet. Their OS is the Browser! If you are running Chrome OS, you need to do all your stuff on the Internet. It has a vague offline support. I do not see any option to install some software. I would say, its not just an OS, its a Cloud Based OS.

Will it replace Windows? – NO WAY!!

Its more of a companion device for Tablets and Netbook. This cannot get into the mainstream OS market at least for the next 4-5 yrs until the Cloud is a reality!

It will only depends on the the web applications developed by Google/3rd party websites. That mean, if you need to open a .doc file, open it in Google Doc. There is no concept of installing .exe files or any other application. You will be dependent on the services provided by Google and some other sites (which will be supported by Chrome OS).

The Features I like:

7 sec boot time.
Its ultra secure since when you login, your ID is authenticated via Google account.

Things I don’t like:

Web only thing (From where I see this now, its only for Internet usage, if you are wondering about Cloud Computing, the buzz word which every body is speaking, then for get it! It’ll take atleast 4-5 more years to host any kind of application on the internet and run it on Chrome OS.)
I cannot install any software on this!

My question is:

If I just need a browser, why can’t I just use Firefox or Chrome on my Windows/Linux machine, why install an OS which works as a browser?!

How to install Chrome OS on your VirtualBox 3.0

Download this torrent.
This will contain a .tar.bz2 file. Unzip it using WinRAR or 7Zip (on windows), Linux supports it natively.
You will get a file chromeos-image-999.999.32309.211410-a1.vmdk. Virtual Machine Disk Format on VMWare but VirtualBox also supports this format.
Install Virtual Box. HowTo – Windows | Linux
Now, in the VirtualBox, click New > Name your OS and select Operating System: Linux; Version: Ubuntu
Set the RAM. (1GB is recommended)
Now in Virtual Hard Disk, select Use Existing HardDisk and Browse.
A new window will open, click Add and add the .vmdk file you just downloaded there. Click Select after adding it.
The finish the whole thing and you are done!
Start your Chrome OS using VirtualBox main window.

Note: This is a developer build We can expect more improved features in the official release some time next year.

I have also attached some screen shots!

If you are a developer, you can also build the source code on Ubuntu. Get more information here.

Screenshots

UPDATE

Source: Tech Source from Bohol

And I Don’t Like It!

Similar Posts:

The Social Media Revolution – Web 2.0

Utkarsh — Wed, 18 Nov 2009 11:31:15 +0000

Share on Facebook

Similar Posts:

Who is lukeywes1234? – The Power of Web 2.0

Amazon RDS – The next move to Cloud

Utkarsh — Tue, 10 Nov 2009 05:32:17 +0000

Share on Facebook

Amazon.com, started in 1994 is one of the greatest example of diversification of an e-commerce player. Started out as an online book store has diversified it self to an online e-commerce website, the Kindle, even door to door grocery delivery service with AmazonFresh and now of the biggest player in the Cloud Computing business!!

So, the Amazon’s cloud with Amazon Web services is expanding even more with the Amazon Relational Database Service. If you are wondering what is AWS? Check these out: Link1, Link2

Amazon Web Services offers loads of remote computing services which cab be used by anyone ranging from a SME, a big firm, a small blog etc.

So, what is this Amazon RDS? Well, to put it in simple words, its just a MySQL 5.1 database hosted on Amazon servers. You do not need to worry about the setup, maintenance and security issues, Amazon will do all that for you!

RDS in the words of Mr Werner Vogles, CTO Amazon.com [BLOG]

Amazon RDS for when the application requires a relational database but you want to reduce the time you spend on database management, Amazon RDS automates common administrative tasks to reduce your complexity and total cost of ownership. Amazon RDS allows you to manage your database compute and storage resources with a simple API call, and only pay for the infrastructure resources they actually consume.

There are few major players competing against Amazon like IBM BlueCloud, Microsoft Azure, Rackspace Mosso to name a few. This is also considered as a boost for MySQL cloud computing solutions.

The Oracle and Amazon love story will be interesting, let see what this relationship holds for us in the future of Cloud.

What do you guys think about the Cloud, Amazon-Oracle tie-up and what is Google and MSFT going to do about it?!

Amazon Relational Database ServiceA

Similar Posts: