MapReduce : Sinking Ship Or Rescue Boat ?

MapReduce: The Ultimate Programming Paradigm for Parallel Processing

MapReduce was one of the major initiatives taken in the field of parallel processing or distributed computing in 2004 by researchers of none other than the tech giant google which is dealing with a tremendous amount of data at that time.

Since then, it is the survival of MapReduce theory for such a long time showcases the hard work and brainstorming of the inventors.

Despite new frameworks getting developed, MapReduce has sustained itself throughout the course of time.

In fact, many of the parallel computing engines followed the path carved by MapReduce theory.

After 18 years down the line, enthusiasts generally get confused about where to start their journey in the era where so many low-code & no-code frameworks are evolving.

But IMHO, any beginner enthusiasts should look to first get their hands dirty on the MapReduce concepts and then move to the trending distributed computation architectures.

Once you get the idea of what happens under the hood in the execution of "MapReduce Hello World - The Word Count Program", look for the advanced concepts.

How Sorting happens in a distributed manner?
How do multiple datasets get Joined?

This will build a solid foundation and lets you achieve a great understanding of the latest tools like Apache Hive, Apache Spark, etc.

So definitely, MapReduce can prove to be the rescue boat for the Big Data Enthusiasts

Get Started Here:

https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf

(Feel free to add any resources and corrections in the comments)

Comments

Post a Comment

Popular posts from this blog

Calculating Top N items per Group (Without Window Functions)

Image
These kinds of questions are more common than you think: Out of a list of Twitter users, find their N recent tweets. Out of all the cities, find the top N cities in each country by population. Out of all orders on the e-commerce website, find the top N recent orders for each customer. Solving these questions becomes pretty easy if we use a window function like row_number(), but solving this without any window function makes the task challenging. Here we will see both approaches. Question: Display the top 3 recent values from every group Table : groups                                            Required Output: Method 1 (Using Window Function): WITH ranked AS ( SELECT group_id,value, ROW_NUMBER() OVER (PARTITION BY group_id ORDER BY value DESC ) AS rn FROM groups ) SELECT group_id,value FROM ranked WHERE rn <= 3 ; Method 2 (Without Window Function): SELECT t1.group_id,t1.value FROM groups t1 INNER JOIN groups t2 ON t1.group_id = t2.group_id AND t1.value
Read more

Single vs Double Quotation marks (In PostgreSQL)

While writing sql queries there can be a lot of confusion when it comes to using single vs double quotation marks, especially in a database like PostgreSQL where both of them can not be used interchangeably (unlike MySQL) Even though it is very basic topic, it is very important and can having clarity on it can save a lot of time while writing and debugging the sql queries in PostgreSQL. Here's the link that i found that can reduce the dilemma upto an extent. https://streamofcoding.com/single-quote-vs-double-quotes-in-postgres/ Please feel free to add your thoughts or resources related to the topic in the comments.
Read more

Leverage the mind smartly, How?

Add new learnings to your mind effectively. Mistakes : While studying/learning any new topic, we all tend to make the same mistake time and again. Whenever we come across an unknown topic, we go through the internet, read any article, or watch any random youtube videos. Yes, this way, we understand the topic and just stop there and after some time, when we need the same thing, we repeat the same process of surfing the internet. As you are reading the same topic from different sources every time you need it, it takes a long time for your mind to grasp the thing. So, What's the solution? What we can do is, after getting sufficient knowledge on the topic, make a note on it which you can revise later point in time and don't need to go through the web. And if the topic is too large and complex to prepare notes on, bookmark the webpage/video you learn it on and visit the same exact thing next time. what happens is when you read something from the same thing again and again, it builds
Read more