Thursday, November 15, 2012

Hadoop File System Metadata Replication By Using Secondary Name Node


FSIMAGE and EDITS are the two most important files of name node. FSIMAGE is having snapshot of all the metadata of the Hadoop cluster whereas EDITS contains the incremental information of metadata. The reason for maintaining the incremental information in another file because it requires the write operation whereas FSIMAGE is served directly from the RAM. Over the time EDITS log file grows and in case of failure of name node, it could take a longer time to load the stuff in FSIMAGE.

How to eliminate the problem in case of failure of primary name node?

The solution would be secondary name node. The logic is as simple as we do replace the two numbers by using the temp variable. Below are the messages exchange between primary name node and secondary name node.

1.Secondary name node sends a message to primary name node to start writing the incremental messages in EDITS.NEW file.
2.Secondary name node copies the FSIMAGE and EDITS file from the primary name node.
3.Secondary name node adds the EDITS to the FSIMAGE and make a new FSIMAGE file.
4.Sends it to the primary name node.
5.Primary name node renames the EDITS.NEW file to EDITS.

People who read this post also read :



6 comments:

Unknown said...

You have done a great job by explaining what is hadoop file system metadata is all about. Thanks for sharing this in here.

Big Data Course in Chennai

priya said...

i gathered knowledge info from your blog about hadoop.
Hadoop Training in chennai

Unknown said...

Thanks for your informative guide on SQL technology. As we all know, SQL is most used database tool and your article offer clear insight about the technology. Your blog is recommended for freshers and experienced ones looking for ideal blog to learn SQL technology.
Java Training in Chennai
.Net Training in Chennai
PHP Training in Chennai

Unknown said...

For management and troubleshooting of enormous databases, corporations square measure searching for qualified and licensed dispersions Hadoop experts for the duty. Hadoop Training in Chennai

Unknown said...

Thanks for sharing informative post. Big data is a term that portrays the substantial volume of information; both organized and unstructured that immerses a business on an everyday premise. To know more details please visit Big Data Training in Chennai | Primavera Training in Chennai

Unknown said...

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.
Regards,
Hadoop Training in Chennai|Big Data Training in Chennai