{"rsdb":{"rid":"309115","subhead":"","postdate":"0","aid":"224277","fid":"114","uid":"1","topic":"1","content":"
\n
\n \n \u7248\u6743\u58f0\u660e\uff1a\u672c\u6587\u4e3a\u535a\u4e3b\u539f\u521b\u6587\u7ae0\uff0c\u672a\u7ecf\u535a\u4e3b\u5141\u8bb8\u4e0d\u5f97\u8f6c\u8f7d\u3002 https:\/\/blog.csdn.net\/hongchenlingtian\/article\/details\/53524705 <\/div>\n \n \n
\n \n

\u4e00 \u3001\u9700\u8981\u77e5\u9053\u7684\u5185\u5bb9<\/span><\/p>\n

1.ResourceManager ------>yarn\u7684\u8001\u5927
\n2.NodeManager ------>yarn\u7684\u5c0f\u5f1f
\n3.ResourceManager\u8c03\u5ea6\u5668 a.\u9ed8\u8ba4\u8c03\u5ea6\u5668------>\u5148\u8fdb\u5148\u51faFIFO
\n <\/span> b.\u516c\u5e73\u8c03\u5ea6\u5668------>\u6bcf\u4e2a\u4efb\u52a1\u90fd\u6709\u6267\u884c\u7684\u673a\u4f1a
<\/span>......
\n4.\u5fc3\u8df3\u673a\u5236 ------>NodeManager\u53ef\u901a\u8fc7\u5fc3\u8df3\u673a\u5236\u5c06\u8282\u70b9\u5065\u5eb7\u72b6\u51b5\u5b9e\u65f6\u6c47\u62a5\u7ed9ResourceManager\uff0c\u800cResourceManager\u5219\u4f1a<\/span> \u6839\u636e\u6bcf\u4e2aNodeManager\u7684\u5065\u5eb7\u72b6\u51b5\u9002\u5f53\u8c03\u6574\u5206\u914d\u7684\u4efb\u52a1\u6570\u76ee\u3002\u5f53NodeManager\u8ba4\u4e3a\u81ea\u5df1\u7684\u5065\u5eb7\u72b6\u51b5\u201c\u6b20<\/span> \u4f73\u201d\u65f6\uff0c\u53ef\u8ba9ResourceManager\u4e0d\u518d\u5206\u914d\u4efb\u52a1\uff0c\u5f85\u5065\u5eb7\u72b6\u51b5\u597d\u8f6c\u65f6\uff0c\u518d\u5206\u914d\u65b0\u4efb\u52a1\u3002
<\/span>
\n5.NodeManager\u5b50\u8fdb\u7a0b------>\u72ec\u7acb\u4e8eNodeManager\uff0c\u4e0d\u5728NodeManager\u5185\u90e8
<\/span><\/p>\n


<\/span><\/p>\n

\u4e8c \u3001Hadoop\u5de5\u4f5c\u6d41\u7a0b\uff1a<\/span><\/p>\n

<\/span>\"\"
<\/p>\n

1.Client\u4e2d\uff0c\u5ba2\u6237\u7aef\u63d0\u4ea4\u4e00\u4e2amr\u7684jar\u5305\u7ed9JobClient(\u63d0\u4ea4\u65b9\u5f0f\uff1ahadoop jar ...)

\n2.JobClient\u6301\u6709ResourceManager\u7684\u4e00\u4e2a\u4ee3\u7406\u5bf9\u8c61\uff0c\u5b83\u5411ResourceManager\u53d1\u9001\u4e00\u4e2aRPC\u8bf7\u6c42\uff0c\u544a\u8bc9ResourceManager\u4f5c\u4e1a\u5f00\u59cb\uff0c
<\/span>\u7136\u540eResourceManager\u8fd4\u56de\u4e00\u4e2aJobID\u548c\u4e00\u4e2a\u5b58\u653ejar\u5305\u7684\u8def\u5f84\u7ed9Client

\n3.Client\u5c06\u5f97\u5230\u7684jar\u5305\u7684\u8def\u5f84\u4f5c\u4e3a\u524d\u7f00\uff0cJobID\u4f5c\u4e3a\u540e\u7f00(path = hdfs\u4e0a\u7684\u5730\u5740 + jobId) \u62fc\u63a5\u6210\u4e00\u4e2a\u65b0\u7684hdfs\u7684\u8def\u5f84\uff0c\u7136\u540eClient\u901a\u8fc7FileSystem\u5411hdfs\u4e2d\u5b58\u653ejar\u5305\uff0c\u9ed8\u8ba4\u5b58\u653e10\u4efd
<\/span>\uff08NameNode\u548cDateNode\u7b49\u64cd\u4f5c\uff09

\n4.\u5f00\u59cb\u63d0\u4ea4\u4efb\u52a1\uff0cClient\u5c06\u4f5c\u4e1a\u7684\u63cf\u8ff0\u4fe1\u606f\uff08JobID\u548c\u62fc\u63a5\u540e\u7684\u5b58\u653ejar\u5305\u7684\u8def\u5f84\u7b49\uff09RPC\u8fd4\u56de\u7ed9ResourceManager

\n5.ResourceManager\u8fdb\u884c\u521d\u59cb\u5316\u4efb\u52a1\uff0c\u7136\u540e\u653e\u5230\u4e00\u4e2a\u8c03\u5ea6\u5668\u4e2d

\n6.ResourceManager\u8bfb\u53d6HDFS\u4e0a\u7684\u8981\u5904\u7406\u7684\u6587\u4ef6\uff0c\u5f00\u59cb\u8ba1\u7b97\u8f93\u5165\u5206\u7247\uff0c\u6bcf\u4e00\u4e2a\u5206\u7247\u5bf9\u5e94\u4e00\u4e2aMapperTask\uff0c\u6839\u636e\u6570\u636e\u91cf\u786e\u5b9a\u8d77\u591a\u5c11\u4e2amapper,\u591a\u5c11\u4e2areducer

\n7.NodeManager \u901a\u8fc7\u5fc3\u8df3\u673a\u5236\u5411ResourceManager\u9886\u53d6\u4efb\u52a1\uff08\u4efb\u52a1\u7684\u63cf\u8ff0\u4fe1\u606f\uff09

\n8.\u9886\u53d6\u5230\u4efb\u52a1\u7684NodeManager\u53bbHdfs\u4e0a\u4e0b\u8f7djar\u5305\uff0c\u914d\u7f6e\u6587\u4ef6\u7b49

\n9.NodeManager\u542f\u52a8\u76f8\u5e94\u7684\u5b50\u8fdb\u7a0byarnchild\uff0c\u8fd0\u884cmapreduce\uff0c\u8fd0\u884cmaptask\u6216\u8005reducetask

\n10.map\u4ecehdfs\u4e2d\u8bfb\u53d6\u6570\u636e\uff0c\u7136\u540e\u4f20\u7ed9reduce\uff0creduce\u5c06\u8f93\u51fa\u7684\u6570\u636e\u7ed9\u56dehdfs<\/span><\/p>\n


<\/span><\/p>\n <\/div>\n <\/div>","orderid":"0","title":"Hadoop\u63d0\u4ea4\u4f5c\u4e1a------>hadoop\u5de5\u4f5c\u6d41\u7a0b","smalltitle":"","mid":"0","fname":"Hadoop","special_id":"0","bak_id":"0","info":"0","hits":"507","pages":"1","comments":"0","posttime":"2019-05-16 00:36:04","list":"1557938164","username":"admin","author":"","copyfrom":"","copyfromurl":"","titlecolor":"","fonttype":"0","titleicon":"0","picurl":"https:\/\/www.cppentry.com\/upload_files\/","ispic":"0","yz":"1","yzer":"","yztime":"0","levels":"0","levelstime":"0","keywords":"Hadoop<\/A> \u63d0\u4ea4<\/A> \u4f5c\u4e1a<\/A> ------><\/A> hadoop<\/A> \u5de5\u4f5c\u6d41\u7a0b<\/A>","jumpurl":"","iframeurl":"","style":"","template":"a:3:{s:4:\"foot\";s:0:\"\";s:8:\"bencandy\";s:0:\"\";s:4:\"head\";s:0:\"\";}","target":"0","ip":"47.106.78.186","lastfid":"0","money":"0","buyuser":"","passwd":"","allowdown":"","allowview":"","editer":"","edittime":"0","begintime":"0","endtime":"0","description":" \u7248\u6743\u58f0\u660e\uff1a\u672c\u6587\u4e3a\u535a\u4e3b\u539f\u521b\u6587\u7ae0\uff0c\u672a\u7ecf\u535a\u4e3b\u5141\u8bb8\u4e0d\u5f97\u8f6c\u8f7d\u3002 https:\/\/blog.csdn.net\/hongchenlingtian\/article\/details\/53524705 \u4e00 \u3001\u9700\u8981\u77e5\u9053\u7684..","lastview":"1714591706","digg_num":"0","digg_time":"0","forbidcomment":"0","ifvote":"0","heart":"","htmlname":"","city_id":"0"},"page":"1"}