【Jsp精品源码栏目提醒】:文章导读:在新的一年中,各位网友都进入紧张的学习或是工作阶段。
网学会员整理了Jsp精品源码-hadoop学习笔记(一) - 软件工程的相关内容供大家参考,祝大家在新的一年里工作和学习顺利!
Hadoop 学习笔记 ——IT进行时Email MSN:zhengxianquanhotmail.com1. 准备工作 Hadoop最新版本为 0.20.1,网址:http://hadoop.apache.org1.1. Supported Platforms GNU/Linux is supported as a development and production platform. Hadoop has beendemonstrated on GNU/Linux clusters with 2000 nodes. Win32 is supported as a development platform. Distributed operation has not been well testedon Win32 so it is not supported as a production platform. Additional requirements for Windows include: Cygwin - Required for shell support in addition to the required software above.1.2. Hadoop 需要的基础东西 1) JDK1.6 2) Cygwin1.3. 安装 Cygwin 注意事项 在windows上跑,需要这个东西来跑shell,构建Linux-like environment for Windows。
下载地址:http://www.cygwin.com/,当前版本为 1.6。
安装过程需要注意选择安装 OPENSSL,在 Net Default 的 Category 中选中: 如下图: 模式上必须选择 For All Users,而不是 Just Me,否则到时候 2.1 步的 SSH 服务会启不来。
2. 单机上配置过程2.1. 配置 SSH2.1.1. 配置服务 1. Open Cygwin command prompt 2. Execute the following commandssh-host-config 3. When asked if privilege separation should be used answer no. 4. When asked if sshd should be installed as a service answer yes. 5. When asked about the value of CYGWIN environment variable enterntsec. 6. Here is the example session of this command note that the input typedby the user is shown in pink and output from the system is shown in gray.2.1.2. 启动 Cygwin sshd 服务2.1.3. Setup authorization keys 1. Open cygwin command prompt 2. Execute the following command to generate keys ssh-keygen 3. When prompted for filenames and pass phrases press ENTER to accept default values.4. After command has finished generating they key enter the following command to change into your .ssh directory cd /.ssh5. Check if the keys where indeed generated by executing the following command ls -l You should see two file id_rsa.pub and id_rsa with the recent creation dates. These files contain authorization keys.6. To register the new authorization keys enter the following command. Note that double brackets they are very important. cat id_rsa.pub authorized_keys7. Now check if the keys where set-up correctly by executing the following command ssh localhost Since it is a new ssh installation you warned that authenticity of the host could not be established and will be prompted whether you really want to connect answer yes and press ENTER. You should see the cygwin prompt again which means that you have successfully connected. 8. Now execute the command again ssh localhost This time you should not be prompted for anything.2.2. 修订/配置/conf/core-site.xml 默认是空的,修订为这样:fs.default.namehdfs://zhengxq:9000The name of the default file system. Either the literal string local or a host:port forDFS.hadoop.tmp.dirc:/temp/hadoopA base for other temporary directories.Multi path can split by dfs.replication1Default block replication. The actual number of replications can be specified whenthe file is created. The default isused if replication is not specified in create time. 曾经按照一些文档在这个文件配置完整而忽略了