I am trying to mount hdfs on my local machine running Ubuntu using the following command :--- sudo mount -t nfs -o vers=3,proto=tcp,nolock 192.168.170.52:/ /mnt/hdfs_mount/ But I am getting this error:- mount.nfs: mount system call failed Output for rpcinfo -p 192.168.170.52 is program vers proto port service 100000 4 tcp 111 portmapper 100000 3 tcp 111 portmapper 100000 2 tcp 111 portmapper 100000 4 udp 111 portmapper 100000 3 udp 111 portmapper 100000 2 udp 111 portmapper 100024 1 udp 48435 status 100024 1 tcp 54261 status 100005 1 udp 4242 mountd 100005 2 udp 4242 mountd 100005 3 udp 4242 mountd 100005 1 tcp 4242 mountd 100005 2 tcp 4242 mountd 100005 3 tcp 4242 mountd 100003 3 tcp 2049 nfs Output for showmount -e 192.168.170.52 is Export list for 192.168.170.52: / * I also tried by adding hadoop.proxyuser.root.groups * hadoop.proxyuser.root.hosts * in my core-site.xml file located in /etc/hadoop/conf.pseudo. But it did not work. Please help me with this.

I am running a shell script stored on HDFS (so that it can be recognized by my oozie workflow). to run this script I am using hadoop fs -cat script.sh |exec sh However I need to pass inline arguments to the script. On the CLI I would simply do this with ./script.sh arg1 Then echo the varaiable with $1. I am trying to figure out how I would do the same with a script stored in HDFS

I'm colaborating on a deployment which aims to provide Hadoop, Hive and Impala for learning and teaching pruposes. We use Ubuntu 22.04 as a base system on a VM. While all related to HDFS, Hadoop and Hive are working fine (including PostgreSQL for Hive Metastore), Impala installation is being a very hard challenge. I applied the commands recommended in [Apache's Documentation](https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala) in order to build Impala, and then ran make install. After building, when trying to run catalogd service, it doesn't start, logging the following:

cat /tmp/catalogd.pc.hadoop.log.ERROR.20250311-114741.4782
Log file created at: 2025/03/11 11:47:41
Running on machine: pc
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0311 11:47:41.450573  4782 logging.cc:256] stderr will be logged to this file.
E0311 11:48:10.130164  4782 catalog.cc:101] NoSuchMethodError: org.apache.hadoop.hive.metastore.IMetaStoreClient.getThriftClient()Lorg/apache/hadoop/hive/metastore/api/ThriftHiveMetastore$Client;
. Impalad exiting.
Picked up JAVA_TOOL_OPTIONS: -Dsun.java.command=catalogd

statestored and admissiond are working. Using Impala commit 34b17db7b473d6729ac6c9cf139fcf410f18d941 and Hive 4.0.1 pre-built.

I have the following code from the Fedora command terminal : [root@localhost python_code]# pwd /root/python_code [root@localhost python_code]# cd .. [root@localhost ~]# ls anaconda-ks.cfg python_code [root@localhost ~]# cd /usr/local [root@localhost local]# ls bin games hadoop-2.6.4.tar.gz input lib libexec sbin src etc hadoop include jdk1.8.0_77 lib64 output share [root@localhost local]# cd hadoop [root@localhost hadoop]# ls bin include libexec logs README.txt share etc lib LICENSE.txt NOTICE.txt sbin [root@localhost hadoop]# pwd /usr/local/hadoop [root@localhost hadoop]# bin/hdfs dfs -copyFromLocal /root/python_code/input_data.csv /usr/local/hadoop/input_localuser copyFromLocal: `/usr/local/hadoop/input_localuser': No such file or directory [root@localhost hadoop]# ls bin include libexec logs README.txt share etc lib LICENSE.txt NOTICE.txt sbin [root@localhost hadoop]# mkdir input_localuser [root@localhost hadoop]# ls bin include lib LICENSE.txt NOTICE.txt sbin etc input_localuser libexec logs README.txt share [root@localhost hadoop]# bin/hdfs dfs -copyFromLocal /root/python_code/input_data.csv /usr/local/hadoop/input_localuser copyFromLocal: `/usr/local/hadoop/input_localuser': No such file or directory [root@localhost hadoop]# bin/hdfs dfs -copyFromLocal /root/python_code/input_data.csv home/usr/local/hadoop/input_localuser copyFromLocal: `home/usr/local/hadoop/input_localuser': No such file or directory [root@localhost hadoop]# pwd /usr/local/hadoop [root@localhost hadoop]# ls bin include lib LICENSE.txt NOTICE.txt sbin etc input_localuser libexec logs README.txt share [root@localhost hadoop]# More Code : [root@localhost hadoop]# cd ~ [root@localhost ~]# ls anaconda-ks.cfg python_code [root@localhost ~]# cd python_code [root@localhost python_code]# ls data_generator.py input_data.csv mapper_project.py reducer_project.py [root@localhost python_code]# As seen above, my python_code directory is in root folder and I am trying to copy files from that folder to the directory input_userlocal which is inside hadoop. But the following command is throwing an error: bin/hdfs dfs -copyFromLocal /root/python_code/input_data.csv /usr/local/hadoop/input_localuser copyFromLocal: `/usr/local/hadoop/input_localuser': No such file or directory

We have Hadoop cluster with 487 data-nodes machines ( each data-node machine include also the Service node-manager ) , all machines are physical machines ( DELL ) , and OS is RHEL 7.9 version. Each data-node machine have 12 disks, each disk is with size of 12T Hadoop cluster type installed from HDP packages ( previously was under Horton-works and now under Cloudera ) Users are complain about slowness from spark applications that run on data-nodes machines And after investigation we seen the following warning from data-node logs 2024-03-18 17:41:30,230 WARN datanode.DataNode (BlockReceiver.java:receivePacket(567)) - Slow BlockReceiver write packet to mirror took 401ms (threshold=300ms), downstream DNs=[172.87.171.24:50010, 172.87.171.23:50010] 2024-03-18 17:41:49,795 WARN datanode.DataNode (BlockReceiver.java:receivePacket(567)) - Slow BlockReceiver write packet to mirror took 410ms (threshold=300ms), downstream DNs=[172.87.171.26:50010, 172.87.171.31:50010] 2024-03-18 18:06:29,585 WARN datanode.DataNode (BlockReceiver.java:receivePacket(567)) - Slow BlockReceiver write packet to mirror took 303ms (threshold=300ms), downstream DNs=[172.87.171.34:50010, 172.87.171.22:50010] 2024-03-18 18:18:55,931 WARN datanode.DataNode (BlockReceiver.java:receivePacket(567)) - Slow BlockReceiver write packet to mirror took 729ms (threshold=300ms), downstream DNs=[172.87.11.27:50010] from above log we can see the warning Slow BlockReceiver write packet to mirror took xxms and also the data-nodes machines as 172.87.171.23,172.87.171.24 etc. from my understanding the exceptions as Slow BlockReceiver write packet to mirror indicate maybe on delay in writing the block to OS cache or disk So I am trying to collect the reasons for this warning / exceptions , and here there are 1. delay in writing the block to OS cache or disk 2. cluster is as or near its resources limit ( memory , CPU or disk ) 3. network issues between machines From my verification I not see **disk** or **CPU** or **memory** problem , we checked all machines From network side I not see special issues that relevant to machines itself And we also used the iperf3 ro check the Bandwidth between one machine to other. here is example between data-node01 to data-node03 ( from my understanding and please Correct me if I am wrong looks like Bandwidth is ok ) From data-node01 iperf3 -i 10 -s [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 7.90 GBytes 6.78 Gbits/sec [ 5] 10.00-20.00 sec 8.21 GBytes 7.05 Gbits/sec [ 5] 20.00-30.00 sec 7.25 GBytes 6.23 Gbits/sec [ 5] 30.00-40.00 sec 7.16 GBytes 6.15 Gbits/sec [ 5] 40.00-50.00 sec 7.08 GBytes 6.08 Gbits/sec [ 5] 50.00-60.00 sec 6.27 GBytes 5.39 Gbits/sec [ 5] 60.00-60.04 sec 35.4 MBytes 7.51 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-60.04 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-60.04 sec 43.9 GBytes 6.28 Gbits/sec receiver From data-node03 iperf3 -i 1 -t 60 -c 172.87.171.84 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 792 MBytes 6.64 Gbits/sec 0 3.02 MBytes [ 4] 1.00-2.00 sec 834 MBytes 6.99 Gbits/sec 54 2.26 MBytes [ 4] 2.00-3.00 sec 960 MBytes 8.05 Gbits/sec 0 2.49 MBytes [ 4] 3.00-4.00 sec 896 MBytes 7.52 Gbits/sec 0 2.62 MBytes [ 4] 4.00-5.00 sec 790 MBytes 6.63 Gbits/sec 0 2.70 MBytes [ 4] 5.00-6.00 sec 838 MBytes 7.03 Gbits/sec 4 1.97 MBytes [ 4] 6.00-7.00 sec 816 MBytes 6.85 Gbits/sec 0 2.17 MBytes [ 4] 7.00-8.00 sec 728 MBytes 6.10 Gbits/sec 0 2.37 MBytes [ 4] 8.00-9.00 sec 692 MBytes 5.81 Gbits/sec 47 1.74 MBytes [ 4] 9.00-10.00 sec 778 MBytes 6.52 Gbits/sec 0 1.91 MBytes [ 4] 10.00-11.00 sec 785 MBytes 6.58 Gbits/sec 48 1.57 MBytes [ 4] 11.00-12.00 sec 861 MBytes 7.23 Gbits/sec 0 1.84 MBytes [ 4] 12.00-13.00 sec 844 MBytes 7.08 Gbits/sec 0 1.96 MBytes Note - Nic card/s are with 10G speed ( we checked this by ethtool ) We also checked the firmware-version of the NIC card ethtool -i p1p1 driver: i40e version: 2.8.20-k firmware-version: 8.40 0x8000af82 20.5.13 expansion-rom-version: bus-info: 0000:3b:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes We also checked from kernel messages ( dmesg ) but no seen something special. from dmesg about CPU dmesg | grep CPU [ 0.000000] smpboot: Allowing 32 CPUs, 0 hotplug CPUs [ 0.000000] smpboot: Ignoring 160 unusable CPUs in ACPI table [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:32 nr_cpu_ids:32 nr_node_ids:2 [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=2 [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=32. [ 0.184771] CPU0: Thermal monitoring enabled (TM1) [ 0.184943] TAA: Vulnerable: Clear CPU buffers attempted, no microcode [ 0.184944] MDS: Vulnerable: Clear CPU buffers attempted, no microcode [ 0.324340] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (fam: 06, model: 4f, stepping: 01) [ 0.327772] smpboot: CPU 1 Converting physical 0 to logical die 1 [ 0.408126] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter. [ 0.436824] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 0.436828] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details. [ 0.464933] Brought up 32 CPUs [ 3.223989] acpi LNXCPU:7e: hash matches [ 49.145592] L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.

I am trying to mount hdfs on my local machine(ubuntu) using nfs by following the below link:-- https://www.cloudera.com/documentation/enterprise/5-2-x/topics/cdh_ig_nfsv3_gateway_configure.html#xd_583c10bfdbd326ba--6eed2fb8-14349d04bee--7ef4 So,at my machine I installed nfs-common using:- sudo apt-get install nfs-common Then,before mounting I have ran these commands:- rpcinfo -p 192.168.170.52 program vers proto port service 100000 4 tcp 111 portmapper 100000 3 tcp 111 portmapper 100000 2 tcp 111 portmapper 100000 4 udp 111 portmapper 100000 3 udp 111 portmapper 100000 2 udp 111 portmapper 100024 1 udp 48435 status 100024 1 tcp 54261 status 100005 1 udp 4242 mountd 100005 2 udp 4242 mountd 100005 3 udp 4242 mountd 100005 1 tcp 4242 mountd 100005 2 tcp 4242 mountd 100005 3 tcp 4242 mountd 100003 3 tcp 2049 nfs showmount -e 192.168.170.52 Export list for 192.168.170.52: / * after that i tried mounting the hdfs using:-- sudo mount -t nfs -o vers=3,proto=tcp,nolock 192.168.170.52:/ /mnt/hdfs_mount/ But i was getting this error:--- mount.nfs: mount system call failed Then i googled for the problem and installed nfs-kernel-server,portmap using sudo apt-get install nfs-kernel-server portmap After executing the above command,the output for:--- rpcinfo -p 192.168.170.52 is:-- 192.168.170.52: RPC: Port mapper failure - Unable to receive: errno 113 (No route to host) and for showmount -e 192.168.170.52 is:--- clnt_create: RPC: Port mapper failure - Unable to receive: errno 113 (No route to host) Also the output for:-- sudo service nfs start comes out to be:-- Failed to start nfs.service: Unit nfs.service not found. Please help me with this.

I am building Hadoop Pseudo cluster in my personal machine using CentOS 9 and Hadoop 3.1.1. I completed the installation of hadoop and performed some operation, everything was fine. In later time, when I do jps, my namenode daemon was not running. So as per the guidelines from google search, I deleted namenode and datanode directory, recreated those and did "hadoop namenode -format", then my namenode (datanode as well) daemon started up and running. But I am not able to get any data stored in hdfs using "hdfs dfs -ls" which used to get me data before formatting the namenode. I understand that formatting a namenode will create new namespace ID and cluster ID in namenode VERSION file due to which we are not able to get the old data from hdfs. I want to know how can we recover the data as well. I tried the below but doesnt work. 1. Replaced the clusterID value from datanode VERSION file to namenode VERSION file, assuming datanode will send the hearbeat and report to namenode thus namenode will build the metadata. But it doesnt help. Any advice to restore the data please?

we have Hadoop cluster with 265 Linux RHEL machines. from total 265 machines, we have 230 data nodes machines with HDFS filesystem. total memory on each data-node is 128G and we run many spark applications on these machines. last month we added another spark applications, so process takes more memory from data-nodes machines. we noticed that cache. memory is very important part, and when more process are running on machines, then the right conclusion is to add more RAM memory. since we can't do memory upgrade to 256G on next 5-6 month, then we are thinking about how to improve the performance of the RHEL machine and memory cash as possible. from our experience, memory Casch is very important for applications stability. one option is to clear the RAM memory cache and buffer as the following. 1. Clear PageCache only. # sync; echo 1 > /proc/sys/vm/drop_caches 2. Clear dentries and inodes. # sync; echo 2 > /proc/sys/vm/drop_caches 3. Clear PageCache, dentries and inodes. # sync; echo 3 > /proc/sys/vm/drop_caches and run them from the cron as following. ( from https://www.wissenschaft.com.ng/blog/how-to-clear-ram-memory-cache-buffer-and-swap-space-on-linux/ ) #!/bin/bash # Note, we are using "echo 3", but it is not recommended in production instead use "echo 1" echo "echo 3 > /proc/sys/vm/drop_caches" Set execute permission on the clearcache.sh file. # chmod 755 clearcache.sh Now you may call the script whenever you required to clear ram cache. Now set a cron to clear RAM cache everyday at 2am. Open crontab for editing. # crontab -e Append the below line, save and exit to run it at 2am daily. 0 2 * * * /path/to/clearcache.sh but since we are talking on production data-nodes machines, then I am not so sure that above settings are safety, and they give (?) some solution until we can increase the memory from 128G to 256G can we get yours ideas about what I wrote? and if the "Clear RAM Memory Cache" is the right temporary solution until memory upgrade

Log file raw data :

READER_1_1_1> BIGQUERYV2_10000 [2022-11-04 01:55:20.724] [INFO] Job statistics - \n Job ID [job_PsfUvYJkPeBfecxeIzUUrIIa9TEc] \n Job creation time [2022-11-04 01:54:54.724] , \n Job start time [2022-11-04 01:54:54.936], \n Job end time [2022-11-04 01:55:10.88], \n Bytes processed [4,081,564,561] .

DIRECTOR> CMN_1740 [2022-11-04 03:25:53.269] Table: [src_BQ_CONSUMER_CURRENT] (Instance Name: [src_BQ_CONSUMER_CURRENT] Instance UI Name: [src_BQ_CONSUMER_CURRENT])

     Output Rows , Affected Rows , Applied Rows , Rejected Rows 
DIRECTOR> CMN_1740 [2022-11-04 03:25:53.269] Table: [Account] (Instance Name: [Account] Instance UI Name: [TGT_ACCOUNT])

     Output Rows , Affected Rows , Applied Rows , Rejected Rows 
DIRECTOR> CMN_1740 [2022-11-04 03:25:53.269] Table: [EU_Delta_Account_txt] (Instance Name: [EU_Delta_Account_txt] Instance UI Name: [tgt_FILE])

     Output Rows , Affected Rows , Applied Rows , Rejected Rows 
DIRECTOR> TM_6020 [2022-11-04 03:25:53.269] Session [s_mtt_0117JZ0Z000000000047] completed at [Fri Nov 04 03:25:53 2022].

Need to capture fields like Table:,Job start time, completed at,appended rows,error rows data into another file. These fields data store it in Hadoop hive table for mail send.

we have 463 RHEL 7.6 machines in the cluster most of then are HDFS machines ( datanode ) from free -g command we can that usually **buff/cach**e is around 30-50 when total memory is 256G as I know - a buffer is an area of memory used to temporarily store data while being moved from one place to another but the available memory is also memory that used for application so I am little confused , what is the diff between **buff/cache** to **available** memory ?

I just installed VirtualBox on my mac, created a new Ubuntu Virtual Machine with "Use an existing virtual hard disk file" of the Cloudera Hadoop disk image. I'm able to start and run the virtual machine, however, I'd prefer to ssh into from my terminal. The following produces the message "connect to host 127.0.0.1 port 2222: Connection refused": ssh root@127.0.0.1 -p 2222 I've also tried -p 22 I've also tried using "cloudera" as the user. Is there a VirtualBox setting I need to change to allow SSH? I've also just tried to create a new linux virtual machine without using Cloudera disk image, and I can SSH into that either.

I am with CentOS 7, and I want to start Kafka standalone producer (File Connector) as a service. The command is: /opt/kafka/bin/connect-standalone.sh /opt/kafka/config/connect-standalone.properties /opt/kafka/config/connect-file-source.properties And, obviously, it has no stop command. Normally, I just Ctrl+C to stop it as a foreground process. But when I test, I found very tedious to open several terminal console session to run several processes (Zookeeper, Kafka server, Kafka producer, Storm jar, etc.) so I change Zookeeper and Kafka server as service, which have their stop script. But in this case, no. I tried systemctl start kafka-producer and systemctl stop kafka-producer, but the service entered failed status, and is not stopped. I had to delete the service, reload the daemon, and switch to manual again. kafka-producer.service: [Unit] Description=Kafka Producer After=network.target [Service] Type=simple ExecStart=/opt/kafka/bin/connect-standalone.sh /opt/kafka/config/connect-standalone.properties /opt/kafka/config/connect-file-source.properties Restart=on-abort [Install] WantedBy=default.target Anyway to manipulate a service without ExecStop command?

I have a folder in HDFS contains 830000 files, and I want to move the last "8797" files enter code here to another folder in HDFS? I tried using xargs but didn't work fine. Any other ideas? Here is the exact split point between all files. I want to move files after "2021-03-09 15:15" ` -rw-rw-r--+ 3 talend_user talend_group 102013 2021-03-09 15:14 /user/file_1 ` ` -rw-rw-r--+ 3 talend_user talend_group 9360 2021-03-09 15:15 /user/file_2 `

I have 3 virtual machines: 1. master@master-virtualbox 2. worker1@worker1-virtualbox 3. worker2@worker2-virtualbox When I try to copy ssh id from worker1 to master, asked for password which I dont know what is it. I dont even know what password!!!. Anyway, I tried the password I created for all the machines (They all are same password) and ended up with worker1@master-virtualbox: Permission denied (publickey,keyboard-interactive). I dont even have master host or master user in the worker machines vice-versa no worker1 or worker2 users in master machine. I dont know where is it trying to connect. By the way I get similar error when I run start-dfs.sh command from my master machine. worker1@worker1-virtualbox: master@worker1-virtualBox: Permission denied (publickey,keyboard-interactive). worker2@worker2-virtualbox: master@worker2-virtualBox: Permission denied (publickey,keyboard-interactive). However when I connect (ssh worker1@worker1-virtualbox or ssh worker2@worker2-virtualbox) from master or from worker machines to connect master (ssh master@master-virtualbox) or worker to worker(ssh worker1@worker1-virtualbox or ssh worker2@worker2-virtualbox) etc. Connecting without password , no problem. Any help will be appreciated. Thanks in advance

Not able to upgrade python on Cloudera as it shows the error whenever **I** run the below command:

$ sudo yum install python27
error: No package python27 available.

$ sudo yum install python36u
error: No package python36u available

I also run yum install mrjob but still not able to upgrade my python in cloudera.

I'm trying to install Hadoop 2.7.3. on Elementary OS (which ~ Ubuntu, I believe) following the instructions in the _BUILDING.txt_ that came with the Hadoop files. The file indicates that I need to install Oracle JDK 1.7 by going through the following steps: $ sudo apt-get purge openjdk* $ sudo apt-get install software-properties-common $ sudo add-apt-repository ppa:webupd8team/java $ sudo apt-get update $ sudo apt-get install oracle-java7-installer All works well until the last step, where it exits with an error message which I'm not sure what to make of (sorry if paste is too long; I'd be happy to edit it but Idk what's relevant):

--2017-05-26 10:58:06--  http://download.oracle.com/otn-pub/java/jdk/7u80- 
b15/jdk-7u80-linux-x64.tar.gz Resolving download.oracle.com
(download.oracle.com)... 199.117.103.169, 199.117.103.171

Connecting to download.oracle.com
(download.oracle.com)|199.117.103.169|:80... connected. HTTP request
sent, awaiting response... 302 Moved Temporarily Location:
https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz 

[...]

[following]

--2017-05-26 10:58:06--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1495814405_54d4699c133b9ceee2666c152616ece6 
Connecting to download.oracle.com
(download.oracle.com)|199.117.103.169|:80... connected.

HTTP request sent, awaiting response... 404 Not Found 2017-05-26
10:58:07 ERROR 404: Not Found.

download failed

Oracle JDK 7 is NOT installed.

dpkg: error processing package oracle-java7-installer (--configure): 
subprocess installed post-installation script returned error exit
status 1 dpkg: dependency problems prevent configuration of
oracle-jdk7-installer:  oracle-jdk7-installer depends on
oracle-java7-installer; however:   Package oracle-java7-installer is
not configured yet.

I figured - the 404 is the main issue - it's looking for files but can't find them So my question is - Is that even true? What is the roadblock here? - How can I fix it, or do I have to take another path? (I am following White's _Hadoop - The Definitive Guide_ for the overall approach. )

I have installed hadoop and ssh. hadoop was working fine, then today I am getting the error below when I run the command sbin/start-dfs.sh: Starting namenodes on [localhost] localhost: ssh: connect to host localhost port 22: Connection refused Starting datanodes localhost: ssh: connect to host localhost port 22: Connection refused Starting secondary namenodes [chbpc-VirtualBox] chbpc-VirtualBox: ssh: connect to host chbpc-virtualbox port 22: Connection refused I've checked the ssh status, and I have the following error: ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled) Active: failed (Result: start-limit-hit) since Tue 2020-02-04 15:34:10 +04; 3h 35min ago Process: 946 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=255) Feb 04 15:34:09 chbpc-VirtualBox systemd: ssh.service: Control process exited, code=exited status=255 Feb 04 15:34:09 chbpc-VirtualBox systemd: Failed to start OpenBSD Secure Shell server. Feb 04 15:34:09 chbpc-VirtualBox systemd: ssh.service: Unit entered failed state. Feb 04 15:34:09 chbpc-VirtualBox systemd: ssh.service: Failed with result 'exit-code'. Feb 04 15:34:10 chbpc-VirtualBox systemd: ssh.service: Service hold-off time over, scheduling restart. Feb 04 15:34:10 chbpc-VirtualBox systemd: Stopped OpenBSD Secure Shell server. Feb 04 15:34:10 chbpc-VirtualBox systemd: ssh.service: Start request repeated too quickly. Feb 04 15:34:10 chbpc-VirtualBox systemd: Failed to start OpenBSD Secure Shell server. Feb 04 15:34:10 chbpc-VirtualBox systemd: ssh.service: Unit entered failed state. Feb 04 15:34:10 chbpc-VirtualBox systemd: ssh.service: Failed with result 'start-limit-hit'. How can I fix this?

i am trying to start my hadoop cluster using the command "start-dfs.sh" but am getting errors as shown below Starting namenodes on [master] master: ssh: connect to host master port 22: Connection refused Starting datanodes master: ssh: connect to host master port 22: Connection refused ive checked the ssh status, it then returns me: ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enab Active: failed (Result: start-limit-hit) since Tue 2020-02-04 14:15:01 +04; 2 Process: 5017 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=255) Feb 04 14:15:00 hadoop-HP-Pro3500-Series systemd: ssh.service: Unit entered f Feb 04 14:15:00 hadoop-HP-Pro3500-Series systemd: ssh.service: Failed with re Feb 04 14:15:01 hadoop-HP-Pro3500-Series systemd: ssh.service: Service hold-o Feb 04 14:15:01 hadoop-HP-Pro3500-Series systemd: Stopped OpenBSD Secure Shel Feb 04 14:15:01 hadoop-HP-Pro3500-Series systemd: ssh.service: Start request Feb 04 14:15:01 hadoop-HP-Pro3500-Series systemd: Failed to start OpenBSD Sec Feb 04 14:15:01 hadoop-HP-Pro3500-Series systemd: ssh.service: Unit entered f how to fix this?

we have two namenode in the cluster hadoop cluster this is good example of one of the name node that return good status first namenode machine IP - 92.3.44.2 curl -v http://92.3.44.2:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus * About to connect() to 92.3.44.2 port 50070 (#0) * Trying 92.3.44.2... * Connected to 92.3.44.2 (92.3.44.2) port 50070 (#0) > GET /jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus HTTP/1.1 > User-Agent: curl/7.29.0 > Host: 92.3.44.2:50070 > Accept: */* > GET /jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus HTTP/1.1 > User-Agent: curl/7.29.0 > Host: 92.3.44.2:50070 > Accept: */* > we can see that command curl is stuck how to identify why curl is stuck ?

we are using HDP version - 2.6.5 , and HDFS Block replication is 3 we are try to understand data nodes disks min requirements for production mode and according to the fact that Block replication=3 since we are talking about production cluster and regrading to HDFS replica = 3 what should be the min disks number per data-node machine?