Some ‘small’ text files that contain about 500000 entries/rows. Each row has also a ‘key’ column. I need to find this keys in a big file (8GB, at least 219 million entries). When found, I need to append the ‘Value’ from the big file into the small file, at the end of the row as … Continue reading How to find this keys in a big file (8GB, at least 219 million entries)
When a node fails, the blocks stored on there no longer count as being available for HDFS. You can easily see this by looking at the number of under replicated blocks when a node fails or is disabled. In general, the system will try to solve underreplication when it occurs (and when capacity is available). … Continue reading What is the procedure of disk or data node failure and recovery in hadoop
What is Machine learning? Machine learning is a method of data analysis that automates analytical model building. Using algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look. Essentially, it is a method of teaching computers to make and improve predictions or behaviors based on … Continue reading Big Data Machine Learning
The Life of a Data Scientist Data scientists are big data wranglers. They take an enormous mass of messy data points (unstructured and structured) and use their formidable skills in math, statistics and programming to clean, massage and organize them. Then they apply all their analytic powers – industry knowledge, contextual understanding, skepticism of existing … Continue reading How to become a Data Scientist
Big Data /Hadoop and SAP HANA from Global Business Solutions SME
