What is the principle of “code moving to data” rather than data to code?
In a recent discussion about distributed processing and streaming I came across the concept of 'code moving to data'. Can someone please help explaining the same. Reference for this phrase is MapReduceWay. In terms of Hadoop, it's stated in a question but still could not figure out an explanation of the principle in a tech agnostic way. The basic idea is easy: if code and data are on different machines, one of them must be moved to the other machine before the code can be executed on the data. If the code is smaller than the data, better to send the code to the machine holding the data than the other way around, if all the machines are equally fast. In the world of Big Data, the code is almost always smaller than the data. I also just came across the sentence “ Moving Computation is Cheaper than Moving Data ” (from the Apache Hadoop documentation) and after some reading I think this refers to the principle of data locality . Data locality is a strategy for task schedulin...