In order to automatically monitor farmers’ activities, we propose a farm operation monitoring system using “Field Servers” and a wearable device equipped with an RFID reader and motion sensors. Our proposed system helps in recognizing farming operations by analyzing the data from the sensors and detected RFID tags that are attached to various objects such as farming materials, facilities, and machinery. This method can be applied to various situations without changing the conventional system. Moreover, this system provides useful information in real-time and controls specific machines in a coordinated manner on the basis of recognized operation.