Path planning of manure-robot cleaners using grid-based reinforcement learning

The use of a robot cleaner for manure removal improves housing conditions for dairy cows in the face of labor shortages. However, current robot cleaners follow programmed fixed routes without considering the dynamic behaviors of cows. This cleaning approach is less efficient and leads to more cow-robot encounters or collisions, thus affecting animal welfare. To address these issues, this paper (1) developed heatmap models for cow locations and defecation behaviors; (2) proposed a dynamic path planning approach for the manure robot cleaner using Grid-based Reinforcement Learning; (3) incorporated cow location information and defecation behavior into the path planning process; (4) compared the performance of the proposed approach with two different cleaning methods: the current fixed programmed cleaning in practice and the ideal path produced by simulated annealing for traveling salesman problem. The simulations mimic the situation in a barn at Dairy Campus of Wageningen Livestock Research located in Leeuwarden (the Netherlands). Obviously, the best performance was achieved when the route was executed without cows present, resulting in no cow-robot collision. However, with cows present, the proposed dynamic path planning strategy achieved a 67.6% reduction in cow-robot encounters while maintaining 85.4% of the cleaning performance compared to the current programmed fixed routes. Compared to the ideal path produced by simulated annealing for traveling salesman problem, the proposed dynamic path planning approach achieved 5% better cleaning performance, at the cost of 25% more cow-robot encounters due to its longer working path. We conclude the proposed grid-based Reinforcement Learning solution for manure robots in barns cleaned most efficient with the least interference with cow traffic.