Without access to modern farming techniques or machinery, let alone science-based climate and weather data, farmers’ livelihood hinge precariously on a changing environment that they are struggling to understand

U.S. Agency for international development      

INTRODUCTION

Though the microfarms are small in nature, they offer a wide range of farming options. Microfarmers grow a wide range of crops and are involved in animal husbandry. Furthermore, they help us produce some of our own food, increase sustainability, and reduce our reliance on commercial markets. Agricultural robots automate slow, repetitive and dull tasks for farmers, allowing them to focus more on improving overall production yields. ROMI project is located at the intersection of microfarming and robotics linking specialized partners spread around Europe. 

A new generation of farmers are starting small innovative market gardens in rural, peri-urban and urban areas across Europe. These farms often grow polycultures of up to 100 different varieties per year on small surfaces between 0.01 to 5 hectares. Polyculture and organic microfarms are proving to be highly productive, sustainable and economical, yet some of the on-the-ground experiences of farming communities are still unknown. ROMI aims to develop a better understanding of this emerging field through research, events and the development of specialised techniques and tools.

CONTEXT

ROMI is a four-year Europe-funded research project committed to promote a sustainable, local, and human-scale agriculture. It is developing an affordable, multipurpose platform adapted to support organic and polyculture market-garden farms.

The platform constitutes robotic tools, research, data and shared documentation and aims to help farming communities increase their production and improve their working conditions.

WP3 AERIAL ROBOTICS

IAAC, in collaboration with Fab Lab Barcelona and Noumena, is developing aerial robotic solutions to tackle the necessity of farmers in heterogeneous environments. Noumena previous experience on this topic rely on the development of N.E.Ro – Networking Environmentals RObotics, a series of autonomous aerial and land robots, developed, designed and programmed in order to accomplish specific tasks: from fields mapping to urban analysis. This workflow allows to automate repetitive tasks and to simplify complex procedures.

The proposed workflow is based on automated robotic collection of multispectral images using an aerial robot that provide precise information to a ground rover equipped with actuators to locally operate on specific plants. 
This solution allow us to remotely gather images, store it online and program the most efficient path to minimize timing and resource usage of the overall operation.

Data processing is developed through a specific customized method, generating one single point cloud, storing RGB, NDVI and Thermal Data over the same geometrical organization. This serves as a base to store the additional layers like the typology of plant, the health status, and the growth rate. 

cablebot
ROMI_black

PROJECT TITLE
ROMI

WP3 LEADER
IaaC – Institute for Advanced Architecture of Catalunya

WP3 COLLABORATORS
Fab Lab Barcelona
Green Fab Lab
Noumena

PERIOD
2018-2022

PARTNERS
PRINCIPAL
EU Commission
Horizon 2020
FINANCIAL
FEI
TECHNOLOGICAL 
SONY CSL
Inria
CNRS
INSTITUTIONS
IaaC
Humboldt University
FARMING COMMUNITY
Can Valldaura
Pepinieres Chatelain

VEGETABLES GARDEN DATA
Type: 3 rows cultivation
Area:  72sqm (14×5 meters)
Distance from the ground: 2.50m
Slope: none

CAMERA INFORMATION
Type: Mapir Survey 3W
Filter: OCN (Orange Cyan Nir)
Tools: Calibration target
Extra: Pi cam live streaming

OUTPUT
Format: RAW
Capture distance: 50cm
Gps: Yes
Color marker: Yes

HARDWARE

WP3 Aerial focus on aerial robotic solutions with the main aim of developing a drone and a cable bot. This last serves as an alternative to drone in restricted and complex environments. The basic setup consists of four vertical poles that serve as support for the cable system. These cables act as the linear axis for the robot movement. The second axis movement, useful to cover surfaces, allow the system to slide. Combining the two axes we can easily locate the cable bot in a confined area. 

NERO Drone v0.1
NERO Drone v0.1 is an open source project that has 4 key components: on-board sensors, automated flight paths, NDVI and Interoperable data. The design of NERO v0.1 drone frame has been developed parametrically in order to be adapted to different motor setups and sizes.

 

romi20

NERO Drone V0.2
This new version of the NERO drone was improved thanks to the adoption of a new frame made of carbon fiber tubes and plates. These new features made the structure of the drone lighter and more rigid, increasing the frame size and allowing for biggers propellers and motor with the aim to increase the active payload as well as to reduced the vibrations.

romi12

NERO Aerobot
The aerobot consists of an aerostatic balloon that carries a camera and it is controlled by four ropes connected to stepper motors. A
carbon fiber tubes structure is used as a connection for the balloon and for the four cables arriving from the motors. There is has a plate for fixing the gimbal and the cameras. A custom firmware runs the motors and allow to direct the Aerobot in a precise location with a known height.

romi15

NERO Cablebot
During the design phase of the cable bot, several inputs were taken into account: weight, resistance, waterproof connection and compactness. This allows to iterate and develop a modular system that allows to easily replace parts or install upgrades. The cable bot relies on a precise GPS – Rtk localization system to move in the environment and geotag the recognized plants.

romi13

SOFTWARE

Our strategy to integrate the core software system was to embed a single-board computer such as a Raspberry-PI or a Jetson TX2 in both the drones and the aerobot. This architecture allows to create an interface with the internet, read real-time data and send messages through the serial port to other robots.

The software is composed of the following modules:

  • cameras-python: This Python library aims to collect all the methods, remote controls, and functionalities for the gimbal and the trigger of the cameras. This set serves to synchronize separated hardware and to be able to store organized data for the post-processing.
  • cable-bot-firmware. This library specifically deals with the firmware and supports the tradition g-code instructions for the movements of the robots. This firmware also deals with the physics trigonometry’s and the electronics in order to make the device fully scalable.
  • pointcloud-fusion-python: This Python library is a set of methods and functionalities for Point clouds and the automation of data fusion of Thermal and Multispectral imagery, a basis for the generation of geometrical model in COLMAP.
  • web-exporter: This is docker container that contains the potree export implementation. This container allow to export PLY files into octree from las, laz, binary ply, xyz or ptx files.

DATA ANALYTICS

A specific task for the aerial robotic solutions is to map and analyze the crop cultivation. To achieve this task, a precise workflow was established to translate the information recovered from the cameras into a point cloud representation of each single field. Working with point clouds is a critical asset for the operations having at disposal many layers of data using a 3 dimensional model, describing in this way the topographic characteristics of the soil, monitoring crops and vegetation health status through infrared cameras, evaluating the status of the irrigation for the field and the temperature of the soil thanks to thermal cameras.

  • photogrammetry: For the generation of the point clouds third-party software, commercial and open source, have been tested to compare performances, time, hardware needs and accuracy of the results. All point clouds are generated with multiple cameras, from multispectral cameras to RGB and thermal ones. The goal is to integrate multiple layers of data  to describe and detect critical issues regarding the crops quality.
  • NDVI: A crucial asset we are implementing in this operation is the possibility to extrapolate from infrared bands NDVI information in relation to the operational context. Normalized Difference Vegetation Index (NDVI) quantifies vegetation by measuring the difference between near-infrared (which the vegetation strongly reflects) and red light (which the vegetation absorbs). Some of these operation were carried out using NERO library developed by Noumena.
  • Data fusion: A synthetic map collects all the different layers of information that allows for data correlation and interpolation. During the first year, the prototype collects data related to RGB, NIR, NDVI, and depth.  Future development will integrate maps related to: humidity, tempearture, rate of growth and taxonomy.

RESEARCH GARDEN REPORT

In the fields of Can Valldaura, 5 different areas were selected as relevant and named the “research gardens”. It has been possible to reconstruct the 3D model of the area through a series of drone missions. The type of data produced are respectively: a RGB pointcloud of the area, from which derives the depth point cloud and a NIR pointcloud, which is used for the calculation of the NDVI pointcloud. An implementation of a thermal pointcloud is still in progress.

Each point cloud is scaled in meters and aligned (registered) with the other cloud corresponding of the same research garden. The aim is to realize a continuative survey of these areas in order to keep track of the evolution in time.

R1 – VEGETABLE GARDEN 

This area hosts different cultivations of low and medium height vegetables, its a narrow and long piece of land located closely to the main building. Its easy to access and to monitor.

R2 – ROBOTIC STAGES

This small 15×4 meters area is dedicated for the direct robot application: this is the stage of the cable bot and rover, the area contains three channel of cultivation suited to make different tests.

R3 – FOOD FOREST 

Several fruits tree covers this area, located nearby the villa in a sloped terrain.

R4 – MELLIFEROUS GARDEN

Portion of the forset characterized by low vegetation and high trees. This area is located in sloped terrain.

R5 – ROOT GARDEN

Small area dedicated to root cultivation: its main product are calzots, typical vegetable from Catalunya, similar to the onion.

PARTNERS

PRINCIPAL

h2020
eu-comm

FINANCIAL

fei

TECHNOLOGICAL

Sony-csl
inria
cnrs

INSTITUTIONS

IAAC
humboldt

FARMING COMMUNITY

valldaura
pepiniers
flag_yellow_high

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 773875