The EUROBENCH team has created the first benchmarking framework for robotic systems in Europe. The framework allows companies and researchers to test the performance of their robots at any stage of development.
Throughout these almost five years of the project , the EUROBENCH team has developed two benchmarking facilities and a benchmarking software allowing research teams to conduct standardized tests on advanced robotic prototypes.
- – Regarding the two benchmarking facilities, one is destinated to wearable robots, including exoskeletons and prostheses, and is located in the Hospital Los Madroños (Madrid), and the other one is destined to humanoid robots and is located in the Italian Institute of Technology (Genoa).
- The unified benchmarking Software allows researchers, developers, and end-users worldwide to design and run the tests in their own laboratory environments.
EUROBENCH has partnered with external entities, known as Third Parties, to develop and validate specific sub-components of the Facilities and Software. This has been carried out by means of a procedure of cascade funding called FINANCIAL SUPPORT TO THIRD PARTIES (FSTP). In this context, the EUROBENCH team organized two competitive Open Calls (FSTP-1 and FSTP-2).
1st FSTP Open Call: “DEVELOPMENT of the benchmarking framework”. The Call sought Third Parties interested in designing and developing benchmarking facilities. We received 53 proposals from more than 100 participants from 15 different countries in Europe. The proposals submitted were called to apply to one of the following options:
- Option 1: develop a benchmarking solution for one specific benchmarking scenario including one or more outcomes: testbed, software routines and/or experimental datasets.
- Option 2: propose a benchmarking device, composed of sensors and/or actuators, that could be applied indistinctly to different benchmarking scenarios.
2nd FSTP Open Call: “VALIDATION of the benchmarking framework”. Through this Open Call, Third Parties had the opportunity to use the benchmarking facilities and software at no cost to test and improve their own robotic systems. This way the EUROBENCH consortium can use Third Parties feedback to validate the usability (efficiency, effectiveness, and satisfaction) of the EUROBENCH framework. Each of the submitted proposals was invited to apply to one of the three funding modalities:
- Option 1 (In situ experiments). Applicants proposed testing experiments to be conducted in one of the two EUROBENCH facilities (either the one for Wearable Robots or the one for Humanoids).
- Option 2 (Remote experiments). Applicants proposed testing experiments to be conducted in their own laboratory by using the EUROBENCH tools.
- Option 3 (Retrospective data analysis). Applicants proposed the use of the EUROBENCH software to process retrospective data on robot and/or humans. All the collected and processed data should be uploaded and integrated in the EUROBENCH database.
At the EUROBENCH closing event, which will be held on 21 & 22 June in Hospital Los Madroños (Madrid), the consortium will present the results achieved throughout the project. Furthermore, the EUROBENCH Summit will host robotics and benchmarking experts involved in the EUROBENCH project. The attendees will have the opportunity to network, know the EUROBENCH facilities and learn about the benchmarking software that is accessible from the cloud. It will be a key event for robotics!