Fraunhofer IPA has developed a benchmark to check humanoid robots such because the Unitree G1. Supply: Fraunhofer IPA
In recent times, humanoid robots have been omnipresent within the media and proceed to fascinate us. On social media in addition to in public areas, they’re assured to draw consideration. The expertise is poised to take over duties in areas the place, attributable to demographic change, human labor will now not be obtainable sooner or later.
But there stays a major data hole between media hype and precise talents. Whereas the robots are being examined in remoted, personal pilot functions, questions persist for many potential customers relating to what talents the robots truly possess, how reliably they function, and to what extent we people can belief humanoids.
“For finish customers and producers alike, it’s subsequently important to look behind the facade generally constructed by advertising and marketing companies,” defined Simon Schmidt, senior supervisor of the automated methods enterprise unit at Fraunhofer. “The market is just too unstable and opaque to permit for a well-founded evaluation and dependable analysis of humanoids for one’s personal functions or compared to different fashions.”
That is exactly why Fraunhofer IPA has developed a benchmark. On this course of, analysis groups from the institute’s automation division put humanoids by numerous challenges and consider the outcomes. The impartial, third-party service is modular, permitting producers, finish customers, or software program suppliers to pick the areas related to their software.
Benchmark follows industrial requirements
The query of talents and trustworthiness is damaged down into six application-relevant standards:
Six check classes for humanoid robots. Supply: Fraunhofer IPA
The benchmark attracts on the analysis groups’ current experience and, the place potential, follows established industrial requirements which were internationally acknowledged for many years. Within the space of cleanliness, for instance, Fraunhofer IPA has examined and certified over 3,000 automation parts in keeping with ISO 14644 in recent times.
The measurement of collision forces and different safety-relevant properties relies on widespread security requirements for force- and power-limited robots akin to ISO 10218 and ISO TS 15066.
The benchmark relies on grasp knowledge such because the robots’ dimension, weight, and attain. It was additionally essential to the analysis groups to outline reproducible exams that may be standardized and stay significant for future generations of humanoids. In any case, extra humanoids will observe.
“With this instrument, humanoids may be in contrast not solely with each other but in addition with recognized automation parts,” mentioned Werner Kraus, head of the analysis division automation and robotics at Fraunhofer IPA. “Customers can interpret the outcomes immediately and thus discover the correct humanoid for the correct software.”
Utilizing the Unitree G1 for instance, Fraunhofer IPA itself put the humanoid by its paces with regard to its suitability to be used in manufacturing and, in doing so, discovered a fantastic deal in regards to the present limitations of the expertise. The technical foundation was the Unitree G1 EDU-4 with Dex3-1 3-finger palms delivered in Could 2025 with firmware Model 1.04.
Editor’s be aware: The 2026 Robotics Summit & Expo this month in Boston will embody a keynote and periods on humanoid robots. Register now to attend.
Six standards for evaluating humanoid robots
1. Expertise and primary talents
The applied sciences utilized in humanoid robots, akin to sensors or AI fashions, enable direct conclusions to be drawn about precision and reliability. Analyzing further primary talents permits detailed assessments of the humanoid’s technological potential and skills.
Amongst different issues, the analysis examines sensor expertise (akin to imaginative and prescient, audio, textual content recognition, speech recognition, human detection), manipulation talents (gripper kind, quantity and mobility of fingers), in addition to power (dealt with hundreds, gripping forces), and strolling velocity.
An examination of the applied sciences is performed by figuring out the put in parts and evaluating them with knowledge sheets. Exams allow the dedication of primary talents. A 3D monitoring system from Vicon is used to find out strolling velocity. Gripping forces are measured utilizing a pressure sensor. Dumbbells of various weights can be found to find out the utmost load that may be dealt with.
Utilizing the G1 for instance, it grew to become obvious, amongst different issues, that its dexterity continues to be far beneath human ranges. As delivered by the producer, it may solely stroll by way of distant management. Customers should implement further primary talents themselves. Extending the arms horizontally even with none further load could cause them to close down and drop after one to 2 minutes as a result of the actuators overheat.
Measurements taken with the Vicon tracker decided that 0.49 m/s (1 mph) is the sluggish strolling velocity and 0.84 m/s (1.8 mph) is the quick strolling velocity. When carrying a 3-kg (6.6 lb.) payload, the robotic doesn’t decelerate, nevertheless it takes a number of tenths of a second longer to speed up and decelerate.
The outcomes present that the humanoid can not but carry out many duties that people can deal with. Nonetheless, future software program and {hardware} updates are anticipated to extend the variety of pre-implemented primary talents.
2. Advanced talents
Constructing on the essential talents, this part focuses on performing small generic duties that require a mixture of applied sciences and abilities. The benchmark permits a complete analysis of humanoids throughout numerous activity domains.
The exams may be categorized into whole-body actions (operating, leaping, climbing, navigating ramps, standing up), manipulative abilities (opening doorways), and navigation by impediment programs, in addition to precision and pressure management, The affect of fixing environmental circumstances and extra hundreds on robotic efficiency may also be measured.
Most of the designed exams couldn’t but be carried out with the G1. For instance, in keeping with the producer’s specs, the robotic isn’t appropriate for climbing stairs. Advanced impediment programs requiring the robotic to navigate are additionally not possible with its onboard talents.
Nonetheless, when strolling over steps (cable duct) and on a slope with 20% inclination, the legged robotic demonstrated good self-stabilization talents. It by no means misplaced its stability through the exams. For the G1, standing up from a supine place requires surfaces with enough friction, akin to carpet. On clean tile or hardwood flooring, the arms could slip, leading to a failure to push the robotic in an upright place.
The exams for complicated talents are deliberately designed to overwhelm present humanoids. Solely future fashions will be capable to absolutely meet these benchmarks. This permits comparability of humanoids throughout a number of mannequin generations. Moreover, the duties that can’t presently be carried out clearly display the boundaries of present expertise to potential customers.
3. Cleanliness
The cleanroom suitability benchmark examines whether or not humanoids and different automation parts can be utilized in delicate manufacturing environments such because the semiconductor, optical, electrical, pharmaceutical, biotechnology, and meals industries with out inflicting crucial contamination.
Particle emission is evaluated at numerous factors on the shifting robotic in accordance with ISO 14644-14, outgassing conduct in keeping with ISO 14644-15, and cleanability and hygienic design in keeping with present tips. The aim is to objectively decide the robotic’s suitability for cleanroom use and, if essential, determine areas for optimization in design or materials choice.
Conducting the benchmark is difficult as a result of complexity of humanoids and their numerous vary of movement. Typical working parameters should be outlined and worst-case situations examined to acquire reasonable outcomes. The restricted battery life should be thought of when figuring out the check period.
Upon completion of the benchmark, clients obtain a qualification bundle with a report and certificates, which serves as a dependable foundation for decision-making relating to use in cleanrooms and permits a validated, trade commonplace analysis.
In exams relating to particle emissions and outgassing, the G1 confirmed promising outcomes. Accordingly, the humanoid is predicted to be appropriate to be used in ISO Class 5 cleanrooms per ISO 14644-1, akin to these used within the semiconductor trade.
Exams on hygienic design revealed that the cleanability of the machine was evidently not a precedence through the improvement of the G1. There are gaps and areas on the quite a few joints which might be inaccessible for wipe cleansing. Consequently, use in areas with excessive hygiene necessities isn’t potential with out additional measures.
On condition that cleanroom suitability and hygienic design will not be presently a precedence for producers, the check outcomes ought to be considered positively. Ought to producers develop fashions sooner or later which might be particularly optimized for cleanliness, the exams developed right here present a useful basis. Producers can fee the intensive exams at Fraunhofer IPA and thus profit from the present cleanroom infrastructure.
Particle emissions exams of the Unitree G1 point out that use in ISO 5 cleanrooms is feasible. Supply: Fraunhofer IPA
4. Practical security
Humanoid robots are supposed to collaborate with people and share widespread workspaces. Subsequently, useful security is of nice significance. The components influencing security in humanoids are numerous. These embody, above all, stability when shifting on totally different surfaces, conduct on slopes, and when lifting hundreds.
Added to this are the detection and avoidance of obstacles and pressure limitation within the occasion of collisions. Additionally related are system conduct throughout communication failures and full battery discharge, dangers arising from {hardware} design, and the accessibility of emergency cease gadgets.
Numerous challenges had been developed to evaluate stability. For instance, a humanoid should navigate a course that includes totally different surfaces akin to steps or cable ducts from numerous method instructions.
To check stability on slopes, the robotic walks up and down ramps and performs stops and adjustments in route. The experiments may be recorded utilizing the 3D monitoring system.
Pressure sensors had been used for collision exams; these are additionally employed for pressure measurements in accordance with ISO TS 15066 for collaborative robots. These sensors are used to analyze each collision actions of the arms and collisions with a shifting robotic (such because the chest or shoulder).
It was discovered, for instance, that the G1 can exert forces exceeding 500 N on people throughout full-body collisions and through speedy arm actions. This considerably exceeds the ache thresholds permitted by the usual for operation close to people, that means that accidents can’t be dominated out.
Conversely, nonetheless, the robotic may be very secure and may hardly be thrown off stability even on tough surfaces.
The exams additionally revealed that the absence of an emergency cease button on the robotic can result in issues if it turns into unstable throughout experiments and must be stopped rapidly. On this case, the one solution to minimize off the facility provide to the G1 is to take away the battery. Potential for enchancment was additionally recognized relating to pinch factors within the space of the joints.
The check outcomes present that you will need to present producers and customers with an goal, evidence-based basis for evaluating and bettering security, as current requirements for human-robot collaboration are solely of restricted applicability.
The ISO isn’t anticipated to publish the ISO 25785-1 commonplace, which incorporates security necessities for humanoids, till 2028. Till then, functions involving humanoids should be made secure on a case-by-case foundation utilizing a sequence of complementary measures.
The G1 can exert forces exceeding 500 N on people throughout full-body collisions and through speedy arm actions. Supply: Fraunhofer IPA
5. Cybersecurity
As complicated {hardware} and software program merchandise, humanoid robots usually obtain updates and thus profit from an web connection. The benchmark responds to rising authorized necessities and the rising significance of cybersecurity in robotics. It helps determine vulnerabilities and open factors of assault at an early stage, which come up specifically from uncared for safety architectures.
The outcomes present a fast overview of the robotic’s resilience in opposition to typical assault patterns and provide producers steering for enhancements. On this means, the benchmark helps to sustainably strengthen the acceptance and safety of humanoid robots out there.
The Cybersecurity Benchmark consists of 4 independently usable modules: vulnerability evaluation, safe lifecycle, connectivity, and stability. Every module examines particular elements akin to:
- Administration and existence of vulnerabilities
- Dealing with of updates and finish of life for {hardware} and software program
- Safety of community interfaces
- The system’s resilience and resistance to penetration, or how safe is it in opposition to unauthorized entry?
The aim is to offer producers with an goal, scientifically sound evaluation of their cybersecurity measures and to display whether or not the safe operation of their merchandise is assured.
On the time of the investigation, the G1 had a documented distant code execution vulnerability inside its Bluetooth interface. This allowed attackers inside Bluetooth vary to realize intensive management over the robotic.
The vulnerability persevered throughout a number of firmware updates, even after an replace changelog had famous its decision. Based mostly on present data, it has since been patched by a more moderen replace.
Moreover, the G1 offers no data relating to end-of-life expectations for its {hardware} and software program, common replace intervals, or warnings for recognized cybersecurity vulnerabilities. Unitree has not but described the continual transmission of sensor knowledge to the producer’s servers (with out the choice to disable this within the administration interface). Throughout load exams, the G1 carried out very stably.
6. Vitality effectivity
Battery life is a key design parameter for the operation of humanoid robots. This benchmark addresses standards akin to battery capability, charging instances, and the vary of the cost relying on the working mode. It affords robotic producers a standardized, reproducible technique for measuring and evaluating the vitality consumption of humanoids beneath outlined, real-world working circumstances.
The benchmark is primarily aimed toward robotic producers and offers impartial vitality metrics that function a reference for exterior communication and aggressive positioning. It additionally helps finish customers higher plan the working time and charging cycles of their robots and particularly optimize vitality effectivity.
Vitality necessities and common energy consumption are measured in numerous situations akin to standing, strolling on flat floor, strolling uphill, and strolling beneath load. Additionally, power-on and power-off procedures in addition to battery charging conduct are taken under consideration.
The outcomes are consolidated right into a significant metric that describes the vitality consumption per hour and the battery runtime beneath standardized circumstances, enabling an goal comparability of various robotic fashions. Distinctions may be made between numerous utilization situations, akin to stationary work at a workstation or predominantly strolling in logistics duties.
When utilized to the Unitree G1, the measurements present a median energy consumption of roughly 154 W in stationary operation. When strolling on a flat floor, energy consumption is roughly 272 W, whereas it rises to round 283 W when strolling on a ten % incline. For the outlined one-hour commonplace situation, this leads to a median energy consumption of roughly 239 W.
The outcomes display the anticipated variations between static operation and motion and ensure that the benchmark persistently displays these results. This leads to most working instances of two hours and 49 minutes on a single battery cost whereas standing nonetheless, and 1 hour and 49 minutes in a typical situation that features each standing and strolling.
Customers and producers can use the benchmark to systematically incorporate vitality metrics into the choice and deployment planning of humanoid robots. Towards this backdrop, Fraunhofer IPA expects that the vitality necessities of humanoids will sooner or later be extra strongly oriented towards dimension, system structure, and the respective software profiles, and might be differentiated accordingly.
Energy consumption and battery lifetime of the Unitree G1. Supply: Fraunhofer IPA
Fraunhofer IPA is able to check future humanoid robots
Many individuals count on humanoids to display their added worth notably the place flexibility is required and the place there’s a have to function in environments designed for people. On this respect, there are numerous potential software situations. These typically require very totally different traits and skills from humanoids.
The developed check strategies and preliminary outcomes present that an impartial analysis of traits and skills is each helpful and essential. Whereas benchmarks for applied sciences and abilities give attention to transparency and comparability, impartial testing can assist construct belief in humanoid robots relating to points akin to useful security and cybersecurity.
Fraunhofer IPA’s aim is to check further humanoids and thus create a comparative database. Producers and customers can contact us relating to the whole lot from preliminary normal inquiries and curiosity in particular person benchmark modules to requests for complete testing.
In regards to the authors
Simon Schmidt has been a senior supervisor of the Automated Programs enterprise unit at Fraunhofer IPA since 2024. His key subjects are cognitive robotics and embodied AI in intralogistics, manufacturing, and for meeting methods.
Dr. Werner Kraus has been head of the Automation and Robotics analysis unit at Fraunhofer IPA since 2019. He’s managing loads of nationwide and worldwide initiatives and is an knowledgeable in cognitive robotics.
Dr. Theo Jacobs has been chief of the Secure Robotic Purposes and Cobots analysis group at Fraunhofer IPA since 2023. He’s additionally a licensed equipment security knowledgeable and engaged in ISO actions round secure humanoid robots.
The publish Fraunhofer IPA affords new check benchmark for humanoids appeared first on The Robotic Report.
