Performance Assessment Framework for Robotic Systems

Objective:  Enable manufacturers to assess and assure robot system assembly task performance by delivering a methodology and tools for characterizing and composing performance of perception, mobility, dexterity, and safety components.

Technical Idea:  The new technical idea is to compose assembly task-driven measures of component performance (perception, mobility, dexterity, and safety) into robot system autonomous assembly performance models. Performance measures will provide manufacturers and systems integrators with data for selecting the appropriate system components. The plan is to create:

• Framework: A decomposition of typical assembly operations into the constituent robotic subsystem capabilities that will drive the research plans for the perception, mobility, dexterity, and safety thrusts.
• Capability Models: A representational framework for modeling performance within each component area that will enable analyses and studies to increase understanding of performance of the overall system within industrial settings.
• Test Methods: A suite of test methods for perception, mobility, dexterity, and safety components of robots derived from an assembly operation taxonomy and requirements, along with a methodology and tools for composing the resulting measurements into robot system performance models.

Research Plan:  

• Framework: The research plan entails development of a performance modeling and analysis framework that utilizes the results from parallel efforts to define key component metrics and test methods for robotic assembly operations. Central to the research plan is a testbed integrating robot arms, mobile robot bases, hands and grippers, and a range of sensor systems. The reconfigurable testbed will support experiments in advanced perception, mobility, dexterity, and safety for robotic assembly. The testbed will be designed to support increasing levels of experimentation complexity in the assembly domain, starting with picking and placing parts for kit-building in static environments and ending, after 5 years, with fastening and joining operations in dynamic unstructured environments.
• Capability Models: The project will produce a robot systems capability model detailing a taxonomy of assembly tasks, decomposed into sub-tasks to define performance metrics and measurement methods for the perception, mobility, dexterity, and safety components of the robot system. This capability model will guide and focus the development of robot systems for use in assembly-centric manufacturing. Test method suites of increasing complexity will be developed:
  • measuring robot performance in identifying, localizing, and tracking parts and other entities in its surroundings and purposefully grasping parts. This test method suite will provide manufacturers and robot developers with the means of assessing and assuring robot performance in assembly tasks using general-purpose grippers and hands.
  • measuring robot performance in joining and fastening operations, including use of assembly tools. This test method suite will provide manufacturers and robot developers the ability to assess and assure a robot systems’ performance in assembly operations using tools.
  • measuring the performance of mobile robots and mobile manipulators in safely navigating unstructured and dynamic environments. Through this test suite, NIST will provide industry with the means of assessing and assuring the performance and safety of mobile robots and manipulators in support of assembly-centric operations.
• Test Methods: At the end of the project lifetime, the results will be consolidated into a general methodology and tool set for composing perception, mobility, dexterity, and safety performance into robot system autonomous assembly performance models. This will enable manufacturers to assess and assure robot system task performance in smart manufacturing assembly-centric applications.