Problem Description

Robotic assembly of modular objects remains a central challenge, particularly with respect to extracting an assembly sequence from visual observations and modelling the physical interactions required to reliably fit complex geometries together. Existing approaches remain inflexible and difficult to adapt to new assembly tasks or object configurations. Assembly failures remain one of the major challenges, as recovering from unexpected situations and selecting an appropriate next action is often difficult.

This project is motivated by the need for structured and programmable assembly strategies that explicitly account for assembly order, object geometry, interaction mechanics, and failure modes, rather than relying solely on end-to-end learning or black-box AI approaches. Recent end-to-end robot learning models, such as Pi0, provide a promising basis for robotic control but generally lack formal guarantees regarding robustness, interpretability, and failure recovery. The overall objective is therefore to investigate hybrid approaches that combine AI-based robot control with formal planning and verification methods.

Using simple yet representative objects (e.g., LEGO-like bricks), the project explores how robots can be instructed to assemble predefined object stacks with minimal assembly failures.

Project Definition

Goal: Design, implement, and evaluate structured assembly methodologies for modular objects using a dual-arm robotic setup.

Objectives:

• Design and evaluate different assembly pipelines using systematic methods (e.g. alternative concepts, execution schedules, and parallel versus sequential task execution).
• Develop innovative hardware solutions or manipulation strategies for gripping and stacking objects.
• Depending on the project scope, students may first develop and validate their approaches in simulation before transferring them to real hardware.
• Students will work within a team throughout the project to define an experimental setup to demonstrate at the end of the project.

Implementation:

• Research and design structured representations of assembly sequences (Precedence Graph, Petri Nets, Behaviour Trees, Task Graph, etc.).
• Design and compare different strategies for picking, aligning, inserting, and stacking objects, considering factors such as insertion difficulty, compliance, and positional uncertainty.
• Use an RGB camera as sensor input as part of an active feedback loop during insertion.
• Collect training data for assembly using a lead-follow teleoperation setup and train AI models using learning-from-demonstration (LfD).
• Model object interactions (e.g. fit versus no-fit using geometric constraint modelling and learning-based methods) and evaluate or optimise gripper designs.
• Program and evaluate assembly strategies in both simulated and real environments.
• Optional: Use of microcontrollers and sensors for additional feedback (e.g. force).

Outcomes: Depending on the selected project scope, assembly strategies will either be implemented and evaluated primarily in simulation (Isaac Sim or PyBullet) before being transferred to a real dual-arm robotic platform (SO100/SO101 or equivalent), or implemented directly on the physical system. Deliverables:

• Documented assembly strategies with experimental results: successes, failure modes, and recovery strategies.
• Comparative evaluation of at least two assembly strategies.
• Well-documented source code together with usage instructions.
• Final presentation demonstrating the effectiveness and limitations of the proposed approaches.

Learning Outcome

After completing the project, students will be able to:

• Explain the challenges associated with autonomous robotic assembly of modular objects.
• Design structured and programmable assembly strategies for modular objects.
• Apply mechanical reasoning (e.g., forces, tolerances, and compliance) when developing robotic assembly software.
• Implement and evaluate robotic assembly algorithms, preferably in Python, in both simulated and real environments.
• Describe the principles of end-to-end robot learning models and assess their advantages and limitations within structured robotic assembly.

Participation Requirements

• Basic to intermediate Python programming skills
• Willingness to work with simulation environments and robotic APIs
• Basic familiarity with 3D models and coordinate systems is beneficial
• Interest in physical systems, experimentation, and systematic problem-solving
• Prior experience with machine learning is not required, but students should be open to evaluating learning-from-demonstration methods as one possible tool among others.

External Partner

Innovation Hub: The project is supported by the Innovation Hub’s Modellfabrik facilities, which provide the development environment, robotic platforms, and infrastructure for implementing and evaluating the proposed robotic assembly strategies.