Thesis Work for Seeing Is Verifying: Simulation-in-the-Loop Development of AI-Generated Robot Programs with Local Vision-Language Models

At ABB, we help industries outrun - leaner and cleaner. Here, progress is an expectation - for you, your team, and the world. As a global market leader, we’ll give you what you need to make it happen. It won’t always be easy, growing takes grit. But at ABB, you’ll never run alone. Run what runs the world.

This role sits within ABB's Robotics business, a leading global robotics company. We're entering an exciting new chapter as we’ve announced the plan for SoftBank Group to acquire ABB Robotics. SoftBank is a globally recognized technology group and investor/operator focused on AI, robotics, and next-generation computing.  By joining us now, you’ll be part of a pioneering team shaping the future of robotics—working alongside world-class experts in a fast-moving, innovation-driven environment.

This Position reports to

R&D Team Lead

Your role and responsibilities

Large language models can generate RAPID robot programs, and RobotStudio gives AI agents access to its station, RAPID editor, virtual controller and simulation through a Model Context Protocol (MCP) server. Text feedback can detect syntax and controller errors, but may miss programs that run and do the wrong thing. This thesis investigates whether rendered simulation images and structured signals can help agents verify task execution and generate or improve programs in a generate–simulate–verify–improve loop. It focuses on small vision-language models running locally, so station data can remain on the customer’s computer.

Details

Period: January to June 2027

Number of credits: 30 ECTS/högskolepoäng (hp)

Number of students for this thesis work: 2

Location: Partly on site at ABB Robotics, Göteborg, Sweden, and partly at the university, as agreed.

In collaboration with Ericsson Research, students will use automatic filter tuning as an end-to-end challenge: a robot adjusts a filter’s tuning screws based on measurement data and filter-test outputs. The agent may generate the robot program from scratch or improve an existing one, using simulation feedback and test results to guide successive adjustments. This case evaluates the broader thesis approach; it does not replace its research focus on simulation-in-the-loop verification of AI-generated programs.

Students will review state-of-the-art methods and build a benchmark with a natural-language task, an initial program (empty or containing a planted error), and an automatic success check. The Ericsson filter-tuning case will serve as an end-to-end demonstrator.

They will develop verification signals (collision reports, tool paths, reachability, cycle time, object poses and rendered keyframes), implement text-, structured-data- and vision-based feedback, and compare local vision-language models with cloud models. Evaluation will measure task success, verifier precision and recall, iterations, latency and hardware needs; the filter case will also assess achieved tuning performance and test-and-adjust cycles. Deliverables include a demo, documented results and recommendations.

Qualifications for the role

You are pursuing a Master’s degree in Computer Science, Robotics, Artificial Intelligence/Machine Learning, Mechatronics, Electrical Engineering, Engineering Physics, or a related field.

Strong programming skills in C# and .NET; experience integrating with SDKs or APIs is an advantage.

Understanding of large language models and computer vision; experience with vision-language models, agent frameworks, or the Model Context Protocol is a plus.

Experience running models locally, for example with Ollama, llama.cpp, ONNX Runtime, or Foundry Local, is an advantage.

Interest in robotics simulation and offline programming; experience with RobotStudio, RAPID, or similar tools is a plus.

Interest in measurement data, signal processing, optimization, or control is an advantage; experience with experimental or iterative tuning is a plus.

Ability to design experiments, evaluate results, and communicate findings clearly in English; collaborative and able to work partly on site in Mölndal.

More about us

This Theses work is offered by ABB Robotics in collaboration with Ericsson and Göteborgs Tekniska College.

ABB Supervisor Contact, Pablo Beigveder Durante,

[email protected]

Recruiting Manager Göran Holmqvist, [email protected]

Positions are filled continuously. Please apply with your CV, academic transcripts, and a cover letter in English.

We look forward to receiving your application!

A Future Opportunity

Please note that this position is part of our talent pipeline and not an active job opening at this time. By applying, you express your interest in future career opportunities with ABB.

We value people from different backgrounds. Could this be your story? Apply today or visit www.abb.com to learn more about us and see the impact of our work across the globe.