Robots, Devices & Interfaces
Our Robots
We explore teleoperation in different scenarios and using various robots (these robots are currently available at ISR-Lisboa).
If you have interest in using our robots for MSc/PhD thesis or collaboration projects, contact us.
Raposa-NG
Type
Search and rescue UGV
Research purpose
Augment user interface in teleoperation scenarios to enhance operator’s Situation Awareness (SA).
Sensors
Laser: Hokuyo UTM-30LX
Stereo Camera: Point Grey Bumblebll2 BB2-03S2C
Depth Camera: Intel RealSense D435
IMU: Microstrain 3DM-GX2
RGB Camera: Generic USB Webcam
Functionalities
Tracked wheel robot
Front body can be tilted
Laser is on a gimbal to keep it leveled
Stereo camera on a pan and tilt gimbal
Controllable by either tether or wireless connection
PhD thesis
Jéssica Corujeira: Augmentation of Situation Awareness Through Multimodal Interfaces in Mobile Robot Teleoperation
Tested scenarios
Has been used in collaboration with Portuguese Search and Rescue corporations, such as GNR GIPS and RSBL, and also with the Portuguese Explosive Ordinance Disposal Squadron (FAP ERIEE)
Trident
Type
Underwater ROV
Research purpose
Development platform to study teleoperation in underwater scenarios, while creating an immersive underwater experience
Awards
Science Exploration Education Initiative, organized by National Geographic and OpenROV
Sensors
DDS-based communication
Off-the-shelf platform (no longer available, previously sold by OpenROV)
Currently installed payload: 360º 3D camera
PhD thesis
Rui Xavier: under development
MSc thesis
João Nascimento: Enhancement of Underwater Teleoperation using a Pseudo-Haptic Attitude Indicator
Research Grantee
Rute Luz: Creating Immersive Underwater Experience
KayJay
Type
Planetary Rover
Story behind the name
Tribute to Katherine Johnson and that also aligned with our group philosophy of promoting inclusion in research
Research purpose
Develop a platform to test our onboard algorithms and remote teleoperation
Build a deployable container (physical or virtual) that can be easily integrated in various ground rovers
Reuse a mobile platform with added modern computing and sensors
Sensors
Differential mobile platform (Pioneer based)
RGB-D camera
GNSS receiver
IMU
Integration with ROS1 and ROS2
Functionalities
Traction Detection algorithm
Onboard autonomous navigation (under development)
Research work developed with this robot
MEROP robotics team @ AMADEE analog missions – (link to amadee page)
MSc thesis
Rui Abrantes: Human-supervised Autonomous Navigation of Planetary Rovers in Rough Terrain
Gonçalo Coelho: Augmented Reality for Improved Teleoperation of Planetary Rovers with Increased Situation Awareness
Margarida Pereira: Towards Perception of Zero Latency in Teleoperation – a Predictive Digital Twin Interface for Remote Planetary Rovers
PhD thesis
Rute Luz: under development
AMADEE analog missions
Operator Control Unit
Our group also developed a custom Teleoperation Operator Control Unit, together with haptic devices that provide feedback to the operator and tackle challenges identified in various field experiments, such as attitude, traction, and traversability awareness.
This system is designed as an easily deployable and self-contained teleoperation console, integrating a GUI interface and haptic devices within a portable case. The case houses a laptop running the MEROP teleoperation software, the haptic devices responsible for attitude and traction feedback, and a joystick for robot control and teleoperation functionalities. Additionally, it includes a video camera and spare parts for the haptic devices, ensuring operational reliability in diverse field conditions.
Photo taken by Gonçalo Gouveia (Instituto Superior Técnico – MDN)
Haptic devices
Haptic Attitude Feedback Device
The Haptic Attitude Feedback Device enhances the operator’s attitude awareness and perception of the terrain texture, when teleoperating an uncrewed ground vehicle through rough/ unstructured environments (it uses upper limb proprioception for the attitude feedback).
Functionalities
Provides absolute attitude orientation (Roll and Pitch)
Provides rate of change of attitude (angular acceleration and velocity)
Sensitive enough to provide terrain texture
“Attitude Perception of an Unmanned Ground Vehicle Using an Attitude Haptic Feedback Device” J. Corujeira, J. L. Silva and R. Ventura, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China, 2018, pp. 356-363
Traction Glove
The Traction Glove allows the operator to feel vibration when the rover is stuck or sliding.
Traction Cylinder Device
Functionalities
Custom made rotation device (cylinder)
Operators can feel on the palm of the hand different rotation patterns when the rover loses traction (depending on the state of the rover)
When the rover is stuck or sliding, the cylinder rotates to warn the operator
Teleoperation Graphical User Interfaces (GUI)
Raposa-NG GUI
The GUI used with Raposa-NG robot can be visualised on a computer monitor or within a Head-Mounted Display (HMD).
Elements
Camera feeds that can be cycled through and camera main camera pan-tilt functionality, either through joystick or head rotation when wearing the HMD
Robot location in the map
Attitude Indicator (turned off when using the haptic attitude feedback device)
MEROP GUI for AMADEE-20
The GUI used within the AMADEE-20 mission can be visualised on a computer monitor.
Elements
Camera feeds that can be cycled through
Robot location in the map
Attitude Indicator (turned off when using the haptic attitude feedback device)
Information about communication latency and bandwidth when these go below a threshold, as well as a warning when there is a connection failure
MEROP GUI for AMADEE-24
The GUI used within the AMADEE-24 mission can also be visualised on a computer monitor.
Elements
Camera feeds that can be cycled through
Robot location in the map (the map also shows georeferenced user notes, points-of-interest, regions of interest, and Wi-Fi signal strength)
Icons represent information about Wi-Fi signal strength and battery state
There are two modes of operation: teleoperation mode and avatar mode (semi-autonomy)
