Ned User Manual Ned User Manual
Ned User Manual Ned User Manual
Ned User Manual Ned User Manual
1)
Ned
Robot
Ned is a 6-axis collaborative robot arm designed for Education and Research.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Ned is designed to reproduce all the movements required in the most advanced uses in industry 4.0, with
a precision of 0.5mm and a repeatability of 0.5mm.
Ned’s aluminum structure makes it exemplary robust, allowing it to accomplish with uidity the
movements required for your robotics projects.
T h i s cobot takes advantage of the capacities of the Raspberry Pi 4, with a 64-bit ARM V8 high
performance processor, 2GB of RAM and an improved connectivity.
Ned is a collaborative robot based on Ubuntu 18.04 and ROS (Robot Operating System) Melodic, an
open-source solution created for robotics. Through ROS, Ned has multiple libraries allowing you to
conceive many programs, from the most simple to the most complex ones, responding then, in a exible
way, to your needs.
Mounting a tool on Ned is made easy with our brand new EasyConnect system. Simply plug your tool,
connect its cable, and it is ready to use.
Ned’s package includes a Custom Gripper. Its standard jaws can manipulate small objects, and you can 3D
print your own custom jaws or buy our set of jaws (index.html#overview-standard-jaws) that contains the
precision jaws, the flat ones and the XL ones.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
The Large Gripper (index.html#large-gripper) which has the ability to grab larger objects while keeping
the ability to close entirely;
The Adaptive Gripper (index.html#adaptive-gripper) which allow Ned to grab non-standard objects with
complex shapes (eg. an egg);
The Vacuum Pump (index.html#vacuum-pump) to grab objects with plane and non-porous surfaces;
The Electromagnet (index.html#electromagnet) that is useful to manipulate metallic objects, from one
to many (eg. screws, bolts…).
Ned’s Ecosystem is designed to let you reproduce advanced use cases of industry 4.0.
Vision Set
The Vision Set gives Ned the ability to see its environment and detect the objects to interact with
based on their characteristics (shape and/or color). This set includes :
A camera that is mountable on Ned’s wrist to give it a dynamic eld of view depending on Ned’s
position;
A workspace which is designed around a specific repositionnable material;
Supports for the workspace and the robot, to build a rigid environment for the vision;
6 objects (3 rounds, 3 squares) of di erent colors to use our built-in recognition system based on colors
and shapes. The squares are designed to be able to act as containers for the circles. Easily create an
industry 4.0 pick & pack application with no particular knowledge of programming;
Vision jaws for the Custom Gripper, that are specifically designed for the provided objects;
A calibration tip to set your workspace up.
Conveyor Belt
The Conveyor Belt is the key add-on to prototype industry 4.0 production lines.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
The Standard version includes the Conveyor Belt, its control box for autonomous use, and an IR sensor to
detect the presence of an object on it.
With Ned’s ecosystem, you have the optimal solution to discover, learn and test industry 4.0
processes.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Safety notice
Warning
You must read, understand, and follow all safety information in this manual.
It is important that the safety instruction on this manual and the electrical and the mechanical instructions
are followed. The robot speci cations should be respected to avoid all damage to the machine or to the
user.
Additional content
You can find additional content by clicking on the link here (https://docs.niryo.com/).
When you order Ned, you receive a box with the following items:
1x Ned Robot
1x Custom Gripper
1x Power cable
1x Universal travel plug adapter
1x SD card reader
1x Allen Key set
Unpacking
Operating conditions
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Parameters Value
This gure shows Ned in its shipping position, which is a recommended transport position to avoid Ned’s
damages.
Safety instructions
This section speci es all dangers that can arise when doing the work described in the User Manual. Each
danger consists of:
A caption specifying the danger level (DANGER, WARNING, or CAUTION) and the type of danger;
A brief description of the danger;
Instruction about how to eliminate danger.
Designation Description
Warns that an accident will occur if the instructions are not followed, resulting in an
DANGER
injury.
Warns that an accident will occur if the instructions are not followed, resulting in a
WARNING
possible injury or product damage.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Safety Overview
This section includes information on general safety risks to be considered when performing installation
and service work on Ned.
These safety instructions must be read and followed by any person who deals with Ned.
The installation and the use of external components is possible (such as Arduino, external sensors,
devices). But the user must respect the electrical speci cations and connections mentioned in the User
Manual.
Niryo is not liable for damages or injuries caused by unauthorized modifications to the robot system or the
misuse of the robot.
Complete robot
Danger
1. Before installing, using, or programming Ned, read the product specifications and Manual;
2. Make sure to install Ned and all electrical equipment according to the speci cations and the
warnings from chapters Installation and commissioning (index.html#installation-and-
commissioning) and Technical specifications (index.html#technical-specifications);
3. Children are not allowed to operate Ned without adult supervision to prevent any possible injury or
improper misuse of Ned;
4. Make sure that changing robot software will not cause hazards, damages to the Ned’s system or
injuries.
Warning
1. Motors and gearboxes are hot after running Ned for a long time. Avoid touching especially with a
higher environment temperature;
2. Removing parts may result in the collapse of Ned. Take the necessary measures to ensure that Ned
does not collapse as parts are removed;
3. Make sure to remove the power cable during assembly, wiring, or repair. Doing so will help to
prevent equipment damage or accidental short-circuit;
4. The cable packages may be sensitive to mechanical damage;
5. Before using Ned, make sure to firmly fix it on a stable surface to avoid collapse or the fall of Ned;
6. Do not use Ned outside;
7. Do not put Ned in a humid environment or near water;
8. Do not install or operate Ned in dangerous environments (e.g., in the presence a strong magnetic
eld, dangerous gases, re or ammables) to avoid dangers which may occur due to external
conditions during operation;
9. Respect the technical speci cations to avoid damaging the motors. Stepper motors and servo
motors may be damaged if excessive force is used;
10. Make sure external change on the motors parameters or the robot software does not cause
damage to Ned.
11. Do not open Ned base when the power cable is connected. Do not touch the electronic board, they
may be hot.
Niryo is not liable for damages or injuries caused by unauthorized modi cations to the robot system or
the misuse of the robot.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Caution
1. Do not turn o Ned during a sequence or a motion unless absolutely necessary, this may reduce
Ned operating age;
2. Try to install Ned arm on a stable surface with enough space to avoid any shock or vibration.
Safety actions
Fire extinguishing
Note
Use a CARBON DIOXIDE (CO2) extinguisher in the event of a fire in Ned’s system.
Manually stopping or overriding the arm: if needed, Ned arm can be stopped manually. This is possible
since Ned arm is light, the arm force is limited, and the drivetrain power is limited.
To prevent unnecessary damage and wear of Ned arm, it is recommended to use the normal stopping
functions of Ned (by the software or the top button).
Note
Ned is not equipped with brakes since its weight and design do not require a holding brake.
Technical specifications
This section describes the technical specifications of Ned.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Parameters Value
Weight 6,5 kg
Payload 300 g
Reach 440 mm
Repeatability 0.5 mm
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Parameters Value
Footprint 200x200 mm
Mounting Table
2 x USB2.0
I/O interface 2 x USB3.0
1 x ETHERNET GIGABIT
Aluminum
Materials
Plastic PLA
Temperature 5 - 45°C
Niryo Studio
PyNiryo
Programming Environment C++
Python
ROS
Tools Overview
3 x Grippers
1 x Electromagnet
1 x Vacuum Pump
Specifications
Custom Gripper
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_STANDARD_SCHEMA_1.jpg)
(_images/GRIPPER_MORS_STANDARD.png)
Overview - XL Jaws
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_XL_SCHEMA_1.jpg)
(_images/GRIPPER_XL_SCHEMA_2.jpg)
(_images/GRIPPER_MORS_XL.png)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_FLAT_SCHEMA_1.jpg)
(_images/GRIPPER_FLAT_SCHEMA_2.jpg)
(_images/GRIPPER_MORS_FLAT.png)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_PRECISION_SCHEMA_1.jpg)
(_images/GRIPPER_MORS_PRECISION.png)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_VISION_SCHEMA_1.jpg)
(_images/GRIPPER_VISION_SCHEMA_2.jpg)
(_images/GRIPPER_MORS_VISION.png)
Parameter Description
Standard Jaws: 85 mm
XL Jaws: 85 mm
Flat Jaws: 90 mm
Picking distance from end effector Precision Jaws: 85 mm
base Vision Jaws: 85 mm
Large Gripper
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_LARGE_SCHEMA_1.jpg)
(_images/GRIPPER_LARGE_SCHEMA_2.jpg)
(_images/GRIPPER_LARGE.png)
Parameter Description
Motor
XL320 Servo Motor
Weight 90 g
Adaptive Gripper
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_ADAPTATIVE_SCHEMA_1.jpg)
(_images/GRIPPER_ADAPTATIVE_SCHEMA_2.jpg)
(_images/GRIPPER_ADAPTATIF.png)
Parameter Description
Weight 110 g
Electromagnet
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_ELECTROMAGNET_SCHEMA_1.jpg)
(_images/AIMANT.png)
Parameter Description
Picking surface 20 mm
Digital
Control interface output
Weight 42 g
Power Supply 5V
Vacuum Pump
(_images/GRIPPER_VACUUM_PUMP_SCHEMA_1.jpg)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
(_images/GRIPPER_VACUUM_PUMP_SCHEMA_2.jpg)
(_images/POMPE.png)
Parameter Description
Weight 10 g
Payload 300 g
Mechanical interface
Mechanical Overview
This chapter introduces the mechanical and the electrical interfaces of Ned.
Ned is a 6-axis collaborative robotic arm. It consists mainly of six robot joints of aluminum with plastic
covers. Ned consists of 7 parts :
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Robot workspace
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Tool mounting
Each tool has the same mechanical connector interface. This option allows a quick-change option of the
tool and good stability. See the picture below.
To mount the Vision Set, all you need to do is attach the camera to Ned’s wrist, set up and x your
workspace to Ned’s structure to have a stable environment and put the vision jaws on the Custom Gripper.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Robot calibration
Ned is delivered without a mechanical calibration. Ned can not execute any motion properly without auto-
calibration because of a mismatch between the origin of each motor and its corresponding origin stored in
the controller.
Note
Make sure to process the calibration in order to align the origins of every joint.
Hint
Electrical interface
Electrical architecture:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Danger
Make sure that all the equipment and the wires are kept dry. If water enters the equipment,
disconnect the power;
Carefully follow the safety instructions of the next section to avoid damaging your robot.
Warning
Do not unplug or power o the robot during use, you may damage the Raspberry Pi 4 inside the robot,
or the micro SD card inside the Raspberry Pi 4
Panel connectors
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
3 Ethernet port
4 Panel LED
5 2 x NiryoStepper
connector
6 XL320 connector
8 2 x SW : 12 V digital output
9 2 x GPIO panel
10 Top button
Power supply
Parameters value
Frequency 50/60 Hz
Output 11.1 V – 6 A
Warning
1. When Ned is turned on, do not disconnect the power supply of Ned. You may damage the
Raspberry Pi 4 inside Ned, or the micro-SD card.
2. If you modify the original power supply of the robot, you should respect the same specifications.
3. Before plugging the power connector and powering on Ned, first make sure that:
Ned is on a flat and stable surface;
The power switch is turned off (position 0);
Ned has enough space to move without hitting something or someone.
Power Switch
The power switch turns Ned on and off. It enables the power supply.
Always switch it off after shutting down Ned and turn it on to power on Ned.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Warning
Before plugging or unplugging the power supply, make sure to turn off the switch.
Do not keep the switch on after shutting down Ned, otherwise , the power will be present on the
motors and on the shield
Digital inputs/outputs
Ned has 6 inputs/outputs that can be con gured either as digital input or digital output with Niryo
Studio (see How to program the digital I/O on the Software Manual
(https://docs.niryo.com/product/niryo-studio/index.html)).
Digital Input
[1A-2A-1B-2B-1C-2B] Voltage 0 5 V
Digital output
[1A-2A-1B-2B-1C-2B] Voltage 0 5 V
This example shows how to connect a load in order to be controlled from a digital output (see below).
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Switch connections
Digital Output
Voltage 0 11.6 V
SW1, SW2
Current 0 0.5 A
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Ethernet
Modbus TCP/IP;
PyNiryo;
Connect Ned to your local area network.
USB Port
The USB can be used to connect the keyboard, mouse, or Vision Set.
Find more information about Vision Set on the Vision Set Manual (https://docs.niryo.com/product/vision-
set/index.html).
NiryoStepper connectors
NiryoStepper connectors interface can be used to drive a NiryoStepper motor. It is based on CAN Bus
communication protocol.
DXL connection
XL320 Connector
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
XL430 Connector
Not used.
Panel LED
Ned’s LED panel has several colors which represent di erent modes and error status. Refer to the
following table:
Alternating
between Motor error on hotspot mode. Check “HARDWARE STATUS” and
Blue/Red logs to troubleshoot the error.
Alternating
white – green
The robot is paused on connected mode N/A
Alternating
white – blue The robot is paused on hotspot mode N/A
Top button
The top button of Ned has multiple functionalities depending on how long you press the button and the
number of presses.
Note
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Note
All functionalities are available for use only after the robot has been fully started (LED is green or blue).
Program is executing
Program is paused
Note
When you pause the robot’s program with the top button, you should either press the button to
continue or press twice to stop it.
Hint
You can test your robot hardware for the rst time without using any software tool, you can follow the
steps below:
Tool wiring
There is a cable on the tool end of Ned with 3 pins. This connector can drive Custom Gripper, Large
Gripper and Adaptive Gripper.
Note
Make sure to respect color order when you are wiring your gripper.
Warning
Not respecting the wiring instructions may damage your gripper or Ned.
Vacuum pump:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned User Manual (v3.1.1)
Plug the suction cup to Ned’s hand and connect the motor wire to theXL320 Connector placed at the back
of Ned.
Electromagnet:
Plug the electromagnet to Ned’s hand and connect the cable on the GPIO1 or GPIO 2 connector on the
panel connector.
Note
Information about programming and using the tool are available on the Software Manual
(https://docs.niryo.com/product/niryo-studio/index.html)
Advanced Programming
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Niryo Studio is a graphical HMI. It allows a fast and direct control of Ned with an external computer.
Its purpose is to provide users with a complete and simple interface for Ned motion, programming
environments and current status of Ned.
Users must read this manual in order to fully understand all Ned and Niryo Studio functionalities.
Note
If you encounter software problems or if you want to access latest updates for your Ned, please give an
internet access to your robot (Ethernet or Wi-Fi) to be able to update it through NiryoStudio.
Instructions here (index.html#ned-software-update)
Changelog
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Features
Direct control
Jog Pose.
Control the pose of the robot in position and orientation. The arrows show the direction of the
translation and rotation command to apply to the robot’s pose. You can adjust the command speed.
Jog Joints.
Control the joints of the robot in orientation. The arrows show the direction of the rotation command
to apply to a joint. You can adjust the command speed.
Frames visualization.
Visualize the TCP and the robot base frames. You can select which frames you want to display.
Programming
The function will try to run what is inside of it. A number of attempts can be set to repeat the operation
if the robot failed to do it. You can set the function to continue or stop the program if the attempts do
not succeed.
Linear function makes the robot move linearly. Try Linear function makes the robot move linearly but
if no linear trajectory can be computed, it will do a Standard movement.
Linear function makes the robot do a translation movement linearly. Try linear makes the robot do a
translation movement linearly but if no linear trajectory can be computed, it will do a Standard
movement.
This function builds a trajectory by giving it waypoints. The robot will try to reach way points in the
given order. The distance smoothing parameter de nes the distance from which the trajectory will be
smoothed around each way point.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
This function changes the TCP (Tool Center Point) coordinates of the robot. TCP coordinates are used
with Pose-type movements.
This function activates or deactivates TCP coordinates. When activated, TCP coordinates are de ned by
the user. When deactivated, the TCP coordinates are equal to the final point of the robot’s arm.
Settings
Angles unit selection between degree and radian from the top bar.
Change TCP (Tool Center Point) coordinates. TCP coordinates are used as references for Pose
movements. Activate TCP allows the modi cation of the TCP coordinates. Scan will check if a tool is
attached to the arm and will update the TCP coordinates according to the connected tool. Reset will
reset the TCP coordinates to the actual tool (0 if no tool). You can manually modify the TCP coordinates
by writing your orientation and translation. Click Apply to apply the new coordinates.
Saved positions
Play position
Move to the desired position from position list by clicking on thePlay button.
Vision set
Camera’s settings
Improvements
Programming
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Overview
Operating interface
1. The top toolbar allows you to change the angle units, the language, and connect to Ned
2. 3D Ned visualization or camera stream video
3. State section and mode selection
4. The left menu allows you to switch between the different sections of the application
5. The main window of the application
Navigation interface
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
The listed icons from top to bottom on the left menu are:
Robot connection
There are three ways to connect Ned to Niryo Studio:
If the LED on the back panel is blue, that means you can connect to Ned via its own Wi-Fi. In this case Ned
IP address is 10.10.10.10.
Ned has its own Wi-Fi network, which starts with “NiryoRobot” followed by a series of numbers and letters.
To connect, follow the steps below:
Connect to the Wi-Fi access point using your laptop with the password “niryorobot”
Click the arrow near “Not connected” to access Ned connection
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
To disconnect from Ned, open the connection panel again and click on “Disconnect”
If Ned is connected to the Wi-Fi network (see the robot settings (index.html#ned-settings) section), the LED
on the back panel is green.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
If you do not know your robot IP address, open the connection panel and click on the “Wi-Fi + Search”
button. This will search for connected robots inside the current Wi-Fi network. Choose the correct IP
address and click “Connect Niryo Robot”,
If you gave a custom name to Ned, select it by its name,
If you already know your Ned’s IP address, click the arrow, enter your IP address and connect to the
robot.
You can also use an Ethernet cable to physically connect Ned with your computer.
Connect Ned to your local network (computer or switch) using the LAN port,
Modify the selected IP address on the connection panel to “Niryo Robot Ethernet (default IP)”,
Connect to Ned.
Click the arrow near “Not connected” to access the robot connection,
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Robots specificities
Calibration
At the start-up
If the steppers motors lose their offset positions and need to re-calibrate
If the 3D view and Ned physical movements are not similar
In most of the cases, Niryo Studio will alert you by showing “CALIBRATION NEEDED” on the top right.
Note
In the case of “CALIBRATION NEEDED”, you can’t perform any command on Ned without calibrating it.
To perform a calibration, click on the button “Request new calibration” in the “ROBOT CALIBRATION”
section.
After clicking, a “Calibration Needed” warning will appear on top of the 3D view.
When the “Calibration Needed” warning appears, you can click on it to perform an auto calibration or you
can click on the “Auto calibration” button in the “ROBOT CALIBRATION” section.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
By choosing auto-calibration, Ned will execute an automatic sequence by moving its axis that needs to be
calibrated until they reach their maximum position, so a correct o set can be applied to each motor. This
sequence should take about 20-30 seconds. When the calibration is over, Ned will come back to a resting
position.
Caution
Before launching the auto-calibration, make sure that there is no obstacle around Ned. You should put
Ned in a proper position before performing the calibration. The position in the picture below is
recommended. This will avoid damaging Ned, collision, and any hazard motion.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Note
If the calibration is not successful, the “Calibration Needed” warning will still be displayed. Try the
calibration again and if this new calibration does not succeed, it can mean that your robot has a
mechanical/hardware issue.
Hint
You can check if the calibration has succeeded by moving the robot and examine the robot motion in
the 3D view.
Learning mode
Niryo robots have two operating modes: the manual mode (Learning mode) and the automatic mode. The
current mode can be determined visually in Niryo Studio, via the Learning Mode button position.
Manual mode/learning mode: when Ned is in learning mode, the toggle on Niryo Studio is on.
In this mode, the user can guide Ned by hand. In this case, there is no torque applied on the joints
and motors are in relaxed mode. This mode is recommended when no program is running.
Automatic Mode (Learning Mode is deactivated): The Learning Mode does not work under
Automatic Mode.
It is not possible to move Ned by hand. This mode is active when a program is running or when Ned
is performing a motion.
Caution
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Do not try to override the motor’s torque when learning mode is disabled: you could damage the robot
and the motors. You can force to eliminate the risk or you can unplug the power supply ONLY in case
of emergency (Ned blocked or application freezed)
Hint
3D View
You can see here the 3D representation of the robot. You can display TCP (Tool Center Point) and Base link
frames.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Direct control
Arm command
On the ARM COMMAND section, you can move Ned’s arm directly either by moving the robot joint
individually or by translating / rotating Ned pose.
You will nd 4 main subsections that this documention will cover below, but we will start with the main
buttons available.
Home Position
Click the “BACK TO HOME” button in order to move Ned to its home position.
If you already saved a position, you can select it via the “Select Saved Position” scrolling menu. Once
selected, it will upload joints value if your are in joints view, or pose if you are in pose view.
Then you can click on “MOVE JOINTS” or “MOVE POSE” to move the robot according to the selected
joints/pose.
Stop command
Joints
This method allows you to control each joint individually from its minimum to its maximum value (see the
Robots specificities (index.html#document-source/specification) section).
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
By clicking on the “SET TO CURRENT” button, you will refresh the interface to the actual joints values.
Modify the values of the joints either by writing the desired joint value or by moving the cursor,
Click on the “MOVE JOINTS” button to execute the move command.
Once the command is done (success or not), you will get a notification at the bottom of the screen.
Important
At any time, you can cancel the current command execution by clicking on the “STOP COMMAND”
button.
Pose
This panel gives you the possibility to move the pose of Ned’s TCP (Tool Center Point).
You can edit the values by clicking on them then use the “MOVE POSE” button.
JOG TCP
This panel will help you move Ned in a more visual way.
The TCP (Tool Center Point) of the robot is the center of the end effector.
Holding a translational arrow will make the robot follow its direction,
Holding a rotational arrow, will make the robot rotate around the TCP.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
You can modify the speed of the jogs. A low value will give you more precision while a higher value will
bring you faster to the position.
Hint
The performance of the jog TCP can be impacted by your network connection. We advise you to use an
Ethernet connection to get the optimal communication and the best experience with the jog TCP
feature.
JOG JOINTS
You can also switch to jog Joints on the ARM COMMAND panel.
The jog Joints allows you to control each joints of the robot by holding down one arrow.
You can also change the movement speed. A low value will give you more precision while a higher value
will bring you faster to the position.
Hint
The performance of the jog joints can be impacted by your network connection. We advise you to use
an Ethernet connection to get the optimal communication and the best experience with the jog Joints
feature.
Tool command
The “SCAN” button will detect and add a tool if there is already a connected tool (with motor).
If you have a gripper:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
You can set the opening and closing speed by writing the value or by dragging the cursor (from
100 and 1000),
You can send open and close commands to the gripper by clicking “OPEN” and “CLOSE” buttons.
If you have a Vacuum Pump, the “ON” button will pull the air with the Vacuum Pump and the “OFF”
button will pull the air out of the pump.
You can monitor and set I/O signals from or to Ned. The “DIGITAL I/O PANEL” displays the current state of
the I/O “SW1”, “SW2”, “GPIO1”, and “GPIO2” connectors on the back of Ned.
Note
“SW1” and “SW2” connectors can be used to plug a fan, a motor, etc.
For “GPIO1” and “GPIO2”, you have, from left to right: GND, 5V, and 3 digital pins. The digital pins can be
used to communicate with another device (ex: an Arduino board. Check out this tutorial
(https://docs.niryo.com/applications/ned/source/tutorials/control_ned_arduino.html) to learn more
about that).
Switches
You can change the “SW1” and “SW2” states to “HIGH” or “LOW” by clicking the corresponding button.
GPIO
Warning
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Warning
You can change the mode (INPUT or OUTPUT) and the state (LOW or HIGH) for each pin, by clicking on the
corresponding mode and state on the “DIGITAL I/O PANEL”.
When the pin is set as an INPUT, you can read the state in this panel.
In OUTPUT mode, you can set the OUTPUT state of the pin (LOW or HIGH) by clicking on the
corresponding button.
Note
By default, all the GPIO pins are in INPUT mode and HIGH state.
Electromagnet control
The Electromagnet can be set up and controlled on the “DIGITAL I/O PANEL”.
You can connect the electromagnet on the GPIO1 or GPIO2 at the back of Ned (seeHardware Manual
(https://docs.niryo.com/product/ned/source/hardware/electrical_interface.html#electromagnet)).
Hint
If you are using the connector provided with the electromagnet, select 1A pin if you connect to the
GPIO1 and 2A if you connect to the GPIO2.
Programming
Project panel
Click the project button on the left menu to start creating and editing Blocky programs.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
In the figure below, the top is the project Toolbar, and the left is the Blockly menu.
1. Create a new project: click the “NEW” button to create a new project.
2. Save program: click the “SAVE” button, to save a program. You can save a program on your computer or
directly on Ned as an XML file.
3. Load project: click the “LOAD” button to import a Blockly program either from computer or from Ned.
The file is a xml.
4. Undo. You can also use CTRL + Z.
5. Redo. You can also use CTRL + MAJ + Z.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
6. Play the program displayed on the workspace. Once the program is done (success or not), you will get a
notification on the bottom of the screen.
7. Stop the current program execution.
8. Add a position block. This will open a pop-up:
This is the most useful way to add a position command block to your workspace. You can choose between
a “Move Joint” block and a “Move Pose” block, and use either the current robot state, or a previously saved
position.
9. Select the save mode when using the top button on Ned.
Hint
While the Niryo Block interface is open, you can add a “Move joint” or “Move pose” block with current
joints values and pose values just by pressing and releasing the top button of the robot.
10. Help.
11. Clicking on this icon will center the workspace on your blocks.
12. Workspace zoom control.
13. Workspace zoom control.
14. To delete a block, simply drag it and drop it onto the trash. You can also select it and press the delete
key on your keyboard.
15. The “Niryo Robot” functions in the block library cover almost all of the possible commands you can
execute on Ned:
Arm commands
Movement commands
Tool commands
I/O setup and control
Vision commands
Conveyor control
Note
The “NIRYO BLOCKS” panel contains everything you need to create complete programs for Ned. This is
probably the panel on which you will spend most of your time.
Note
You cannot execute a new sequence while another sequence is running. If you want to start a new
sequence, you will have to stop the previous one.
Hint
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Niryo blocks along with the learning mode is the perfect combination to easily create programs in no
time. Activate the learning mode, then for each position you want, add a position block corresponding
to the current state, and that’s it.
Saved positions
In the “SAVED POSITION” panel, you can find all the saved positions.
Name
Description
Values: list of joints values in radian and the position and orientation of the TCP (Tool Center Point).
You can:
Program settings
In the “Program Settings”, you can set an autorunning program, upload a Python program and display a
program.
Program Autorun
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
The “PROGRAM AUTORUN” functionality allows you to execute a set of sequences, directly by pressing the
top button of the robot.
Hint
This is very useful when you use Ned in an exhibition event or for a demo. You will just need to power
on Ned, wait for the LED to turn blue or green, and then press the button to run a given set of
programs (that you previously programmed).
a. ONE SHOT: when you press and release the top button, the given set of sequences will be run once.
After that, Ned will go back to a “resting position” and activate the “Learning Mode”. You can start
another run by pressing the button again. If you press the button while the set of sequences is
running, it will stop the execution.
b. LOOP: when you press and release the top button, the selected program will run in a loop, forever.
You can press the button again to stop the execution.
Upload program
Programs list
You can pick a previously saved program from the select box. Click on a sequence to see all the properties
and actions available for this program.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
1. Play the selected program (same as the “PLAY” button in Niryo blocks).
2. Stop the current program execution (same as the “STOP” button in Niryo blocks).
3. Open the program in Niryo blocks. This will make you switch to the “Niryo blocks” panel, and the
program will be added to the current workspace.
Hint
This functionality is very useful when you want to duplicate and create a new program from an existing
one.
4. Edit the program. You can modify the name, the description, and the Blockly XML.
5. Delete the program.
Addons
Conveyor Belt
Vision Set
Discover how to use the vision in the Vision Set documentation ( https://docs.niryo.com/product/vision-
set/source/setup.html#software).
Ned settings
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Robot Name
This name will be used when Ned is in hotspot mode to create the Wi-Fi network name.
In Niryo Studio, when Ned is in connected mode, you will see that Ned is represented both by its IP
address and its custom name.
You can set a custom name under “ROBOT SETTINGS”. After choosing the name, you need to reboot Ned.
The TCP (Tool Center Point) of the robot corresponds to the working point of the tool. From here, you can
change the frame of the TCP.
When the TCP is enabled, every position move is set corresponding to the TCP frame.
When you plug a tool and if the TCP is enabled, the TCP frame will automatically update to the
corresponding tool TCP.
If Ned is already connected to a Wi-Fi network and you want to change your Wi-Fi network:
Note
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Note
If you are not connected to Ned, you will not be able to change its Wi-Fi network.
If successfully connected: the robot will reboot itself, the LED will become red then green. Connect
to the same wi network and then connect to Ned (Help: Using Ned on your Wi-Fi network
(index.html#using-ned-on-your-wi-fi-network).)
If failed to connect to the SSID: the LED will remain blue (the LED didn’t change after ~30 seconds).
Verify your password and try to connect again.
If the SSID can’t be found currently by the Raspberry Pi: you won’t be disconnected from Niryo
Studio, but an error noti cation will be shown. You can retry within a few seconds (the wi- card
may be disturbed by other wi-fi devices) or move your Raspberry Pi closer to the Wi-fi router.
Warning
If you changed Ned network con guration from Wi- to hotspot, troubles can occur when reconnecting
via Niryo Studio, please reboot your robot in this case.
Language selection
The default language of Niryo Studio is English but you can change the current language with the language
button on the top toolbar.
Note
The default angle unit in robotics is radian. But you can change to degree with the corresponding button in
the top toolbar.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Raspberry PI settings
ROS logs:
The available disk size on the Raspberry Pi. If the number is displayed in orange or red, it means there
is no space enough on the micro-SD card.
The size of the stored ROS logs: after some times, they can take a lot of space.
You can choose to remove all previous ROS logs when Ned boots. This is very useful for a normal usage
of Ned. Basically, you do not need all the ROS logs that are produced and stored, unless you are
developing or debugging a speci c point (in this case you might want to switch the option to “NO” and
then turn it back to “YES” when you’re done).
Caution
If you never purge ROS logs (on boot or manually), the micro-SD card will eventually become full and
might not work as expected. So only change the option to “NO” if you know what you are doing. In any
case, if you do not really understand this functionality, always leave the option to “YES”.
You can also remove all ROS logs immediately by clicking on the “Purge ROS logs” button.
Shutdown Ned.
You can shut down Ned with Niryo Studio. The “SHUTDOWN NIRYO ROBOT” will shut down the
Raspberry Pi (same behaviour as if you press the physical top button for 3 seconds). Be sure to wait for
the LED to turn red before completely powering off Ned.
You can reboot Ned by clicking “Reboot”.
Note
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Note
If you reboot Ned, do not power it o . The LED will turn to purple, then red, then will become blue or
green again.
Software versions
You can check that Ned and relative softwares are up to date by comparing the current and latest version
for each of them.
Note
Note
If an incompatible software version is detected, if the version you have is not supported anymore or an
update is available, you will get a pop-up noti cation when you connect to Ned. Please follow the
displayed instructions.
a. Message displayed when Ned is not connected b. An update is available to update Ned
to internet. software.
Ned software inside the robot can be updated through Niryo Studio in only a few steps.
First, your Ned must have an access to internet, so you have to connect it to an ethernet (you have to edit
via ssh the network con guration of Ned) or a Wi-fi (index.html#using-ned-on-your-wi-fi-network) network
that provides internet.
When your robot is properly con gured, at the next boot of the robot, it will check if an update is available.
A pop-up will then be displayed at each following connection from Niryo Studio with Ned to inform you
that an update is available for your robot.
You then only have to click on the update button of the pop-up or the update button in thesoftware
versions panel.
Warning
Ned will reboot itself at the end of the process (in case of update success) so it’s recommended not to
do any other operation with Ned before starting an update.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Once the update has started, a pop-up will indicate that Ned is updating and that it’s downloading /
applying the update.
Note
Update time is heavily impacted by the internet speed of your connection so if you connect to a wi-
network, be sure that the raspberry is not too far away from the wi-fi modem.
Once the update has successfully ended, another pop-up will appear (closing the previous one) telling that
your robot will reboot and you may be disconnected from Niryo Studio. Now you just have to wait for the
robot to initialize properly (the red LED turning into green or blue) before reconnecting through Niryo
Studio, the update process is done.
Warning
In case the update failed, it could be a faulty connection, try to do the update once more and if you still
encounter a problem, please contact us at [email protected] (mailto:contact%40niryo.com).
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Logs
At the top of the Robot Status page, you can find 2 sections:
Hint
You can save Ned logs or Niryo Studio logs on a text file.
You can send this file to our technical support to help you in case of problem.
Hardware status
In the “HARDWARE STATUS” panel, you can find the hardware status of Ned.
In this section, you can nd details about the controller temperature, motors voltage, temperature, and
error code.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Logic
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Loops
Math
Division remainder
block Returns the remainder from the
division of two numbers.
Lists
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Variables
Functions
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Utility
Arm
Movement
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
I/O
Set I/O mode block Sets I/O pin mode to input or output.
Set output state block Sets output pin state to high or low.
Tool
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Niryo Studio User Manual (v3.2.1)
Activate
electromagnet block Activates the electromagnet power.
Vision
The Vision blocks and Vision templates are detailed in the Vision Set Documentation
(https://docs.niryo.com/product/vision-set/source/how_to_use_blockly.html#blocks-description).
Conveyor blocks
Advanced Programming
Python (https://docs.niryo.com/dev/pyniryo/index.html)
ROS (https://docs.niryo.com/dev/ros/index.html)
Modbus (https://docs.niryo.com/dev/modbus/index.html)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Preamble
Before diving into the software documentation, you can learn more about the robot development in theOverwiew (index.html#document-source/overview) section.
Also, if you want to use a simulated version of Ned, browse the Simulation (index.html#document-source/simulation) section.
Important
To control the robot directly with ROS, you will need either to be connected in SSH to the physic robot, or to use the simulation.
to send command via the terminal in order to call services, trigger action, …
to write a entire Python/C++ node in a script to realize a full process.
See ROS (index.html#document-source/ros) section to see all Topic & Services available.
Important
To use Python ROS Wrapper, you will need either to be connected in SSH to the physic robot, or to use the simulation.
The Python ROS Wrapper is built on top of ROS to allow a faster development than ROS. Programs are run directly on the robot which allows to trigger them with the robot’s
button once a computer is no longer needed.
See Python ROS Wrapper (index.html#document-source/ros_wrapper) to see which functions are accessible and examples on how to use them.
More ways
Other methods are available to control the robot allowing the user to code and run programs outside his terminal.
Ned is a robot based on Raspberry, Arduino & ROS. It uses ROS to make the interface between Hardware and high-level bindings.
On the following figure, you can see a global overview of the Niryo’s robot software in order to understand where are placed each part of the software.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
The simulation allows to control a virtual Ned directly from your computer.
Note
You can use Niryo Studio with the simulation (https://docs.niryo.com/product/ned/source/software/niryo_studio.html#connecting-simulation-to-niryo-studio/). To do so,
you just have to connect Niryo Studio to “Localhost”.
Attention
The whole ROS Stack is developed and tested on ROS Melodic which requires Ubuntu 18.04 to run correctly. The using of another ROS version or OS may lead to
malfunctions of some packages.
To allow the simulation to run on your computer, you will need to install ROS and some packages.
Installation index:
Prepare environment
Install ROS dependencies
Setup Ned ROS environment
Prepare environment
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Note
Place yourself in the folder of your choice and create a folder catkin_ws_niryo_ned as well as a sub-folder src:
mkdir -p catkin_ws_niryo_ned/src
Then go to the folder catkin_ws_niryo_ned and clone Ned repository in the folder src. For the future operation, be sure to stay in the catkin_ws_niryo_ned folder:
cd catkin_ws_niryo_ned
git clone https://github.com/NiryoRobotics/ned_ros src
Install ROS¶
You firstly need to install ROS Melodic. To do so, follow the ROS official tutorialhere (http://wiki.ros.org/melodic/Installation/Ubuntu) and chose the Desktop-Full Install.
rosdep update
rosdep install --from-paths src --ignore-src --default-yes --rosdistro melodic --skip-keys "python-rpi.gpio"
build-essential
catkin
python-catkin-pkg
python-pymodbus
python-rosdistro
python-rospkg
python-rosdep-modules
python-rosinstall python-rosinstall-generator
python-wstool
moveit
control
controllers
tf2-web-republisher
rosbridge-server
joint-state-publisher-gui
Note
Then perform the make of Ned’s ROS Stack via the command:
catkin_make
If no errors occurred during the make phase, the setup of your environment is almost complete!
It is necessary to source the configuration file to add all Ned packages to ROS environment. To do so, run the command:
source devel/setup.bash
It is necessary to run this command each time you launch a new terminal. If you want to make this sourcing appends for all your futur terminals, you can add it to yourbashrc
file:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Simulation utilization
Important
If you haven’t follow the step of Simulation Environment Installation. you won’t be able to use the simulation.
Hardware features won’t be accessible.
The simulation is a powerful tool which allow to test new programs directly on your computer which prevent to transfer new code on the robot.
It also helps for developing purpose → no need to transfer code, compile and restart the robot which is way slower than doing it on a desktop computer.
Rviz should open with a window containing 6 trackbars. Each of these trackbars allows to control the corresponding joint.
To run it:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
For the simulation, Ned uses Gazebo, a well known tool among the ROS community. It allows:
Collision.
World creation → An virtual environment in which the robot can deal with objects.
Gripper & Camera using.
The Niryo Gripper 1 has been replicated in Gazebo. The Camera is also implemented.
Note
Gazebo also generates camera distortion, which brings the simulation even closer from the reality!
Launch simulation¶
A specific world has been created to use Ned in Gazebo with 2 workspaces.
To run it:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Note
You can edit Gazebo world to do your own! It’s placed in the folderworlds of the package niryo_robot_gazebo.
Simulation option¶
The user can disable 3 things by adding the specific string to the command line:
Hint
Gazebo can be very slow. If your tests do not need Gripper and Camera, consider using Rviz to alleviate your CPU.
ROS Logo
ROS (Robot Operating System) is an Open-Source Robotic Framework which allows to ease robot software development. The framework is used in almost each part of Ned
software.
The high-level packages (motion planner, vision, …) are coded in Python to give good readability whereas communication with Hardware is developed in C++ to ensure speed.
Note
In this section, you will have access to all information about each Niryo Robot’s ROS packages.
Niryo_robot_bringup
This packages provides config and launch files to start Ned and ROS packages with various parameters.
Launch files are placed in the launch folder. Only files with .launch extension can be executed.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
On RaspberryPI
Ned¶
The file niryo_ned_robot.launch allows to launch ROS on a Raspberry Pi 4.
This file is automatically launched when Ned boots (Ned RPi4B image).
On Desktop (Simulation)
Gazebo simulation¶
Run Gazebo simulation. The robot can do everything that is not hardware-related:
move, get_pose.
use the camera (to disable it, set “camera” parameter to ‘false’).
use the Gripper 1 (to disable it, set “simu_gripper” parameter to ‘false’).
save/run programs, go to saved pose, …
Rviz simulation¶
Run Rviz simulation. You can access same features as Gazebo except Camera & Gripper.
niryo_robot_base files setup many rosparams, these files should be launched before any other package.
Niryo_robot_arm_commander
This package is the one dealing with all commander related stuff.
It is composed of only one node, which is running separately the arm commander and the tool commander.
Commander node
Parameters - Commander¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Commander’s Parameters
Name Description
simu_gripper If you are using the simulated Gripper and want to control the Gripper
robot_action RobotMove Command the arm and tools through an action server
Services - Commander¶
Messages - Commander¶
Name Description
Dependencies - Commander¶
actionlib
actionlib_msgs
control_msgs
geometry_msgs
MoveIt!
moveit_msgs
Niryo_robot_msgs
Niryo robot tools commander package
python-numpy
ros_controllers
rosbridge_server
sensor_msgs
std_msgs
tf2_web_republisher
trajectory_msgs
RobotMove (Action)¶
# goal
niryo_robot_arm_commander/ArmMoveCommand cmd
---
# result
int32 status
string message
---
# feedback
niryo_robot_msgs/RobotState state
GetFK (Service)¶
float32[] joints
---
niryo_robot_msgs/RobotState pose
GetIK (Service)¶
niryo_robot_msgs/RobotState pose
---
bool success
float32[] joints
JogShift (Service)¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
int32 JOINTS_SHIFT = 1
int32 POSE_SHIFT = 2
int32 cmd
float32[] shift_values
---
int32 status
string message
ArmMoveCommand (Message)¶
int32 JOINTS = 0 # uses joints
int32 POSE = 1 # uses position and rpy
int32 POSITION = 2 # uses position
int32 RPY = 3 # uses rpy
int32 POSE_QUAT = 4 # uses position and orientation
int32 LINEAR_POSE = 5 # uses position and rpy
int32 SHIFT_POSE = 6 # uses shift
int32 SHIFT_LINEAR_POSE = 7 # uses shift
int32 EXECUTE_TRAJ = 8 # uses dist_smoothing, list_poses
int32 DRAW_SPIRAL = 9
int32 cmd_type
float64[] joints
geometry_msgs/Point position
niryo_robot_msgs/RPY rpy
geometry_msgs/Quaternion orientation
niryo_robot_arm_commander/ShiftPose shift
geometry_msgs/Pose[] list_poses
float32 dist_smoothing
PausePlanExecution (Message)¶
int8 STANDBY = 0
int8 PLAY = 1
int8 PAUSE = 2
int8 RESUME = 3
int8 CANCEL = 4
int8 state
ShiftPose (Message)¶
int32 axis_number
float64 value
Niryo_robot_description
This package contains URDF files and meshes (collada + stl) for Ned.
Note : 3D visualization is not available on Ned Raspberry Pi4 image. To use the following commands, you must have setup Ned ros stack on your computer.
Niryo_robot_gazebo
Gazebo
Usage
When launching the Gazebo version of the ROS Stack, the fileniryo_robot_gazebo_world.launch.xml will be called to generate the Gazebo world.
Create your world’s file and put it on the folder worlds. Once it is done, you have to change the parameter world_name in the file niryo_robot_gazebo_world.launch.xml.
You can take a look at the Gazebo world by launching it without robot by precising the world name in the argworld_name:
Niryo_robot_msgs
This package contains standard messages which can be used by all other packages.
Niryo messages
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Ned Messages
Name Description
Niryo services
Ned Services
Name Description
geometry_msgs
CommandStatus¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
int32 val
# overall behavior
int32 SUCCESS = 1
int32 CANCELLED = 2
int32 PREEMPTED = 3
int32 FAILURE = -1
int32 ABORTED = -3
int32 STOPPED = -4
# - Hardware
int32 HARDWARE_FAILURE = -110
int32 HARDWARE_NOT_OK = -111
int32 LEARNING_MODE_ON = -112
int32 CALIBRATION_NOT_DONE = -113
int32 DIGITAL_IO_PANEL_ERROR = -114
int32 LED_MANAGER_ERROR = -115
int32 BUTTON_ERROR = -116
int32 WRONG_MOTOR_TYPE = -117
int32 DXL_WRITE_ERROR = -118
int32 DXL_READ_ERROR = -119
int32 CAN_WRITE_ERROR = -120
int32 CAN_READ_ERROR = -121
int32 NO_CONVEYOR_LEFT = -122
int32 NO_CONVEYOR_FOUND = -123
int32 CONVEYOR_ID_INVALID = -124
int32 CALIBRATION_IN_PROGRESS = -125
# - Vision
int32 VIDEO_STREAM_ON_OFF_FAILURE = -170
int32 VIDEO_STREAM_NOT_RUNNING = -171
int32 OBJECT_NOT_FOUND = -172
int32 MARKERS_NOT_FOUND = -173
# - Commander
# Arm Commander
int32 ARM_COMMANDER_FAILURE = -220
int32 GOAL_STILL_ACTIVE = -221
int32 JOG_CONTROLLER_ENABLED = -222
int32 CONTROLLER_PROBLEMS = -223
int32 SHOULD_RESTART = -224
int32 JOG_CONTROLLER_FAILURE = -225
# Tool Commander
int32 TOOL_FAILURE = -251
int32 TOOL_ID_INVALID = -252
int32 TOOL_NOT_CONNECTED = -253
int32 TOOL_ROS_INTERFACE_ERROR = -254
# - Pose Handlers
int32 POSES_HANDLER_CREATION_FAILED = -300
int32 POSES_HANDLER_REMOVAL_FAILED = -301
int32 POSES_HANDLER_READ_FAILURE = -302
int32 POSES_HANDLER_COMPUTE_FAILURE = -303
# - Programs Manager
int32 PROGRAMS_MANAGER_FAILURE = -320
int32 PROGRAMS_MANAGER_READ_FAILURE = -321
int32 PROGRAMS_MANAGER_UNKNOWN_LANGUAGE = -325
int32 PROGRAMS_MANAGER_NOT_RUNNABLE_LANGUAGE = -326
int32 PROGRAMS_MANAGER_EXECUTION_FAILED = -327
int32 PROGRAMS_MANAGER_STOPPING_FAILED = -328
int32 PROGRAMS_MANAGER_AUTORUN_FAILURE = -329
int32 PROGRAMS_MANAGER_WRITING_FAILURE = -330
int32 PROGRAMS_MANAGER_FILE_ALREADY_EXISTS = -331
int32 PROGRAMS_MANAGER_FILE_DOES_NOT_EXIST = -332
# - Serial
int32 SERIAL_FILE_ERROR = -400
int32 SERIAL_UNKNOWN_ERROR = -401
ObjectPose¶
float64 x
float64 y
float64 z
float64 roll
float64 pitch
float64 yaw
RobotState¶
geometry_msgs/Point position
niryo_robot_msgs/RPY rpy
geometry_msgs/Quaternion orientation
RPY¶
# roll, pitch and yaw
float64 roll
float64 pitch
float64 yaw
HardwareStatus¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
std_msgs/Header header
# Raspberry Pi board
int32 rpi_temperature
# Motors
bool connection_up
string error_message
bool calibration_needed
bool calibration_in_progress
string[] motor_names
string[] motor_types
int32[] temperatures
float64[] voltages
int32[] hardware_errors
string[] hardware_errors_message
SoftwareVersion¶
string rpi_image_version
string ros_niryo_robot_version
string[] motor_names
string[] stepper_firmware_versions
GetBool¶
---
bool value
GetInt¶
---
int32 value
GetNameDescriptionList¶
---
string[] name_list
string[] description_list
GetStringList¶
---
string[] string_list
SetBool¶
bool value
---
int32 status
string message
SetInt¶
int32 value
---
int32 status
string message
SetString¶
string value
---
int32 status
string message
Trigger¶
---
int32 status
string message
Niryo_robot_modbus
Niryo_robot_poses_handlers
This package is in charge of dealing with transforms, workspace, grips and trajectories.
Workspaces¶
A workspace is de ned by 4 markers that form a rectangle. With the help of the robot’s calibration tip, the marker positions are learned. The camera returns poses (x, y, yaw)
relative to the workspace. We can then infer the absolute object pose in robot coordinates.
Grips¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
When we know the object pose in robot coordinates, we can’t directly send this pose to the robot because we specify the target pose of the tool_link and not of the actual TCP
(tool center point). Therefore we introduce the notion of grip. Each end e ector has its own grip that speci es where to place the robot with respect to the object. Currently,
the notion of grip is not part of the python/tcp/blockly interface because it would add an extra layer of complexity that is not really necessary for the moment. Therefore we
have a default grip for all tools that is selected automatically based on the current tool id. However, everything is ready if you want to de ne custom grips, e.g. for custom
tools or for custom grip positions.
Name Description
GetPose (Service)¶
string name
---
int32 status
string message
niryo_robot_poses_handlers/NiryoPose pose
GetTargetPose (Service)¶
string workspace
float32 height_offset
float32 x_rel
float32 y_rel
float32 yaw_rel
---
int32 status
string message
niryo_robot_msgs/RobotState target_pose
GetTrajectory (Service)¶
string name
---
int32 status
string message
geometry_msgs/Pose[] list_poses
GetWorkspaceRatio (Service)¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
string workspace
---
int32 status
string message
float32 ratio # width/height
GetWorkspaceRobotPoses (Service)¶
string name
---
int32 status
string message
niryo_robot_msgs/RobotState[] poses
ManagePose (Service)¶
int32 cmd
int32 SAVE = 1
int32 DELETE = -1
niryo_robot_poses_handlers/NiryoPose pose
---
int32 status
string message
ManageTrajectory (Service)¶
int32 cmd
int32 SAVE = 1
int32 DELETE = -1
string name
string description
geometry_msgs/Pose[] poses
---
int32 status
string message
ManageWorkspace (Service)¶
int32 SAVE = 1
int32 SAVE_WITH_POINTS = 2
int32 DELETE = -1
int32 cmd
niryo_robot_poses_handlers/Workspace workspace
---
int32 status
string message
NiryoPose (Message)¶
string name
string description
float64[] joints
geometry_msgs/Point position
niryo_robot_msgs/RPY rpy
geometry_msgs/Quaternion orientation
Trajectory (Message)¶
string name
geometry_msgs/Pose[] poses
Workspace (Message)¶
string name # maximum lenght of workspace's name is 30 characters
string description
geometry_msgs/Point[] points
niryo_robot_msgs/RobotState[] poses
Niryo_robot_programs_manager
The ROS Node is made of several services to deal with the storage and running of programs.
Calls are not available from the Python ROS Wrapper, as it made to run its programs with the Python ROS Wrapper.
Name Description
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
All these services are available as soon as the node is started whereas on standalone mode or not.
ExecuteProgram (Service)¶
bool execute_from_string
string name
string code_string
niryo_robot_programs_manager/ProgramLanguage language
---
int16 status
string message
GetProgram (Service)¶
string name
niryo_robot_programs_manager/ProgramLanguage language
---
int32 status
string message
string code
string description
GetProgramAutorunInfos (Service)¶
---
int32 status
string message
niryo_robot_programs_manager/ProgramLanguage language
string name
# Mode
int8 ONE_SHOT = 1
int8 LOOP = 2
int8 mode
GetProgramList (Service)¶
niryo_robot_programs_manager/ProgramLanguage language
---
string[] programs_names
niryo_robot_programs_manager/ProgramLanguageList[] list_of_language_list
string[] programs_description
ManageProgram (Service)¶
# Command
int32 SAVE = 1
int32 DELETE = -1
int8 cmd
# Program Name
string name
# - Creation
niryo_robot_programs_manager/ProgramLanguage language
string code
string description
bool allow_overwrite
---
int16 status
string message
SetProgramAutorun (Service)¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
# Program language
niryo_robot_programs_manager/ProgramLanguage language
# Program Name
string name
# Mode
int8 DISABLE = 0
int8 ONE_SHOT = 1
int8 LOOP = 2
int8 mode
---
int16 status
string message
ProgramLanguage (Message)¶
int8 NONE = -1
int8 ALL = 0
# Runnable
int8 PYTHON2 = 1
int8 PYTHON3 = 2
# Not Runnable
int8 BLOCKLY = 66
int8 used
ProgramLanguageList (Message)¶
niryo_robot_programs_manager/ProgramLanguage[] language_list
ProgramList (Message)¶
string[] programs_names
niryo_robot_programs_manager/ProgramLanguageList[] list_of_language_list
string[] programs_description
Niryo_robot_rpi
This package deals with Raspberry Pi related stuff (Button, fans, I/O, leds, …).
Raspberry Pi Node
Note that this package should not be used when you are using Ned ROS stack on your computer in simulation mode. Executes actions when the button is pressed.
pause_state PausePlanExecution Publish the current execution state launched when button is pressed
/niryo_robot/blockly/save_current_point std_msgs/Int32 Publish current point when user is in Blockly page to save block by pressing button
digital_io_state DigitalIOState Publish the I/Os state by giving for each it’s pin / name / mode / state
ros_log_status LogStatus Publish the current log status (log size / available disk / boolean if should delete ros log on startup)
RPI Services
/niryo_robot/rpi/change_button_mode SetInt Change top button mode (autorun program, blockly, nothing, …)
set_led_custom_blinker LedBlinker Set the led in blink mode with the color given
set_purge_ros_log_on_startup SetInt Modify the permanent settings that tells if robot should purge it’s ROS log at each boot
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
ChangeMotorConfig (Service)¶
int32[] can_required_motor_id_list
int32[] dxl_required_motor_id_list
---
int32 status
string message
GetDigitalIO (Service)¶
int32 pin
---
int32 status
string message
int32 pin
string name
int32 mode
int32 state
LedBlinker (Service)¶
uint8 LED_OFF = 0
uint8 LED_BLUE = 1
uint8 LED_GREEN = 2
uint8 LED_BLUE_GREEN = 3
uint8 LED_RED = 4
uint8 LED_PURPLE = 5
uint8 LED_RED_GREEN = 6
uint8 LED_WHITE = 7
bool activate
uint8 frequency # between 1hz and 100Hz
uint8 color
float32 blinker_duration # 0 for infinite
---
int32 status
string message
SetDigitalIO (Service)¶
uint8 pin
uint8 value
---
int32 status
string message
DigitalIOState (Service)¶
# GPIO pin
int32[] pins
# PIN names seen by user to make it simpler
string[] names
# IN/OUT
int32[] modes
# HIGH/LOW
int32[] states
LogStatus (Service)¶
std_msgs/Header header
# in MB
int32 log_size
int32 available_disk_size
bool purge_log_on_startup
This package allows to manage the TCP (Tool Center Point) of the robot. If the functionality is activated, all the movements (in Cartesian coordinates [x, y, z, roll, pitch, yaw]) of
the robot will be performed according to this TCP. The same program can then work with several tools by adapting the TCP transformation to them. By default this feature is
disabled, but can be enabled through the robot services.
The ROS Node is made of services to equip tool, an action server for tool command and topics for the current tool or the tool state.
Publisher - tools¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
tcp TCP Publish if the TCP (Tool Center Point) is enabled and transformation between the tool_link and the TCP
Services - tools¶
update_tool std_srvs/Trigger Ping/scan for a dxl motor flashed with an ID corresponding to a tool and equip it (if found)
equip_electromagnet SetInt Equip the electromagnet with the motor ID given as parameter
set_tcp SetTCP Activate the TCP (Tool Center Point) functionality and defines a new TCP transformation.
reset_tcp std_srvs/Trigger Reset the TCP transformation. By default it will be the one of the equipped tool.
Dependencies - tools¶
Niryo_robot_msgs
std_msgs
geometry_msgs
ToolAction (Action)¶
# goal
niryo_robot_tools_commander/ToolCommand cmd
---
# result
int32 status
string message
---
# feedback
int32 progression
ToolCommand (Message)¶
# Gripper
uint8 OPEN_GRIPPER = 1
uint8 CLOSE_GRIPPER = 2
# Vacuump pump
uint8 PULL_AIR_VACUUM_PUMP = 10
uint8 PUSH_AIR_VACUUM_PUMP = 11
uint8 cmd_type
# if gripper close
uint16 gripper_close_speed
# if gripper open
uint16 gripper_open_speed
TCP (Message)¶
bool enabled
geometry_msgs/Point position
niryo_robot_msgs/RPY rpy
geometry_msgs/Quaternion orientation
SetTCP (Service)¶
geometry_msgs/Point position
Niryo_robot_unit_tests
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
This package provides config and launches files to start Ned unit tests.
Niryo_robot_poses_handlers
Niryo_robot_programs_manager
Python ROS Wrapper
Using a launch le make the task easy as you do not have to bother about your current directory. Nevertheless, it starts a roscore, and so, the process won’t end as “/rosout”
will be still alive. To overcome this issue, the test node is set as required, so do not worry if you see some red lines at the execution’s end.
Using a python file implies to use the correct relative directory, but it also allows to get status code at the end of the script.
Niryo_robot_user_interface
This packages handle high-level user interface commands coming TCP requests and also system-related features like I/Os, LED and fans.
TCP Server
Ned is permanently running a TCP Server to acquire requests. This server is built on top of the Ned Python ROS Wrapper (index.html#document-source/ros_wrapper).
It o ers a simple way for developers to create programs for robot to control them via remote communication on a computer, on a mobile or any device with network
facilities.
Programs can communicate through network TCP with the robots in any language available.
Connection¶
To access the server, you will have to use to robot’s IP adress and communicate via the port 40001.
Communication¶
Only one client can communicate with the server (reconnection effective but no multi clients).
The server answers only after the command is done, so it can’t deal with multiple commands at the same time.
Packet convention¶
General format¶
For easier usage and easier debugging, the communication is based on JSON format.
Request¶
Format - Request¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Examples - Request¶
Calibrate auto: {'param_list': ['AUTO'], 'command': 'CALIBRATE'}
Move joints: {'param_list': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'command': 'MOVE_JOINTS'}
Answer¶
Format - Answer¶
Firstly, answers indicate to the user if its command has been well executed. This is indicated in the JSON by the parameter “status”.
{'status': 'OK', 'list_ret_param': [<param_1>, <param_2>, ....], 'payload_size': <payload_size_int>, 'command': <command_str>}<payload_str>
An unsuccessful answer will have the format: {'status': 'KO', 'message': <message_str>}
Examples - Answer¶
Calibrate Auto: {'status': 'OK', 'list_ret_param': [], 'payload_size': 0, 'command': 'CALIBRATE'}
Get Pose: {'status': 'OK', 'list_ret_param': [0.2, 0.15, 0.35, 0.5, -0.6, 0.1], 'payload_size': 0, 'command': 'GET_POSE'}
CALIBRATE= 0
SET_LEARNING_MODE= 1
GET_LEARNING_MODE= 2
SET_ARM_MAX_VELOCITY= 3
SET_JOG_CONTROL= 4
GET_JOINTS= 10
GET_POSE= 11
GET_POSE_QUAT= 12
MOVE_JOINTS= 20
MOVE_POSE= 21
SHIFT_POSE= 22
MOVE_LINEAR_POSE= 23
SHIFT_LINEAR_POSE= 24
JOG_JOINTS= 25
JOG_POSE= 26
FORWARD_KINEMATICS= 27
INVERSE_KINEMATICS= 28
GET_POSE_SAVED= 50
SAVE_POSE= 51
DELETE_POSE= 52
GET_SAVED_POSE_LIST= 53
PICK_FROM_POSE= 60
PLACE_FROM_POSE= 61
PICK_AND_PLACE= 62
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
GET_TRAJECTORY_SAVED= 80
EXECUTE_TRAJECTORY_FROM_POSES= 81
EXECUTE_TRAJECTORY_SAVED= 82
SAVE_TRAJECTORY= 83
DELETE_TRAJECTORY= 84
GET_SAVED_TRAJECTORY_LIST= 85
EXECUTE_TRAJECTORY_FROM_POSES_AND_JOINTS= 86
UPDATE_TOOL= 120
OPEN_GRIPPER= 121
CLOSE_GRIPPER= 122
PULL_AIR_VACUUM_PUMP= 123
PUSH_AIR_VACUUM_PUMP= 124
SETUP_ELECTROMAGNET= 125
ACTIVATE_ELECTROMAGNET= 126
DEACTIVATE_ELECTROMAGNET= 127
GET_CURRENT_TOOL_ID= 128
GRASP_WITH_TOOL= 129
RELEASE_WITH_TOOL= 130
ENABLE_TCP= 140
SET_TCP= 141
RESET_TCP= 142
TOOL_REBOOT= 145
SET_PIN_MODE= 150
DIGITAL_WRITE= 151
DIGITAL_READ= 152
GET_DIGITAL_IO_STATE= 153
GET_HARDWARE_STATUS= 154
SET_CONVEYOR= 180
UNSET_CONVEYOR= 181
CONTROL_CONVEYOR= 182
GET_CONNECTED_CONVEYORS_ID= 183
GET_IMAGE_COMPRESSED= 200
GET_TARGET_POSE_FROM_REL= 201
GET_TARGET_POSE_FROM_CAM= 202
VISION_PICK= 203
MOVE_TO_OBJECT= 205
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
DETECT_OBJECT= 204
GET_CAMERA_INTRINSICS= 210
SAVE_WORKSPACE_FROM_POSES= 220
SAVE_WORKSPACE_FROM_POINTS= 221
DELETE_WORKSPACE= 222
GET_WORKSPACE_RATIO= 223
GET_WORKSPACE_LIST= 224
SET_IMAGE_BRIGHTNESS= 230
SET_IMAGE_CONTRAST= 231
SET_IMAGE_SATURATION= 232
GET_IMAGE_PARAMETERS= 235
Niryo_robot_vision
This package is the one dealing with all vision related stuff.
Vision Node
The ROS Node is made of several services to deal with video streaming, object detection… The node is working exactly the same way if you chose to use it on simulation or
reality.
This node can be launched locally in a standalone mode via the command:
Configuration (Frame Per Second, Camera Port, Video Resolution) can be edited in the config file:
Parameters - Vision¶
Name Description
Publisher - Vision¶
video_stream_parameters ImageParameters Publishes the brightness, contrast and saturation settings of the video stream
Services - Vision¶
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
debug_colors DebugColorDetection Returns an annotated image to emphasize what happened with color detection
debug_markers DebugMarkers Returns an annotated image to emphasize what happened with markers detection
Dependencies - Vision¶
Niryo_robot_msgs
sensor_msgs
ImageParameters (Topic)¶
float64 brightness_factor
float64 contrast_factor
float64 saturation_factor
DebugColorDetection (Service)¶
string color
---
sensor_msgs/CompressedImage img
DebugMarkers (Service)¶
---
bool markers_detected
sensor_msgs/CompressedImage img
ObjDetection (Service)¶
string obj_type
string obj_color
float32 workspace_ratio
bool ret_image
---
int32 status
niryo_robot_msgs/ObjectPose obj_pose
string obj_type
string obj_color
sensor_msgs/CompressedImage img
TakePicture (Service)¶
string path
---
bool success
SetImageParameter (Service)¶
float64 factor
---
int32 status
string message
ROS Logo
ROS (Robot Operating System) is an Open-Source Robotic Framework which allows to ease robot software development. The framework is used in almost each part of Ned’s
software.
The high-level packages (motion planner, vision, …) are coded in Python to give good readability whereas communication with Hardware is developed in C++ to ensure speed.
Note
In this section, you will have access to all information about each Niryo robot’s ROS hardware stack packages.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Niryo_robot_hardware_interface
Name Description
Joints_interface
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Name Description
/niryo_robot/joints_interface/calibrate_motors SetInt Starst motors calibration - value can be 1 for auto calibration, 2 for manual
Conveyor_interface
Name Description
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
ControlConveyor (Service)¶
uint8 id
bool control_on
int16 speed
int8 direction
---
int16 status
string message
SetConveyor (Service)¶
uint8 cmd
uint8 id
uint8 ADD = 1
uint8 REMOVE = 2
---
int16 id
int16 status
string message
ConveyorFeedbackArray (Message)¶
conveyor_interface/ConveyorFeedback[] conveyors
ConveyorFeedback (Message)¶
#Conveyor id ( either 12 or 18)
uint8 conveyor_id
#Conveyor Connection state ( if it is enabled)
bool connection_state
# Conveyor Controls state : ON or OFF
bool running
# Conveyor Speed ( 1-> 100 %)
int16 speed
# Conveyor direction ( backward or forward)
int8 direction
Tools_interface
Name Description
List of tools id
id_list
Default: ‘[11,12,13,14,31]’
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
PingDxlTool (Service)¶
---
int32 state
uint8 id
OpenGripper (Service)¶
uint8 id
int16 open_position
int16 open_speed
int16 open_hold_torque
---
uint8 state
CloseGripper (Service)¶
uint8 id
int16 close_position
int16 close_speed
int16 close_hold_torque
int16 close_max_torque
---
uint8 state
PullAirVacuumPump (Service)¶
uint8 id
int16 pull_air_position
int16 pull_air_hold_torque
---
uint8 state
PushAirVacuumPump (Service)¶
uint8 id
int16 push_air_position
---
uint8 state
CPU_interface
Name Description
Fake_interface
This package provides fakes hardware interfaces when the robot is used in simulation.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
tools interface
Conveyor Belt interface
joints interface
/niryo_robot/joints_interface/calibrate_motors SetInt Starts motors calibration - value can be 1 for auto calibration, 2 for manual
Dynamixel_driver
Name Description
Writes frequency.
dxl_hardware_write_frequency
Default: ‘50.0’
Reads frequency.
dxl_hardware_read_data_frequency
Default: ‘15.0’
List of dynamixels ID
dxl_motor_id_list
Default: ‘[2,3,6]’
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
SendCustomDxlValue (Service)¶
# Check XL-320 and XL-430 reference doc for
# the complete register table
DxlMotorHardwareStatus (Message)¶
niryo_robot_msgs/MotorHeader motor_identity
uint32 temperature
float64 voltage
uint32 error
string error_msg
DxlMotorCommand (Message)¶
uint8 cmd_type
uint8 CMD_TYPE_POSITION=1
uint8 CMD_TYPE_VELOCITY=2
uint8 CMD_TYPE_EFFORT=3
uint8 CMD_TYPE_TORQUE=4
uint8[] motors_id
uint32[] params
DxlArrayMotorHardwareStatus (Message)¶
std_msgs/Header header
dynamixel_driver/DxlMotorHardwareStatus[] motors_hw_status
Stepper_driver
Name Description
Writes frequency.
can_hw_write_frequency
Default: ‘50.0’
List of steppers ID
stepper_motor_id_list
Default: ‘[1,2,3]’
StepperMotorHardwareStatus (Message)¶
niryo_robot_msgs/MotorHeader motor_identity
string firmware_version
int32 temperature
int32 voltage
int32 error
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
StepperMotorCommand (Message)¶
uint8 cmd_type
uint8 CMD_TYPE_POSITION=1
uint8 CMD_TYPE_VELOCITY=2
uint8 CMD_TYPE_EFFORT=3
uint8 CMD_TYPE_TORQUE=4
uint8[] motors_id
int32[] params
StepperArrayMotorHardwareStatus (Message)¶
std_msgs/Header header
stepper_driver/StepperMotorHardwareStatus[] motors_hw_status
Dxl_debug_tools
This package offers scripts to change ping/scan DXL motors and changes register values of these motors.
Python Logo
In order to control Ned more easily than calling each topics & services one by one, a Python ROS Wrapper has been built on top of ROS.
For instance, a script realizing a moveJ via Python ROS Wrapper will look like:
niryo_robot = NiryoRosWrapper()
niryo_robot.move_joints(0.1, -0.2, 0.0, 1.1, -0.5, 0.2)
It generates a RobotMove Action Goal and set it as a joint command with the corresponding joints value.
Sends goal to the Commander Action Server.
Waits for the Commander Action Server to set Action as finished.
Checks if action finished with a success.
In this section, we will give some examples on how to use the Python ROS Wrapper to control Ned, as well as a complete documentation of the functions available in the Ned
Python ROS Wrapper.
Hint
The Python ROS Wrapper forces the user to write his code directly in the robot, or, at least, copy the code on the robot via a terminal command. If you do not want that,
and run code directly from your computer you can use the python Package PyNiryo (index.html#pyniryo).
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
The Python interpreter needs to have all used packages in the environment variable PYTHONPATH, to do that, you need to have sourced your ROS environment:
If you are coding directly on your robot, it is made directly in every terminal.
If your are using simulation, be sure to have followed the setup from Setup Ned ROS environment.
To run, your program will need some imports & initialization. We give you below the piece of code you must use to make Python ROS Wrapper work:
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
niryo_robot = NiryoRosWrapper()
You have now everything you need to control the robot through its Python ROS Wrapper. To run a script, simply use the command python my_script.py .
Examples: Basics
In this file, two short programs are implemented & commented in order to help you understand the philosophy behind the Python ROS Wrapper.
Danger
If you are using the real robot, make sure the environment around is clear.
The following example shows a first use case. It’s a simple MoveJ.
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
# Moving joint
niryo_robot.move_joints(0.1, -0.2, 0.0, 1.1, -0.5, 0.2)
#!/usr/bin/env python
import rospy
niryo_robot = NiryoRosWrapper()
Once the connection is done, we calibrate the robot using its calibrate_auto()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.calibrate_auto) function:
niryo_robot.calibrate_auto()
As the robot is now calibrated, we can do a Move Joints by giving the 6 axis positions in radians! To do so, we use move_joints()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.move_joints):
For our second example, we are going to develop an algorithm of pick and place:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
# Updating tool
niryo_robot.update_tool()
#!/usr/bin/env python
rospy.init_node('niryo_robot_example_python_ros_wrapper')
niryo_robot = NiryoRosWrapper()
niryo_robot.calibrate_auto()
niryo_robot.update_tool()
Now that our initialization is done, we can open the Gripper (or push air from the Vacuum pump) with release_with_tool()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.release_with_tool), go to the picking pose via move_pose()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.move_pose) & then catch an object with grasp_with_tool()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.grasp_with_tool)!
You may not have fully understood how to move the robot and use tools of Ned and that is totally fine because you will find more details on another examples page!
The important thing to remember from this page is how to import the library & connect to the robot.
Examples: Movement
This document shows how to control Ned in order to make Move Joints & Move Pose.
If you want see more, you can look at API - Joints & Pose (index.html#joints-pose).
Danger
If you are using the real robot, make sure the environment around is clear.
Joints
To do a moveJ, you should pass 6 oats: (j1, j2, j3, j4, j5, j6) to the method move_joints()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.move_joints):
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
niryo_robot = NiryoRosWrapper()
niryo_robot.move_joints(0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
joints = niryo_robot.get_joints()
j1, j2, j3, j4, j5, j6 = joints
Pose
To do a moveP, you should pass 6 oats: (x, y, z, roll, pitch, yaw) to the method move_pose()
(index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.move_pose).
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
niryo_robot = NiryoRosWrapper()
niryo_robot.move_pose(0.25, 0.0, 0.25, 0.0, 0.0, 0.0)
This page shows how to control Ned’s tools via the Python ROS Wrapper.
If you want see more, you can look at API - Tools (index.html#tools).
Danger
If you are using the real robot, make sure the environment around it is clear.
Tool control
Equip tool¶
In order to use a tool, it should be mechanically plugged to the robot but also connected software wise.
niryo_robot.update_tool()
Grasping¶
To grasp with any tool, you can use the function: grasp_with_tool() (index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.grasp_with_tool). This
action corresponds to:
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
# Updating tool
niryo_robot.update_tool()
# Grasping
niryo_robot.grasp_with_tool()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
# Updating tool
tool_used = ToolID.XXX
niryo_robot.update_tool()
Releasing¶
To release with any tool, you can use the function: release_with_tool() (index.html#niryo_robot_python_ros_wrapper.ros_wrapper.NiryoRosWrapper.release_with_tool).
This action correspond to:
niryo_robot.release_with_tool()
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
# Updating tool
tool_used = ToolID.XXX
niryo_robot.update_tool()
There are a plenty of ways to realize a pick and place with the ROS Wrapper. Methods will be presented from the lowest to highest level.
# Imports
from niryo_robot_python_ros_wrapper import *
def pick_n_place_version_x(niryo_ned):
# -- SOME CODE -- #
if __name__ == '__main__':
niryo_robot = NiryoRosWrapper()
niryo_robot.calibrate_auto()
pick_n_place_version_x(niryo_robot)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
def pick_n_place_version_1(niryo_ned):
height_offset = 0.05 # Offset according to Z-Axis to go over pick & place poses
gripper_speed = 400
# Raising
niryo_ned.move_pose(pick_pose[0], pick_pose[1], pick_pose[2] + height_offset,
pick_pose[3], pick_pose[4], pick_pose[5])
Second solution: pick from pose & place from pose functions¶
We use predefined functions:
def pick_n_place_version_3(niryo_ned):
# Pick
niryo_ned.pick_from_pose(*pick_pose)
# Place
niryo_ned.place_from_pose(*place_pose)
def pick_n_place_version_4(niryo_ned):
# Pick & Place
niryo_ned.pick_and_place(pick_pose, place_pose)
If you want see more about Ned’s Conveyor Belt functions, you can look at API - Conveyor.
Note
Imports & initialisation are not mentioned, but you should not forget it!
This short example shows how to connect a Conveyor Belt, activate the connection and launch its motor:
niryo_robot = NiryoRosWrapper()
This example shows how to do a certain amount of pick & place by using the Conveyor Belt with the infrared sensor:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
# -- Setting variables
sensor_pin_id = PinID.GPIO_1A
catch_nb = 5
# -- MAIN PROGRAM
niryo_robot = NiryoRosWrapper()
for i in range(catch_nb):
run_conveyor(niryo_robot, conveyor_id)
while niryo_robot.digital_read(sensor_pin_id) == PinState.LOW:
niryo_robot.wait(0.1)
Examples: Vision
If you want see more about Ned’s Vision functions, you can look at API - Vision (index.html#vision).
Beforehand
As the examples start always the same, there is the code you need to add at the beginning of all of them:
#!/usr/bin/env python
# Imports
from niryo_robot_python_ros_wrapper import *
import rospy
niryo_robot = NiryoRosWrapper()
# - Constants
workspace_name = "workspace_1" # Robot's Workspace Name
# - Main Program
# Calibrate robot if robot needs calibration
niryo_robot.calibrate_auto()
# Changing tool
niryo_robot.update_tool()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
niryo_robot.move_pose(*observation_pose)
# Trying to pick target using camera
ret = niryo_robot.vision_pick(workspace_name,
height_offset=0.0,
shape=ObjectShape.ANY,
color=ObjectColor.ANY)
obj_found, shape_ret, color_ret = ret
if obj_found:
niryo_robot.place_from_pose(*place_pose)
niryo_robot.set_learning_mode(True)
This file presents the different Functions, Classes & Enums available with the API.
API functions
Enums
API functions
This class allows you to control the robot via internal API.
By controlling, we mean:
calibrate_auto()
Call service to calibrate motors then wait for its end. If failed, raise NiryoRosWrapperException
calibrate_manual()
Call service to calibrate motors then wait for its end. If failed, raise NiryoRosWrapperException
get_learning_mode()
set_learning_mode(set_bool)
set_arm_max_velocity(percentage)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
get_joints()
get_pose()
get_pose_as_list()
Parameters: j1 (float) –
j2 (float) –
j3 (float) –
j4 (float) –
j5 (float) –
j6 (float) –
move_to_sleep_pose()
Move to Sleep pose which allows the user to activate the learning mode without the risk of the robot hitting something because of gravity
Move robot end effector pose to a (x, y, z, roll, pitch, yaw) pose.
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
shift_pose(axis, value)
Parameters: axis (ShiftPose) – Value of RobotAxis enum corresponding to where the shift happens
value (float) – shift value
shift_linear_pose(axis, value)
Parameters: axis (ShiftPose) – Value of RobotAxis enum corresponding to where the shift happens
value (float) – shift value
Move robot end effector pose to a (x, y, z, roll, pitch, yaw) pose, with a linear trajectory
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
set_jog_use_state(state)
jog_joints_shift(shift_values)
Parameters: shift_values (list[float]) – list corresponding to the shift to be applied to each joint
jog_pose_shift(shift_values)
Parameters: shift_values (list[float]) – list corresponding to the shift to be applied to the position
Parameters: j1 (float) –
j2 (float) –
j3 (float) –
j4 (float) –
j5 (float) –
j6 (float) –
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
Saved poses¶
class NiryoRosWrapper
move_pose_saved(pose_name)
get_pose_saved(pose_name)
Get saved pose from robot intern storage Will raise error if position does not exist
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
delete_pose(name)
get_saved_pose_list()
Execute a picking from a position. If an error happens during the movement, error will be raised. A picking is described as : - going over the object - going down until
height = z - grasping with tool - going back over the object
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
Execute a placing from a position. If an error happens during the movement, error will be raised. A placing is described as : - going over the place - going down until
height = z - releasing the object with tool - going back over the place
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
Execute a pick and place. If an error happens during the movement, error will be raised. -> Args param is for development purposes
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Trajectories¶
class NiryoRosWrapper
get_trajectory_saved(trajectory_name)
Get saved trajectory from robot intern storage Will raise error if position does not exist
execute_trajectory_saved(trajectory_name)
execute_trajectory_from_poses(list_poses_raw, dist_smoothing=0.0)
Parameters: list_poses_raw (list[list[float]]) – list of [x, y, z, qx, qy, qz, qw] or list of [x, y, z, roll, pitch, yaw]
dist_smoothing (float) – Distance from waypoints before smoothing trajectory
save_trajectory(trajectory_name, list_poses_raw)
delete_trajectory(trajectory_name)
get_saved_trajectory_list()
Tools¶
class NiryoRosWrapper
get_current_tool_id()
Returns: Tool Id
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
update_tool()
grasp_with_tool(pin_id=-1 )
Grasp with the tool linked to tool_id. This action correspond to - Close gripper for Grippers - Pull Air for Vacuum pump - Activate for Electromagnet
Parameters: pin_id (PinID) – [Only required for electromagnet] Pin ID of the electromagnet
release_with_tool(pin_id=-1 )
Release with the tool associated to tool_id. This action correspond to - Open gripper for Grippers - Push Air for Vacuum pump - Deactivate for Electromagnet
Parameters: pin_id (PinID) – [Only required for electromagnet] Pin ID of the electromagnet
open_gripper(speed=500)
close_gripper(speed=500)
pull_air_vacuum_pump()
Pull air
push_air_vacuum_pump()
Pull air
setup_electromagnet(pin_id)
activate_electromagnet(pin_id)
deactivate_electromagnet(pin_id)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
enable_tcp(enable=True )
Enables or disables the TCP function (Tool Center Point). If activation is requested, the last recorded TCP value will be applied. The default value depends on the
gripper equipped. If deactivation is requested, the TCP will be coincident with the tool_link.
Activates the TCP function (Tool Center Point) and defines the transformation between the tool_link frame and the TCP frame.
Parameters: x (float) –
y (float) –
z (float) –
roll (float) –
pitch (float) –
yaw (float) –
reset_tcp()
Reset the TCP (Tool Center Point) transformation. The TCP will be reset according to the tool equipped.
Hardware¶
class NiryoRosWrapper
set_pin_mode(pin_id, pin_mode)
digital_write(pin_id, digital_state )
digital_read(pin_id)
Returns: state
get_hardware_status()
Get hardware status : Temperature, Hardware version, motors names & types …
get_digital_io_state()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Conveyor Belt¶
class NiryoRosWrapper
set_conveyor()
Scan for conveyor on can bus. If conveyor detected, return the conveyor ID
Raises: NiryoRosWrapperException –
Returns: ID
unset_conveyor(conveyor_id )
Raises: NiryoRosWrapperException –
Control conveyor associated to conveyor_id. Then stops it if bool_control_on is False, else refreshes it speed and direction
Vision¶
class NiryoRosWrapper
get_compressed_image()
set_brightness(brightness_factor)
Parameters: brightness_factor (float) – How much to adjust the brightness. 0.5 will give a darkened image, 1 will give the original image while 2 will enhance the brightness by a
factor of 2.
set_contrast(contrast_factor)
Parameters: contrast_factor (float) – While a factor of 1 gives original image. Making the factor towards 0 makes the image greyer, while factor>1 increases the contrast of the image.
set_saturation(saturation_factor)
Parameters: saturation_factor (float) – How much to adjust the saturation. 0 will give a black and white image, 1 will give the original image while 2 will enhance the saturation by a
factor of 2.
Given a pose (x_rel, y_rel, yaw_rel) relative to a workspace, this function returns the robot pose in which the current tool will be able to pick an object at this pose. The
height_offset argument (in m) defines how high the tool will hover over the workspace. If height_offset = 0, the tool will nearly touch the workspace.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Returns: target_pose
First detects the specified object using the camera and then returns the robot pose in which the object can be picked with the current tool
Picks the speci ed object from the workspace. This function has multiple phases: 1. detect object using the camera 2. prepare the current tool for picking 3. approach
the object 4. move down to the correct picking pose 5. actuate the current tool 6. lift the object
get_camera_intrinsics()
save_workspace_from_poses(name, list_poses_raw)
Save workspace by giving the poses of the robot to point its 4 corners with the calibration Tip. Corners should be in the good order
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
save_workspace_from_points(name, list_points_raw)
Save workspace by giving the poses of its 4 corners in the good order
delete_workspace(name)
get_workspace_poses(name)
get_workspace_ratio(name)
Returns: ratio
get_workspace_list()
Enums
class ShiftPose
AXIS_X= 0
AXIS_Y= 1
AXIS_Z= 2
ROT_ROLL= 3
ROT_PITCH= 4
ROT_YAW= 5
class ToolID
NONE= 0
GRIPPER_1= 11
GRIPPER_2= 12
GRIPPER_3= 13
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
GRIPPER_4= 14
ELECTROMAGNET_1= 30
VACUUM_PUMP_1= 31
class PinMode
OUTPUT= 0
INPUT= 1
class PinState
LOW= 0
HIGH= 1
class PinID
Pins ID
GPIO_1A= 2
GPIO_1B= 3
GPIO_1C= 16
GPIO_2A= 26
GPIO_2B= 19
GPIO_2C= 6
SW_1= 12
SW_2= 13
class ConveyorID
NONE= 0
ID_1= 12
ID_2= 13
class ConveyorDirection
FORWARD= 1
BACKWARD= -1
class ObjectColor
RED= 'RED'
GREEN= 'GREEN'
BLUE= 'BLUE'
ANY= 'ANY'
class ObjectShape
CIRCLE= 'CIRCLE'
SQUARE= 'SQUARE'
ANY= 'ANY'
todo: vu avec Etienne, supprimer la partie Modbus d’ici et mettre en place une redirection vers doc Modbus.
Ned is permanently running a Modbus TCP Server that enables Ned to communicate with a PLC, or another computer in the same network.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Coil.
Discrete Input.
Holding Register.
Input Register.
Each datastore has a different set of functionalities. Note that each datastore contains a completely different set of data.
Coil
READ/WRITE (the stored values correspond to the last given command, not the current robot state).
0x01: READ_COILS
0x05: WRITE_SINGLE_COIL
This datastore can be used to set Digital I/O mode and state.
0/100: 1A
1/101: 1B
2/102: 1C
3/103: 2A
4/104: 2B
5/105: 2C
Address Description
Discrete input
READ-ONLY
0x02: READ_DISCRETE_INPUTS
0/100: 1A
1/101: 1B
2/102: 1C
3/103: 2A
4/104: 2B
5/105: 2C
Address Description
This datastore can be used to read Digital I/O mode and state of the robot. See Coil above for digital I/O number mapping.
Holding register
READ/WRITE (the stored values correspond to the last given command, not the current robot state)
0x03: READ_HOLDING_REGISTERS
0x06: WRITE_SINGLE_REGISTER
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
Address Description
101 Sends Pose Move command with stored position and orientation
102 Sends Pose Linear Move command with stored position and orientation
152 Contains data retrieved from last cmd (depends of the cmd)
159 Vision - Shape of the object found (-1: ANY, 1: CIRCLE, 2: SQUARE, 3: TRIANGLE, 0: NONE)
160 Vision - Color of the object found (-1: ANY, 1: BLUE, 2: RED, 3: GREEN, 0: NONE)
500 Updates the tool id according to the gripper plugged (gripper 1: 11, gripper 2: 12, gripper 3: 13, vaccum pump: 31)
520 Enables a Conveyor Belt newly connected [on success: store its ID at 152]
521 Detaches / disables Conveyor Belt with the Conveyor Belt ID given at 525
522 Control Conveyor Belt with the Conveyor Belt ID given at 525
525 [related to 520/521/522/526] Stores the Conveyor Belt ID for all related command
526 Stops Conveyor Belt with the Conveyor Belt ID given at 525
600 TCP - Enables or disables the TCP function (Tool Center Point).
601 Activates the TCP function (Tool Center Point) and defines the transformation between the tool_link frame and the TCP frame.
610 Vision - Gets target pose from relative pose, with stored relative pose and height_offset
611 Vision - Gets target pose from camera, with stored workspace name, height offset, shape and color
612 Vision - Vision pick, with stored workspace name, height offset, shape and color
613 Vision - Moves to object, with stored workspace name, height offset, shape and color
614 Vision - Detects object, with stored workspace name, shape and color
624 Vision - Stores requested shape (-1: ANY, 1: CIRCLE, 2: SQUARE, 3: TRIANGLE)
625 Vision - Stores requested color (-1: ANY, 1: BLUE, 2: RED, 3: GREEN)
626 - max 641 Vision - Stores workspace’s name, as a string encoded in 16 bits hex (see examples on how to store a workspace name from a client)
Input Register
READ-ONLY
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Ned ROS Documentation (v3.2.0)
0x04: READ_INPUT_REGISTERS
Address Description
pymodbus library
Niryo_robot_msgs
std_msgs
If you are a beginner, look at Blockly section to understand the programming fundamentals.
If you want to go further, maybe experience your own image processing, multi-robot, AI… You can go to PyNiryo for more information.
Blockly
PyNiryo
As explained in the page Use Ned’s TCP server (index.html#use-ned-s-tcp-server), a TCP Server is running on Ned, which allows it to receive commands from any external
device.
PyNiryo is a Python package available on Pip which allows to command the Niryo Robots with easy Python Binding.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
The Modbus/TCP server is running on port 5020 by default. It has been built on top of the pymodbus
(https://pymodbus.readthedocs.io/en/latest/index.html) library. This enables you to make Ned
communicates with a PLC, or another computer on the same network.
Note
Download Python on the o cial Python website (https://www.python.org/) and nd more information
about the installation on this website (https://realpython.com/installing-python/).
This installation requires the use of pip (https://pypi.org/project/pip/), the package manager included in
Python.
To use the Modbus API, you also need to install Modbus python library pymodbus
(https://pymodbus.readthedocs.io/en/latest/index.html):
Attention
Pip can require administrator authorizations to install packages. In this case, add
sudo
If pip is not automatically installed with Python, please visit the following website: pip installation
(https://pypi.org/project/pip/).
Introduction
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
All 4 Modbus datastores are implemented: Coils, Discrete inputs, Holding registers, Input registers. Each
datastore has a different set of functionalities. Note that each datastore contains a completely different set
of data.
Discrete Input and Input register are READ-ONLY tables. Those have been used to keep the robot state.
Coil and Holding Register are READ/WRITE tables. Those have been used to give user commands to the
robot. Hence, those 2 tables do not contain the robot state, but the last given command.
Coils
READ/WRITE (the stored values correspond to the last given command, not the current robot state)
0x01: READ_COILS
0x05: WRITE_SINGLE_COIL
This datastore can be used to set Digital I/O mode and state. Digital I/O numbers used for Modbus:
0: 1A
1: 1B
2: 1C
3: 2A
4: 2B
5: 2C
Address Description
Discrete inputs
Each address contains a 1bit value.
READ-ONLY
0x02: READ_DISCRETE_INPUTS
This datastore can be used to read Digital I/O mode and state. See the Coils section above for digital I/O
number mapping.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
Address Description
Holding registers
Each address contains a 16bit value.
READ/WRITE (the stored values correspond to the last given command, not the current robot state)
0x03: READ_HOLDING_REGISTERS
0x06: WRITE_SINGLE_REGISTER
Address Description
101 Sends Pose Move command with stored position and orientation
102 Sends Linear Pose Move command with stored position and orientation
153 -
Vision - Target pose result
158
159 Vision - Shape of the object found (-1: ANY, 1: CIRCLE, 2: SQUARE, 3: TRIANGLE, 0: NONE)
160 Vision - Color of the object found (-1: ANY, 1: BLUE, 2: RED, 3: GREEN, 0: NONE)
Updates the tool id according to the gripper plugged (gripper 1: 11, gripper 2: 12, gripper
500 3: 13, vaccum pump: 31)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
Address Description
600 TCP - Enables or disables the TCP function (Tool Center Point).
Activates the TCP function (Tool Center Point) and defines the transformation between
601 the tool_link frame and the TCP frame.
610 Vision - Gets target pose from relative pose, with stored relative pose and height_offset
Vision - Gets target pose from camera, with stored workspace name, height offset, shape
611
and color
612 Vision - Vision pick, with stored workspace name, height offset, shape and color
613 Vision - Moves to object, with stored workspace name, height offset, shape and color
614 Vision - Detects object, with stored workspace name, shape and color
624 Vision - Stores requested shape (-1: ANY, 1: CIRCLE, 2: SQUARE, 3: TRIANGLE)
625 Vision - Stores requested color (-1: ANY, 1: BLUE, 2: RED, 3: GREEN)
626 - Vision - Stores workspace’s name, as a string encoded in 16 bits hex (see examples on
max 641 how to store a workspace name from a client)
‘*’ The “Last command result” gives you more information about the last executed command:
0: no result yet
1: success
2: command was rejected (invalid params, …)
3: command was aborted
4: command was canceled
5: command had an unexpected error
6: command timeout
7: internal error
Input registers
Each address contains a 16bit value.
READ-ONLY.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
0x04: READ_INPUT_REGISTERS
Address Description
Repository
Examples of Modbus python lib can be found here Python Modbus examples
(https://github.com/NiryoRobotics/ned_ros/tree/master/niryo_robot_modbus/examples/).
Examples
In the examples folder, you can nd several example scripts that control Ned. These scripts are
commented to help you understand every step.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
This script shows how to activate the Conveyor Belt through the Modbus Python API, set a direction, a
speed, and start and stop the device.
This script shows how to use the vision pick method from a Modbus Client, through the Modbus Python
API. Ned picks a red object seen in its workspace and releases it on its left. Note that we use the
string_to_register method to convert a string into an object storable in registers.
#!/usr/bin/env python
# Enums for shape and color. Those enums are the one used by the modbus server to receive requests
@unique
class ColorEnum(Enum):
ANY = -1
BLUE = 1
RED = 2
GREEN = 3
NONE = 0
@unique
class ShapeEnum(Enum):
ANY = -1
CIRCLE = 1
SQUARE = 2
TRIANGLE = 3
NONE = 0
def raw_data_to_number(val):
if (val >> 15) == 1:
val = - (val & 0x7FFF)
return val
def string_to_register(string):
# code a string to 16 bits hex value to store in register
builder = BinaryPayloadBuilder()
builder.add_string(string)
payload = builder.to_registers()
return payload
def back_to_observation():
# To change
# joint_real = [0.057, 0.604, -0.576, -0.078, -1.384,0.253]
joint_simu = [0, -0.092, 0, 0, -1.744, 0]
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
time.sleep(0.01)
def register_workspace_name(ws_name):
workspace_request_register = string_to_register(ws_name)
client.write_registers(626, workspace_request_register)
def register_height_offset(height_offset):
client.write_registers(620, int(number_to_raw_data(height_offset * 1000)))
def auto_calibration():
print "Calibrate Robot if needed ..."
client.write_register(311, 1)
# Wait for end of calibration
while client.read_input_registers(402, 1).registers[0] == 1:
time.sleep(0.05)
def get_current_tool_id():
return client.read_input_registers(200, count=1).registers[0]
def open_tool():
tool_id = get_current_tool_id()
if tool_id == 31:
client.write_register(513, 1)
else:
client.write_register(510, 1)
while client.read_holding_registers(150, count=1).registers[0] == 1:
time.sleep(0.05)
client.write_registers(624, number_to_raw_data(shape_int))
client.write_registers(625, number_to_raw_data(color_int))
if __name__ == '__main__':
print "--- START"
# update tool
client.write_registers(500, 1)
print 'VISION PICK - pick a red pawn, lift it and release it'
shape = ShapeEnum.ANY.value
color = ColorEnum.RED.value
shape_picked, color_picked = vision_pick(workspace_name, height_offset, shape, color)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
Modbus User Manual (v3.2.0)
client.write_registers(0, joints_to_send)
client.write_register(100, 1)
open_tool()
back_to_observation()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
PyNiryo Documentation
This documentation presents Ned’s PyPi package, which is a TCP API made with Python.
It o ers a simple way for developers to create programs for robot and to control them via remote
communication from their computers. Contrary to the Python ROS Wrapper, the user will not need to be
connected on the robot through a terminal.
Note
This package is able to control Ned in simulation as well as the physical robot.
Ned
This documentation also contains everything you need to know if you want to use Ned through simulation.
Sections organization
This document is organized in 4 main sections.
Setup
Install & Setup your environment in order to use Ned with PyNiryo.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Installation
The library uses Python, which must be installed and available in your working environment.
The version should be equal or above:
Hint
To check your Python version, use the command python --version if you are using Python 2 and
python3 --version if you are using Python3.
The below sections explain how to install the library with pip, the package installer for Python.
Attention
If you have both Python 2 & Python 3 installed on your computer, the command pip will install
packages in Python 2 version. You should use pip3 instead in order to target Python 3.
You can find more information about the PyPi package here (https://pypi.org/project/pyniryo/).
If you also want to use Vision functions to do your own Image Processing pipeline install OpenCV via the
command:
Uninstall
In order to use your robot through TCP connection, you will rst need to connect to it, which implies that
you know its IP address.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
The next sections explain how to find your robot IP according to your configuration:
Hotspot mode
Simulation or directly on the robot
Direct ethernet connection
Computer and robot connected on the same router
Make IP permanent
Hotspot mode
If you are directly connected to your robot through its wi- , the IP address you will need to use is
10.10.10.10 .
In this situation, the robot is running on the same computer as the client, the IP address will be the
localhost address 127.0.0.1 .
If you are directly connected to your robot with an ethernet cable, the static IP of your robot will be
169.254.200.200 .
The reader should note that he may need to change his wired settings to allow the connection. See how to
Connect to Ned via Ethernet on Ubuntu
(https://docs.niryo.com/applications/ned/source/tutorials/setup_connect_ned_ethernet.html).
You will need to nd the robot’s address using nmap , or you can also use search button of Niryo Studio
to see which robots are available.
You can also Make IP permanent so that you will not have to search for it next time
Make IP permanent
Step 1
ssh niryo@<robot_ip_address>
On Windows, you can use Putty (https://www.putty.org/). Robot username is niryo and password is
robotics .
Step 2
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
ifconfig
Your proxy key is written on the first line and should look something like eth0 .
Step 3
Select arbitrarily a number between 50 & 255. It will be your IP address’ last number.
auto <robot_proxy_key>
iface <robot_proxy_key> inet static
address 192.168.1.<your_ip_address_last_number>
broadcast 192.168.1.255
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 192.168.1.1
From its next reboot, the robot will appear under the IP 192.168.1.<your_ip_address_last_number> .
In order to verify your computer’s setup, we are going to run a program from it, and see if the robot
answers as expected.
Note
Before verifying your setup, be sure that your physical robot (or simulation) is turned on.
Firstly, go in the folder of your choice and create an empty le named “pyniryo_test.py”. This le will
contain the checking code.
robot_ip_address = "10.10.10.10"
Attention
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Still on your computer, open a terminal, and place your current directory in the same folder than your le.
Then, run the command:
python pyniryo_test.py
Note
If you are using Python 3, you may need to change python to python3 .
If your robot starts calibrating, then moves, and nally, goes to learning mode, your setup is validated, you
can now start coding!
Examples
Examples: Basics
In this le, two short programs are implemented & commented in order to help you understand the
philosophy behind the PyNiryo package.
Danger
If you are using the real robot, make sure the environment around it is clear.
The following example shows a first use case. It’s a simple MoveJ.
robot = NiryoRobot("10.10.10.10")
robot.calibrate_auto()
robot.close_connection()
Then, we instantiate the connection and link the variable robot to the robot at the IP Address
10.10.10.10 .
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
robot = NiryoRobot("10.10.10.10")
Once the connection is done, we calibrate the robot using its calibrate_auto()
(index.html#api.tcp_client.NiryoRobot.calibrate_auto) function.
robot.calibrate_auto()
As the robot is now calibrated, we can do a Move Joints by giving the 6 axis positions in radians! To do so,
we use move_joints() (index.html#api.tcp_client.NiryoRobot.move_joints).
Our process is now over, we can close the connection with quit() .
robot.close_connection()
In the second example, we are going to develop a pick and place algorithm.
robot = NiryoRobot("10.10.10.10")
robot.calibrate_auto()
robot.update_tool()
robot.release_with_tool()
robot.move_pose(0.2, -0.1, 0.25, 0.0, 1.57, 0.0)
robot.grasp_with_tool()
robot.close_connection()
First of all, we import the library and start the connection between our computer and the robot. We also
calibrate the robot.
robot = NiryoRobot("10.10.10.10")
robot.calibrate_auto()
robot.update_tool()
Now that our initialization is done, we can open the gripper (or push air from the vacuum) with
release_with_tool() (index.html#api.tcp_client.NiryoRobot.release_with_tool), go to the picking pose
v i a move_pose() (index.html#api.tcp_client.NiryoRobot.move_pose) & then catch an object with
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
grasp_with_tool() (index.html#api.tcp_client.NiryoRobot.grasp_with_tool)!
robot.release_with_tool()
robot.move_pose(0.2, -0.1, 0.25, 0.0, 1.57, 0.0)
robot.grasp_with_tool()
robot.close_connection()
Notes
You may not have fully understood how to move the robot and use PyNiryo and that is totally ne because
you will find more details on the next examples page!
The important thing to remember from this page is how to import the library, connect to the robot & call
functions.
Examples: Movement
This document shows how to control Ned in order to make Move Joints & Move Pose.
If you want to see more, you can look at PyNiryo - Joints & Pose (index.html#joints-pose)
Important
In the following sections, you are supposed to be already connected to a calibrated robot. The robot’s
instance is saved in the variable robot . To know how to do so, go look at section Examples: Basics
(index.html#document-source/examples/examples_basics).
Danger
If you are using the real robot, make sure the environment around it is clear.
Joints
Move Joints
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
You should note that these 4 commands are doing exactly the same thing! In your future scripts, chose the
one you prefer, but try to remain consistent to keep a good readability.
Get Joints
To get actual joint positions, you can use the function get_joints()
(index.html#api.tcp_client.NiryoRobot.get_joints) or the joints getter. Both will return a list of the 6 joints
position:
Hint
As we are developing in Python, we can unpack list very easily, which means that we can retrieve joints
value in 6 variables by writing j1, j2, j3, j4, j5, j6 = robot.get_joints() .
Pose
Move Pose
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Get Pose
To get end e ector actual pose, you can use the function get_pose()
(index.html#api.tcp_client.NiryoRobot.get_pose) or the pose getter. Both will return a PoseObject
(index.html#api.objects.PoseObject):
The PoseObject (index.html#api.objects.PoseObject) is a Python object which allows to store all poses’
6 coordinates (x, y, z, roll, pitch, yaw) in one single instance. It can be converted into a list if needed with
the method to_list() (index.html#api.objects.PoseObject.to_list).
It also allows to create new PoseObject (index.html#api.objects.PoseObject) with some o set, much
easier than copying list and editing only 1 or 2 values. For instance, imagine that we want to shift the place
pose by 5 centimeters at each iteration of a loop, you can use the copy_with_offsets()
(index.html#api.objects.PoseObject.copy_with_offsets) method:
pick_pose = PoseObject(
x=0.30, y=0.0, z=0.15,
roll=0, pitch=1.57, yaw=0.0
)
first_place_pose = PoseObject(
x=0.0, y=0.2, z=0.15,
roll=0, pitch=1.57, yaw=0.0
)
for i in range(5):
robot.move_pose(pick_pose)
new_place_pose = first_place_pose.copy_with_offsets(x_offset=0.05 * i)
robot.move_pose(new_place_pose)
If you want to see more, you can look at PyNiryo - Tools (index.html#tools)
Important
In this section, you are already supposed to be connected to a calibrated robot. The robot instance is
saved in the variable robot .
Danger
If you are using the real robot, make sure the environment around it is clear.
Tool control
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Equip Tool
In order to use a tool, it should be plugged mechanically to the robot but also connected to the software
wise.
robot.update_tool()
Note
For the Grasping and Releasing sections, this command should be added in your codes! If you want to
use a specific tool, you need to store the ToolID you are using in a variable named tool_used .
Grasping
To grasp with any tool, you can use the function grasp_with_tool()
(index.html#api.tcp_client.NiryoRobot.grasp_with_tool). This action corresponds to:
robot.grasp_with_tool()
Releasing
To release with any tool, you can use the function release_with_tool()
(index.html#api.tcp_client.NiryoRobot.release_with_tool). This action corresponds to:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
A Pick & Place is an action which consists in going to a certain pose in order to pick an object and then,
going to another pose to place it.
There are plenty of ways to perform a pick and place with PyNiryo. Methods will be presented from the
lowest to highest level.
Code Baseline
For the sake of brevity, every piece of code beside the Pick & Place function will not be rewritten for every
method. So that, you will need to use the code and implement the Pick & Place function to it.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# Imports
from pyniryo import *
def pick_n_place_version_x(robot):
# -- -------------- -- #
# -- CODE GOES HERE -- #
# -- -------------- -- #
if __name__ == '__main__':
# Connect to robot
client = NiryoRobot(robot_ip_address)
# Calibrate robot if robot needs calibration
client.calibrate_auto()
# Changing tool
client.update_tool()
pick_n_place_version_x(client)
# Releasing connection
client.close_connection()
For this first function, every steps are done by hand, as well as poses computing.
Note
In this example, the tool used is a Gripper. If you want to use another tool than a gripper, do not forget
to adapt grasp & release functions!
def pick_n_place_version_1(robot):
height_offset = 0.05 # Offset according to Z-Axis to go over pick & place poses
gripper_speed = 400
# Raising
robot.move_pose(pick_pose.x, pick_pose.y, pick_pose.z + height_offset,
pick_pose.roll, pick_pose.pitch, pick_pose.yaw)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Note
def pick_n_place_version_2(robot):
height_offset = 0.05 # Offset according to Z-Axis to go over pick & place poses
pick_pose_high = pick_pose.copy_with_offsets(z_offset=height_offset)
place_pose_high = place_pose.copy_with_offsets(z_offset=height_offset)
Third Solution: Pick from pose & Place from pose functions
For those who have already seen the API Documentation, you may have seen pick & place dedicated
functions!
def pick_n_place_version_3(robot):
# Pick
robot.pick_from_pose(pick_pose)
# Place
robot.place_from_pose(place_pose)
The example exposed in the previous section could be useful if you want to do an action between the pick
& the place phases.
For those who want to do everything in one command, you can use the pick_and_place()
(index.html#api.tcp_client.NiryoRobot.pick_and_place) function!
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
def pick_n_place_version_4(robot):
# Pick & Place
robot.pick_and_place(pick_pose, place_pose)
If you want to see more about Ned’s Conveyor Belt functions, you can look at PyNiryo - Conveyor
(index.html#conveyor)
Danger
If you are using the real robot, make sure the environment around it is clear.
This short example shows how to connect a conveyor and launch its motor (control it by setting its speed
and direction):
# Connecting to robot
robot = NiryoRobot(<robot_ip_address>)
# Running the Conveyor Belt at 50% of its maximum speed, in forward direction
robot.run_conveyor(conveyor_id, speed=50, direction=ConveyorDirection.FORWARD)
# Waiting 3 seconds
robot.wait(3)
This example shows how to do a certain amount of pick & place by using the Conveyor Belt with the
infrared sensor:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# -- Setting variables
sensor_pin_id = PinID.GPIO_1A
catch_nb = 5
# -- MAIN PROGRAM
for i in range(catch_nb):
robot.run_conveyor(conveyor_id)
while robot.digital_read(sensor_pin_id) == PinState.LOW:
robot.wait(0.1)
Examples: Vision
If you want to see more about Ned’s Vision functions, you can look at PyNiryo - Vision
If you want to see how to do Image Processing, go check out the Image Processing section.
Note
Even if you do not own a Vision Set, you can still realize these examples with the Gazebo simulation
version.
Danger
If you are using the real robot, make sure the environment around it is clear.
Important
In order to achieve the following examples, you need to create a vision workspace. In this page, the
workspace used is named workspace_1 . To create it, the user should go on Niryo Studio!
As the examples start always the same, add the following lines at the beginning of codes:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# Imports
from pyniryo import *
# - Constants
workspace_name = "workspace_1" # Robot's Workspace Name
robot_ip_address = "x.x.x.x"
# - Initialization
# Connect to robot
robot = NiryoRobot(robot_ip_address)
# Calibrate robot if the robot needs calibration
robot.calibrate_auto()
# Updating tool
robot.update_tool()
robot.close_connection()
Hint
All the following examples are only a part of what can be made with the API in terms of Vision. We
advise you to look at API - Vision (index.html#vision) to understand more deeply
The goal of a Vision Pick & Place is the same as a classical Pick & Place, with a close di erence: the camera
detects where the robot has to go in order to pick!
This short example shows how to do your rst Vision pick using the vision_pick()
(index.html#api.tcp_client.NiryoRobot.vision_pick) function:
robot.move_pose(observation_pose)
# Trying to pick target using camera
obj_found, shape_ret, color_ret = robot.vision_pick(workspace_name)
if obj_found:
robot.place_from_pose(place_pose)
robot.set_learning_mode(True)
To execute a Vision pick, we firstly need to go to a place where the robot will be able to see the workspace:
robot.move_pose(observation_pose)
Then, we try to perform a Vision pick in the workspace with the vision_pick()
(index.html#api.tcp_client.NiryoRobot.vision_pick) function:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Variables shape_ret and color_ret are respectively of type ObjectShape and ObjectColor , and
store the shape and the color of the detected object! We will not use them for this first example.
The obj_found variable is a boolean which indicates whereas an object has been found and picked, or
not. Thus, if the pick worked, we can place the object at the place pose.
if obj_found:
robot.place_from_pose(place_pose)
robot.set_learning_mode(True)
Note
In most of use cases, the robot will need to perform more than one Pick & Place. In this example, we will
see how to condition multiple objects according to a straight line:
# Initializing variables
offset_size = 0.05
max_catch_count = 4
catch_count += 1
robot.go_to_sleep()
We want to catch max_catch_count objects, and space each of them by offset_size meter:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
offset_size = 0.05
max_catch_count = 4
catch_count = 0
while catch_count < max_catch_count:
For each iteration, we rstly go to the observation pose and then, try to make a Vision pick in the
workspace:
robot.move_pose(observation_pose)
If the Vision pick failed, we wait 0.1 second and then, start a new iteration:
if not obj_found:
robot.wait(0.1)
continue
Else, we compute the new place position according to the number of catches, and then, go placing the
object at that place:
catch_count += 1
robot.go_to_sleep()
During a conditioning task, objects may not always be placed as the same place according to their type. In
this example, we will see how to align object according to their color, using the color element
ObjectColor returned by vision_pick() (index.html#api.tcp_client.NiryoRobot.vision_pick) function:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
try_without_success = 0
# Loop until too much failures
while try_without_success < max_failure_count:
# Moving to observation pose
robot.move_pose(observation_pose)
# Trying to get object via Vision Pick
obj_found, shape, color = robot.vision_pick(workspace_name)
if not obj_found:
try_without_success += 1
robot.wait(0.1)
continue
# Increment count
count_dict[color] += 1
try_without_success = 0
robot.go_to_sleep()
We want to catch objects until Vision Pick failed max_failure_count times. Each of the object will be put
on a speci c column according to its color. The number of catches for each color will be stored on a
dictionary count_dict .
try_without_success = 0
# Loop until too much failures
while try_without_success < max_failure_count:
For each iteration, we rstly go to the observation pose and then, try to make a Vision pick in the
workspace:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
robot.move_pose(observation_pose)
If the Vision pick failed, we wait 0.1 second and then, start a new iteration, without forgetting to increment
the failure counter:
if not obj_found:
try_without_success += 1
robot.wait(0.1)
continue
Else, we compute the new place position according to the number of catches, and then, go place the object
at that place:
count_dict[color] += 1
try_without_success = 0
robot.go_to_sleep()
An interesting way to bring objects to the robot, is the use of a Conveyor Belt. In this examples, we will see
how to catch only a certain type of object by stopping the conveyor as soon as the object is detected on
the workspace.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# Initializing variables
offset_size = 0.05
max_catch_count = 4
shape_expected = ObjectShape.CIRCLE
color_expected = ObjectColor.RED
conveyor_id = robot.set_conveyor()
catch_count = 0
while catch_count < max_catch_count:
# Turning conveyor on
robot.run_conveyor(conveyor_id)
# Moving to observation pose
robot.move_pose(observation_pose)
# Check if object is in the workspace
obj_found, pos_array, shape, color = robot.detect_object(workspace_name,
shape=shape_expected,
color=color_expected)
if not obj_found:
robot.wait(0.5) # Wait to let the conveyor turn a bit
continue
# Stopping conveyor
robot.stop_conveyor(conveyor_id)
# Making a vision pick
obj_found, shape, color = robot.vision_pick(workspace_name,
shape=shape_expected,
color=color_expected)
if not obj_found: # If visual pick did not work
continue
catch_count += 1
robot.go_to_sleep()
Firstly, we initialize your process: we want the robot to catch 4 red circles. To do so, we set variables
shape_expected and color_expected with ObjectShape.CIRCLE and ObjectColor.RED .
offset_size = 0.05
max_catch_count = 4
shape_expected = ObjectShape.CIRCLE
color_expected = ObjectColor.RED
We activate the connection with the Conveyor Belt and start a loop until the robot has caught
max_catch_count objects:
conveyor_id = robot.set_conveyor()
catch_count = 0
while catch_count < max_catch_count:
For each iteration, we rstly run the Conveyor Belt (if the latter is already running, nothing will happen),
then go to the observation pose:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
We then check if an object corresponding to our criteria is in the workspace. If not, we wait 0.5 second and
then, start a new iteration:
Else, stop the Conveyor Belt and try to make a Vision pick:
If Vision Pick succeed, compute new place pose, and place the object:
catch_count += 1
Once the target catch number is achieved, we stop the Conveyor Belt and go to sleep:
robot.go_to_sleep()
Code templates
As code structures are always the same, we wrote down few templates for you to start your code le with
a good form.
# Releasing connection
robot.close_connection()
Advanced template
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
This template let the user de ne his own process but it handles connection, calibration, tool equipping,
and makes the robot go to sleep at the end:
def process(niryo_edu):
# --- --------- --- #
# --- YOUR CODE --- #
# --- --------- --- #
if __name__ == '__main__':
# Connect to robot
robot = NiryoRobot(robot_ip_address)
# Calibrate robot if robot needs calibration
robot.calibrate_auto()
# Equip tool
robot.update_tool()
# Launching main process
process(client)
# Ending
robot.go_to_sleep()
# Releasing connection
robot.close_connection()
robot.stop_conveyor()
if __name__ == '__main__':
# Connect to robot
robot = NiryoRobot(robot_ip_address)
# Calibrate robot if robot needs calibration
robot.calibrate_auto()
# Equip tool
robot.update_tool()
# Activating connexion with conveyor
conveyor_id = robot.set_conveyor()
# Launching main process
process(robot, conveyor_id)
# Ending
robot.go_to_sleep()
# Deactivating connexion with conveyor
robot.unset_conveyor(conveyor_id)
# Releasing connection
robot.close_connection()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
def process(robot):
robot.move_pose(observation_pose)
catch_count = 0
while catch_count < 3:
ret = robot.get_target_pose_from_cam(workspace_name,
height_offset=0.0,
shape=ObjectShape.ANY,
color=ObjectColor.ANY)
obj_found, obj_pose, shape, color = ret
if not obj_found:
continue
catch_count += 1
# --- --------- --- #
# --- YOUR CODE --- #
# --- --------- --- #
robot.place_from_pose(place_pose)
if __name__ == '__main__':
# Connect to robot
robot = NiryoRobot(robot_ip_address)
# Calibrate robot if robot needs calibration
robot.calibrate_auto()
# Equip tool
robot.update_tool()
# Launching main process
process(client)
# Ending
robot.go_to_sleep()
# Releasing connection
robot.close_connection()
API Documentation
Master controls with PyNiryo with full the detailed functions here (index.html#document-
source/api_doc/api).
This file presents the different Command functions, Enums & Python object classes available with the API.
Command functions are used to deal directly with the robot. It could be move_joints() ,
get_hardware_status() vision_pick() , or also run_conveyor()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Enums are used to pass speci c arguments to functions. For instance PinState ,
ConveyorDirection , …
Python object classes, as PoseObject , ease some operations
Command functions
This section references all existing functions to control your robot, which includes:
All functions to control the robot are accessible via an instance of the class NiryoRobot
robot = NiryoRobot(<robot_ip_address>)
TCP Connection
Main purpose functions
Joints & Pose
Saved Poses
Pick & Place
Trajectories
Tools
Hardware
Conveyor
Vision
TCP Connection
NiryoRobot.connect(ip_address )
Parameters:
ip_address (str (https://docs.python.org/3/library/stdtypes.html#str)) – IP Address
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.close_connection()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.calibrate(calibrate_mode)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Calibrate (manually or automatically) motors. Automatic calibration will do nothing if motors are
already calibrated
Parameters:
calibrate_mode (CalibrateMode) – Auto or Manual
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.calibrate_auto()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.need_calibration()
Return type:
bool (https://docs.python.org/3/library/functions.html#bool)
NiryoRobot.get_learning_mode()
Returns:
True if learning mode is on
Return type:
bool (https://docs.python.org/3/library/functions.html#bool)
NiryoRobot.set_learning_mode(enabled)
Parameters:
enabled (bool (https://docs.python.org/3/library/functions.html#bool)) – True or False
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.set_arm_max_velocity(percentage_speed )
Parameters:
percentage_speed (int (https://docs.python.org/3/library/functions.html#int)) – Should be between
1 & 100
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.set_jog_control(enabled)
Parameters:
enabled (bool (https://docs.python.org/3/library/functions.html#bool)) – True or False
Return type:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
None (https://docs.python.org/3/library/constants.html#None)
static NiryoRobot.wait(duration )
Parameters:
duration (float (https://docs.python.org/3/library/functions.html#float)) – duration in seconds
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_joints()
joints = robot.get_joints()
joints = robot.joints
Returns:
List of joints value
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]
NiryoRobot.get_pose()
Get end e ector link pose as [x, y, z, roll, pitch, yaw]. x, y & z are expressed in meters / roll, pitch & yaw
are expressed in radians You can also use a getter
pose = robot.get_pose()
pose = robot.pose
Return type:
PoseObject (index.html#api.objects.PoseObject)
NiryoRobot.get_pose_quat()
Returns:
Position and quaternion coordinates concatenated in a list : [x, y, z, qx, qy, qz, qw]
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]
NiryoRobot.move_joints(*args)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)]]) – either 6 args (1 for each joints) or a list
of 6 joints
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.move_pose(*args)
Move robot end e ector pose to a (x, y, z, roll, pitch, yaw) pose. x, y & z are expressed in meters / roll,
pitch & yaw are expressed in radians
Parameters:
args (Union[tuple (https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.move_linear_pose(*args)
Move robot end effector pose to a (x, y, z, roll, pitch, yaw) pose with a linear trajectory
Parameters:
args (Union[tuple (https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.shift_pose(axis, shift_value)
Parameters:
axis (RobotAxis) – Axis along which the robot is shifted
shift_value (float (https://docs.python.org/3/library/functions.html# oat)) – In meter for X/Y/Z
and radians for roll/pitch/yaw
Return type:
None (https://docs.python.org/3/library/constants.html#None)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
NiryoRobot.shift_linear_pose(axis, shift_value)
Shift robot end effector pose along one axis, with a linear trajectory
Parameters:
axis (RobotAxis) – Axis along which the robot is shifted
shift_value (float (https://docs.python.org/3/library/functions.html# oat)) – In meter for X/Y/Z
and radians for roll/pitch/yaw
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.jog_joints(*args)
Jog robot joints’. Jog corresponds to a shift without motion planning. Values are expressed in radians.
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)]]) – either 6 args (1 for each joints) or a list
of 6 joints offset
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.jog_pose(*args)
Jog robot end e ector pose Jog corresponds to a shift without motion planning Arguments are [dx, dy,
dz, d_roll, d_pitch, d_yaw] dx, dy & dz are expressed in meters / d_roll, d_pitch & d_yaw are expressed
in radians
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)]]) – either 6 args (1 for each coordinates) or
a list of 6 offset
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.move_to_home_pose()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.go_to_sleep()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.forward_kinematics(*args)
Compute forward kinematics of a given joints configuration and give the associated spatial pose
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)]]) – either 6 args (1 for each joints) or a list
of 6 joints
Return type:
PoseObject (index.html#api.objects.PoseObject)
NiryoRobot.inverse_kinematics(*args)
Parameters:
args (Union[tuple (https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Returns:
List of joints value
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]
Saved Poses
NiryoRobot.get_pose_saved(pose_name)
Parameters:
pose_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – Pose name in robot’s
memory
Returns:
Pose associated to pose_name
Return type:
PoseObject (index.html#api.objects.PoseObject)
NiryoRobot.save_pose(pose_name, *args)
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
None (https://docs.python.org/3/library/constants.html#None)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
NiryoRobot.delete_pose(pose_name)
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_saved_pose_list()
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[str
(https://docs.python.org/3/library/stdtypes.html#str)]
NiryoRobot.pick_from_pose(*args)
A picking is described as :
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.place_from_pose(*args)
A placing is described as :
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
None (https://docs.python.org/3/library/constants.html#None)
Parameters:
pick_pose (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – Pick Pose : [x, y, z, roll, pitch, yaw] or PoseObject
place_pos (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – Place Pose : [x, y, z, roll, pitch, yaw] or PoseObject
dist_smoothing (float (https://docs.python.org/3/library/functions.html# oat)) – Distance from
waypoints before smoothing trajectory
Return type:
None (https://docs.python.org/3/library/constants.html#None)
Trajectories
NiryoRobot.get_trajectory_saved(trajectory_name)
Returns:
Trajectory
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]
NiryoRobot.execute_trajectory_from_poses(list_poses, dist_smoothing=0.0)
Parameters:
list_poses (list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]) – List of [x,y,z,qx,qy,qz,qw] or list of
[x,y,z,roll,pitch,yaw]
dist_smoothing (float (https://docs.python.org/3/library/functions.html# oat)) – Distance from
waypoints before smoothing trajectory
Return type:
None (https://docs.python.org/3/library/constants.html#None)
Parameters:
list_pose_joints (list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]) – List of [x,y,z,qx,qy,qz,qw] or list of
[x,y,z,roll,pitch,yaw] or a list of [j1,j2,j3,j4,j5,j6]
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.execute_trajectory_saved(trajectory_name)
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.save_trajectory(trajectory_name, list_poses)
Parameters:
list_poses (list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]) – List of [x,y,z,qx,qy,qz,qw] or list of
[x,y,z,roll,pitch,yaw]
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.delete_trajectory(trajectory_name)
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_saved_trajectory_list()
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[str
(https://docs.python.org/3/library/stdtypes.html#str)]
Tools
NiryoRobot.get_current_tool_id()
Return type:
ToolID
NiryoRobot.update_tool()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
NiryoRobot.grasp_with_tool()
Grasp with tool | This action correspond to | - Close gripper for Grippers | - Pull Air for Vacuum pump
| - Activate for Electromagnet
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.release_with_tool()
Release with tool | This action correspond to | - Open gripper for Grippers | - Push Air for Vacuum
pump | - Deactivate for Electromagnet
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.open_gripper(speed=500)
Parameters:
speed (int (https://docs.python.org/3/library/functions.html#int)) – Between 100 & 1000
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.close_gripper(speed=500)
Parameters:
speed (int (https://docs.python.org/3/library/functions.html#int)) – Between 100 & 1000
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.pull_air_vacuum_pump()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.push_air_vacuum_pump()
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.setup_electromagnet(pin_id)
Parameters:
pin_id (PinID) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.activate_electromagnet(pin_id)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Parameters:
pin_id (PinID) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.deactivate_electromagnet(pin_id)
Parameters:
pin_id (PinID) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.enable_tcp(enable=True)
Enables or disables the TCP function (Tool Center Point). If activation is requested, the last recorded
TCP value will be applied. The default value depends on the gripper equipped. If deactivation is
requested, the TCP will be coincident with the tool_link.
Parameters:
enable (Bool) – True to enable, False otherwise.
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.set_tcp(*args)
Activates the TCP function (Tool Center Point) and de nes the transformation between the tool_link
frame and the TCP frame.
Parameters:
args (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], tuple
(https://docs.python.org/3/library/stdtypes.html#tuple)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) – either 6 args (1 for each coordinates) or a list of 6
coordinates or a PoseObject
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.reset_tcp()
Reset the TCP (Tool Center Point) transformation. The TCP will be reset according to the tool equipped.
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.tool_reboot()
Reboot the motor of the tool equipped. Useful when an Overload error occurs. (cf HardwareStatus)
Return type:
None (https://docs.python.org/3/library/constants.html#None)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Hardware
NiryoRobot.set_pin_mode(pin_id, pin_mode)
Parameters:
pin_id (PinID) –
pin_mode (PinMode) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.digital_write(pin_id, digital_state )
Parameters:
pin_id (PinID) –
digital_state (PinState) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.digital_read(pin_id)
Parameters:
pin_id (PinID) –
Return type:
PinState
NiryoRobot.get_hardware_status()
Get hardware status : Temperature, Hardware version, motors names & types …
Returns:
Infos contains in a HardwareStatusObject
Return type:
HardwareStatusObject (index.html#api.objects.HardwareStatusObject)
NiryoRobot.get_digital_io_state()
Returns:
List of DigitalPinObject instance
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[DigitalPinObject
(index.html#api.objects.DigitalPinObject)]
Conveyor
NiryoRobot.set_conveyor()
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Returns:
New conveyor ID
Return type:
ConveyorID
NiryoRobot.unset_conveyor(conveyor_id )
Parameters:
conveyor_id (ConveyorID) – Basically, ConveyorID.ONE or ConveyorID.TWO
Parameters:
conveyor_id (ConveyorID) –
speed (int (https://docs.python.org/3/library/functions.html#int)) –
direction (ConveyorDirection) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.stop_conveyor(conveyor_id )
Parameters:
conveyor_id (ConveyorID) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
Parameters:
conveyor_id (ConveyorID) –
control_on (bool (https://docs.python.org/3/library/functions.html#bool)) –
speed (int (https://docs.python.org/3/library/functions.html#int)) – New speed which is a
percentage of maximum speed
direction (ConveyorDirection) – Conveyor direction
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_connected_conveyors_id()
Returns:
List of the connected conveyors’ ID
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[ConveyorID]
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Vision
NiryoRobot.get_img_compressed()
Get image from video stream in a compressed format. Use uncompress_image from the vision
package to uncompress it
Returns:
string containing a JPEG compressed image
Return type:
str (https://docs.python.org/3/library/stdtypes.html#str)
NiryoRobot.set_brightness(brightness_factor)
Parameters:
brightness_factor (float (https://docs.python.org/3/library/functions.html# oat)) – How much to
adjust the brightness. 0.5 will give a darkened image, 1 will give the original image while 2 will
enhance the brightness by a factor of 2.
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.set_contrast(contrast_factor)
Parameters:
contrast_factor (float (https://docs.python.org/3/library/functions.html# oat)) – While a factor of 1
gives original image. Making the factor towards 0 makes the image greyer, while factor>1 increases
the contrast of the image.
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.set_saturation(saturation_factor)
Parameters:
saturation_factor (float (https://docs.python.org/3/library/functions.html# oat)) – How much to
adjust the saturation. 0 will give a black and white image, 1 will give the original image while 2 will
enhance the saturation by a factor of 2.
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_image_parameters()
Get last stream image parameters: Brightness factor, Contrast factor, Saturation factor.
Brightness factor: How much to adjust the brightness. 0.5 will give a darkened image, 1 will give the
original image while 2 will enhance the brightness by a factor of 2.
Contrast factor: A factor of 1 gives original image. Making the factor towards 0 makes the image
greyer, while factor>1 increases the contrast of the image.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Saturation factor: 0 will give a black and white image, 1 will give the original image while 2 will enhance
the saturation by a factor of 2.
Returns:
Brightness factor, Contrast factor, Saturation factor
Return type:
float (https://docs.python.org/3/library/functions.html# oat), float
(https://docs.python.org/3/library/functions.html#float), float
(https://docs.python.org/3/library/functions.html#float)
Given a pose (x_rel, y_rel, yaw_rel) relative to a workspace, this function returns the robot pose in which
the current tool will be able to pick an object at this pose.
The height_o set argument (in m) de nes how high the tool will hover over the workspace. If
height_offset = 0, the tool will nearly touch the workspace.
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – name of the
workspace
height_offset (float (https://docs.python.org/3/library/functions.html# oat)) – o set between
the workspace and the target height
x_rel (float (https://docs.python.org/3/library/functions.html# oat)) – x relative pose (between 0
and 1)
y_rel (float (https://docs.python.org/3/library/functions.html# oat)) – y relative pose (between 0
and 1)
yaw_rel (float (https://docs.python.org/3/library/functions.html#float)) – Angle in radians
Returns:
target_pose
Return type:
PoseObject (index.html#api.objects.PoseObject)
First detects the speci ed object using the camera and then returns the robot pose in which the object
can be picked with the current tool
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – name of the
workspace
height_offset (float (https://docs.python.org/3/library/functions.html# oat)) – o set between
the workspace and the target height
shape (ObjectShape) – shape of the target
color (ObjectColor) – color of the target
Returns:
object_found, object_pose, object_shape, object_color
Return type:
(bool (https://docs.python.org/3/library/functions.html#bool), PoseObject
(index.html#api.objects.PoseObject), ObjectShape, ObjectColor)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Picks the specified object from the workspace. This function has multiple phases:
Example:
robot = NiryoRobot(ip_address="x.x.x.x")
robot.calibrate_auto()
robot.move_pose(<observation_pose>)
obj_found, shape_ret, color_ret = robot.vision_pick(<workspace_name>,
height_offset=0.0,
shape=ObjectShape.ANY,
color=ObjectColor.ANY)
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – name of the
workspace
height_offset (float (https://docs.python.org/3/library/functions.html# oat)) – o set between
the workspace and the target height
shape (ObjectShape) – shape of the target
color (ObjectColor) – color of the target
Returns:
object_found, object_shape, object_color
Return type:
(bool (https://docs.python.org/3/library/functions.html#bool), ObjectShape, ObjectColor)
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – name of the
workspace
height_offset (float (https://docs.python.org/3/library/functions.html# oat)) – o set between
the workspace and the target height
shape (ObjectShape) – shape of the target
color (ObjectColor) – color of the target
Returns:
object_found, object_shape, object_color
Return type:
(bool (https://docs.python.org/3/library/functions.html#bool), ObjectShape, ObjectColor)
Parameters:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Returns:
object_found, object_pose, object_shape, object_color
Return type:
(bool (https://docs.python.org/3/library/functions.html#bool), PoseObject
(index.html#api.objects.PoseObject), str (https://docs.python.org/3/library/stdtypes.html#str), str
(https://docs.python.org/3/library/stdtypes.html#str))
NiryoRobot.get_camera_intrinsics()
Returns:
camera intrinsics, distortions coefficients
Return type:
(list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)] ] , list
(https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]])
Save workspace by giving the poses of the robot to point its 4 corners with the calibration Tip. Corners
should be in the good order. Markers’ pose will be deduced from these poses
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – workspace name,
maximum lenght 30 char.
pose_origin (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) –
pose_2 (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) –
pose_3 (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) –
pose_4 (Union[list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)], PoseObject
(index.html#api.objects.PoseObject)]) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Save workspace by giving the points of worskpace’s 4 corners. Points are written as [x, y, z] Corners
should be in the good order.
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – workspace name,
maximum lenght 30 char.
point_origin (list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]) –
point_2 (list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]) –
point_3 (list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]) –
point_4 (list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.delete_workspace(workspace_name)
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) –
Return type:
None (https://docs.python.org/3/library/constants.html#None)
NiryoRobot.get_workspace_ratio(workspace_name)
Parameters:
workspace_name (str (https://docs.python.org/3/library/stdtypes.html#str)) –
Return type:
float (https://docs.python.org/3/library/functions.html#float)
NiryoRobot.get_workspace_list()
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[str
(https://docs.python.org/3/library/stdtypes.html#str)]
Enums
robot.shift_pose(RobotAxis.Y, 0.15)
List of enums:
CalibrateMode
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
RobotAxis
ToolID
PinMode
PinState
PinID
ConveyorID
ConveyorDirection
ObjectColor
ObjectShape
class CalibrateMode(value)
AUTO= 0
MANUAL= 1
class RobotAxis(value)
X= 0
Y= 1
Z= 2
ROLL= 3
PITCH= 4
YAW= 5
class ToolID(value)
NONE= 0
GRIPPER_1= 11
GRIPPER_2= 12
GRIPPER_3= 13
ELECTROMAGNET_1= 30
VACUUM_PUMP_1= 31
class PinMode(value)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
INPUT= 0
OUTPUT= 1
class PinState(value)
LOW= 0
HIGH= 1
class PinID(value)
GPIO_1A= 0
GPIO_1B= 1
GPIO_1C= 2
GPIO_2A= 3
GPIO_2B= 4
GPIO_2C= 5
class ConveyorID(value)
NONE= 0
ID_1= 12
ID_2= 13
class ConveyorDirection(value)
FORWARD= 1
BACKWARD= -1
class ObjectColor(value)
RED= 'RED'
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
BLUE= 'BLUE'
GREEN= 'GREEN'
ANY= 'ANY'
class ObjectShape(value)
SQUARE= 'SQUARE'
CIRCLE= 'CIRCLE'
ANY= 'ANY'
Special objects
Create a new pose from copying from copying actual pose with offsets
Return type:
PoseObject (index.html#api.objects.PoseObject)
to_list()
Return a list [x, y, z, roll, pitch, yaw] corresponding to the pose’s parameters
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
This le illustrates few image processing pipeline using vision module from niryo_edu package. This
module is based in OpenCV (https://opencv.org/) and its functions are detailed in Functions
documentation (index.html#document-source/vision/image_processing_api).
The package niryo_edu comes up with the vision module which contains image processing functions
including thresholding, blob detection, …
To use it, add to your imports from pyniryo.vision import * .
Note
It is also possible to merge both import lines by using from pyniryo import * .
We are firstly going to take a look at robot’s functions which can be find at API - Vision (index.html#vision)
Ned can share its video stream through TCP. As sending raw images will lead to heavy packets which can
saturate the network, it sends compressed images. You access it through the robot’s function:
get_img_compressed() (index.html#api.tcp_client.NiryoRobot.get_img_compressed). Once your
image is received, you rstly need to uncompress via uncompress_image()
(index.html#vision.image_functions.uncompress_image) and you can then display it with
show_img_and_wait_close() (index.html#vision.image_functions.show_img_and_wait_close).
# Connecting to robot
robot = NiryoRobot("10.10.10.10")
# Getting image
img_compressed = robot.get_img_compressed()
# Uncompressing image
img = uncompress_image(img_compressed)
# Displaying
show_img_and_wait_close("img_stream", img)
Note
In this section, we are going to display the raw video stream & the undistorted video stream.
As Ned’s camera is passing raw images to the robot, these images are distorted due to the camera lens. In
order to undistort them, we need to use Ned’s camera intrinsics.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Once, we have both raw & undistorted images, we can concatenate them in order to display them in once
with concat_imgs() (index.html#vision.image_functions.concat_imgs). Finally, we display the image
show_img() (index.html#vision.image_functions.show_img).
observation_pose = PoseObject(
x=0.18, y=0.0, z=0.35,
roll=0.0, pitch=1.57, yaw=-0.2,
)
# Connecting to robot
robot = NiryoRobot("10.10.10.10")
robot.calibrate_auto()
# - Display
# Concatenating raw image and undistorted image
concat_ims = concat_imgs((img_raw, img_undistort))
# Showing images
key = show_img("Images raw & undistorted", concat_ims, wait_ms=30)
if key in [27, ord("q")]: # Will break loop if the user press Escape or Q
break
Note
Attention
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Attention
Color thresholding
Color thresholding is very useful in order to detect object with an uniform color. The implemented
function to realize this operation is threshold_hsv()
(index.html#vision.image_functions.threshold_hsv).
show_img("img_threshold_red", img_threshold_red)
show_img_and_wait_close("img_threshold_blue", img_threshold_blue)
Images result
Blue
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Red
Morphological transformations
Morphological transformations are some simple operations based on the image shape. It is normally
performed on binary images. It needs two inputs, one is our original image, second one is called
structuring element or kernel which decides the nature of operation. Two basic morphological operators
a r e Erosion (https://en.wikipedia.org/wiki/Mathematical_morphology#Erosion) and Dilation
(https://en.wikipedia.org/wiki/Mathematical_morphology#Dilation).
show_img("img_threshold", img_threshold)
show_img("img_erode", img_erode)
show_img_and_wait_close("img_close", img_close)
Images result
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
None
Erode
Close
Contours finder
Contours can be explained simply as a curve joining all the continuous points (along the boundary), having
same color or intensity. The contours are a useful tool for shape analysis and object detection and
recognition. See more on OpenCV Documentation
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
(https://docs.opencv.org/3.4/d3/d05/tutorial_py_table_of_contents_contours.html).
The code to extract and draw the 3 biggest contours from an image is the following:
cnts = biggest_contours_finder(img_threshold, 3)
show_img("init", img_threshold)
show_img_and_wait_close("img with contours", img_contours)
Images result
Thresh +
Opening
3
contours
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
In order to catch an object, we need to nd a pose from where the end e ector can grasp the object. The
following method uses contours which have been found in the previous section and nds their barycenter
and orientation via the functions get_contour_barycenter()
(index.html#vision.image_functions.get_contour_barycenter) & get_contour_angle()
(index.html#vision.image_functions.get_contour_angle).
cnt = biggest_contour_finder(img_threshold)
cnt_barycenter = get_contour_barycenter(cnt)
cnt_angle = get_contour_angle(cnt)
Images result
Thresh +
Opening
Barycenter
+ Angle
Note
The drawn vector is normal to the contour’s length because we want Ned to catch the object by the
width rather than the length. Indeed, it leads to least cases where the gripper cannot open enough.
Markers extraction
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Images result
Original
Extracted
Debug mode
If Ned’s functions are failing, you can use Debug functions which are debug_threshold_color()
(index.html#vision.image_functions.debug_threshold_color) & debug_markers()
(index.html#vision.image_functions.debug_markers) in order to display what the robot sees.
show_img("init", img_test)
show_img("debug_color", debug_color)
show_img_and_wait_close("debug_markers", debug_markers_im)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Images result
Original
Debug
red
Debug
Markers
Now that you are a master in image processing, let’s look at full examples.
In the current state, the following code will display the video stream and the extracted workspace image.
You can add your own image processing functions maybe to apply additional process.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# Connecting to robot
robot = NiryoRobot("10.10.10.10")
robot.calibrate_auto()
# - Display
# Concatenating raw image and undistorted image
concat_ims = concat_imgs((img_raw, img_undistort))
# Concatenating extracted workspace with markers annotation
if img_workspace is not None:
resized_img_workspace = resize_img(img_workspace, height=res_img_markers.shape[0])
res_img_markers = concat_imgs((res_img_markers, resized_img_workspace))
# Showing images
show_img("Images raw & undistorted", concat_ims)
key = show_img("Markers", res_img_markers, wait_ms=30)
if key in [27, ord("q")]: # Will break loop if the user press Escape or Q
break
You may want to send coordinate to Ned in order to pick the object of your choice! To do that, use the
f u n c t i o n get_target_pose_from_rel()
(index.html#api.tcp_client.NiryoRobot.get_target_pose_from_rel) which converts a relative pose in the
workspace into a pose in the robot’s world!
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
# Initializing variables
obj_pose = None
try_without_success = 0
count = 0
color_hsv_setting = ColorHSV.ANY.value
img_compressed = robot.get_img_compressed()
img = uncompress_image(img_compressed)
img = undistort_image(img, mtx, dist)
# extracting working area
im_work = extract_img_workspace(img, workspace_ratio=1.0)
if im_work is None:
print("Unable to find markers")
try_without_success += 1
if display_stream:
cv2.imshow("Last image saw", img)
cv2.waitKey(25)
continue
if display_stream:
show_img("Last image saw", img, wait_ms=0)
show_img("Image thresh", img_thresh, wait_ms=30)
# Getting biggest contour/blob from threshold image
contour = biggest_contour_finder(img_thresh)
if contour is None or len(contour) == 0:
print("No blob found")
obj_found = False
else:
img_thresh_rgb = cv2.cvtColor(img_thresh, cv2.COLOR_GRAY2BGR)
draw_contours(img_thresh_rgb, [contour])
show_img("Image thresh", img_thresh, wait_ms=30)
if not obj_found:
try_without_success += 1
continue
# Placing
robot.place_from_pose(place_pose)
try_without_success = 0
count += 1
Functions documentation
This file presents the different functions and Enums Image Processing available for image processing.
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Pure image processing are used to deal with the thresholding, contours detection, ..
Workspaces wise section contains functions to extract workspace and deal with the relative position in
the workspace
The section Show allows to display images
Image Editing contains lot of function which can compress images, add text to image, …
Take BGR image (OpenCV imread result) and return thresholded image according to values on HSV
(Hue, Saturation, Value) Pixel will worth 1 if a pixel has a value between min_v and max_v for all
channels
Parameters:
img (numpy.array) – image BGR if rgb_space = False
list_min_hsv (list (https://docs.python.org/3/library/stdtypes.html#list)[int
(https://docs.python.org/3/library/functions.html#int)]) – list corresponding to
[min_value_H,min_value_S,min_value_V]
list_max_hsv (list (https://docs.python.org/3/library/stdtypes.html#list)[int
(https://docs.python.org/3/library/functions.html#int)]) – list corresponding to
[max_value_H,max_value_S,max_value_V]
use_s_prime (bool (https://docs.python.org/3/library/functions.html#bool)) – True if you want to
use S channel as S’ = S x V else classic
reverse_hue (bool (https://docs.python.org/3/library/functions.html#bool)) – Useful for Red
color cause it is at both extremum
Returns:
threshold image
Return type:
numpy.array
image_functions.debug_threshold_color(color_hsv)
Parameters:
img (numpy.array) – OpenCV image
color_hsv (ColorHSV (index.html#vision.enums.ColorHSV)) – Color used for debug
Returns:
Masked image
Return type:
numpy.array
Parameters:
im_thresh (numpy.array) – Black & White Image
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Returns:
image after processing
Return type:
numpy.array
image_functions.get_contour_barycenter()
Parameters:
contour (OpenCV Contour) –
Returns:
Barycenter X, Barycenter Y
Return type:
int (https://docs.python.org/3/library/functions.html#int), int
(https://docs.python.org/3/library/functions.html#int)
image_functions.get_contour_angle()
Return orientation of a contour according to the smallest side in order to be well oriented for gripper
Parameters:
contour (OpenCV Contour) – contour
Returns:
Angle in radians
Return type:
float (https://docs.python.org/3/library/functions.html#float)
image_functions.biggest_contour_finder()
image_functions.biggest_contours_finder(nb_contours_max=3)
Parameters:
img (numpy.array) – Binary Image
nb_contours_max (int (https://docs.python.org/3/library/functions.html#int)) – maximal number
of contours which will be returned
Returns:
biggest contours found
Return type:
list (https://docs.python.org/3/library/stdtypes.html#list)[OpenCV Contour]
image_functions.draw_contours(contours)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Parameters:
img (numpy.array) – Image
contours (list (https://docs.python.org/3/library/stdtypes.html#list)[OpenCV Contour]) – contours
list
Returns:
Image with drawing
Return type:
numpy.array
Workspaces wise
image_functions.extract_img_workspace(workspace_ratio=1.0)
Parameters:
img (numpy.array) – OpenCV image which contain 4 Niryo’s markers
workspace_ratio (float (https://docs.python.org/3/library/functions.html#float)) – Ratio between
the width and the height of the area represented by the markers
Returns:
extracted and warped working area image
Return type:
numpy.array
image_functions.debug_markers(workspace_ratio=1.0)
Parameters:
img (numpy.array) – OpenCV image which contain Niryo’s markers
workspace_ratio (float (https://docs.python.org/3/library/functions.html#float)) – Ratio between
the width and the height of the area represented by the markers
Returns:
(status, annotated image)
Return type:
numpy.array
image_functions.relative_pos_from_pixels(x_pixels, y_pixels)
Parameters:
img (numpy.array) – Image where the object is detected
x_pixels (int (https://docs.python.org/3/library/functions.html#int)) – coordinate X
y_pixels (int (https://docs.python.org/3/library/functions.html#int)) – coordinate Y
Returns:
X relative, Y relative
Return type:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Show
image_functions.show_img_and_check_close(img)
Parameters:
window_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – window’s name
img (numpy.array) – Image
Returns:
boolean indicating if the user wanted to leave
Return type:
bool (https://docs.python.org/3/library/functions.html#bool)
image_functions.show_img(img, wait_ms=1)
Parameters:
window_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – window’s name
img (numpy.array) – Image
wait_ms (int (https://docs.python.org/3/library/functions.html#int)) – Wait time in milliseconds
Returns:
value of the key pressed during the display
Return type:
int (https://docs.python.org/3/library/functions.html#int)
image_functions.show_img_and_wait_close(img)
Parameters:
window_name (str (https://docs.python.org/3/library/stdtypes.html#str)) – window’s name
img (numpy.array) – Image
Returns:
None
Return type:
None (https://docs.python.org/3/library/constants.html#None)
Image Editing
image_functions.compress_image(quality=90)
Parameters:
img (numpy.array) – OpenCV Image
quality (int (https://docs.python.org/3/library/functions.html#int)) – integer between 1 - 100. The
higher it is, the less information will be lost, but the heavier the compressed image will be
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Returns:
status & string representing compressed image
Return type:
bool (https://docs.python.org/3/library/functions.html#bool), str
(https://docs.python.org/3/library/stdtypes.html#str)
image_functions.uncompress_image()
Parameters:
compressed_image (str (https://docs.python.org/3/library/stdtypes.html#str)) – compressed image
Returns:
OpenCV image
Return type:
numpy.array
image_functions.add_annotation_to_image(text, write_on_top=True)
Parameters:
img (numpy.array) – Image
text (str (https://docs.python.org/3/library/stdtypes.html#str)) – text string
write_on_top (bool (https://docs.python.org/3/library/functions.html#bool)) – if you write the
text on top
Returns:
img with text written on it
Return type:
numpy.array
image_functions.undistort_image(mtx, dist)
Parameters:
img (numpy.array) – Raw Image
mtx (list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]) – Camera Intrinsics matrix
dist (list (https://docs.python.org/3/library/stdtypes.html#list)[list
(https://docs.python.org/3/library/stdtypes.html#list)[float
(https://docs.python.org/3/library/functions.html#float)]]) – Distortion Coefficient
Returns:
Undistorted image
Return type:
numpy.array
Resize an image. The user should precise only width or height if he wants to keep image’s ratio
Parameters:
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
Returns:
resized image
Return type:
numpy.array
image_functions.concat_imgs(axis=1)
Parameters:
tuple_imgs (tuple (https://docs.python.org/3/library/stdtypes.html#tuple)[numpy.array]) – tuple
of images
axis (int (https://docs.python.org/3/library/functions.html#int)) – 0 means vertically and 1 means
horizontally
Returns:
Concat image
Return type:
numpy.array
List of enums:
ColorHSV
ColorHSVPrime
ObjectType
MorphoType
KernelType
class ColorHSV(value)
class ColorHSVPrime(value)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
class ObjectType(value)
An enumeration.
SQUARE= 4
TRIANGLE= 3
CIRCLE= -1
ANY= 0
class MorphoType(value)
An enumeration.
ERODE= 0
DILATE= 1
OPEN= 2
CLOSE= 3
class KernelType(value)
An enumeration.
RECT= 0
ELLIPSE= 2
CROSS= 1
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS
PyNiryo (v1.0.5)
2021, Niryo All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Niryo SAS