1. Introduction
In the past 20 years, as a result of the trend of containerization of international trade, the throughput and tonnage of container ports have been gradually improved. Moreover, due to the influence of COVID-19, a large number of ships berthed at port to wait for unloading, thus requiring an improvement in port handling capacity [
1]. Meanwhile, as a type of container handling equipment, port cranes have strong load periodicity, wide moving range, and large loads during operation, which indicate the need for higher requirements for its safety and reliability [
2].
In order to improve the safety and reliability of cranes, many researches have studied the crane monitoring system [
3,
4]. These studies mainly focused on the aspects of information communication, system integration, and visualization. In terms of information communication, RFID, ZigBee, Wi-Fi, 5G and other technologies were applied in various crane monitoring systems. For example, Lee et al. [
5] combined RFID and machine vision technology to provide crane operators with more detailed information about the working environment and objects to be transported during operation. Gao et al. [
6] designed the swing angle measurement system of the crane based on a micro-electro mechanical system (MEMS) and sent the swing angle signal by ZigBee. Yang et al. [
7] used multi-type sensors to collect the crane’s information, which was transmitted by Wi-Fi technology, and used MATLAB and ANSYS to analyze and visually display the stress and deformation of the crane. Shi et al. [
8] used narrow band Internet of Things (NB-IoT) to transmit the data of the crane load and the motor current, which were uploaded to the cloud database for fault diagnosis and condition monitoring. Awad et al. [
9] discussed the different applications of smart sensor networks on the cranes, and used an IoT-PLC control system to monitor the crane on the cloud. Li et al. [
10] integrated 5G network and edge computing technology into the remote control of the crane, reducing the end-to-end control delay. In terms of system integration and visualization, digital mock-up technology has developed rapidly under the impetus of computer technology. For example, Price et al. [
11] obtained crane working environment data through encoders, cameras, and laser scanning, so as to monitor the cranes in real time. They also established a graphic user interface (GUI) in Unity 3D to provide operators with three-dimensional visualization of crane status data, and used multi-sensor data for obstacle distance estimation and collision warning. Li and Liu [
12] designed a data acquisition and transmission system based on a CAN bus, established a virtual model of tower crane in a parametric and modular manner, and used database data to drive the virtual crane movement. At the same time, they integrated the construction site data and virtual model to replace the camera monitoring system.
As a result of the development of a new generation of information and communication technologies, such as multi-sensor fusion technology, IoT, and cloud computing, digital twins have made considerable progress as a new technology that breaks the barrier between virtual and real data [
13,
14]. The digital twin is the mirror image of a physical entity in an information system through data and simulation software. It can be used to simulate, monitor, diagnose, and predict the formation process, state, and behavior of physical entities in the real environment, and provide more decision-making feedback for the actual system. The cyber-physical system (CPS) can realize the interactive linkage between the information virtual entity and the physical entity by constructing a closed-loop channel for data interaction between the information space and the physical space [
15]. The digital twin and CPS have different characteristics in aspects of cyber-physical mapping, hierarchy, and core elements [
16]. Simulation technology is a model technology that applies simulation hardware and software to reflect a system’s behavior or process with the help of numerical calculations and problem solving. Traditional simulation technology can only simulate the physical world offline, without the function of analysis and optimization [
17]. Compared with traditional simulation, the digital twin relies on methods including simulation, measurement, and data analysis to perceive, diagnose, and predict the physical entity state, so as to optimize the physical entity and update the digital model to better reflect the current state of device. Compared with the hardware-in-the-loop simulation system, the digital twin adopts data for the whole system to drive the virtual system, and obtains more comprehensive information and feedback [
18]. The digital twin acts in different application forms in different fields, but its model should generally conform to the basic model of real space–virtual space–linking mechanism proposed by Grieves [
19]. Their common feature is the effective feedback provided to decision makers, physical systems, and information systems.
The digital twin has been utilized in the management and design of product lifecycles. Tao et al. [
20] proposed a new method of product design, manufacture, and service driven by the digital twin and its corresponding application framework to solve the problem of serious data fragmentation between the product life cycle data and the various stages of the cycle in the virtual and real space. Liu et al. [
21] established a digital twin model in the machining process based on biomimicry, simulating machining process from the three aspects of geometry, behavior, and environment to support machining decision making. Liu et al. [
22] proposed a configuration, motion, control, and optimization (CMCO) architecture, and used the established digital twin model to conduct a hardware-in-the-loop simulation during the scheme design stage to detect and address potential design errors.
The digital twin is also used to improve monitoring systems. For example, Soares et al. [
23] established a digital twin model of a sugarcane processing device, which has the ability to collect real-time data, such as the employee’s location and the device status for analysis, and provide twin information for the real-time decision-making process. Moi et al. [
24] used the nonlinear finite element analysis method to simulate the stress process of the crane and established a virtual strain gauge to measure the strain data at the crane’s key positions. Lozano [
25] developed a digital twin monitoring system for offshore cranes to provide real-time structural monitoring, and combined real-time sensing data and the digital twin model to realize early warning of the crane’s state. Cai et al. [
26] fused the spindle vibration data and machining feature information into the digital twin three-axis vertical milling machine, so that it can better reflect the working state of the actual milling machine in the simulation machining process. Ding et al. [
27] emphasized the interconnection and interoperability between physical and virtual workshops. They established a digital twin cyber-physical production system (DT-CPPS), which transformed the real-time data of the workshops into engineering information for the decision-making level, and improved the decision-making ability of the intelligent manufacturing system. Liu et al. [
28] proposed a hoisting safety risk management framework based on a digital twin model, which realized the real-time perception and virtual–real interaction of multi-source information during the hoisting process of large-scale prefabricated buildings.
Combined with the application of the digital twin in other fields, it can be found that most of the existing crane monitoring system research has the following main problems:
Generally, only the data chart and the camera are used to reflect the equipment status. Firstly, due to the limited number of cameras, it is impossible to obtain a comprehensive view of the device. Secondly, it is difficult to understand the overall status of the equipment with limited measurement data. Thirdly, there is a lack of simple and effective means of human–machine interaction.
In terms of data collection and interaction, only data transmission methods are generally considered. However, when using hardware devices from different manufacturers, different software drivers need to be redesigned according to the hardware type to collect data, which leads to the low degree of openness of data collection.
The actual operating data and theoretical model data are not fused, resulting in a lack of online data simulation and analysis capabilities and a long prototype system development cycle.
Therefore, this paper proposes a port crane monitoring system framework based on the digital twin to support crane control algorithm testing and online monitoring of its operation process. Aiming at the problem (1), the digital twin crane is constructed from the aspects of geometry, underactuated motion, and anti-swing control. Combined with the data collected by various sensors, the status monitoring interface, and the digital twin model, the current status of the crane can be displayed from multiple viewpoints. In view of problem (2), this paper comprehensively considers the characteristics of various hardware products and designs a data interaction method combined with the information model of OPC UA, which can interact with various data across platforms, with low delay and scalability. In response to problem (3), multi-source data can be collected by multi-sensing methods, and various types of data are exchanged with information models to drive digital twin port cranes online. In addition, through in-depth fusion of virtual and real data, various control algorithms are verified online, reducing algorithm development time. Compared with the existing crane monitoring system, the monitoring system mentioned in this paper has the advantages of simple data collection and high platform openness, and can integrate virtual and real data for control algorithm testing, which improves the automation and intelligence level of port cranes.
2. Port Crane Monitoring System Framework Based on the Digital Twin
The composition framework of the monitoring system proposed in this paper is shown in
Figure 1. In the physical system, the multi-type sensors collect status data of the crane and surrounding environment data, and upload the data to the monitoring system software platform via industrial ethernet to complete the data collection and interaction. The crane control system receives the scheduling commands from the software platform and transmits the commands to each terminal actuator to complete the operation task of the crane. In the information system, through the establishment of a digital twin crane model, the control algorithm can be tested and optimized in advance. During the operation of the crane, the state of the crane is synchronously mapped to the virtual model with real-time data, and the early warning information is fed back to the physical system in time, which can improve the operation safety of the crane. The shape and size of the geometric model are built based on the physical model, and the consistency of the virtual and real motion attributes is ensured through the variables and actions bound by the online data-association model (data-constraint), so as to realize the ‘twin’ in vision and kinematics. The underactuated model describes the dynamic properties of the hoisting system, and the anti-swing control model optimizes the operation commands to reduce the residual swing angle of the lifting load. As participants, technical staff can assemble the crane operation knowledge into a knowledge base to provide data support for big data analysis, and remotely control the crane under necessary conditions according to operation experience. In addition, the software platform of the monitoring system can be used to operate the digital twin crane for pre-job operation training.
When technical staff interacts with the information system, not only can the data chart on the monitoring system software platform be used to view the current crane operation data, but the equipment’s running posture can also be accurately observed through the digital twin crane that is running synchronously. In addition, by using the software platform of the monitoring system, the digital twin crane can be controlled virtually to test the control algorithm, and the crane operation training can be carried out using a suitable good human–machine interface.
The functions in the framework mainly include reproduction of historical operations, control program testing, synchronous mapping simulation, and remote control. reproduction of historical operations means that the digital twin crane performs simulated movements according to the historical data of the database, for managers to analyze emergency handling methods in historical conditions, and operator training. The control program test uses the software platform to send control instructions to drive the digital twin crane, combined with the designed control model to virtually test the control algorithm. A synchronous mapping simulation drives the digital twin crane through actual crane data, continuously updates and maps the movement trajectory of the physical crane, and visually displays the movement status of the crane. Remote control is used to issue control instructions through the software platform to remotely operate the physical crane under specific circumstances.
4. Verification of the Scheme
4.1. Monitoring Process of Digital Twin-Driven RMGC
The digital twin-driven RMGC monitoring system takes the digital twin RMGC as the core, and combines various functional applications to provide a 3D visualization scene and control algorithm test environment for the safe and efficient operation of the RMGC. Thus truly and synchronously reflects the operation status of the physical RMGC, and can perform fault diagnosis based on monitoring data. On this basis, the remote control function can also be used to send control commands to the control system of the physical RMGC to solve the abnormal problem. The real-time monitoring process is shown in
Figure 7.
In the monitoring process, the environmental parameters and the motion properties of each mechanism of the geometric model of the RMGC established by CoppeliaSim (V-REP) should be set first. Then, the control command parameters obtained using the anti-swing control model are tested and optimized in the virtual scene. After the optimization test is completed, the software platform sends the control command parameters to IPC via industrial ethernet with OPC UA as the data link layer, and starts to control the physical RMGC for operation. During operation, the multi-sensor acquisition and interaction system collects the real-time position, speed, stress, vibration, sling swing angle, and other multi-source heterogeneous data of the RMGC for analysis. Combined with CoppeliaSim (V-REP) synchronous simulation, the state of the RMGC can be monitored from multiple angles. If an abnormal state occurs, the software platform will display alarm information and send a signal to suspend the operation to avoid possible injury to the RMGC or operators. After troubleshooting the corresponding exceptions, the operation task can be continued until the task is completed.
During real-time monitoring, the data acquisition system uploads the data to the software platform, which stores and visualizes the data separately and connects to CoppeliaSim (V-REP) for synchronous simulation. The stored data can be used to reproduce the historical operation process for further data analysis or conduct pre-job safety training. Among the data, the visual monitoring data in the form of graphs or tables can be combined with the status prompts after data analysis to jointly reflect the current operating status of the RMGC, so that users can view and stop dangerous operations in time. At the same time, the geometric model in CoppeliaSim (V-REP) is simulated iteratively according to the real-time operation data of each component, so as to realize the 3D visual monitoring effect of the RMGC operation.
4.2. OPC UA Data Interaction Delay Analysis
On the client of the software platform, OPC UA and Socket protocols are used to deliver and receive control signals, respectively, so as to verify the advantages of OPC UA communication protocol in data acquisition and processing. Here, the motion control of the crane is taken as an example. Ten groups of experiments on communication time delay were carried out for comparison, starting at the time when the control signal was sent and ending at the time when the truck position data changed in the data packet after data parsing. The experimental results are shown in
Figure 8.
It can be seen from
Figure 8 that the average delay time of OPC UA in receiving data is 0.01670 s, whereas the average delay time of Socket in receiving data is 0.6487 s. This indicates that OPC UA has the characteristic of low delay in the data interaction between the monitoring host and controller designed in this paper.
4.3. Virtual–Real Consistency Test
In order to verify the accuracy and the reliability of the data-constrained driving mode, the crane traveling mechanism, the trolley mechanism, and the lifting mechanism were commanded to move 0.3 m, respectively. The encoder on each servo motor is used to record the current position of each mechanism of the physical RMGC, and the collected data are used to drive the movement of the digital twin RMGC. At the same time, the function sim.getObjectPosition (Entity1, Entity2) in CoppeliaSim (V-REP) is used to obtain the position of the digital twin RMGC. The comparison of the displacement results of each mechanism is shown in
Figure 9.
It can be seen from
Figure 9 that the motion attributes of the virtual and physical entities are basically identical. Since the virtual–real data interaction and the CoppeliaSim (V-REP) simulation calculation take a period of time, the movement in the virtual entity occurs later than that in the physical entity. The delay time is closely related to the performance of the computer and the design of the data interaction program.
4.4. Virtual Test of Anti-Swing Control for Digital Twin-Driven RMGC
Generally, the test of the control algorithm requires the use of a standard controller to directly control the physical entity, which requires good testing conditions and large initial investment. As a result of the introduction of the digital twin, the developer can test the control algorithm in the virtual environment in the early stage of algorithm development, and judge the control effect beforehand. In the middle stage of development, the actual controller can be used to control the digital twin entity to check the control method, thus reducing the construction cost of the prototype. In the later stage of development, the data obtained by the digital twin can be used to reflect the running state of the actual control system. It should be noted that, considering the real-time performance and security of the control system, the digital twin control system in this paper does not directly participate in the real-time control of the actual controller, but provides information for decision makers, and then feeds information back to the actual controller, which is indirect data feedback.
In order to verify the practicability and feasibility of the port crane monitoring system framework based on the digital twin, experiments were carried out according to the designed RMGC monitoring process. The first stage of the process was the virtual optimization test of the control program by the digital twin RMGC. Here, the trolley operation control was tested. The operation conditions were as follows: the trolley operation displacement was 1.0 m, the maximum operation speed was 0.2 m/s, and three groups of experiments were carried out with accelerations of 0.4 m/s2 (condition 1), 0.8 m/s2 (condition 2) and 1.0 m/s2 (condition 3).
According to the anti-swing control model, the monitoring system software platform sends the control parameters under three working conditions to the digital twin RMGC, and then the digital twin RMGC shapes the acceleration of the trolley using the ZV method. The acceleration curves of the trolley before and after shaping are shown in
Figure 10. The virtual test process of the digital twin RMGC is shown in
Figure 11 and the obtained load swing angle curves are shown in
Figure 12.
It can be seen from
Figure 12 that after shaping, the maximum swing angles during operation under the three working conditions are reduced by 74.63%, 73.44%, and 73.05%, respectively, and the maximum residual swing angles are reduced by 99.2%, 98.16%, and 97.7%, respectively. This shows that the digital twin RMGC can simulate and test its operation control and visualize the operation process, laying a foundation for the actual operation control of the physical RMGC.
4.5. Online Monitoring Experiment of RMGC Based on Digital Twin
To further verify the applicability of the proposed monitoring system, the accuracy of the control program test, and the real-time performance of virtual real mapping, an on-line monitoring experimental study of virtual-real combination was carried out. The monitoring system software platform sends the control parameters obtained by simulation optimization to the physical RMGC to remotely control its movement. During the movement, the strain of the important structure of the RMGC, the vibration of the span structure, the running position and speed of each mechanism, and the swing angle of the spreader are measured. At the same time, using the above multi-source heterogeneous data interaction scheme, the digital twin RMGC is directly driven in the form of a data motion joint, and the physical RMGC is monitored online in combination with the mapping results of the digital twin RMGC and the monitoring data of the software platform. The online monitoring system interface is shown in
Figure 13.
To verify the effectiveness and accuracy of the virtual test of the digital twin-based anti-swing control, the real swing angle data (Condition Real) of the physical RMGC measured by the gyroscope under three working conditions were extracted and compared with the swing angle data obtained by the digital twin RMGC virtual test (Condition DTs) using the ZV shaping method. The results are shown in
Figure 14.
As can be seen from
Figure 14, under the three working conditions, the measured swing angle of the physical RMGC is basically consistent with that obtained from the virtual test, indicating that the digital twin RMGC can effectively test the control algorithm.
In addition, to determine the delay of data transmission, in this study, the time stamps carried by the data during 20 data transmissions were statistically compared, and the average time for data to complete one transmission in the system was calculated to be 0.11765 s, which can meet the transmission requirements of RMGC monitoring data.
5. Conclusions
In this paper, a digital twin-based port crane monitoring system framework is proposed, and the feasibility and effectiveness of the proposed framework are verified with the online monitoring of RMGC as an example. Compared with the existing monitoring system, the new monitoring system has a high system integration degree, which enriches the interactive means of the port crane monitoring process and the test environment of the control algorithm, and improves the openness of data collection and the degree of data visualization. In the case study of the monitoring system, a multi-sensor real-time monitoring physical platform of an RMGC was built based on the IEC61131-3 standard. Then, the digital twin model of the RMGC was built, including the geometric model of the RMGC based on CoppeliaSim (V-REP), the dynamic model of the load swing system, the anti-swing control model based on the input shaping method, and the virtual and real data interaction based on OPC UA. The monitoring system has the functions of real-time online monitoring, historical operation reproduction, motion control program testing, synchronous mapping simulation, and remote control. Combined with the multi-sensor acquisition and interaction system, it can realize the synchronous mapping of virtual and real models. The experimental results indicate that the monitoring system proposed in this paper has high real-time performance, and the digital twin RMGC can map the motion process of the physical RMGC and test the control algorithm. In addition, the model visualization effect is good, and the remote control of the software platform is accurate and effective.
In future work, OPC UA will be compared with other industrial protocols in terms of data transmission stability, delay and CPU operation, and storage consumption in the case of massive data interaction, so as to provide a reference for its application in different digital twin scenarios. In addition, although this study found that the proposed digital twin-based operation status monitoring system can improve the safety of the crane, the crane safety problem itself was not discussed in detail in the current paper. Thus, this can be the focus of the following research, including using the digital twin model to study the anti-collision features of the crane.