Abstract
The integration of both linear and circular processes in one production system poses significant challenges. In particular, the reprocessing of end-of-life products is associated with uncertainties at all levels of the production system, from the initial planning and control through to the executing production hardware and intralogistics. To address these challenges, this article presents approaches for self-learning and autonomously adapting production equipment for the Circular Factory. Initially, hardware and software solutions are developed to cover the necessary processes. Reprocessing is covered by modular and reconfigurable manufacturing cells, which also include new process chains such as the combination of additive-subtractive processes. The provided capabilities must be applied to ever new products, for example by transferring human procedures for unknown products to the production equipment. Lastly, an overall robust and dynamic production planning and control system is developed that maintains continuous operation even in unforeseen situations. The resulting highly dynamic overall system is connected by an autonomous intralogistics system.
Zusammenfassung
Die Integration von sowohl linearen und als auch zirkulären Prozessen in einem Produktionssystem stellt eine große Herausforderung dar. Insbesondere die Wiederaufarbeitung von Altprodukten ist mit Unsicherheiten auf allen Ebenen des Produktionssystems verbunden, von der anfänglichen Planung und Steuerung bis hin zur ausführenden Produktionshardware und Intralogistik. Um diesen Herausforderungen zu begegnen, werden in diesem Beitrag Ansätze für selbstlernende und sich autonom adaptierende Produktionsanlagen für die Kreislauffabrik vorgestellt. Zunächst werden Hard- und Softwarelösungen entwickelt, die die notwendigen Prozesse bereitstellen. Die Wiederaufarbeitung wird durch modulare und rekonfigurierbare Fertigungszellen abgedeckt, die auch neue Prozessketten wie die Kombination von additiv-subtraktiven Verfahren umfassen. Die bereitgestellten Fähigkeiten müssen auf immer neue Produkte angewandt werden, zum Beispiel indem menschliche Vorgehensweisen bei unbekannten Produkten auf die Produktionsanlagen übertragen werden. Schließlich wird ein insgesamt robustes und dynamisches Produktionsplanungs- und -steuerungssystem entwickelt, das auch in unvorhergesehenen Situationen einen kontinuierlichen Betrieb aufrechterhält. Das so entstandene hochdynamische Gesamtsystem wird durch ein autonomes Intralogistiksystem verknüpft.
Funding source: Carl Zeiss Foundation
Award Identifier / Grant number: project ID: 471687386
About the authors
Prof. Dr.-Ing. Jürgen Fleischer is director of the research group Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for the areas of intelligent machines and components, agile and reversible production systems and the automation of immature processes.
Prof. Dr.-Ing. Frederik Zanger is director of the research group Manufacturing and Material Technology at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for the area digitalization of process development for additive manufacturing.
Prof. Dr.-Ing. habil. Volker Schulze is director of the research group Manufacturing and Material Technology at wbk Institute of Production Science as well as member of the cooperative board of management at the Institute for Applied Material – Materials Science and Engineering at Karlsruhe Institute of Technology (KIT). His research focuses on processes and surface engineering.
Prof. Dr. Gerhard Neumann is a full professor at KIT and heading the chair „Autonomous Learning Robots“. Research focus: Intersection of machine learning, robotics and human-robot interaction.
Prof. Dr.-Ing. Nicole Stricker is a full professor at Hochschule Aalen. Research focus: Operations Management.
Prof. Dr.-Ing. Kai Furmans is head of the Institute for Material Handling and Logistics (IFL). Research focus: Stochastic models in production and logistics, plug & play material flow systems logistics – Industry 4.0 in logistics and SCM, risk management in supply chains.
Dr. Julius Pfrommer is the head of the department for cognitive industrial systems at Fraunhofer IOSB. Research focus: Intelligent cyber-physical systems and adaptive production systems.
Prof. Dr.-Ing. Gisela Lanza is director of the research group Production Systems at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for quality management, production system planning and global production strategies.
Malte Hansjosten, M.Sc. is a research associate in the group of Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Smart components for machines and automated disassembly systems.
Patrick Fischmann, M.Sc. is a research associate in the group of Manufacturing and Material Technology at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research area: Additive manufacturing using selective laser melting.
Julia Dvorak, M.Sc. is a research associate in the field of production system planning at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research area: Human Centricity and Assistance Systems in Production.
Jan-Felix Klein, M.Sc. is a research associate in the Robotics and Interactive Systems department at the Institute of Materials Handling and Logistics Systems (IFL) at Karlsruhe Institute of Technology (KIT). Main research areas: Autonomous mobile robotics, intralogistics of decentralized distributed systems, remanufacturing.
Felix Rauscher, M.Sc. is a research associate in the Remote Handling and Logistics under Extreme Boundary Conditions department at the Institute of Materials Handling and Logistics Systems (IFL) at Karlsruhe Institute of Technology (KIT). Main research areas: Simulation of the material flow and maintenance logistics.
Andreas Ebner, M.Sc. is group leader for adaptive production systems at Fraunhofer IOSB.
Marvin May, M.Sc. is chief engineer for Production System Planning at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Machine learning in production planning and control as well as control of flexible production systems.
Philipp Gönnheimer, M.Sc. is chief engineer for Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Control and communication technology in machines and equipment as well as modular concepts for production systems.
-
Research ethics: Not applicable.
-
Author contributions: The authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: The authors state no conflict of interest.
-
Research funding: The project AgiProbot is funded by the Carl Zeiss Foundation and the work described therein served to prepare the SFB 1574 Circular Factory for the Perpetual Product (project ID: 471687386), which has since been approved by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) with a start date of April 1, 2024.
-
Data availability: Not applicable.
References
[1] M. Wurster, B. Häfner, D. Gauder, N. Stricker, and G. Lanza, “Fluid automation – a definition and an application in remanufacturing production systems,” Proc. CIRP, vol. 97, pp. 508–513, 2021, https://doi.org/10.1016/j.procir.2020.05.267.Suche in Google Scholar
[2] J. Rickert, S. Blömeke, M. Mennenga, F. Cerdas, S. Thiede, and C. Herrmann, “Refining circulation factories: classification scheme and supporting product and factory features for closed-loop production integration,” in Production at the Leading Edge of Technology, B.-A. Behrens, A. Brosius, W. Hintze, S. Ihlenfeldt, and J. P. Wulfsberg, Eds., Berlin, Heidelberg, Springer Berlin Heidelberg, 2021, pp. 591–601.10.1007/978-3-662-62138-7_59Suche in Google Scholar
[3] F. Cerdas, et al.., “Defining circulation factories – a pathway towards factories of the future,” Proc. CIRP, vol. 29, pp. 627–632, 2015, https://doi.org/10.1016/j.procir.2015.02.032.Suche in Google Scholar
[4] K. Urano and S. Takata, “Module reconfiguration management for circular factories without discriminating between virgin and reused products,” in Re-Engineering Manufacturing for Sustainability, A. Y. C. Nee, B. Song, and S.-K. Ong, Eds., Singapore, Springer Singapore, 2013, pp. 603–608.10.1007/978-981-4451-48-2_98Suche in Google Scholar
[5] J. Fleischer, et al.., “Agile produktion elektrischer traktionsmotoren als antwort auf volatile märkte und technologien,” Z. für Wirtsch. Fabr., vol. 116, no. 3, pp. 128–132, 2021. https://doi.org/10.1515/zwf-2021-0025.Suche in Google Scholar
[6] J. Fleischer, F. Fraider, F. Kößler, D. Mayer, and F. Wirth, “Agile production systems for electric mobility,” Proc. CIRP, vol. 107, pp. 1251–1256, 2022, https://doi.org/10.1016/j.procir.2022.05.140.Suche in Google Scholar
[7] J. Baumgärtner, P. Gönnheimer, and J. Fleischer, “Optimal robot workpiece placement for maximized repeatability,” in Advances in System-Integrated Intelligence, M. Valle, et al.., Eds., Cham, Springer International Publishing, 2023, pp. 252–261.10.1007/978-3-031-16281-7_25Suche in Google Scholar
[8] J. Baumgärtner, M. Hansjosten, D. Schönhofen, and P. D.-I. J. Fleischer, “Pybullet industrial: a process-aware robot simulation,” J. Open Source Softw., vol. 8, no. 85, p. 5174, 2023. https://doi.org/10.21105/joss.05174.Suche in Google Scholar
[9] M. Hansjosten and J. Fleischer, “Towards autonomous adaptive disassembly of permanent-magnet synchronous motors with industrial robots,” Manuf. Lett., vol. 35, pp. 1336–1346, 2023, https://doi.org/10.1016/j.mfglet.2023.08.084.Suche in Google Scholar
[10] J.-F. Klein, M. Wurster, N. Stricker, G. Lanza, and K. Furmans, “Towards ontology-based autonomous intralogistics for agile remanufacturing production systems,” in 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), IEEE, 2021, pp. 1–7.10.1109/ETFA45728.2021.9613486Suche in Google Scholar
[11] J. Pfrommer, et al.., “An ontology for remanufacturing systems,” Automatisierungstechnik, vol. 70, no. 6, pp. 534–541, 2022. https://doi.org/10.1515/auto-2021-0156.Suche in Google Scholar
[12] S. Vongbunyong, S. Kara, and M. Pagnucco, “Application of cognitive robotics in disassembly of products,” CIRP Ann., vol. 62, no. 1, pp. 31–34, 2013. https://doi.org/10.1016/j.cirp.2013.03.037.Suche in Google Scholar
[13] S. Vongbunyong, S. Kara, and M. Pagnucco, “Learning and revision in cognitive robotics disassembly automation,” Robot. Comput.-Integr. Manuf., vol. 34, pp. 79–94, 2015, https://doi.org/10.1016/j.rcim.2014.11.003.Suche in Google Scholar
[14] A. Mandlekar, et al.., “Roboturk: a crowdsourcing platform for robotic skill learning through imitation,” in Conference on Robot Learning, 2018.Suche in Google Scholar
[15] N. Das, S. Bechtle, T. Davchev, D. Jayaraman, A. Rai, and F. Meier, “Model-based inverse reinforcement learning from visual demonstrations,” in Proceedings of the 2020 Conference on Robot Learning, 2021.Suche in Google Scholar
[16] O. Zenkri, N. A. Vien, and G. Neumann, “Hierarchical policy learning for mechanical search,” in IEEE International Conference on Robotics and Automation (ICRA), 2022.10.1109/ICRA46639.2022.9811572Suche in Google Scholar
[17] M. Mittal, et al.., “Orbit: a unified simulation framework for interactive robot learning environments,” IEEE Rob. Autom. Lett., vol. 8, no. 6, pp. 3740–3747, 2023. https://doi.org/10.1109/lra.2023.3270034.Suche in Google Scholar
[18] E. Todorov, T. Erez, and Y. Tassa, “Mujoco: a physics engine for model-based control,” in IROS, IEEE, 2012.10.1109/IROS.2012.6386109Suche in Google Scholar
[19] Y. Narang, et al.., “Factory: fast contact for robotic assembly,” in Robotics: Science and Systems, 2022.10.15607/RSS.2022.XVIII.035Suche in Google Scholar
[20] M. Liu, et al.., “One-2-3-45: any single image to 3d mesh in 45 seconds without per-shape optimization,” in Thirty-Seventh Conference on Neural Information Processing Systems, 2023.Suche in Google Scholar
[21] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. Battaglia, “Learning mesh-based simulation with graph networks,” in International Conference on Learning Representations, 2021.Suche in Google Scholar
[22] O. X.-E. Collaboration, et al.., “Open X-Embodiment: robotic learning datasets and RT-X models,” arXiv preprint arXiv:2310.08864, 2023.Suche in Google Scholar
[23] Y. Zhu, et al.., “Reinforcement and imitation learning for diverse visuomotor skills,” in Proceedings of Robotics: Science and Systems, 2018.10.15607/RSS.2018.XIV.009Suche in Google Scholar
[24] R. Julian, B. Swanson, G. Sukhatme, S. Levine, C. Finn, and K. Hausman, “Never stop learning: the effectiveness of fine-tuning in robotic reinforcement learning,” in Proceedings of the 2020 Conference on Robot Learning, 2021.Suche in Google Scholar
[25] A. Rajeswaran, et al.., “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” in Proceedings of Robotics: Science and Systems (RSS), 2018.10.15607/RSS.2018.XIV.049Suche in Google Scholar
[26] P. J. Ball, L. Smith, I. Kostrikov, and S. Levine, “Efficient online reinforcement learning with offline data,” in International Conference on Machine Learning, 2023.Suche in Google Scholar
[27] O. Eberhard, J. Hollenstein, C. Pinneri, and G. Martius, “Pink noise is all you need: colored noise exploration in deep reinforcement learning,” in The Eleventh International Conference on Learning Representations, 2023.Suche in Google Scholar
[28] A. Raffin, J. Kober, and F. Stulp, “Smooth exploration for robotic reinforcement learning,” in 5th Annual Conference on Robot Learning, 2021.Suche in Google Scholar
[29] G. Li, et al.., “Open the black box: step-based policy updates for temporally-correlated episodic reinforcement learning,” in Submitted to The Twelfth International Conference on Learning Representations, 2023, under review.Suche in Google Scholar
[30] A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Probabilistic movement primitives,” Adv. Neural Inf. Process. Syst., vol. 26, pp. 2616–2624, 2013.Suche in Google Scholar
[31] G. Li, Z. Jin, M. Volpp, F. Otto, R. Lioutikov, and G. Neumann, “Prodmp: a unified perspective on dynamic and probabilistic movement primitives,” IEEE Rob. Autom. Lett., vol. 8, no. 4, pp. 2325–2332, 2023. https://doi.org/10.1109/lra.2023.3248443.Suche in Google Scholar
[32] F. Otto, O. Celik, H. Zhou, H. Ziesche, V. A. Ngo, and G. Neumann, “Deep black-box reinforcement learning with movement primitives,” in Conference on Robot Learning, 2022.Suche in Google Scholar
[33] J. Gao, Z. Tao, N. Jaquier, and T. Asfour, “K-vil: keypoints-based visual imitation learning,” IEEE Trans. Rob., vol. 39, no. 5, pp. 3888–3908, 2023. https://doi.org/10.1109/tro.2023.3286074.Suche in Google Scholar
[34] T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, “An algorithmic perspective on imitation learning,” Found. Trends® Rob., vol. 7, no. 1–2, pp. 1–179, 2018. https://doi.org/10.1561/2300000053.Suche in Google Scholar
[35] C. Wirth, J. Fürnkranz, and G. Neumann, “Model-free preference-based reinforcement learning,” in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2016.10.1609/aaai.v30i1.10269Suche in Google Scholar
[36] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Adv. Neural Inf. Process. Syst., vol. 30, pp. 4299–4307, 2017.Suche in Google Scholar
[37] G. Warnell, N. Waytowich, V. Lawhern, and P. Stone, “Deep tamer: interactive agent shaping in high-dimensional state spaces,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, 2018.10.1609/aaai.v32i1.11485Suche in Google Scholar
[38] D. P. Losey, A. Bajcsy, M. K. O’Malley, and A. D. Dragan, “Physical interaction as communication: learning robot objectives online from human corrections,” Int. J. Rob. Res., vol. 41, no. 1, pp. 20–44, 2022. https://doi.org/10.1177/02783649211050958.Suche in Google Scholar
[39] P. Sharma, et al.., “Correcting robot plans with natural language feedback,” in Proceedings of Robotics: Science and Systems, 2022.10.15607/RSS.2022.XVIII.065Suche in Google Scholar
[40] A. Taranovic, A. G. Kupcsik, N. Freymuth, and G. Neumann, “Adversarial imitation learning with preferences,” in The Eleventh International Conference on Learning Representations, 2023.Suche in Google Scholar
[41] E. Bıyık, D. P. Losey, M. Palan, N. C. Landolfi, G. Shevchuk, and D. Sadigh, “Learning reward functions from diverse sources of human feedback: optimally integrating demonstrations and preferences,” Int. J. Rob. Res., vol. 41, no. 1, pp. 45–67, 2022. https://doi.org/10.1177/02783649211041652.Suche in Google Scholar
[42] D. S. Brown, W. Goo, and S. Niekum, “Better-than-demonstrator imitation learning via automatically-ranked demonstrations,” in Proceedings of the 3rd Conference on Robot Learning, 2019.Suche in Google Scholar
[43] E. Biyik and D. Sadigh, “Batch active preference-based learning of reward functions,” in Proceedings of The 2nd Conference on Robot Learning, 2018.Suche in Google Scholar
[44] N. Freymuth, N. Schreiber, A. Taranovic, P. Becker, and G. Neumann, “Inferring versatile behavior from demonstrations by matching geometric descriptors,” in 6th Annual Conference on Robot Learning, 2022.Suche in Google Scholar
[45] E. Özceylan, C. B. Kalayci, A. Güngör, and S. M. Gupta, “Disassembly line balancing problem: a review of the state of the art and future directions,” Int. J. Prod. Res., vol. 57, nos. 15–16, pp. 4805–4827, 2019, https://doi.org/10.1080/00207543.2018.1428775.Suche in Google Scholar
[46] F. T. Altekin and C. Akkan, “Task-failure-driven rebalancing of disassembly lines,” Int. J. Prod. Res., vol. 50, no. 18, pp. 4955–4976, 2012. https://doi.org/10.1080/00207543.2011.616915.Suche in Google Scholar
[47] N. Stricker, M. Micali, D. Dornfeld, and G. Lanza, “Considering interdependencies of kpis – possible resource efficiency and effectiveness improvements,” Proc. Manuf., vol. 8, pp. 300–307, 2017, https://doi.org/10.1016/j.promfg.2017.02.038.Suche in Google Scholar
[48] J. Kurilova-Palisaitiene, E. Sundin, and B. Poksinska, “Remanufacturing challenges and possible lean improvements,” J. Clean. Prod., vol. 172, pp. 3225–3236, 2018, https://doi.org/10.1016/j.jclepro.2017.11.023.Suche in Google Scholar
[49] A. Kuhnle, M. C. May, L. Schäfer, and G. Lanza, “Explainable reinforcement learning in production control of job shop manufacturing system,” Int. J. Prod. Res., vol. 60, no. 19, pp. 5812–5834, 2022. https://doi.org/10.1080/00207543.2021.1972179.Suche in Google Scholar
[50] M. Wurster, M. Michel, M. C. May, A. Kuhnle, N. Stricker, and G. Lanza, “Modelling and condition-based control of a flexible and hybrid disassembly system with manual and autonomous workstations using reinforcement learning,” J. Intell. Manuf., vol. 33, no. 2, pp. 575–591, 2022. https://doi.org/10.1007/s10845-021-01863-3.Suche in Google Scholar
[51] G. Fragapane, R. de Koster, F. Sgarbossa, and J. O. Strandhagen, “Planning and control of autonomous mobile robots for intralogistics: literature review and research agenda,” Eur. J. Oper. Res., vol. 294, no. 2, pp. 405–426, 2021. https://doi.org/10.1016/j.ejor.2021.01.019.Suche in Google Scholar
[52] C. Lieberoth-Leden, J. Fischer, B. Vogel-Heuser, and J. Fottner, “Implementation, self-configuration and coordination of logistical functions for autonomous logistics modules in flexible automated material flow systems,” Int. J. Mech. Eng. Rob. Res., vol. 8, no. 4, pp. 498–505, 2019. https://doi.org/10.18178/ijmerr.8.4.498-505.Suche in Google Scholar
[53] W. Günthner, E. Klenk, and P. Tenerowicz-Wirth, “Adaptive logistiksysteme als wegbereiter der industrie 4.0,” in Handbuch Industrie 4.0 Bd.4, VDI Springer Reference, B. Vogel-Heuser, T. Bauernhansl, and M. ten Hompel, Eds., Berlin, Heidelberg, Springer Berlin Heidelberg, 2016, pp. 99–125.10.1007/978-3-662-53254-6_6Suche in Google Scholar
[54] M. de Ryck, D. Pissoort, T. Holvoet, and E. Demeester, “Decentral task allocation for industrial agv-systems with resource constraints,” J. Manuf. Syst., vol. 59, pp. 310–319, 2021, https://doi.org/10.1016/j.jmsy.2021.03.008.Suche in Google Scholar
[55] M. de Ryck, M. Versteyhe, and F. Debrouwere, “Automated guided vehicle systems, state-of-the-art control algorithms and techniques,” J. Manuf. Syst., vol. 54, pp. 152–173, 2020, https://doi.org/10.1016/j.jmsy.2019.12.002.Suche in Google Scholar
[56] M. Schranz, M. Umlauft, M. Sende, and W. Elmenreich, “Swarm robotic behaviors and current applications,” Front. Rob. AI, vol. 7, p. 36, 2020, https://doi.org/10.3389/frobt.2020.00036.Suche in Google Scholar PubMed PubMed Central
[57] V. R. Guide, “Production planning and control for remanufacturing: industry practice and research needs,” J. Oper. Manag., vol. 18, no. 4, pp. 467–483, 2000. https://doi.org/10.1016/s0272-6963(00)00034-6.Suche in Google Scholar
[58] V. R. Guide, M. E. Kraus, and R. Srivastava, “Scheduling policies for remanufacturing,” Int. J. Prod. Econ., vol. 48, no. 2, pp. 187–204, 1997. https://doi.org/10.1016/s0925-5273(96)00091-6.Suche in Google Scholar
[59] S. Groß, W. Gerke, P. Plapper, and M. Vette-Steinkamp, “Agile and autonomous production control for remanufacturing,” in 2021 9th International Conference on Control, Mechatronics and Automation (ICCMA), IEEE, 2021, pp. 231–236.10.1109/ICCMA54375.2021.9646213Suche in Google Scholar
[60] P. Grimmel, J. Wessel, M. Mennenga, and C. Herrmann, “Potentials of ontology-based knowledge discovery in data bases for learning factories,” Proceedings of the 12th Conference on Learning Factories (CLF), 2022. https://doi.org/10.2139/ssrn.4073026.Suche in Google Scholar
© 2024 Walter de Gruyter GmbH, Berlin/Boston
