Abstract
The integration of both linear and circular processes in one production system poses significant challenges. In particular, the reprocessing of end-of-life products is associated with uncertainties at all levels of the production system, from the initial planning and control through to the executing production hardware and intralogistics. To address these challenges, this article presents approaches for self-learning and autonomously adapting production equipment for the Circular Factory. Initially, hardware and software solutions are developed to cover the necessary processes. Reprocessing is covered by modular and reconfigurable manufacturing cells, which also include new process chains such as the combination of additive-subtractive processes. The provided capabilities must be applied to ever new products, for example by transferring human procedures for unknown products to the production equipment. Lastly, an overall robust and dynamic production planning and control system is developed that maintains continuous operation even in unforeseen situations. The resulting highly dynamic overall system is connected by an autonomous intralogistics system.
Zusammenfassung
Die Integration von sowohl linearen und als auch zirkulären Prozessen in einem Produktionssystem stellt eine große Herausforderung dar. Insbesondere die Wiederaufarbeitung von Altprodukten ist mit Unsicherheiten auf allen Ebenen des Produktionssystems verbunden, von der anfänglichen Planung und Steuerung bis hin zur ausführenden Produktionshardware und Intralogistik. Um diesen Herausforderungen zu begegnen, werden in diesem Beitrag Ansätze für selbstlernende und sich autonom adaptierende Produktionsanlagen für die Kreislauffabrik vorgestellt. Zunächst werden Hard- und Softwarelösungen entwickelt, die die notwendigen Prozesse bereitstellen. Die Wiederaufarbeitung wird durch modulare und rekonfigurierbare Fertigungszellen abgedeckt, die auch neue Prozessketten wie die Kombination von additiv-subtraktiven Verfahren umfassen. Die bereitgestellten Fähigkeiten müssen auf immer neue Produkte angewandt werden, zum Beispiel indem menschliche Vorgehensweisen bei unbekannten Produkten auf die Produktionsanlagen übertragen werden. Schließlich wird ein insgesamt robustes und dynamisches Produktionsplanungs- und -steuerungssystem entwickelt, das auch in unvorhergesehenen Situationen einen kontinuierlichen Betrieb aufrechterhält. Das so entstandene hochdynamische Gesamtsystem wird durch ein autonomes Intralogistiksystem verknüpft.
Funding source: Carl Zeiss Foundation
Award Identifier / Grant number: project ID: 471687386
About the authors
Prof. Dr.-Ing. Jürgen Fleischer is director of the research group Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for the areas of intelligent machines and components, agile and reversible production systems and the automation of immature processes.
Prof. Dr.-Ing. Frederik Zanger is director of the research group Manufacturing and Material Technology at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for the area digitalization of process development for additive manufacturing.
Prof. Dr.-Ing. habil. Volker Schulze is director of the research group Manufacturing and Material Technology at wbk Institute of Production Science as well as member of the cooperative board of management at the Institute for Applied Material – Materials Science and Engineering at Karlsruhe Institute of Technology (KIT). His research focuses on processes and surface engineering.
Prof. Dr. Gerhard Neumann is a full professor at KIT and heading the chair „Autonomous Learning Robots“. Research focus: Intersection of machine learning, robotics and human-robot interaction.
Prof. Dr.-Ing. Nicole Stricker is a full professor at Hochschule Aalen. Research focus: Operations Management.
Prof. Dr.-Ing. Kai Furmans is head of the Institute for Material Handling and Logistics (IFL). Research focus: Stochastic models in production and logistics, plug & play material flow systems logistics – Industry 4.0 in logistics and SCM, risk management in supply chains.
Dr. Julius Pfrommer is the head of the department for cognitive industrial systems at Fraunhofer IOSB. Research focus: Intelligent cyber-physical systems and adaptive production systems.
Prof. Dr.-Ing. Gisela Lanza is director of the research group Production Systems at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT) and responsible for quality management, production system planning and global production strategies.
Malte Hansjosten, M.Sc. is a research associate in the group of Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Smart components for machines and automated disassembly systems.
Patrick Fischmann, M.Sc. is a research associate in the group of Manufacturing and Material Technology at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research area: Additive manufacturing using selective laser melting.
Julia Dvorak, M.Sc. is a research associate in the field of production system planning at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research area: Human Centricity and Assistance Systems in Production.
Jan-Felix Klein, M.Sc. is a research associate in the Robotics and Interactive Systems department at the Institute of Materials Handling and Logistics Systems (IFL) at Karlsruhe Institute of Technology (KIT). Main research areas: Autonomous mobile robotics, intralogistics of decentralized distributed systems, remanufacturing.
Felix Rauscher, M.Sc. is a research associate in the Remote Handling and Logistics under Extreme Boundary Conditions department at the Institute of Materials Handling and Logistics Systems (IFL) at Karlsruhe Institute of Technology (KIT). Main research areas: Simulation of the material flow and maintenance logistics.
Andreas Ebner, M.Sc. is group leader for adaptive production systems at Fraunhofer IOSB.
Marvin May, M.Sc. is chief engineer for Production System Planning at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Machine learning in production planning and control as well as control of flexible production systems.
Philipp Gönnheimer, M.Sc. is chief engineer for Machines, Equipment and Process Automation at wbk Institute of Production Science at Karlsruhe Institute of Technology (KIT). Main research areas: Control and communication technology in machines and equipment as well as modular concepts for production systems.
-
Research ethics: Not applicable.
-
Author contributions: The authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: The authors state no conflict of interest.
-
Research funding: The project AgiProbot is funded by the Carl Zeiss Foundation and the work described therein served to prepare the SFB 1574 Circular Factory for the Perpetual Product (project ID: 471687386), which has since been approved by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) with a start date of April 1, 2024.
-
Data availability: Not applicable.
References
[1] M. Wurster, B. Häfner, D. Gauder, N. Stricker, and G. Lanza, “Fluid automation – a definition and an application in remanufacturing production systems,” Proc. CIRP, vol. 97, pp. 508–513, 2021, https://doi.org/10.1016/j.procir.2020.05.267.Search in Google Scholar
[2] J. Rickert, S. Blömeke, M. Mennenga, F. Cerdas, S. Thiede, and C. Herrmann, “Refining circulation factories: classification scheme and supporting product and factory features for closed-loop production integration,” in Production at the Leading Edge of Technology, B.-A. Behrens, A. Brosius, W. Hintze, S. Ihlenfeldt, and J. P. Wulfsberg, Eds., Berlin, Heidelberg, Springer Berlin Heidelberg, 2021, pp. 591–601.10.1007/978-3-662-62138-7_59Search in Google Scholar
[3] F. Cerdas, et al.., “Defining circulation factories – a pathway towards factories of the future,” Proc. CIRP, vol. 29, pp. 627–632, 2015, https://doi.org/10.1016/j.procir.2015.02.032.Search in Google Scholar
[4] K. Urano and S. Takata, “Module reconfiguration management for circular factories without discriminating between virgin and reused products,” in Re-Engineering Manufacturing for Sustainability, A. Y. C. Nee, B. Song, and S.-K. Ong, Eds., Singapore, Springer Singapore, 2013, pp. 603–608.10.1007/978-981-4451-48-2_98Search in Google Scholar
[5] J. Fleischer, et al.., “Agile produktion elektrischer traktionsmotoren als antwort auf volatile märkte und technologien,” Z. für Wirtsch. Fabr., vol. 116, no. 3, pp. 128–132, 2021. https://doi.org/10.1515/zwf-2021-0025.Search in Google Scholar
[6] J. Fleischer, F. Fraider, F. Kößler, D. Mayer, and F. Wirth, “Agile production systems for electric mobility,” Proc. CIRP, vol. 107, pp. 1251–1256, 2022, https://doi.org/10.1016/j.procir.2022.05.140.Search in Google Scholar
[7] J. Baumgärtner, P. Gönnheimer, and J. Fleischer, “Optimal robot workpiece placement for maximized repeatability,” in Advances in System-Integrated Intelligence, M. Valle, et al.., Eds., Cham, Springer International Publishing, 2023, pp. 252–261.10.1007/978-3-031-16281-7_25Search in Google Scholar
[8] J. Baumgärtner, M. Hansjosten, D. Schönhofen, and P. D.-I. J. Fleischer, “Pybullet industrial: a process-aware robot simulation,” J. Open Source Softw., vol. 8, no. 85, p. 5174, 2023. https://doi.org/10.21105/joss.05174.Search in Google Scholar
[9] M. Hansjosten and J. Fleischer, “Towards autonomous adaptive disassembly of permanent-magnet synchronous motors with industrial robots,” Manuf. Lett., vol. 35, pp. 1336–1346, 2023, https://doi.org/10.1016/j.mfglet.2023.08.084.Search in Google Scholar
[10] J.-F. Klein, M. Wurster, N. Stricker, G. Lanza, and K. Furmans, “Towards ontology-based autonomous intralogistics for agile remanufacturing production systems,” in 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), IEEE, 2021, pp. 1–7.10.1109/ETFA45728.2021.9613486Search in Google Scholar
[11] J. Pfrommer, et al.., “An ontology for remanufacturing systems,” Automatisierungstechnik, vol. 70, no. 6, pp. 534–541, 2022. https://doi.org/10.1515/auto-2021-0156.Search in Google Scholar
[12] S. Vongbunyong, S. Kara, and M. Pagnucco, “Application of cognitive robotics in disassembly of products,” CIRP Ann., vol. 62, no. 1, pp. 31–34, 2013. https://doi.org/10.1016/j.cirp.2013.03.037.Search in Google Scholar
[13] S. Vongbunyong, S. Kara, and M. Pagnucco, “Learning and revision in cognitive robotics disassembly automation,” Robot. Comput.-Integr. Manuf., vol. 34, pp. 79–94, 2015, https://doi.org/10.1016/j.rcim.2014.11.003.Search in Google Scholar
[14] A. Mandlekar, et al.., “Roboturk: a crowdsourcing platform for robotic skill learning through imitation,” in Conference on Robot Learning, 2018.Search in Google Scholar
[15] N. Das, S. Bechtle, T. Davchev, D. Jayaraman, A. Rai, and F. Meier, “Model-based inverse reinforcement learning from visual demonstrations,” in Proceedings of the 2020 Conference on Robot Learning, 2021.Search in Google Scholar
[16] O. Zenkri, N. A. Vien, and G. Neumann, “Hierarchical policy learning for mechanical search,” in IEEE International Conference on Robotics and Automation (ICRA), 2022.10.1109/ICRA46639.2022.9811572Search in Google Scholar
[17] M. Mittal, et al.., “Orbit: a unified simulation framework for interactive robot learning environments,” IEEE Rob. Autom. Lett., vol. 8, no. 6, pp. 3740–3747, 2023. https://doi.org/10.1109/lra.2023.3270034.Search in Google Scholar
[18] E. Todorov, T. Erez, and Y. Tassa, “Mujoco: a physics engine for model-based control,” in IROS, IEEE, 2012.10.1109/IROS.2012.6386109Search in Google Scholar
[19] Y. Narang, et al.., “Factory: fast contact for robotic assembly,” in Robotics: Science and Systems, 2022.10.15607/RSS.2022.XVIII.035Search in Google Scholar
[20] M. Liu, et al.., “One-2-3-45: any single image to 3d mesh in 45 seconds without per-shape optimization,” in Thirty-Seventh Conference on Neural Information Processing Systems, 2023.Search in Google Scholar
[21] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. Battaglia, “Learning mesh-based simulation with graph networks,” in International Conference on Learning Representations, 2021.Search in Google Scholar
[22] O. X.-E. Collaboration, et al.., “Open X-Embodiment: robotic learning datasets and RT-X models,” arXiv preprint arXiv:2310.08864, 2023.Search in Google Scholar
[23] Y. Zhu, et al.., “Reinforcement and imitation learning for diverse visuomotor skills,” in Proceedings of Robotics: Science and Systems, 2018.10.15607/RSS.2018.XIV.009Search in Google Scholar
[24] R. Julian, B. Swanson, G. Sukhatme, S. Levine, C. Finn, and K. Hausman, “Never stop learning: the effectiveness of fine-tuning in robotic reinforcement learning,” in Proceedings of the 2020 Conference on Robot Learning, 2021.Search in Google Scholar
[25] A. Rajeswaran, et al.., “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” in Proceedings of Robotics: Science and Systems (RSS), 2018.10.15607/RSS.2018.XIV.049Search in Google Scholar
[26] P. J. Ball, L. Smith, I. Kostrikov, and S. Levine, “Efficient online reinforcement learning with offline data,” in International Conference on Machine Learning, 2023.Search in Google Scholar
[27] O. Eberhard, J. Hollenstein, C. Pinneri, and G. Martius, “Pink noise is all you need: colored noise exploration in deep reinforcement learning,” in The Eleventh International Conference on Learning Representations, 2023.Search in Google Scholar
[28] A. Raffin, J. Kober, and F. Stulp, “Smooth exploration for robotic reinforcement learning,” in 5th Annual Conference on Robot Learning, 2021.Search in Google Scholar
[29] G. Li, et al.., “Open the black box: step-based policy updates for temporally-correlated episodic reinforcement learning,” in Submitted to The Twelfth International Conference on Learning Representations, 2023, under review.Search in Google Scholar
[30] A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Probabilistic movement primitives,” Adv. Neural Inf. Process. Syst., vol. 26, pp. 2616–2624, 2013.Search in Google Scholar
[31] G. Li, Z. Jin, M. Volpp, F. Otto, R. Lioutikov, and G. Neumann, “Prodmp: a unified perspective on dynamic and probabilistic movement primitives,” IEEE Rob. Autom. Lett., vol. 8, no. 4, pp. 2325–2332, 2023. https://doi.org/10.1109/lra.2023.3248443.Search in Google Scholar
[32] F. Otto, O. Celik, H. Zhou, H. Ziesche, V. A. Ngo, and G. Neumann, “Deep black-box reinforcement learning with movement primitives,” in Conference on Robot Learning, 2022.Search in Google Scholar
[33] J. Gao, Z. Tao, N. Jaquier, and T. Asfour, “K-vil: keypoints-based visual imitation learning,” IEEE Trans. Rob., vol. 39, no. 5, pp. 3888–3908, 2023. https://doi.org/10.1109/tro.2023.3286074.Search in Google Scholar
[34] T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, “An algorithmic perspective on imitation learning,” Found. Trends® Rob., vol. 7, no. 1–2, pp. 1–179, 2018. https://doi.org/10.1561/2300000053.Search in Google Scholar
[35] C. Wirth, J. Fürnkranz, and G. Neumann, “Model-free preference-based reinforcement learning,” in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2016.10.1609/aaai.v30i1.10269Search in Google Scholar
[36] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Adv. Neural Inf. Process. Syst., vol. 30, pp. 4299–4307, 2017.Search in Google Scholar
[37] G. Warnell, N. Waytowich, V. Lawhern, and P. Stone, “Deep tamer: interactive agent shaping in high-dimensional state spaces,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, 2018.10.1609/aaai.v32i1.11485Search in Google Scholar
[38] D. P. Losey, A. Bajcsy, M. K. O’Malley, and A. D. Dragan, “Physical interaction as communication: learning robot objectives online from human corrections,” Int. J. Rob. Res., vol. 41, no. 1, pp. 20–44, 2022. https://doi.org/10.1177/02783649211050958.Search in Google Scholar
[39] P. Sharma, et al.., “Correcting robot plans with natural language feedback,” in Proceedings of Robotics: Science and Systems, 2022.10.15607/RSS.2022.XVIII.065Search in Google Scholar
[40] A. Taranovic, A. G. Kupcsik, N. Freymuth, and G. Neumann, “Adversarial imitation learning with preferences,” in The Eleventh International Conference on Learning Representations, 2023.Search in Google Scholar
[41] E. Bıyık, D. P. Losey, M. Palan, N. C. Landolfi, G. Shevchuk, and D. Sadigh, “Learning reward functions from diverse sources of human feedback: optimally integrating demonstrations and preferences,” Int. J. Rob. Res., vol. 41, no. 1, pp. 45–67, 2022. https://doi.org/10.1177/02783649211041652.Search in Google Scholar
[42] D. S. Brown, W. Goo, and S. Niekum, “Better-than-demonstrator imitation learning via automatically-ranked demonstrations,” in Proceedings of the 3rd Conference on Robot Learning, 2019.Search in Google Scholar
[43] E. Biyik and D. Sadigh, “Batch active preference-based learning of reward functions,” in Proceedings of The 2nd Conference on Robot Learning, 2018.Search in Google Scholar
[44] N. Freymuth, N. Schreiber, A. Taranovic, P. Becker, and G. Neumann, “Inferring versatile behavior from demonstrations by matching geometric descriptors,” in 6th Annual Conference on Robot Learning, 2022.Search in Google Scholar
[45] E. Özceylan, C. B. Kalayci, A. Güngör, and S. M. Gupta, “Disassembly line balancing problem: a review of the state of the art and future directions,” Int. J. Prod. Res., vol. 57, nos. 15–16, pp. 4805–4827, 2019, https://doi.org/10.1080/00207543.2018.1428775.Search in Google Scholar
[46] F. T. Altekin and C. Akkan, “Task-failure-driven rebalancing of disassembly lines,” Int. J. Prod. Res., vol. 50, no. 18, pp. 4955–4976, 2012. https://doi.org/10.1080/00207543.2011.616915.Search in Google Scholar
[47] N. Stricker, M. Micali, D. Dornfeld, and G. Lanza, “Considering interdependencies of kpis – possible resource efficiency and effectiveness improvements,” Proc. Manuf., vol. 8, pp. 300–307, 2017, https://doi.org/10.1016/j.promfg.2017.02.038.Search in Google Scholar
[48] J. Kurilova-Palisaitiene, E. Sundin, and B. Poksinska, “Remanufacturing challenges and possible lean improvements,” J. Clean. Prod., vol. 172, pp. 3225–3236, 2018, https://doi.org/10.1016/j.jclepro.2017.11.023.Search in Google Scholar
[49] A. Kuhnle, M. C. May, L. Schäfer, and G. Lanza, “Explainable reinforcement learning in production control of job shop manufacturing system,” Int. J. Prod. Res., vol. 60, no. 19, pp. 5812–5834, 2022. https://doi.org/10.1080/00207543.2021.1972179.Search in Google Scholar
[50] M. Wurster, M. Michel, M. C. May, A. Kuhnle, N. Stricker, and G. Lanza, “Modelling and condition-based control of a flexible and hybrid disassembly system with manual and autonomous workstations using reinforcement learning,” J. Intell. Manuf., vol. 33, no. 2, pp. 575–591, 2022. https://doi.org/10.1007/s10845-021-01863-3.Search in Google Scholar
[51] G. Fragapane, R. de Koster, F. Sgarbossa, and J. O. Strandhagen, “Planning and control of autonomous mobile robots for intralogistics: literature review and research agenda,” Eur. J. Oper. Res., vol. 294, no. 2, pp. 405–426, 2021. https://doi.org/10.1016/j.ejor.2021.01.019.Search in Google Scholar
[52] C. Lieberoth-Leden, J. Fischer, B. Vogel-Heuser, and J. Fottner, “Implementation, self-configuration and coordination of logistical functions for autonomous logistics modules in flexible automated material flow systems,” Int. J. Mech. Eng. Rob. Res., vol. 8, no. 4, pp. 498–505, 2019. https://doi.org/10.18178/ijmerr.8.4.498-505.Search in Google Scholar
[53] W. Günthner, E. Klenk, and P. Tenerowicz-Wirth, “Adaptive logistiksysteme als wegbereiter der industrie 4.0,” in Handbuch Industrie 4.0 Bd.4, VDI Springer Reference, B. Vogel-Heuser, T. Bauernhansl, and M. ten Hompel, Eds., Berlin, Heidelberg, Springer Berlin Heidelberg, 2016, pp. 99–125.10.1007/978-3-662-53254-6_6Search in Google Scholar
[54] M. de Ryck, D. Pissoort, T. Holvoet, and E. Demeester, “Decentral task allocation for industrial agv-systems with resource constraints,” J. Manuf. Syst., vol. 59, pp. 310–319, 2021, https://doi.org/10.1016/j.jmsy.2021.03.008.Search in Google Scholar
[55] M. de Ryck, M. Versteyhe, and F. Debrouwere, “Automated guided vehicle systems, state-of-the-art control algorithms and techniques,” J. Manuf. Syst., vol. 54, pp. 152–173, 2020, https://doi.org/10.1016/j.jmsy.2019.12.002.Search in Google Scholar
[56] M. Schranz, M. Umlauft, M. Sende, and W. Elmenreich, “Swarm robotic behaviors and current applications,” Front. Rob. AI, vol. 7, p. 36, 2020, https://doi.org/10.3389/frobt.2020.00036.Search in Google Scholar PubMed PubMed Central
[57] V. R. Guide, “Production planning and control for remanufacturing: industry practice and research needs,” J. Oper. Manag., vol. 18, no. 4, pp. 467–483, 2000. https://doi.org/10.1016/s0272-6963(00)00034-6.Search in Google Scholar
[58] V. R. Guide, M. E. Kraus, and R. Srivastava, “Scheduling policies for remanufacturing,” Int. J. Prod. Econ., vol. 48, no. 2, pp. 187–204, 1997. https://doi.org/10.1016/s0925-5273(96)00091-6.Search in Google Scholar
[59] S. Groß, W. Gerke, P. Plapper, and M. Vette-Steinkamp, “Agile and autonomous production control for remanufacturing,” in 2021 9th International Conference on Control, Mechatronics and Automation (ICCMA), IEEE, 2021, pp. 231–236.10.1109/ICCMA54375.2021.9646213Search in Google Scholar
[60] P. Grimmel, J. Wessel, M. Mennenga, and C. Herrmann, “Potentials of ontology-based knowledge discovery in data bases for learning factories,” Proceedings of the 12th Conference on Learning Factories (CLF), 2022. https://doi.org/10.2139/ssrn.4073026.Search in Google Scholar
© 2024 Walter de Gruyter GmbH, Berlin/Boston
