Technology Transfer – from Research to Innovation

The research results of the Competence Center ML2R are directly used for practice-oriented applications and lead to new data-based products and services. Our range of transfer offers encompasses education and training programs as well as agile cooperation formats, designed for successful collaborations with companies. We hereby take the entire lifecycle of AI- and ML-based technologies into consideration: from strategy development, planning, implementation and integration to safeguarding, operation and optimization. Examples of successful cooperation projects with partners from business and industry are presented below.

Offers for Companies

Many companies have already recognized the potential of AI and ML applications, but they face key challenges: They need comprehensible, trustworthy technologies that can be integrated into corporate processes in a flexible manner. They lack specialists who can develop such technologies and implement them strategically within the company.

Here, we provide you with customized offers and work closely with you to set the course for sustainable success in international competition. Small and medium-sized enterprises (SMEs) in particular benefit from access to strategies, knowledge, experience and the latest findings from our research.

Learn more about Opportunities for Cooperation

Manager for Technology Transfer

Dr. Stefan Rüping

“At ML2R, we are convinced that true innovation comes from the interplay between application and research. That is why we strive to support companies and together ensure that excellent research lays the foundation for intelligent solutions.”

Successful Projects from Strategic Fields of Application

We accompany you on the way to implementing ML technologies and AI solutions in practice. We have selected some examples of successful projects and cooperation with companies for you. The focus lies on the following core application fields:

Industry 4.0

AI and ML technologies have the potential to shorten development cycles, optimize the use of resources and make industrial production more efficient overall. We are happy to support and offer guidance to medium-sized companies, especially with respect to optimizing their processes with AI and ML methods. Our offers for cooperation hereby cover the entire value chain: from feasibility analysis to strategy development, consultation on technology deployment to sustainable implementation and continuous improvement.

Experimental Designs using AI: Accelerated Development and Optimized Use of Resources

Regardless of whether the challenge is to find new products, materials and formulations or to optimize existing ones: Our AI-based methodology helps you in the planning and optimal execution of trials. In this way, you can use your resources efficiently and save time and money.

In order to successfully compete in the global market, manufacturing companies must develop and optimize their products, materials and formulations ever faster. Traditional Designs of Experiment (DoEs), often find it difficult to guarantee high levels of velocity while maintaining the highest quality standards and optimal use of resources.

Experiments are essential for the development of new formulations and materials. Their parameters often allow for a wide range of possible combinations. The testing of each of these possibilities requires a great deal of human and material resources and is therefore time-consuming and costly.

We have developed an Artificial Intelligence-based DoE methodology that optimizes experimental designs as well as the use of resources, thus saving time and reducing costs. With our DoE methodology, we help you reach your goal faster, more reliably and more efficiently than with conventional methods.

Depending on the initial state, AI-based technology can save up to 50 percent of trials compared to traditional concepts of test planning. In addition, problems previously considered too complex too solve can be resolved using AI-based methods. Based on Bayesian optimization, our DoE methodology generates iterative experiment plans that lead you to your goal as quickly as possible.

The technology we have developed is based on our research of hybrid methods that combine Machine Learning with expert knowledge. In this way, we also support you in anchoring and optimally using existing knowledge in your company in the long term.

Data Summaries: Smart Data Summaries for More Efficient Production

For an efficient production, machine and sensor data must be collected, combined and evaluated. In a cooperative project with the company Achenbach Buschhütten, ML2R researchers developed new methods by which data from rolling mills was condensed into data summaries that can be easily analyzed and processed.

The aim of interconnecting machines and components in the context of Industry 4.0 is primarily targeted at monitoring and predicting machine behavior. This makes it possible to avoid unplanned downtime and to increase the efficiency of production. The key hereby lies within the efficient communication of machines among each other and with the cloud. In many companies, the implementation of the respective data communication is still in its early stages, but at Achenbach Buschhütten it is already firmly integrated into day-to-day business. Achenbach Buschhütten is an independent family-owned company looking back on more than 565 years of history and 125 years of experience in rolling mill equipment. The company is a pioneer in the interconnection of industrial plants with their own cloud-based platform Achenbach OPTILINK®. Achenbach Buschhütten is hence an ideal research partner for the Competence Center ML2R.

If machine data is available in large quantities through an interlinkage, it should be analyzed profitably. One challenge is that the large amounts of data can hardly be stored and analyzed in a timely manner. In addition, it is clear that a profitable analysis must always be carried out in close cooperation with the plant engineers.

Data summaries are an ideal approach to reducing the amount of data and making it comprehensible for users, such as plant engineers. A data summary is a small, well-chosen excerpt from the overall data that presents as much information and special features of the whole data set as possible. Formally, summaries can be represented using submodular functions, such as the “Informative Vector Machine”. Submodular functions provide a theoretically sound framework for the analysis of data summarization techniques while formalizing the notion of summarization in a mathematically precise way. This can then be used to select small and expressive summaries in linear time. On the other hand, the quality of the summaries and the selection process can be objectified and compared with the subjective impressions of the operating engineers.

At TU Dortmund University, methods for the summarization of data from astrophysics have been developed, which allow physicists to gain a quick overview of large measurement series. These techniques were further developed at ML2R in cooperation with Achenbach Buschhütten and adapted to the use on sensor data from rolling mills, which are available as time series. Further research on methods for data summarization is a vital part of the cooperation between Achenbach Buschhütten and the Competence Center ML2R.

Cognitive Process

 

Automation

In any company, understanding, summarizing and categorizing texts is an essential part of many work processes. We support you in automating cognitive processes: Benefit from Natural Language Understanding to understand and interpret texts using AI. This saves you time and money and makes your workflows more efficient. Our solutions have also proven themselves to be well-suited for specialist and industry-specific texts, such as court rulings, medical documentation or maintenance reports.

Insight Financials: Generating Financial and Economic Analyses swiftly and by automation

With our tool designed for financial and business professionals, you’ll have information at your fingertips quickly in order to create automated analyses. Our search and analysis engine allows you to automatically find company reports published on the web and extract data. You are hereby able to save time when researching, analyzing and preparing financial and economic data. You are able to use this data for example for market analyses, benchmarking and your strategic corporate planning.

Thousands of company reports, such as stock exchange prospectuses and annual financial statements, are published every year. These reports are based on different legal requirements. The financial figures they contain are available in an unstructured form, are complex and can be defined differently. Therefore, manual information research for benchmarking and market analysis based on company reports is very costly and time-consuming.

Insight Financials is an intelligent, AI-based, cross-industry and cross-company tool that facilitates the work of analysts, consultants, corporate strategists and investors. Based on methods of Artificial Intelligence and Machine Learning, our tool allows for the automated sighting of financial reports from thousands of companies using autonomous web crawlers.

All information essential to you as a user is extracted and provided in real-time through an intuitive web application. The tool offers a granular search process, various filtering options and a benchmarking tool. It is also able to detect outliers and make predictions based on the extracted, temporary data. You can hence make a direct comparison between competitors, which poses valuable information for a company’s management as well as for auditors and investors.

The neural network-based analysis, or so-called parsing, of automatically downloaded company reports was developed by researchers at the Competence Center ML2R. In addition, the ML2R focuses on resource-efficient ML technologies: The language and extraction models, used for the automatic extraction of the financial figures, do not require any specialized, cost-intensive infrastructure.

RICO: Artificial Intelligence facilitates Coding and Billing of Hospital Services

The “RightCoding” (RICO) software simplifies and automates billing processes. Hospital employees use RICO to code diagnoses, seamlessly record treatments and bill health insurance providers for services. The respective processes are hence conducted in a manner that is simpler, more secure and in compliance with data protection regulations.

In order for hospitals to be able to bill health insurance providers for their services in the treatment of patients, the conducted services must be coded. This process is essential for the securing of hospital revenues. However, it demands a high degree of personnel resources and is time-consuming as well as at times prone to errors.

Together with GSG Consulting GmbH and with the participation of ML2R scientists, Fraunhofer IAIS has developed the AI-backed software RICO, which supports hospital staff in the coding process. RICO stands for “RightCoding”: It helps the hospital team to include all services when billing and checks whether all relevant supporting documents are available. If supporting documents are missing, the software automatically searches for them in the medical records.

RICO, the Artificial Intelligence-based coding support software, greatly simplifies the coding and billing process and implements it in a way that withstands medical service audits. RICO was developed in accordance with the highest data protection standards, is immediately operational without additional measures necessary, and is already in use in multiple hospitals.

ML2R conducts research on hybrid ML methods that combine expert knowledge with Machine Learning. This involves, for example, linking contextual text understanding with medical knowledge as sets of rules such as treatment and disease codes or the German drug directory “Rote Liste”. The RICO software for AI-based automation of billing processes in hospitals is one result of this research.

Astrophysics

Machine Learning is indispensable in modern astrophysics to process and evaluate data. Only in this way is it possible to gain insights into the physical world and our universe from the observed raw data. In the course of this, researchers use ML methods that are interpretable, take background knowledge into account, and can process even large amounts of data in real time. Through our cooperation and training programs, users from business and industry also benefit from this experience.

Field of Application Astrophysics: Terrabyte Data Streams for Science and Education

Modern astrophysics relies on Machine Learning methods which are interpretable, take into account background knowledge, and adhere to real-time requirements. We use our long-term experience in this field of application to investigate the methodological foundations of such learning procedures and to train stakeholders from commercial enterprises in dealing with the corresponding requirements.

The exploration of trustworthy, hybrid, and resource-aware learning requires applications that place corresponding demands on Machine Learning techniques. Astrophysics encompasses several such applications, using complex and high-volume data streams. In the context of ML2R, astrophysics becomes a data provider which supports basic and advanced research.

The telescopes whose data we obtain use the Earth’s atmosphere to detect high-energy particles from the cosmos. Special telescope cameras record the atmospheric interactions of these particles in images. Only with Machine Learning on the basis of simulated data is it possible to draw conclusions from the visual observations to the relevant physical properties of the cosmic particles. Astrophysicists are interested in these properties in order to better understand our universe.

In the analysis of astrophysical data as in other use cases, it is essential for the prediction of the Machine Learning model to be trustworthy; otherwise, the analysis results would not allow any conclusions to be drawn about the actual relationships represented in the data. The inclusion of background knowledge also overlaps with the requirements of other use cases: Hybrid learning feeds learning procedures with knowledge that would otherwise not be inferable from the data. Consideration of computational resources is essential due to a data volume of up to 100 terabytes per night; power consumption caused by simulation and analysis has so far been a significant cost factor in the operation of astrophysical telescopes.

Fundamental research at ML2R benefits from these diverse challenges because they motivate innovative solutions to problems that recur in other fields of application. In addition, we can use a significant part of the astrophysical data to train stakeholders from commercial enterprises in dealing with these challenges. Last but not least, our long-term collaboration partners from physics also benefit from our contribution to the state of the art in Machine Learning.

Multimedia Analysis

Broadcasters, media houses, and enterprises often have a huge amount of audio and video data, such as radio and TV broadcasts, which are often only rudimentarily tagged with keywords or other metadata. AI technologies such as audio mining and related video techniques can automatically make these contributions accessible and thus searchable. Automatic speech recognition can be used to transcribe the spoken word, and speaker recognition allows to tag known individuals, for example to retrieve original quotes.

Speech recognition systems learn new expressions autonomously

Language is subject to constant change. Particularly in daily media reporting on new topics, new names and terms often emerge that haven’t been used before or were uncommon. In the context of media coverage on the Corona epidemic, for example the word Covid19 or the name Christian Drosten frequently appeared. Automatic speech recognition systems trained with Machine Learning must therefore constantly learn new names and terms to stay current.

Within the framework of the cooperation with ARD (German public broadcaster) under the leadership of Westdeutscher Rundfunk, scientists from ML2R and Fraunhofer IAIS have developed a self-learning speech recognition system. A particular challenge in the cooperation with ARD was that with the regional programs such as WDR, NDR, HR or SWR, all accents and dialects of the German language had to be covered. Neologisms and loan words from other languages such as “download” or “fun fact” also play an increasingly important role.

In solving this task, ML2R research in hybrid learning was particularly relevant. Thus, already transcribed audio data, but also pronunciation lexica and text data from the Internet have been used for continuous learning. By combining a broad range of knowledge sources in the context of hybrid Machine Learning, a significant improvement in recognition performance could be achieved.

Artificial Intelligence recognizes emotions in interviews with contemporary witnesses

In a research project conducted with the foundation “Haus der Geschichte“, scientists from ML2R and Fraunhofer IAIS are developing a technology that recognizes and categorizes emotions in interviews with contemporary witnesses. With the help of Machine Learning, algorithms for emotion recognition are being trained to enable experts and citizens interested in history to search specifically for emotions in interviews with contemporary witnesses, for example on the topic of the fall of the Berlin Wall.

In the video interviews with contemporary witnesses, the following aspects, among others, are analyzed with the help of computers: the spoken word and the corresponding transcript as well as the voice pitch, speaking velocity and facial expressions of the speaker. For this purpose, intelligent video analysis makes particular use of speech, image, and text recognition technologies. Initially, these analysis processes run separately, but ultimately, they are to be combined, because the AI cannot always unambiguously recognize an emotion on the basis of just one of the previously mentioned modalities.

The challenge of this project in the field of digital humanities lies in the fact that only a small amount of training data is available for the use case, with which modern methods such as Deep Learning can only be inadequately trained. Therefore, with the approach of hybrid learning and especially transfer learning, methods are used to generate knowledge from other datasets that do not originate directly from the field of digital humanities and to transfer this knowledge to the use case. To this end, research is being conducted in particular in the area of multilingual embeddings and multitask learning.

Logistics

Logistics – as the science of moving things in a meaningful and purposeful way – follows simple rules. The complexity of many logistics tasks only arises from the multitude of possible options. In this respect, there are parallels to the game Go, where Machine Learning in the form of deep neural networks has led to groundbreaking successes. In our projects, we use AI to support people in logistics tasks and solve real-world problems, for example by optimizing processes or designing them to be self-adaptive.

Specialized AI for a new generation of autonomous robots in logistics

Autonomous robots – illustrated in our project by the example of the LoadRunner – are becoming increasingly important for the reliable transport of goods in warehouses and production. AI solutions developed at ML2R enable precise and fast localization of the robots, which move at speeds of up to 100 km/h. In this way, the LoadRunners can be used efficiently and safely even in a dynamic environment.

Flexible and resilient logistics processes are key success factors for many companies. Warehouses, where goods are stored and prepared for onward transport with the help of autonomous robots, play a central role in this. For autonomous robots to be used, they have to find their way around and navigate in a dynamic environment. An important prerequisite for navigation is global localization, which means determining the respective position as precisely as possible, even in larger halls with few orientation points.

ML2R scientists use specialized AI applications and a new generation of autonomous robots to solve these intralogistics tasks. Also known as LoadRunners, these autonomous robots can transport smaller packages of goods, are equipped with a chassis that is mobile in all directions, and attain a top speed of 100 km/h. In addition, the individual LoadRunners can operate autonomously in a swarm. Due to the high speed and dynamics, conventional sensors are unsuitable for localizing the LoadRunners.

Therefore, as part of the research at ML2R, a method based on Machine Learning was developed to determine the position within a hall. The LoadRunners are equipped with a floor camera that enables to determine the global position and orientation of the robot up to 400 times per second with an accuracy of a few centimeters. To do this, the AI solution processes the camera images in such a way that each image is assigned a position in space and the orientation is determined.

For the ML2R research work, a special floor covering was laid in the hall that has colored spots. The LoadRunner’s camera extracts all relevant information from the images of the floor and uses the colored spots as recognizable features, also referred to as keypoints. In the next step, a fingerprint, which is as unique an identifier as possible, is generated from the keypoints.

A key element of the AI solution is an autoencoder developed by ML2R scientists, which allows the fingerprints to be efficiently and uniquely found in a database. The results from our research on resource-aware Machine Learning have also made it possible to run the ML procedures in real time on the limited hardware resources of the robot. In this way, we have been able to determine the position of the LoadRunner with great accuracy even at high speeds. This laid the foundation for the use of a new generation of autonomous robots in logistics.

For further explanations on the localization of autonomous robots by means of floor cameras, please also refer to our ML2R blog post (in German) on Machine Learning.

Industry 4.0

AI and ML technologies have the potential to shorten development cycles, optimize the use of resources and make industrial production more efficient overall. We are happy to support and offer guidance to medium-sized companies, especially with respect to optimizing their processes with AI and ML methods. Our offers for cooperation hereby cover the entire value chain: from feasibility analysis to strategy development, consultation on technology deployment to sustainable implementation and continuous improvement.

Experimental Designs using AI: Accelerated Development and Optimized Use of Resources

Regardless of whether the challenge is to find new products, materials and formulations or to optimize existing ones: Our AI-based methodology helps you in the planning and optimal execution of trials. In this way, you can use your resources efficiently and save time and money.

In order to successfully compete in the global market, manufacturing companies must develop and optimize their products, materials and formulations ever faster. Traditional Designs of Experiment (DoEs), often find it difficult to guarantee high levels of velocity while maintaining the highest quality standards and optimal use of resources.

Experiments are essential for the development of new formulations and materials. Their parameters often allow for a wide range of possible combinations. The testing of each of these possibilities requires a great deal of human and material resources and is therefore time-consuming and costly.

We have developed an Artificial Intelligence-based DoE methodology that optimizes experimental designs as well as the use of resources, thus saving time and reducing costs. With our DoE methodology, we help you reach your goal faster, more reliably and more efficiently than with conventional methods.

Depending on the initial state, AI-based technology can save up to 50 percent of trials compared to traditional concepts of test planning. In addition, problems previously considered too complex too solve can be resolved using AI-based methods. Based on Bayesian optimization, our DoE methodology generates iterative experiment plans that lead you to your goal as quickly as possible.

The technology we have developed is based on our research of hybrid methods that combine Machine Learning with expert knowledge. In this way, we also support you in anchoring and optimally using existing knowledge in your company in the long term.

Data Summaries: Smart Data Summaries for More Efficient Production

For an efficient production, machine and sensor data must be collected, combined and evaluated. In a cooperative project with the company Achenbach Buschhütten, ML2R researchers developed new methods by which data from rolling mills was condensed into data summaries that can be easily analyzed and processed.

The aim of interconnecting machines and components in the context of Industry 4.0 is primarily targeted at monitoring and predicting machine behavior. This makes it possible to avoid unplanned downtime and to increase the efficiency of production. The key hereby lies within the efficient communication of machines among each other and with the cloud. In many companies, the implementation of the respective data communication is still in its early stages, but at Achenbach Buschhütten it is already firmly integrated into day-to-day business. Achenbach Buschhütten is an independent family-owned company looking back on more than 565 years of history and 125 years of experience in rolling mill equipment. The company is a pioneer in the interconnection of industrial plants with their own cloud-based platform Achenbach OPTILINK®. Achenbach Buschhütten is hence an ideal research partner for the Competence Center ML2R.

If machine data is available in large quantities through an interlinkage, it should be analyzed profitably. One challenge is that the large amounts of data can hardly be stored and analyzed in a timely manner. In addition, it is clear that a profitable analysis must always be carried out in close cooperation with the plant engineers.

Data summaries are an ideal approach to reducing the amount of data and making it comprehensible for users, such as plant engineers. A data summary is a small, well-chosen excerpt from the overall data that presents as much information and special features of the whole data set as possible. Formally, summaries can be represented using submodular functions, such as the “Informative Vector Machine”. Submodular functions provide a theoretically sound framework for the analysis of data summarization techniques while formalizing the notion of summarization in a mathematically precise way. This can then be used to select small and expressive summaries in linear time. On the other hand, the quality of the summaries and the selection process can be objectified and compared with the subjective impressions of the operating engineers.

At TU Dortmund University, methods for the summarization of data from astrophysics have been developed, which allow physicists to gain a quick overview of large measurement series. These techniques were further developed at ML2R in cooperation with Achenbach Buschhütten and adapted to the use on sensor data from rolling mills, which are available as time series. Further research on methods for data summarization is a vital part of the cooperation between Achenbach Buschhütten and the Competence Center ML2R.

Cognitive Process Automation

In any company, understanding, summarizing and categorizing texts is an essential part of many work processes. We support you in automating cognitive processes: Benefit from Natural Language Understanding to understand and interpret texts using AI. This saves you time and money and makes your workflows more efficient. Our solutions have also proven themselves to be well-suited for specialist and industry-specific texts, such as court rulings, medical documentation or maintenance reports.

Insight Financials: Generating Financial and Economic Analyses swiftly and by automation

With our tool designed for financial and business professionals, you’ll have information at your fingertips quickly in order to create automated analyses. Our search and analysis engine allows you to automatically find company reports published on the web and extract data. You are hereby able to save time when researching, analyzing and preparing financial and economic data. You are able to use this data for example for market analyses, benchmarking and your strategic corporate planning.

Thousands of company reports, such as stock exchange prospectuses and annual financial statements, are published every year. These reports are based on different legal requirements. The financial figures they contain are available in an unstructured form, are complex and can be defined differently. Therefore, manual information research for benchmarking and market analysis based on company reports is very costly and time-consuming.

Insight Financials is an intelligent, AI-based, cross-industry and cross-company tool that facilitates the work of analysts, consultants, corporate strategists and investors. Based on methods of Artificial Intelligence and Machine Learning, our tool allows for the automated sighting of financial reports from thousands of companies using autonomous web crawlers.

All information essential to you as a user is extracted and provided in real-time through an intuitive web application. The tool offers a granular search process, various filtering options and a benchmarking tool. It is also able to detect outliers and make predictions based on the extracted, temporary data. You can hence make a direct comparison between competitors, which poses valuable information for a company’s management as well as for auditors and investors.

The neural network-based analysis, or so-called parsing, of automatically downloaded company reports was developed by researchers at the Competence Center ML2R. In addition, the ML2R focuses on resource-efficient ML technologies: The language and extraction models, used for the automatic extraction of the financial figures, do not require any specialized, cost-intensive infrastructure.

RICO: Artificial Intelligence facilitates Coding and Billing of Hospital Services

The “RightCoding” (RICO) software simplifies and automates billing processes. Hospital employees use RICO to code diagnoses, seamlessly record treatments and bill health insurance providers for services. The respective processes are hence conducted in a manner that is simpler, more secure and in compliance with data protection regulations.

In order for hospitals to be able to bill health insurance providers for their services in the treatment of patients, the conducted services must be coded. This process is essential for the securing of hospital revenues. However, it demands a high degree of personnel resources and is time-consuming as well as at times prone to errors.

Together with GSG Consulting GmbH and with the participation of ML2R scientists, Fraunhofer IAIS has developed the AI-backed software RICO, which supports hospital staff in the coding process. RICO stands for “RightCoding”: It helps the hospital team to include all services when billing and checks whether all relevant supporting documents are available. If supporting documents are missing, the software automatically searches for them in the medical records.

RICO, the Artificial Intelligence-based coding support software, greatly simplifies the coding and billing process and implements it in a way that withstands medical service audits. RICO was developed in accordance with the highest data protection standards, is immediately operational without additional measures necessary, and is already in use in multiple hospitals.

ML2R conducts research on hybrid ML methods that combine expert knowledge with Machine Learning. This involves, for example, linking contextual text understanding with medical knowledge as sets of rules such as treatment and disease codes or the German drug directory “Rote Liste”. The RICO software for AI-based automation of billing processes in hospitals is one result of this research.

Astrophysics

Machine Learning is indispensable in modern astrophysics to process and evaluate data. Only in this way is it possible to gain insights into the physical world and our universe from the observed raw data. In the course of this, researchers use ML methods that are interpretable, take background knowledge into account, and can process even large amounts of data in real time. Through our cooperation and training programs, users from business and industry also benefit from this experience.

Field of Application Astrophysics: Terrabyte Data Streams for Science and Education

Modern astrophysics relies on Machine Learning methods which are interpretable, take into account background knowledge, and adhere to real-time requirements. We use our long-term experience in this field of application to investigate the methodological foundations of such learning procedures and to train stakeholders from commercial enterprises in dealing with the corresponding requirements.

The exploration of trustworthy, hybrid, and resource-aware learning requires applications that place corresponding demands on Machine Learning techniques. Astrophysics encompasses several such applications, using complex and high-volume data streams. In the context of ML2R, astrophysics becomes a data provider which supports basic and advanced research.

The telescopes whose data we obtain use the Earth’s atmosphere to detect high-energy particles from the cosmos. Special telescope cameras record the atmospheric interactions of these particles in images. Only with Machine Learning on the basis of simulated data is it possible to draw conclusions from the visual observations to the relevant physical properties of the cosmic particles. Astrophysicists are interested in these properties in order to better understand our universe.

In the analysis of astrophysical data as in other use cases, it is essential for the prediction of the Machine Learning model to be trustworthy; otherwise, the analysis results would not allow any conclusions to be drawn about the actual relationships represented in the data. The inclusion of background knowledge also overlaps with the requirements of other use cases: Hybrid learning feeds learning procedures with knowledge that would otherwise not be inferable from the data. Consideration of computational resources is essential due to a data volume of up to 100 terabytes per night; power consumption caused by simulation and analysis has so far been a significant cost factor in the operation of astrophysical telescopes.

Fundamental research at ML2R benefits from these diverse challenges because they motivate innovative solutions to problems that recur in other fields of application. In addition, we can use a significant part of the astrophysical data to train stakeholders from commercial enterprises in dealing with these challenges. Last but not least, our long-term collaboration partners from physics also benefit from our contribution to the state of the art in Machine Learning.

Multimedia Analysis

Machine Learning is indispensable in modern astrophysics to process and evaluate data. Only in this way is it possible to gain insights into the physical world and our universe from the observed raw data. In the course of this, researchers use ML methods that are interpretable, take background knowledge into account, and can process even large amounts of data in real time. Through our cooperation and training programs, users from business and industry also benefit from this experience.

Speech recognition systems learn new expressions autonomously

Language is subject to constant change. Particularly in daily media reporting on new topics, new names and terms often emerge that haven’t been used before or were uncommon. In the context of media coverage on the Corona epidemic, for example the word Covid19 or the name Christian Drosten frequently appeared. Automatic speech recognition systems trained with Machine Learning must therefore constantly learn new names and terms to stay current.

Within the framework of the cooperation with ARD (German public broadcaster) under the leadership of Westdeutscher Rundfunk, scientists from ML2R and Fraunhofer IAIS have developed a self-learning speech recognition system. A particular challenge in the cooperation with ARD was that with the regional programs such as WDR, NDR, HR or SWR, all accents and dialects of the German language had to be covered. Neologisms and loan words from other languages such as “download” or “fun fact” also play an increasingly important role.

In solving this task, ML2R research in hybrid learning was particularly relevant. Thus, already transcribed audio data, but also pronunciation lexica and text data from the Internet have been used for continuous learning. By combining a broad range of knowledge sources in the context of hybrid Machine Learning, a significant improvement in recognition performance could be achieved.

Artificial Intelligence recognizes emotions in interviews with contemporary witnesses

In a research project conducted with the foundation “Haus der Geschichte“, scientists from ML2R and Fraunhofer IAIS are developing a technology that recognizes and categorizes emotions in interviews with contemporary witnesses. With the help of Machine Learning, algorithms for emotion recognition are being trained to enable experts and citizens interested in history to search specifically for emotions in interviews with contemporary witnesses, for example on the topic of the fall of the Berlin Wall.

In the video interviews with contemporary witnesses, the following aspects, among others, are analyzed with the help of computers: the spoken word and the corresponding transcript as well as the voice pitch, speaking velocity and facial expressions of the speaker. For this purpose, intelligent video analysis makes particular use of speech, image, and text recognition technologies. Initially, these analysis processes run separately, but ultimately, they are to be combined, because the AI cannot always unambiguously recognize an emotion on the basis of just one of the previously mentioned modalities.

The challenge of this project in the field of digital humanities lies in the fact that only a small amount of training data is available for the use case, with which modern methods such as Deep Learning can only be inadequately trained. Therefore, with the approach of hybrid learning and especially transfer learning, methods are used to generate knowledge from other datasets that do not originate directly from the field of digital humanities and to transfer this knowledge to the use case. To this end, research is being conducted in particular in the area of multilingual embeddings and multitask learning.

Multimedia Analysis

Logistics – as the science of moving things in a meaningful and purposeful way – follows simple rules. The complexity of many logistics tasks only arises from the multitude of possible options. In this respect, there are parallels to the game Go, where Machine Learning in the form of deep neural networks has led to groundbreaking successes. In our projects, we use AI to support people in logistics tasks and solve real-world problems, for example by optimizing processes or designing them to be self-adaptive.

Specialized AI for a new generation of autonomous robots in logistics

Autonomous robots – illustrated in our project by the example of the LoadRunner – are becoming increasingly important for the reliable transport of goods in warehouses and production. AI solutions developed at ML2R enable precise and fast localization of the robots, which move at speeds of up to 100 km/h. In this way, the LoadRunners can be used efficiently and safely even in a dynamic environment.

Flexible and resilient logistics processes are key success factors for many companies. Warehouses, where goods are stored and prepared for onward transport with the help of autonomous robots, play a central role in this. For autonomous robots to be used, they have to find their way around and navigate in a dynamic environment. An important prerequisite for navigation is global localization, which means determining the respective position as precisely as possible, even in larger halls with few orientation points.

ML2R scientists use specialized AI applications and a new generation of autonomous robots to solve these intralogistics tasks. Also known as LoadRunners, these autonomous robots can transport smaller packages of goods, are equipped with a chassis that is mobile in all directions, and attain a top speed of 100 km/h. In addition, the individual LoadRunners can operate autonomously in a swarm. Due to the high speed and dynamics, conventional sensors are unsuitable for localizing the LoadRunners.

Therefore, as part of the research at ML2R, a method based on Machine Learning was developed to determine the position within a hall. The LoadRunners are equipped with a floor camera that enables to determine the global position and orientation of the robot up to 400 times per second with an accuracy of a few centimeters. To do this, the AI solution processes the camera images in such a way that each image is assigned a position in space and the orientation is determined.

For the ML2R research work, a special floor covering was laid in the hall that has colored spots. The LoadRunner’s camera extracts all relevant information from the images of the floor and uses the colored spots as recognizable features, also referred to as keypoints. In the next step, a fingerprint, which is as unique an identifier as possible, is generated from the keypoints.

A key element of the AI solution is an autoencoder developed by ML2R scientists, which allows the fingerprints to be efficiently and uniquely found in a database. The results from our research on resource-aware Machine Learning have also made it possible to run the ML procedures in real time on the limited hardware resources of the robot. In this way, we have been able to determine the position of the LoadRunner with great accuracy even at high speeds. This laid the foundation for the use of a new generation of autonomous robots in logistics.

For further explanations on the localization of autonomous robots by means of floor cameras, please also refer to our ML2R blog post (in German) on Machine Learning.