New aquaponics project gives AI not only a brain, but also eyes

The so-called ‘synaesthetic AI’ in a new SDIS project brings together sensor inputs and acquired knowledge in aquaponics and thus facilitates the construction and operation of such systems. However, the approach of combining live data and existing data sets for artificial intelligence can be used in all cyber-physical systems.

A new project by Studio SDIS and Pro-Aquaponik-GmbH is conducting research at the interface between ‘Generative Artificial Intelligence’ (GenAI), ‘EdgeAI’ and ‘Federated Learning’ in the context of ‘cyber-physical systems’ and ‘IoT’. In ‘Virtual Aquaponics Consultant’, a system is to be developed that uses technologies from the aforementioned areas to automate the operation of plants and thus facilitate the scalability of these systems. The use case in this project is aquaponics, which is ideally suited to the project objective due to its high level of complexity.

Cyber-physical systems combine physical and software-based components. These systems connect the physical world with the digital world, for example by using sensors to monitor and control complex processes in real time.

As the name suggests, generative AI (GenAI) generates content such as images or text; the approach has become known through models such as ChatGPT and Dall-E. To do this, the artificial intelligence draws on background knowledge that has been fed into it. GenAI is, however, completely dependent on this formerly aquired datasets. It cannot process input from its surroundings. However, many applications could benefit from this kind of environmental perception, for example production facilities in which the AI could monitor processes and ultimately make recommendations for action.

Such data could be made available to the AI via sensors that are attached to the machines. The AI’s vast background knowledge therefore meets the current data situation, allowing the artificial intelligence to include an unimagined number of factors in its assessment.

Data protection vs computing power

Unfortunately, there is a challenge: powerful GenAI such as ChatGPT or Google Gemini are cloud-based, as they have to provide high computing power. However, data protection concerns prevent operating institutions of such systems from integrating their systems into cloud services.

This is why the new SDIS project is focussing on precisely this problem area and investigating how GenAI can be fed with sensor data without the raw data being transferred to the cloud. Edge AI and federated learning are important for this.

Put simply, small models are trained and then uploaded to the cloud. The original data cannot be retrieved from the trained models, or only with immense effort. Both are emerging technologies in the field of artificial intelligence and machine learning, which are often combined with each other to enable more efficient, more secure and more data protection-friendly applications.

GenAI, EdgeAI and Federated learning are all well-known and widely researched fields. However, the combination of GenAI with sensor technology and data-protected training on edge nodes has not been researched at all or only to a very limited extent. This project will therefore conduct basic research by investigating how GenAI can be fed with sensor data while maintaining high computing power and avoiding data protection concerns.

Well informed and cleary communicated ideas proposals

A sensor hub is being developed specifically for this purpose in the project, in which sensor data is pre-processed in an edge node with the help of federated learning. The sensor hub is connected to a cloud-based GenAI and passes its findings on to it. This creates a system that significantly facilitates the operation of cyber-physical systems. The system independently interprets the incoming sensor data with the help of the background knowledge available to it and can present suggested solutions in an easily understandable way.

The project team has dubbed this fusion of GenAI, edge AI and federated learning ‘synaesthetic AI’ or ‘Synaesthetic AI: SynAI’. Synaesthesia in this context describes the novel and as yet non-existent ability of AI to record and understand data streams from sensors like a ‘sensory perception’ and to link them with the knowledge embedded in the GenAI.

This creates completely new ways for SynAI to interact with its environment. In this project, it will be available to the operators of cyber-physical systems to support them in their operations. SynAI monitors the processes in the system, recognises irregularities and deviations from the norm and is available to the operators with recommendations for action and warnings.
This is how a conversation with such a SynAI could look like:

Edge AI refers to the execution of AI algorithms directly on devices at the edge of the network – so-called edge devices – instead of in central cloud data centres. These edge devices can be smartphones, IoT (Internet of Things) devices, sensors, cameras, drones and other embedded systems. Federated learning is a machine learning process in which a centralised model is trained using data stored on multiple decentralised devices or servers. Instead of collecting and storing data centrally, the data remains on the local devices.

The wastewater from the fish is used to irrigate the plants after pre-treatment. These benefit from the high nutrient content that is still present in the wastewater. As the plants consume the nutrients, the water is purified and can be returned to the fish’s habitat. Aquaponics is characterised by high yields of both vegetables and fish, with a simultaneous reduction in the amount of fresh water required, wastewater produced and the use of fertilisers, pesticides and antibiotics.

The specific project demonstrates the application in aquaponics systems. Aquaponics benefits from SynAI for two main reasons: On the one hand, it makes it easier to get started and set up aquaponics systems. This is because the optimal operation of aquaponics systems requires a great deal of expertise and experience as well as constant consideration of the conditions in the systems. It takes time to acquire the necessary knowledge, which is why the large-scale roll-out of these systems is still faltering. This is precisely where SynAI comes in: It already brings the necessary expertise to the table and can use sensor technology to record the ambient conditions in the system. This makes it easier for newcomers in particular to get started with aquaponics and operate the systems without having to build up years of experience and expertise.

On the other hand, SynAI helps to comply with the legal framework and to carry out the required official documentation. SynAI is a professional for everything from hygiene regulations to consumer health and at the same time enters the necessary data into the fish book and aquaculture register on an ongoing basis. A huge relief for operators.

This video on the demonstrator for our project EdeN shows the basics of aquaponics: