The SoProMon project (Sonification for Process Monitoring) investigates how to best support users in the real-time monitoring of processes by presenting process-relevant data using both visuals and audio. The project is being carried out as a joint cooperation between the research groups Workflow Systems and Technology and the Ambient Intelligence Group at CITEC/Bielefeld University, Germany (https://www.cit-ec.de/en/ami).
As process monitoring is a task that is often performed in parallel to other tasks, we specifically investigate the usage of sonification, the presentation of data as (non-speech) sound, to enhance current visual-based monitoring approaches. Auditory alerts and alarms are already in use in many control centers but come with several drawbacks, such as the possibility of so-called "alarm floods", in which operators do not take alarms as seriously as they are or ignore them altogether due to an excessive amount of occurring alerts.
However, apart from simple auditory cues, more complex sonifications are capable to represent continuous values by mapping data to auditory features of sound, such as pitch, level, duration, timbral characteristics, brightness, roughness or spatial location of sound events in a stereo or multichannel display.
The SoProMon project thus aims at investigating into the usage of sonification to enhance current visual process monitoring systems for a), its application in control centers, such as e.g. in
- Industrial production and logistics
- Power Plants
- Road traffic or Mass Transit Control Centers
but also for b), areas that typically do not feature full-time monitoring personnel, but rather employees that need to monitor processes in parallel to their main work, such as e.g.
- Office clerks that need to monitor administrative processes, such as e.g. insurance claims
- Managers and supervisors that want to keep an overview over their business processes
- IT administrators that monitor e-commerce processes or need to be informed about potential network intrusions
The SoProMon project specifically tries to answer the question which kind of sound-assisted monitoring is best suited for what kind of usage scenarios, as well as how different types of sound-enhanced process monitoring attract and direct attention and concentration. To reach these goals, we will
- develop specific methods for specific information types
- develop tools for their integration into practical multi-modal monitoring scenarios
- evaluate of their effectiveness, acceptance and utility to organize a user’s focus of attention
Dr. Thomas Hermann, Bielefeld University
Supplementary material and data:
- video recording of the process monitoring component in Sota mode (State-of-the-Art, auditory alerts)
- video recording of the process monitoring component in Son mode (continuous soundscape sonification)
Sonification for Process Analysis
Besides real-time process monitoring, sonification also lends itself to the retrospective analysis of process execution data.
We have developed a ProM plugin for multi-modal process data analysis to investigate the potential of sonification in this domain.
Supplementary material and data:
- process_log.xes: The log file that the recorded sonifications are based on, contains around 12000 events in 12 hours.
- process_sonifications.zip: Archive with different audio recordings of the plugin.
- tracescomplete.mp3: Sonification of "start page" activity, only piano, compressed to 20 seconds.
- order.mp3: Sonification of "order" activity, only piano, compressed to 10 seconds.
- browsechoose.mp3: Sonification of activities "browse" and "choose" (mapped to pitch), trace mapped to instrument, 20 seconds.
- browsechoosesequence.mp3: similiar to browsechoose.mp3, except that one trace is played after the other.
- complete.mp3: several activities mapped to melodies, trace mapped to instrument.
- completereversed.mp3: several activities mapped to instrument, trace is not mapped.
Alpha version of the ProM Plugin:
- Alpha version of the ProM Plugin plugin for multi-modal process data analysis.
- source code and instructions for the integration of our ProM Plugin: https://github.com/felixamerbauer/business_process_sonification_package
(Optional) Soundbank for Standalone-App / ProM-Package
- the standalone-app / ProM-package support using soundbanks in the SF2 format
- go to http://www.schristiancollins.com/generaluser.php and download the latest version of the soundfont in the Current section of the page (https://dl.dropboxusercontent.com/u/8126161/GeneralUser_GS_1.47.zip)
- extract zip and copy <code>GeneralUser GS v1.47.sf2</code> (only this file is required) to the folder containing the standalone JAR file or to the ProM installation folder (the one containing e.g. <code>ProM66.bat</code>/<code>ProM66.sh</code>)
|internal research projects|
|Research Group Workflow Systems and Technology|