Large-scale Multi-agent Simulation and Crowd Sensing with Humans in the Loop
Creating Augmented Virtuality
Overview
Introduction to Augmented Virtuality
Main topic of this talk is Fusion of Real and Virtual worlds creating Augmented Virtuality by using Mobile Agents!
Simulation of Socio-Technical Systems
-
Socio-technical systems are characterized by interactions of:
- Human-Human (initiated by a human)
- Human-Machine (initiated by a human)
- Machine-Human (initiated by a machine, e.g., a chat bot)
- Machine-Machine (initiated by a machine)
-
The simulation of social ensemble behaviour requires simplification of interactions and individual behaviour
-
Commonly simulations are performed with less than 1000 entities (humans, machines, ..) in a sandbox world
-
Agent-based Modelling (ABM) is a suitable behaviour model for simulation
Augmented Reality
- Augmented reality is well known for extending the real world by adding computer-generated perceptual information and overlaid sensory information
[1]
Field Studies
- Experimental field studies are commonly used in social science to test social models or to derive social models
- The ensemble size in field studies is often limited to less than 1000 individuals or entities
Data Mining and Machine Learning are important tools to derive meaningful information from experimental and aggregated data.
Taxonomy of Data Mining
[3]
Crowd Sensing
-
Crowd data can be used in field studies to extend the information data base or replace classical (survey) field studies
-
Mobile Crowd Sensing combines aggregation of user data and mobile computing, i.e., creating spatially annotated data traces
-
Among data supplied by users explicitly, sensor data of mobile devices can be used, too. But: Weakly correlated data!
[2]
Augmented Virtuality
- Simulation worlds are commonly closed and rely on artificial sensory information generated by the simulator program or using data collected off-line (→ field studies).
A new Simulation paradigm providing augmented virtuality
Integration of Crowd Sensing and social Data Mining in simulation worlds
- The simulation world interacts with real world environments, humans, machines, and other virtual worlds in real-time using …
Agent-based Modelling and Computation
- Agents can represent artificial humans, bots, machines ..
- Agents can be used for distributed mobile computing → Crowd Sensing
Augmented Virtuality
Augmented Virtuality
Multi-Virtual Worlds
- Multiple virtual worlds can be connected by the augmented virtuality approach.
- Each virtual world can be considered as a world and model part
- Multiple virtual worlds can be simulated independently and in parallel → Speed-up!
- Virtual and physical world fusion by mobile agents …
Challenges
Time
-
Different time scales in real- and virtual worlds
- Simulation of distributed computing is much slower than Distributed Computing
- Simulation is discrete (processed step-wise) with respect to the time scale
- Interaction (reactivity) is different in virtual and real worlds
-
Short-term versa long-term:
- A simulation is performed commonly on a short time interval (window with defined start and end point)
- Real world environments with interaction between humans and machines do not pose a well defined starting or end point → long-term execution
-
Big Data and Information Strength
Challenges
Space
-
Different spatial scales in real and virtual worlds
-
Three different spatial models:
- Real and virtual worlds cover non-overlapping areas
- Real and virtual worlds cover overlapping areas
- Real world area is mapped on virtual world
Sensor Aggregation
Operational Layers
Sensor Aggregation
Sensor Aggregation
Virtual Sensor
-
A virtual sensor consists of different components:
- The environment of a sensor is a set of input streams of data generated from physical or virtual sensors.
- The environment defines the context within the virtual sensor operates.
- An aggregator processes the input streams and performs sensor data fusion
- A filter produces a set of output streams.
-
Agents can be used to implement virtual sensors
Sensor Aggregation
Sensor Aggregation
Virtual Sensor Network
-
Networks of Virtual sensors compose processing chains (user defined or ad-hoc and self-organizing).
-
Virtual sensors can operate in real-world environments (e.g., executed on mobile devices) or in virtual worlds (simulation).
-
Agents implement the sensor data aggregator and filter function, performing the fusion, storage, and communication and represent mobile transport entities.
-
Agent processing is virtualized by an unified Agent Processing Platform.
Sensor Aggregation
Agents and Simulation
Agent-based Modelling
Emergence
-
Complex systems are characterized by emergent phenomena — patterns that appear to be quite complex can often be generated by simple rules.
-
Emergence is a property classically exhibited by many agent-based models
-
It occurs when an attribute that can be described at a system level is not specifically encoded at the individual level.
Modelling
-
Agent-based modelling investigates the emergence behaviour of complex systems by defining a set of simple behaviour rules for individuals.
-
Artificial Agents are characterized by:
- Autonomy, loosely coupled, social
- Reactivity, Pro-activity, goal-driven
- Learning, Adaptation, Dynamic behaviour
- Self-* (organization, adaptation, learning, ..)
Agent-based Computation
Distributed Computing
-
Agents can be used for distributed and mobile computation in heterogeneous environments, too!
-
Mobile agents can migrate between different host platforms:
- Mobile Devices and Embedded Devices (Sensornetworks)
- Stationary Computers
- Servers and Clouds
-
Computational agents are loosely coupled to their environment and agent processing platforms
Multi-Agent Systems (MAS)
- Large collection of loosely coupled computational processes interacting with each other. MAS can be used to implement:
- autonomous,
- reliable, and
- adaptive data processing in distributed networks.
Sensor Clouds and the Internet
- Deployment of Agents can overcome interface and network barriers
- Closing the gap arising between different platforms and environments
- Integration of Sensor Networks and Crowd Sensing in Clouds
Agent Processing Platform
JavaScript Agent Machine (JAM)
-
Agents are programmed in JavaScript; JAM is programmed in JavaScript ⇒
-
One JAM can process up to 1000 agents/s and can be embedded in
- WEB pages;
- Mobile Apps (Android, iOS); or executed standalone on
- Embedded system devices, desktop, notebooks, and server computer
- Simulations
-
Agents behaviour is modelled with Activity-Transition Graphs (ATG)
Agent Processing Platform
-
JAM agents are mobile (carrying data and behaviour / code)
-
JAM agents can interact (communicate) with eachother via:
- Tuple spaces (public)
- Signals (lightweight messages, private)
Simulation Environment for JAM
Simulation Environment for JAM
