STRETCH: Virtual Shared-Nothing Parallelism for Scalable and Elastic Stream Processing
Stream processing applications extract value from raw data through Directed Acyclic Graphs of data analysis tasks. Shared-nothing (SN) parallelism is the de-facto standard to scale stream processing applications. Given an application, SN parallelism instantiates several copies of each analysis task, making each instance responsible for a dedicated portion of the overall analysis, and relies on dedicated queues to exchange data among connected instances. On the one hand, SN parallelism can scale the execution of applications both up and out since threads can run task instances within and across processes/nodes. On the other hand, its lack of sharing can cause unnecessary overheads and hinder the scaling up when threads operate on data that could be jointly accessed in shared memory. This trade-off motivated us in studying a way for stream processing applications to leverage shared memory and boost the scale up (before the scale out) while adhering to the widely-adopted and SN-based APIs for stream processing applications.
We introduce STRETCH, a framework that maximizes the scale up and offers instantaneous elastic reconfigurations (without state transfer) for stream processing applications. We propose the concept of Virtual Shared-Nothing (VSN) parallelism and elasticity and provide formal definitions and correctness proofs for the semantics of the analysis tasks supported by STRETCH, showing they extend the ones found in common Stream Processing Engines. We also provide a fully implemented prototype and show that STRETCH's performance exceeds that of state-of-the-art baselines (Apache Flink and ScaleJoin) and offers, to the best of our knowledge, unprecedented ultra-fast reconfigurations, taking less than 40 ms even when provisioning tens of new task instances.
Ontology-Based Skill Description Learning for Flexible Production Systems
The increasing importance of resource-efficient production entails that manufacturing companies have to create a more dynamic production environment, with flexible manufacturing machines and processes. To fully utilize this potential of dynamic manufacturing through automatic production planning, formal skill descriptions of the machines are essential. However, generating those skill descriptions in a manual fashion is labor-intensive and requires extensive domain-knowledge. In this contribution an ontology-based semi-automatic skill description system that utilizes production logs and industrial ontologies through inductive logic programming is introduced and benefits and drawbacks of the proposed solution are evaluated.
Coverage Analysis for Satellite Downlink Networks
Satellite networks are promising to provide ubiquitous and high-capacity global wireless connectivity. Traditionally, satellite networks are modeled by placing satellites on a grid of multiple circular orbit geometries. Such a network model, however, requires intricate system-level simulations to evaluate coverage performance, and analytical understanding of the satellite network is limited. Continuing the success of stochastic geometry in a tractable analysis for terrestrial networks, in this paper, we develop novel models that are tractable for the coverage analysis of satellite networks using stochastic geometry. By modeling the locations of satellites and users using Poisson point processes on the surfaces of concentric spheres, we characterize analytical expressions for the coverage probability of a typical downlink user as a function of relevant parameters, including path-loss exponent, satellite height, density, and Nakagami fading parameter. Then, we also derive a tight lower bound of the coverage probability in closed-form expression while keeping full generality. Leveraging the derived expression, we identify the optimal density of satellites in terms of the height and the path-loss exponent. Our key finding is that the optimal average number of satellites decreases logarithmically with the network height to maximize the coverage performance. Simulation results verify the exactness of the derived expressions.