Kimchi

tags: #geo-distributed #article

source: Kimchi: Network Cost-aware Geo-distributed Data Analytics System [link]

• Problem:
  • Data transfer cost across distributed sites should also be considered as its one of the most expensive metrics.
• Approach/Intuition:
  • Schedules placement of tasks based on data transfer cost, input data size, locations and desired cost performance tradeoff
  • improves cost-aware task placement, a cost-aware task adjustment, and a cost-aware push-based mechanism.
• Implementation:
  • GDA MapReduce models: synchronous barrier and asynchronous push-based shuffle.