Distributed systems are computer systems where multiple computer systems are working together effectively as a single unit. Examples of distributed systems include content distribution networks used to serve websites and online videos, supercomputers built out of multiple powerful mathematical processors and distributed databases used in many online systems. Distributed systems can be more resilient, more powerful and speedier than single-computer systems, but they can also require more maintenance and careful thought about organization and problems that can go wrong.
Advantages of a Distributed System
There are a number of potential advantages to using a distributed system. One of the easiest to understand is redundancy and resiliency. If a company is serving its website from a distributed set of servers, rather than a single server, it may be able to stay up even if one server physically fails. If data is distributed between multiple servers or disks, a common occurrence in modern distributed systems, there may not be any data loss even if a storage device ceases to work.
Speed and Content Distribution
Distributed systems can also be faster than single-computer systems. One of the advantages of a distributed database is that queries can be routed to a server with a particular user's information, rather than all requests having to go to a single machine that can be overloaded.
Requests can also be routed to servers physically close or on a speedy network connection to whoever wants the data, which can mean less time and other resources allocated to dealing with network traffic and bottlenecks. That's a common occurrence in content distribution networks used for online media.
Scaling and Parallelism
Once distributed systems are set up to distribute data among the servers involved, they can also be easily scalable. If they're well designed, it can be as simple as adding some new hardware and telling the network to add it to the distributed system.
Distributed systems can also be designed for parallelism. This is common in mathematical operations for things like weather modeling and scientific computing, where multiple powerful processors can divide up independent parts of complex simulations and get the answer faster than they would running them in series.
Distributed Computing Challenges
One big challenge with distributed computing is that it can be hard for programmers to reason about. There can be challenges in how to distribute data to ensure that resiliency requirements are met under various unexpected conditions.
If devices need to synchronize, there can be difficult-to-spot bugs that cause them to wait on each other to transmit data or accidentally try to read or write the same piece of data at the same time, causing errors.
Security and privacy can also become an issue with distributed systems, since people's data is stored across multiple computers, sometimes in multiple physical locations. Distributed systems can also be overkill for some tasks, using more physical resources and engineering time than is necessary.
- IBM: What Is Distributed Computing?
- Yale: Distributed Computing
- Weiner Lecture Archives at UC Berkeley: CS 61A Lecture Notes: Chapter 4: Distributed and Parallel Computing
- EJB Tutorial: Challenges for a Distributed System
- Cloudflare: What is a CDN?
- Advanced Clustering Technologies: HPC CLUSTERS
- Coursera: Distributed Database