Perhaps the best-known example of mainstream distributed (or grid) computing is Amazon’s Elastic Compute Cloud (EC2). In EC2, massive clusters of computers work together to process thousands of jobs and additional resources from within the clusters can be rapidly allocated as needed to support spikes in demand. A web application, for example, running at normal capacity in a virtualized server cluster on EC2 might get a mention from Oprah. The resulting increase in traffic could trigger quick scaling of resources.
Other examples range from co-opting end-user computers on the Internet to process human genome data when they aren’t in use to handling natural language processing or modeling climate change. When the computers are specifically dedicated to a cluster, then distributed computing becomes high performance computing (HPC). Other scenarios just take advantage of large numbers of available computers, combining spare processor clock signals to build a virtual, massively parallel computing platform.
The main criterion for defining distributed computing is that processing jobs are spread among multiple machines, none of which knows about the job as a whole or could complete the job independently. Sophisticated software (including Apple’s Xgrid technology that is available on all OS X Macs) then distributes pieces of the job to each “node” in the grid and assembles the results.
So does this have any application in the enterprise? Or is this just for geeks looking for very large prime numbers or donating processing time behind their screen savers to help researchers look for novel drugs? In fact, the potential uses in the enterprise are manifold. Here’s the top three:
Of course, Amazon has really pioneered the sale of their distributed computing capabilities to third parties, enabling countless applications to flexibly run in the cloud. Netflix and Pinterest, for example, rely on EC2. However, many businesses are building out private clouds running on their own distributed computing grids in either on-premise or virtual data centers. They can apply similar technologies to rapidly scale and shrink resource allocation via virtual machines in their own clusters to deal with shifting capacity needs.
The same technology that lets BOINC (Berkeley Open Infrastructure for Network Computing) harness the PCs of volunteers around the world to work on long-running scientific computing projects can be used by business to create local “desktop grids”. In this case, the desktop PCs throughout an organization can be tied together in a cluster that is transparent to end users and then leveraged to handle spikes in computing demand or crunch numbers in a variety of applications, freeing data center resources for major infrastructure needs and baseline capacity.
For example, a company could have a small datacenter for infrastructure, storage, and data warehousing needs, but offload transactional data processing to employees’s desktop computers overnight. A media company could offload rendering tasks in the same way.
Communication and collaboration
A final use of distributed computing in the enterprise is one that many companies have not adequately leveraged. However, distributed computing can provide fault-tolerant access to a variety of resources and facilitate robust, low-latency communications. Skype, for example, uses peer-to-peer technology (a form of distributed computing) to enhance its quality and functionality. Certain applications using Adobe’s Flash Media Server also use peer-to-peer connections for the same reason.
Torrents have gotten a bad name because of they way they are used to illegally distribute copyrighted materials, but they also represent a form of distributed computing and are very useful for rapid distribution and shared hosting of files within and outside of an organization.
Distributed computing began as a research tool and remains a darling of computer science today. However, the enterprise applications of related technologies are considerable and well worth the attention of IT departments looking to maximize the return on investments in computing resources at the desktop and data center levels and ensure fault tolerance across the enterprise.
Chris Dawson is a writer, speaker, and analyst with particular interests in educational technology, healthcare IT, and the intersection of the two with the cloud and BI. He is a contributing editor at ZDNet, Ziff Davis, and UBM Channel, and a senior editor at Edukwest. You can follow him on Twitter (@mrdatahs) and Google+ (+Christopher Dawson).Tags: Cloud,Technology