Docker ’swarm’ for FME’s geoprocessing

'Ugly details’ of parallel processing hidden. Swarms of workers perform compute intensive tasks.

FME’s trials with Docker continue. Bloggers Don Murray and Grant Arnold report on how Docker’s virtualization technology is changing the way FME is deployed both on-premises and in the cloud. Docker is the ‘ultimate abstraction layer’ in which to deploy fault-tolerant, scalable solutions. First Docker abstracted compute, then it abstracted networking, and soon it will abstract storage.

FME has been experimenting with Docker ‘swarms.’ The blog provides videos and code samples to show how a straightforward, but potentially compute-intensive task can be submitted to a swarm of ‘workers,’ instances of the FME calculation engine. The initiating job doesn’t wait for the task to finish but simply submits all requests to the workers. In fact the driver does not know it is talking to a swarm, the operation is identical to a submittal to a standalone FME engine. Docker swarm ‘magically’ looks after the networking details of sharing the connection with the workers, regardless of whether they are on a single machine or spread across multiple machines. Docker Swarm ‘hides and deals with all the ugly details.’

Click here to comment on this article

Click here to view this article in context on a desktop

© Oil IT Journal - all rights reserved.