I've got some questions on my previous blog on the same topic asking basically for more specific explanation why it is critical to expose the grid topology. I guess when you work in this field for a long time you are getting blind to certain things - hence I will try to provide more detailed explanations below...
Example #1: Your grid is heterogeneous, i.e. nodes are different in terms of CPU, memory, network bandwidth, IO, etc. In this situation split should always be "weighted" by node's characteristics meaning that original task cannot be split into equal jobs but rather each job should be of appropriate "size". Failure to do so will result in unbalanced grid and drastically reduced performance.
Example #2: Your task distribution is heterogeneous, i.e. not all grid nodes can or should execute the jobs from this task. In other words, when you split you pick only a specific (often dynamically changing) sub-set of grid node for your task distribution. This scenario happens practically in any situation: you have existing silos of blades that handle specific type of tasks, you have sub-grid dedicated to a new application, specific segment of your network is dedicated to on-demand computing only, etc. Failure to provide this topology awareness really makes these cases hard or impossible to implement.
Now, examples I provided are really kind of basic examples and each particular real-life application(s) would likely have its quirks and requirements. Here's one real-life example from GridGain usage: our user wanted to do a dynamic CPU scavenging but only for desktops in a specific subnet; all other desktops in the office should be automatically made available to the grid from 9pm to 6am (basically, overnight) and not available during the rest of the day. With GridGain this is the matter of 5-10 lines of code - and it's beautifully elegant.