changes.
| | * [#Why Segment Nodes?] |
| | ** [#Multiple Sub-Grids] |
| | ** [#Dynamic Sub-Grids] |
| | * [#Node Segmentation Example] |
| | |
| | h2. Why Segment Nodes? |
| | Often in deployments you need to segment your grid nodes into several groups, having each group perform one or more subsets of jobs only. For example, let's say you have a scenario where you have some nodes only submitting jobs to grid (masters), and other groups of nodes only executing these jobs (workers). Then you would segment your grid into 2 groups, masters and workers, and have each group do only what it is supposed to do. |
| | |
| | h3. Multiple Sub-Grids |
| | Node segmentation allows you to create multiple sub-grids within your grid. Every sub-grid may have it's own static physical characteristics and logical responsibilities. All node characteristics, physical or logical, if they are static, can be specified in Spring Configuration and used in your [Topology SPI] or {{GridTask.map(..)}} {javadoc:GridTask.html} logic to implement the segmentation (this is shown in example below). |
| | |
| | Note, that based on its attributes, every node can participate in one or multiple segments. |
| | |
| | h3. Dynamic Sub-Grids |
| | You may also wish to segment your grid based on dynamic characteristics, not static. For example, what if you only want to include nodes that have less than 50% CPU utilization. In GridGain you can achieve this by using dynamic {{GridNodeMetrics}} {javadoc:GridNodeMetrics.html} provided by {{GridNodes}} {javadoc:GridNode.html}. All you would have to do is grab current CPU utilization from node metrics and in your {{GridTask.map(..)}} {javadoc:GridTask.html} method only pick the nodes with CPU's loaded under 50%. |
| | |
| | h2. Node Segmentation Example |
| | This example shows how you can segment your grid into static segments using GridGain. In GridGain such segmentation can be easily achieved with *{{node attributes}}* (see {{GridNode.getAttribute(String)}} {javadoc:GridNode.html#getAttribute(java.lang.String)}). Let's say that you want to segment your grid into 3 segments: {{'master'}}, {{'worker1'}}, and {{'worker2'}}. |
| | |
| | Every node at startup should get a certain number of attributes assigned to it. Here is how this can be done from Spring XML configuration: |
| | {code:xml} |
| | <bean id="grid.cfg" class="org.gridgain.grid.GridConfigurationAdapter" scope="singleton"> |
| | ... |
| | <property name="userAttributes"> |
| | <map> |
| | <!-- |
| | In our example, segment value can be either |
| | 'master', 'worker1', or 'worker2'. |
| | --> |
| | <entry key="segment" value="worker1"/> |
| | </map> |
| | </property> |
| | ... |
| | </bean> |
| | {code} |
| | | Then you can restrict the topology passed into {{GridTask.map(..)}} method by properly configuring [GridNodeFilterTopologySpi] to only include nodes from segments {{'segment.worker1'}} and {{'segment.worker2'}} and always exclude nodes belonging to {{'segment.master'}} segment. Here is an example: |
| | | Then you can restrict the topology passed into {{GridTask.map(..)}} method by properly configuring [GridNodeFilterTopologySpi] to only include nodes from segments {{'worker1'}} and {{'worker2'}} and always exclude nodes belonging to {{'master'}} segment. Here is an example: |
| | {code} |
| | <bean id="grid.custom.cfg" class="org.gridgain.grid.GridConfigurationAdapter" singleton="true"> |
| | ... |
| | <property name="topologySpi"> |
| | <bean class="org.gridgain.grid.spi.topology.nodefilter.GridNodeFilterTopologySpi"> |
| | <property name="filter"> |
| | <bean class="org.gridgain.grid.GridJexlNodeFilter"> |
| | <property name="expression"> |
| | <value> |
| | <![CDATA[ |
| | node.attributes['segment'] == 'worker1' || |
| | node.attributes['segment'] == 'worker2' |
| | ]]> |
| | </value> |
| | </property> |
| | </bean> |
| | </property> |
| | </bean> |
| | </property> |
| | ... |
| | </bean> |
| | {code} |
| | Alternatively, you can also implement your {{GridTask.map(..)}} {javadoc:GridTask.html#map(java.util.List, T)} method to map your jobs only to worker node segments. You can check which node segment a node belongs to by checking its attributes via {{GridNode.getAttribute(String)}} {javadoc:GridNode.html#getAttribute(java.lang.String)} method. Here is an example: |
| | {code} |
| | public class FooBarGridTask extends GridTaskAdapter<String, String> { |
| | ... |
| | public Map<GridJob, GridNode> map(List<GridNode> topology, String arg) { |
| | Map<GridJob, GridNode> jobs = new HashMap<GridJob, GridNode>(topology.size()); |
| | |
| | for (GridNode node : topology) { |
| | String segment = node.getAttribute("segment"); |
| | |
| | if (segment != null) { |
| | if (segment.equals("worker1")) { |
| | // This type of job should only execute on 'worker1' segment. |
| | jobs.put(new FooBarWorker1Job(arg), node); |
| | } |
| | else if (segment.equals("worker2")) { |
| | // This type of job should only execute on 'segment.worker2' segment. |
| | jobs.put(new FooBarWorker2Job(arg), node); |
| | } |
| | } |
| | else { |
| | throw new GridException("Node does not belong to any segment."); |
| | } |
| | } |
| | |
| | return jobs; |
| | } |
| | ... |
| | } |
| | {code} |
Page Operations
Browse Space
