Package org.apache.heron.api.utils
Class TopologyUtils
- java.lang.Object
-
- org.apache.heron.api.utils.TopologyUtils
-
public final class TopologyUtils extends Object
Utility to process TopologyAPI.Topology proto
-
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static String
getAdditionalClassPath(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static org.apache.heron.common.basics.ByteAmount
getCheckpointManagerRam(org.apache.heron.api.generated.TopologyAPI.Topology topology)
protected static Map<String,String>
getComponentConfigMap(org.apache.heron.api.generated.TopologyAPI.Topology topology, String key)
This is a util function to parse cpumap, rammap and diskmap.static Map<String,Double>
getComponentCpuMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Parses the value in Config.TOPOLOGY_COMPONENT_CPUMAP, and returns a map containing only component specified.static Map<String,org.apache.heron.common.basics.ByteAmount>
getComponentDiskMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Parses the value in Config.TOPOLOGY_COMPONENT_DISKMAP, and returns a map containing only component specified.static String
getComponentJvmOptions(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static Map<String,Integer>
getComponentParallelism(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static Map<String,org.apache.heron.common.basics.ByteAmount>
getComponentRamMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Parses the value in Config.TOPOLOGY_COMPONENT_RAMMAP, and returns a map containing only component specified.static Boolean
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, boolean defaultValue)
static Double
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Double defaultValue)
static Integer
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Integer defaultValue)
static Long
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Long defaultValue)
static String
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, String defaultValue)
static org.apache.heron.common.basics.ByteAmount
getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, org.apache.heron.common.basics.ByteAmount defaultValue)
static String
getConfigWithException(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key)
static String
getInstanceJvmOptions(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static int
getNumContainers(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static org.apache.heron.api.generated.TopologyAPI.Topology
getTopology(String topologyDefnFile)
static boolean
getTopologyRemoteDebuggingEnabled(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static int
getTotalInstance(Map<String,Integer> parallelismMap)
static int
getTotalInstance(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static String
lookUpTopologyDefnFile(String dir, String filename)
static String
makeClassPath(org.apache.heron.api.generated.TopologyAPI.Topology topology, String originalPackageFile)
static boolean
shouldStartCkptMgr(org.apache.heron.api.generated.TopologyAPI.Topology topology)
static void
validateTopology(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Throw a IllegalArgumentException if verifyTopology returns falsestatic boolean
verifyTopology(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Verify if the given topology has all the necessary information
-
-
-
Method Detail
-
getTopology
public static org.apache.heron.api.generated.TopologyAPI.Topology getTopology(String topologyDefnFile) throws InvalidTopologyException
- Throws:
InvalidTopologyException
-
getConfigWithDefault
public static String getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, String defaultValue)
-
getConfigWithDefault
public static Long getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Long defaultValue)
-
getConfigWithDefault
public static Integer getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Integer defaultValue)
-
getConfigWithDefault
public static Double getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, Double defaultValue)
-
getConfigWithDefault
public static org.apache.heron.common.basics.ByteAmount getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, org.apache.heron.common.basics.ByteAmount defaultValue)
-
getConfigWithDefault
public static Boolean getConfigWithDefault(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key, boolean defaultValue)
-
getConfigWithException
public static String getConfigWithException(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config, String key)
-
getComponentParallelism
public static Map<String,Integer> getComponentParallelism(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
getInstanceJvmOptions
public static String getInstanceJvmOptions(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
getComponentJvmOptions
public static String getComponentJvmOptions(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
getTotalInstance
public static int getTotalInstance(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
shouldStartCkptMgr
public static boolean shouldStartCkptMgr(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
getCheckpointManagerRam
public static org.apache.heron.common.basics.ByteAmount getCheckpointManagerRam(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
validateTopology
public static void validateTopology(org.apache.heron.api.generated.TopologyAPI.Topology topology) throws InvalidTopologyException
Throw a IllegalArgumentException if verifyTopology returns false- Parameters:
topology
- to validate- Throws:
InvalidTopologyException
-
verifyTopology
public static boolean verifyTopology(org.apache.heron.api.generated.TopologyAPI.Topology topology)
Verify if the given topology has all the necessary information
-
getAdditionalClassPath
public static String getAdditionalClassPath(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
getComponentCpuMapConfig
public static Map<String,Double> getComponentCpuMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology) throws RuntimeException
Parses the value in Config.TOPOLOGY_COMPONENT_CPUMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set- Parameters:
topology
- the topology def- Returns:
- a map (componentName -> cpu required)
- Throws:
RuntimeException
-
getComponentRamMapConfig
public static Map<String,org.apache.heron.common.basics.ByteAmount> getComponentRamMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology) throws RuntimeException
Parses the value in Config.TOPOLOGY_COMPONENT_RAMMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set- Parameters:
topology
- the topology def- Returns:
- a map (componentName -> RAM required)
- Throws:
RuntimeException
-
getComponentDiskMapConfig
public static Map<String,org.apache.heron.common.basics.ByteAmount> getComponentDiskMapConfig(org.apache.heron.api.generated.TopologyAPI.Topology topology) throws RuntimeException
Parses the value in Config.TOPOLOGY_COMPONENT_DISKMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set- Parameters:
topology
- the topology def- Returns:
- a map (componentName -> disk required)
- Throws:
RuntimeException
-
getComponentConfigMap
protected static Map<String,String> getComponentConfigMap(org.apache.heron.api.generated.TopologyAPI.Topology topology, String key) throws RuntimeException
This is a util function to parse cpumap, rammap and diskmap. A config example: "spout1:1,spout2:1,bolt1:5". The function validates component name and throws exception if any component name is wrong in the config.- Throws:
RuntimeException
-
getNumContainers
public static int getNumContainers(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
makeClassPath
public static String makeClassPath(org.apache.heron.api.generated.TopologyAPI.Topology topology, String originalPackageFile)
-
getTopologyRemoteDebuggingEnabled
public static boolean getTopologyRemoteDebuggingEnabled(org.apache.heron.api.generated.TopologyAPI.Topology topology)
-
-