Class TopologyUtils


  • public final class TopologyUtils
    extends Object
    Utility to process TopologyAPI.Topology proto
    • Method Detail

      • getConfigWithDefault

        public static String getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                  String key,
                                                  String defaultValue)
      • getConfigWithDefault

        public static Long getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                String key,
                                                Long defaultValue)
      • getConfigWithDefault

        public static Integer getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                   String key,
                                                   Integer defaultValue)
      • getConfigWithDefault

        public static Double getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                  String key,
                                                  Double defaultValue)
      • getConfigWithDefault

        public static org.apache.heron.common.basics.ByteAmount getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                                                     String key,
                                                                                     org.apache.heron.common.basics.ByteAmount defaultValue)
      • getConfigWithDefault

        public static Boolean getConfigWithDefault​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                   String key,
                                                   boolean defaultValue)
      • getConfigWithException

        public static String getConfigWithException​(List<org.apache.heron.api.generated.TopologyAPI.Config.KeyValue> config,
                                                    String key)
      • getComponentParallelism

        public static Map<String,​Integer> getComponentParallelism​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • getInstanceJvmOptions

        public static String getInstanceJvmOptions​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • getComponentJvmOptions

        public static String getComponentJvmOptions​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • getTotalInstance

        public static int getTotalInstance​(Map<String,​Integer> parallelismMap)
      • getTotalInstance

        public static int getTotalInstance​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • shouldStartCkptMgr

        public static boolean shouldStartCkptMgr​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • getCheckpointManagerRam

        public static org.apache.heron.common.basics.ByteAmount getCheckpointManagerRam​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • validateTopology

        public static void validateTopology​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
                                     throws InvalidTopologyException
        Throw a IllegalArgumentException if verifyTopology returns false
        Parameters:
        topology - to validate
        Throws:
        InvalidTopologyException
      • verifyTopology

        public static boolean verifyTopology​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
        Verify if the given topology has all the necessary information
      • getAdditionalClassPath

        public static String getAdditionalClassPath​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • getComponentCpuMapConfig

        public static Map<String,​Double> getComponentCpuMapConfig​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
                                                                 throws RuntimeException
        Parses the value in Config.TOPOLOGY_COMPONENT_CPUMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set
        Parameters:
        topology - the topology def
        Returns:
        a map (componentName -> cpu required)
        Throws:
        RuntimeException
      • getComponentRamMapConfig

        public static Map<String,​org.apache.heron.common.basics.ByteAmount> getComponentRamMapConfig​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
                                                                                                    throws RuntimeException
        Parses the value in Config.TOPOLOGY_COMPONENT_RAMMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set
        Parameters:
        topology - the topology def
        Returns:
        a map (componentName -> RAM required)
        Throws:
        RuntimeException
      • getComponentDiskMapConfig

        public static Map<String,​org.apache.heron.common.basics.ByteAmount> getComponentDiskMapConfig​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
                                                                                                     throws RuntimeException
        Parses the value in Config.TOPOLOGY_COMPONENT_DISKMAP, and returns a map containing only component specified. Returns a empty map if the Config is not set
        Parameters:
        topology - the topology def
        Returns:
        a map (componentName -> disk required)
        Throws:
        RuntimeException
      • getComponentConfigMap

        protected static Map<String,​String> getComponentConfigMap​(org.apache.heron.api.generated.TopologyAPI.Topology topology,
                                                                        String key)
                                                                 throws RuntimeException
        This is a util function to parse cpumap, rammap and diskmap. A config example: "spout1:1,spout2:1,bolt1:5". The function validates component name and throws exception if any component name is wrong in the config.
        Throws:
        RuntimeException
      • getNumContainers

        public static int getNumContainers​(org.apache.heron.api.generated.TopologyAPI.Topology topology)
      • makeClassPath

        public static String makeClassPath​(org.apache.heron.api.generated.TopologyAPI.Topology topology,
                                           String originalPackageFile)
      • lookUpTopologyDefnFile

        public static String lookUpTopologyDefnFile​(String dir,
                                                    String filename)
      • getTopologyRemoteDebuggingEnabled

        public static boolean getTopologyRemoteDebuggingEnabled​(org.apache.heron.api.generated.TopologyAPI.Topology topology)