Package org.apache.heron.api
Class Config
- java.lang.Object
-
- java.util.AbstractMap<K,V>
-
- java.util.HashMap<String,Object>
-
- org.apache.heron.api.Config
-
- All Implemented Interfaces:
Serializable
,Cloneable
,Map<String,Object>
public class Config extends HashMap<String,Object>
Topology configs are specified as a plain old map. This class provides a convenient way to create a topology config map by providing setter methods for all the configs that can be set. It also makes it easier to do things like add serializations.Note that you may put other configurations in any of the configs. Heron will ignore anything it doesn't recognize, but your topologies are free to make use of them by reading them in the prepare method of Bolts or the open method of Spouts.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Config.TopologyReliabilityMode
What is the reliability mode under which we are running this topology Topology writers must set TOPOLOGY_RELIABILITY_MODE to one one of the following modes-
Nested classes/interfaces inherited from class java.util.AbstractMap
AbstractMap.SimpleEntry<K extends Object,V extends Object>, AbstractMap.SimpleImmutableEntry<K extends Object,V extends Object>
-
-
Field Summary
Fields Modifier and Type Field Description static String
HERON_JAVA_SERIALIZER_CLASS_NAME
The serializers available for TOPOLOGY_SERIALIZER_CLASSNAME.static String
HERON_KRYO_SERIALIZER_CLASS_NAME
static String
TOPOLOGY_ADDITIONAL_CLASSPATH
Any user defined classpath that needs to be passed to instances should be set in to config through this key.static String
TOPOLOGY_AUTO_TASK_HOOKS
A list of task hooks that are automatically added to every spout and bolt in the topology.static String
TOPOLOGY_CAP_TICKET
Cap ticket (if filed) for the topology.static String
TOPOLOGY_COMPONENT_CPUMAP
Per component RAM requirement.static String
TOPOLOGY_COMPONENT_DISKMAP
Per component RAM requirement.static String
TOPOLOGY_COMPONENT_JVMOPTS
Per component JVM options.static String
TOPOLOGY_COMPONENT_OUTPUT_BPS
The per component output bytes per second (rate limit) in this topology.static String
TOPOLOGY_COMPONENT_PARALLELISM
The per component parallelism for a component in this topology.static String
TOPOLOGY_COMPONENT_RAMMAP
Per component RAM requirement.static String
TOPOLOGY_CONTAINER_CPU_PADDING
Amount of CPU to pad each container.static String
TOPOLOGY_CONTAINER_CPU_REQUESTED
Number of CPU cores per container to be reserved for this topologystatic String
TOPOLOGY_CONTAINER_DISK_REQUESTED
Amount of disk per container to be reserved for this topology.static String
TOPOLOGY_CONTAINER_MAX_CPU_HINT
Hint for max number of CPU cores per container to be reserved for this topologystatic String
TOPOLOGY_CONTAINER_MAX_DISK_HINT
Hint for max amount of disk per container to be reserved for this topology.static String
TOPOLOGY_CONTAINER_MAX_NUM_INSTANCES
Max number of instances per container for this topology.static String
TOPOLOGY_CONTAINER_MAX_RAM_HINT
Hint for max amount of RAM per container to be reserved for this topology.static String
TOPOLOGY_CONTAINER_PADDING_PERCENTAGE
Percent of resource to pad each container.static String
TOPOLOGY_CONTAINER_RAM_PADDING
Amount of RAM to pad each container.static String
TOPOLOGY_CONTAINER_RAM_REQUESTED
Amount of RAM per container to be reserved for this topology.static String
TOPOLOGY_DEBUG
When set to true, Heron will log every message that's emitted.static String
TOPOLOGY_DROPTUPLES_UPON_BACKPRESSURE
Do we want to drop tuples instead of initiating Spout BackPressurestatic String
TOPOLOGY_ENABLE_ACKING
Deprecated.useTOPOLOGY_RELIABILITY_MODE
instead.static String
TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
True if Heron should timeout messages or not.static String
TOPOLOGY_ENVIRONMENT
Topology-specific environment properties to be added to an Heron instance.static String
TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
Whether or not to fallback to Java serialization in a topology.static String
TOPOLOGY_KRYO_DECORATORS
A list of classes that customize storm's kryo instance during start-up.static String
TOPOLOGY_KRYO_FACTORY
Class that specifies how to create a Kryo instance for serialization.static String
TOPOLOGY_KRYO_REGISTER
A list of serialization registrations if KryoSerializor is used.static String
TOPOLOGY_MAX_SPOUT_PENDING
The maximum number of tuples that can be pending on a spout task at any given time.static String
TOPOLOGY_MESSAGE_TIMEOUT_SECS
The maximum amount of time given to the topology to fully process a message emitted by a spout.static String
TOPOLOGY_METRICSMGR_RAM
Metrics Manager RAM requirementstatic String
TOPOLOGY_NAME
Name of the topology.static String
TOPOLOGY_PACKING_FFD_SORTING_STRATEGY
Sorting strategy for FirstFitDecreasingPacking algorithm.static String
TOPOLOGY_PROJECT_NAME
Project name of the topology, to help us with tagging which topologies are part of which project.static String
TOPOLOGY_RELIABILITY_MODE
A Heron topology can be run in any one of the TopologyReliabilityMode mode.static String
TOPOLOGY_REMOTE_DEBUGGING_ENABLE
Enable Remote debugging for java heron instancesstatic String
TOPOLOGY_SERIALIZER_CLASSNAME
The serialization class that is used to serialize/deserialize tuplesstatic String
TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
Whether or not Heron should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation.static String
TOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDS
What's the checkpoint interval for stateful topologies in secondsstatic String
TOPOLOGY_STATEFUL_CKPTMGR_RAM
Checkpoint Manager RAM requirement.static String
TOPOLOGY_STATEFUL_SPILL_STATE
Whether spill the state to disk for transferringstatic String
TOPOLOGY_STATEFUL_SPILL_STATE_LOCATION
The local disk location where the state is spilledstatic String
TOPOLOGY_STATEFUL_START_CLEAN
Boolean flag that says that the stateful topology should start from clean state, i.e.static String
TOPOLOGY_STMGR_RAM
Stream Manager RAM requirementstatic String
TOPOLOGY_STMGRS
The number of stmgr instances that should spin up to service this topology.static String
TOPOLOGY_TEAM_EMAIL
Email of the team which owns this topology.static String
TOPOLOGY_TEAM_ENVIRONMENT
Name of the of the environment this topology should run in.static String
TOPOLOGY_TEAM_NAME
Name of the team which owns this topology.static String
TOPOLOGY_TICK_TUPLE_FREQ_MS
How often (in milliseconds) a tick tuple from the "__system" component and "__tick" stream should be sent to tasks.static String
TOPOLOGY_TIMER_EVENTS
Timer events registered for a topology.static String
TOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECS
Amount of time to wait after deactivating a topology before updating itstatic String
TOPOLOGY_UPDATE_REACTIVATE_WAIT_SECS
After updating a topology, amount of time to wait for it to come back up before reactivating itstatic String
TOPOLOGY_WORKER_CHILDOPTS
Topology-specific options for the worker child process.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
addClasspath(Map<String,Object> conf, String classpath)
Set<String>
getApiVars()
List<String>
getAutoTaskHooks()
static List<String>
getAutoTaskHooks(Map<String,Object> conf)
void
registerDecorator(Class<? extends IKryoDecorator> klass)
static void
registerDecorator(Map conf, Class<? extends IKryoDecorator> klass)
void
registerKryoSerialization(Class klass)
void
registerKryoSerialization(Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
static void
registerKryoSerialization(Map<String,Object> conf, Class klass)
static void
registerKryoSerialization(Map<String,Object> conf, Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
void
registerTopologyTimerEvents(String name, Duration interval, Runnable task)
static void
registerTopologyTimerEvents(Map<String,Object> conf, String name, Duration interval, Runnable task)
Registers a timer event that executes periodicallyvoid
setAutoTaskHooks(List<String> hooks)
static void
setAutoTaskHooks(Map<String,Object> conf, List<String> hooks)
static void
setCheckpointManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setCheckpointManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setComponentCpu(String component, double cpu)
static void
setComponentCpu(Map<String,Object> conf, String component, double cpu)
void
setComponentCpuMap(String cpuMap)
static void
setComponentCpuMap(Map<String,Object> conf, String cpuMap)
void
setComponentDisk(String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
static void
setComponentDisk(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
void
setComponentDiskMap(String diskMap)
static void
setComponentDiskMap(Map<String,Object> conf, String diskMap)
void
setComponentJvmOptions(String component, String jvmOptions)
static void
setComponentJvmOptions(Map<String,Object> conf, String component, String jvmOptions)
void
setComponentParallelism(int parallelism)
static void
setComponentParallelism(Map<String,Object> conf, int parallelism)
void
setComponentRam(String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
static void
setComponentRam(Map<String,Object> conf, String component, long ramInBytes)
Deprecated.use setComponentRam(Map<String, Object> conf, String component, ByteAmount ramInBytes)static void
setComponentRam(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setComponentRamMap(String ramMap)
static void
setComponentRamMap(Map<String,Object> conf, String ramMap)
void
setContainerCpuPadding(double ncpus)
static void
setContainerCpuPadding(Map<String,Object> conf, double ncpus)
void
setContainerCpuRequested(double ncpus)
static void
setContainerCpuRequested(Map<String,Object> conf, double ncpus)
static void
setContainerDiskRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerDiskRequested(Map<String, Object> conf, ByteAmount nbytes)static void
setContainerDiskRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerDiskRequested(org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerMaxCpuHint(double ncpus)
static void
setContainerMaxCpuHint(Map<String,Object> conf, double ncpus)
static void
setContainerMaxDiskHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerMaxDiskHint(org.apache.heron.common.basics.ByteAmount nbytes)
static void
setContainerMaxRamHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerMaxRamHint(org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerPaddingPercentage(int percentage)
static void
setContainerPaddingPercentage(Map<String,Object> conf, int percentage)
static void
setContainerRamPadding(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerRamPadding(org.apache.heron.common.basics.ByteAmount nbytes)
static void
setContainerRamRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerRamRequested(Map<String, Object> conf, ByteAmount nbytes)static void
setContainerRamRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
void
setContainerRamRequested(org.apache.heron.common.basics.ByteAmount nbytes)
void
setDebug(boolean isOn)
static void
setDebug(Map<String,Object> conf, boolean isOn)
void
setEnableAcking(boolean acking)
Deprecated.static void
setEnableAcking(Map<String,Object> conf, boolean acking)
Deprecated.static void
setEnvironment(Map<String,Object> conf, Map env)
void
setFallBackOnJavaSerialization(boolean fallback)
static void
setFallBackOnJavaSerialization(Map conf, boolean fallback)
void
setFFDSortingStrategy(String sortingStrategy)
static void
setFFDSortingStrategy(Map<String,Object> conf, String sortingStrategy)
void
setKryoFactory(Class<? extends IKryoFactory> klass)
static void
setKryoFactory(Map conf, Class<? extends IKryoFactory> klass)
static void
setMaxNumInstancesPerContainer(Map<String,Object> conf, int n)
void
setMaxSpoutPending(int max)
static void
setMaxSpoutPending(Map<String,Object> conf, int max)
void
setMessageTimeoutSecs(int secs)
static void
setMessageTimeoutSecs(Map<String,Object> conf, int secs)
static void
setMetricsmgrRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setMetricsmgrRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setNumStmgrs(int stmgrs)
static void
setNumStmgrs(Map<String,Object> conf, int stmgrs)
void
setSerializationClassName(String className)
static void
setSerializationClassName(Map<String,Object> conf, String className)
void
setSkipMissingKryoRegistrations(boolean skip)
static void
setSkipMissingKryoRegistrations(Map conf, boolean skip)
static void
setStreamManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setStreamManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
void
setTeamEmail(String teamEmail)
static void
setTeamEmail(Map<String,Object> conf, String teamEmail)
void
setTeamName(String teamName)
static void
setTeamName(Map<String,Object> conf, String teamName)
void
setTickTupleFrequency(int seconds)
static void
setTickTupleFrequency(Map<String,Object> conf, int seconds)
static void
setTickTupleFrequencyMs(Map<String,Object> conf, long millis)
void
setTopologyCapTicket(String ticket)
static void
setTopologyCapTicket(Map<String,Object> conf, String ticket)
void
setTopologyComponentOutputBPS(long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead.static void
setTopologyComponentOutputBPS(Map<String,Object> conf, long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead.void
setTopologyContainerMaxNumInstances(int n)
void
setTopologyDropTuplesUponBackpressure(boolean dropTuples)
void
setTopologyProjectName(String project)
static void
setTopologyProjectName(Map<String,Object> conf, String project)
static void
setTopologyReliabilityMode(Map<String,Object> conf, Config.TopologyReliabilityMode mode)
void
setTopologyReliabilityMode(Config.TopologyReliabilityMode mode)
void
setTopologyRemoteDebugging(boolean isOn)
void
setTopologyStatefulCheckpointIntervalSecs(int secs)
static void
setTopologyStatefulCheckpointIntervalSecs(Map<String,Object> conf, int secs)
void
setTopologyStatefulSpillState(String spillState)
static void
setTopologyStatefulSpillState(Map<String,Object> conf, String spillState)
void
setTopologyStatefulSpillStateLocation(String location)
static void
setTopologyStatefulSpillStateLocation(Map<String,Object> conf, String location)
void
setTopologyStatefulStartClean(boolean clean)
static void
setTopologyStatefulStartClean(Map<String,Object> conf, boolean clean)
void
setUpdateDeactivateWaitDuration(int seconds)
void
setUpdateReactivateWaitDuration(int seconds)
-
Methods inherited from class java.util.HashMap
clear, clone, compute, computeIfAbsent, computeIfPresent, containsKey, containsValue, entrySet, forEach, get, getOrDefault, isEmpty, keySet, merge, put, putAll, putIfAbsent, remove, remove, replace, replace, replaceAll, size, values
-
Methods inherited from class java.util.AbstractMap
equals, hashCode, toString
-
-
-
-
Field Detail
-
TOPOLOGY_WORKER_CHILDOPTS
public static final String TOPOLOGY_WORKER_CHILDOPTS
Topology-specific options for the worker child process. This is used in addition to WORKER_CHILDOPTS.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_JVMOPTS
public static final String TOPOLOGY_COMPONENT_JVMOPTS
Per component JVM options. The format of this flag is something like spout0:jvmopt_for_spout0,spout1:jvmopt_for_spout1. Mostly should be used in conjunction with setComponentJvmOptions(). This is used in addition to TOPOLOGY_WORKER_CHILDOPTS. While TOPOLOGY_WORKER_CHILDOPTS applies for all components, this is per component- See Also:
- Constant Field Values
-
TOPOLOGY_TICK_TUPLE_FREQ_MS
public static final String TOPOLOGY_TICK_TUPLE_FREQ_MS
How often (in milliseconds) a tick tuple from the "__system" component and "__tick" stream should be sent to tasks. Meant to be used as a component-specific configuration.- See Also:
- Constant Field Values
-
TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
public static final String TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
True if Heron should timeout messages or not. Defaults to true. This is meant to be used in unit tests to prevent tuples from being accidentally timed out during the test.- See Also:
- Constant Field Values
-
TOPOLOGY_DEBUG
public static final String TOPOLOGY_DEBUG
When set to true, Heron will log every message that's emitted.- See Also:
- Constant Field Values
-
TOPOLOGY_STMGRS
public static final String TOPOLOGY_STMGRS
The number of stmgr instances that should spin up to service this topology. All the executors will be evenly shared by these stmgrs.- See Also:
- Constant Field Values
-
TOPOLOGY_MESSAGE_TIMEOUT_SECS
public static final String TOPOLOGY_MESSAGE_TIMEOUT_SECS
The maximum amount of time given to the topology to fully process a message emitted by a spout. If the message is not acked within this time frame, Heron will fail the message on the spout. Some spouts implementations will then replay the message at a later time.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_PARALLELISM
public static final String TOPOLOGY_COMPONENT_PARALLELISM
The per component parallelism for a component in this topology. Note:- If you are changing this, please change the utils.h as well- See Also:
- Constant Field Values
-
TOPOLOGY_MAX_SPOUT_PENDING
public static final String TOPOLOGY_MAX_SPOUT_PENDING
The maximum number of tuples that can be pending on a spout task at any given time. This config applies to individual tasks, not to spouts or topologies as a whole.A pending tuple is one that has been emitted from a spout but has not been acked or failed yet. Note that this config parameter has no effect for unreliable spouts that don't tag their tuples with a message id.
- See Also:
- Constant Field Values
-
TOPOLOGY_AUTO_TASK_HOOKS
public static final String TOPOLOGY_AUTO_TASK_HOOKS
A list of task hooks that are automatically added to every spout and bolt in the topology. An example of when you'd do this is to add a hook that integrates with your internal monitoring system. These hooks are instantiated using the zero-arg constructor.- See Also:
- Constant Field Values
-
TOPOLOGY_SERIALIZER_CLASSNAME
public static final String TOPOLOGY_SERIALIZER_CLASSNAME
The serialization class that is used to serialize/deserialize tuples- See Also:
- Constant Field Values
-
HERON_JAVA_SERIALIZER_CLASS_NAME
public static final String HERON_JAVA_SERIALIZER_CLASS_NAME
The serializers available for TOPOLOGY_SERIALIZER_CLASSNAME.- See Also:
- Constant Field Values
-
HERON_KRYO_SERIALIZER_CLASS_NAME
public static final String HERON_KRYO_SERIALIZER_CLASS_NAME
- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_FACTORY
public static final String TOPOLOGY_KRYO_FACTORY
Class that specifies how to create a Kryo instance for serialization. Heron will then apply topology.kryo.register. The default implementation implements topology.fall.back.on.java.serialization and turns references off.- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_REGISTER
public static final String TOPOLOGY_KRYO_REGISTER
A list of serialization registrations if KryoSerializor is used. In Kryo, the serialization can be the name of a class (in which case Kryo will automatically create a serializer for the class that saves all the object's fields), or an implementation of Kryo Serializer.See Kryo's documentation for more information about writing custom serializers.
- See Also:
- Constant Field Values
-
TOPOLOGY_KRYO_DECORATORS
public static final String TOPOLOGY_KRYO_DECORATORS
A list of classes that customize storm's kryo instance during start-up. Each listed class name must implement IKryoDecorator. During start-up the listed class is instantiated with 0 arguments, then its 'decorate' method is called with storm's kryo instance as the only argument.- See Also:
- Constant Field Values
-
TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
public static final String TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
Whether or not Heron should skip the loading of kryo registrations for which it does not know the class or have the serializer implementation. Otherwise, the task will fail to load and will throw an error at runtime. The use case of this is if you want to declare your serializations on the heron.yaml files on the cluster rather than every single time you submit a topology. Different applications may use different serializations and so a single application may not have the code for the other serializers used by other apps. By setting this config to true, Heron will ignore that it doesn't have those other serializations rather than throw an error.- See Also:
- Constant Field Values
-
TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
public static final String TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
Whether or not to fallback to Java serialization in a topology.- See Also:
- Constant Field Values
-
TOPOLOGY_ENABLE_ACKING
@Deprecated public static final String TOPOLOGY_ENABLE_ACKING
Deprecated.useTOPOLOGY_RELIABILITY_MODE
instead.Is the topology running in atleast-once mode?If this is set to false, then Heron will immediately ack tuples as soon as they come off the spout, effectively disabling reliability.
- See Also:
- Constant Field Values
-
TOPOLOGY_RELIABILITY_MODE
public static final String TOPOLOGY_RELIABILITY_MODE
A Heron topology can be run in any one of the TopologyReliabilityMode mode. The format of this flag is the string encoded values of the underlying TopologyReliabilityMode value.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_CPU_REQUESTED
public static final String TOPOLOGY_CONTAINER_CPU_REQUESTED
Number of CPU cores per container to be reserved for this topology- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_RAM_REQUESTED
public static final String TOPOLOGY_CONTAINER_RAM_REQUESTED
Amount of RAM per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_DISK_REQUESTED
public static final String TOPOLOGY_CONTAINER_DISK_REQUESTED
Amount of disk per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_CPU_HINT
public static final String TOPOLOGY_CONTAINER_MAX_CPU_HINT
Hint for max number of CPU cores per container to be reserved for this topology- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_RAM_HINT
public static final String TOPOLOGY_CONTAINER_MAX_RAM_HINT
Hint for max amount of RAM per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_DISK_HINT
public static final String TOPOLOGY_CONTAINER_MAX_DISK_HINT
Hint for max amount of disk per container to be reserved for this topology. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_MAX_NUM_INSTANCES
public static final String TOPOLOGY_CONTAINER_MAX_NUM_INSTANCES
Max number of instances per container for this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_PADDING_PERCENTAGE
public static final String TOPOLOGY_CONTAINER_PADDING_PERCENTAGE
Percent of resource to pad each container.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_RAM_PADDING
public static final String TOPOLOGY_CONTAINER_RAM_PADDING
Amount of RAM to pad each container. In bytes.- See Also:
- Constant Field Values
-
TOPOLOGY_CONTAINER_CPU_PADDING
public static final String TOPOLOGY_CONTAINER_CPU_PADDING
Amount of CPU to pad each container. In decimal form of percent of CPU shares.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_CPUMAP
public static final String TOPOLOGY_COMPONENT_CPUMAP
Per component RAM requirement. The format of this flag is something like spout0:0.2,spout1:0.2,bolt1:0.5.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_RAMMAP
public static final String TOPOLOGY_COMPONENT_RAMMAP
Per component RAM requirement. The format of this flag is something like spout0:12434,spout1:345353,bolt1:545356.- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_DISKMAP
public static final String TOPOLOGY_COMPONENT_DISKMAP
Per component RAM requirement. The format of this flag is something like spout0:12434,spout1:345353,bolt1:545356.- See Also:
- Constant Field Values
-
TOPOLOGY_PACKING_FFD_SORTING_STRATEGY
public static final String TOPOLOGY_PACKING_FFD_SORTING_STRATEGY
Sorting strategy for FirstFitDecreasingPacking algorithm. RAM_FIRST (default), or CPU_FIRST- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDS
public static final String TOPOLOGY_STATEFUL_CHECKPOINT_INTERVAL_SECONDS
What's the checkpoint interval for stateful topologies in seconds- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_START_CLEAN
public static final String TOPOLOGY_STATEFUL_START_CLEAN
Boolean flag that says that the stateful topology should start from clean state, i.e. ignore any checkpoint state- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_CKPTMGR_RAM
public static final String TOPOLOGY_STATEFUL_CKPTMGR_RAM
Checkpoint Manager RAM requirement.- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_SPILL_STATE
public static final String TOPOLOGY_STATEFUL_SPILL_STATE
Whether spill the state to disk for transferring- See Also:
- Constant Field Values
-
TOPOLOGY_STATEFUL_SPILL_STATE_LOCATION
public static final String TOPOLOGY_STATEFUL_SPILL_STATE_LOCATION
The local disk location where the state is spilled- See Also:
- Constant Field Values
-
TOPOLOGY_STMGR_RAM
public static final String TOPOLOGY_STMGR_RAM
Stream Manager RAM requirement- See Also:
- Constant Field Values
-
TOPOLOGY_METRICSMGR_RAM
public static final String TOPOLOGY_METRICSMGR_RAM
Metrics Manager RAM requirement- See Also:
- Constant Field Values
-
TOPOLOGY_NAME
public static final String TOPOLOGY_NAME
Name of the topology. This config is automatically set by Heron when the topology is submitted.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_NAME
public static final String TOPOLOGY_TEAM_NAME
Name of the team which owns this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_EMAIL
public static final String TOPOLOGY_TEAM_EMAIL
Email of the team which owns this topology.- See Also:
- Constant Field Values
-
TOPOLOGY_TEAM_ENVIRONMENT
public static final String TOPOLOGY_TEAM_ENVIRONMENT
Name of the of the environment this topology should run in.- See Also:
- Constant Field Values
-
TOPOLOGY_CAP_TICKET
public static final String TOPOLOGY_CAP_TICKET
Cap ticket (if filed) for the topology. If the topology is in prod this has to be set or it cannot be deployed.- See Also:
- Constant Field Values
-
TOPOLOGY_PROJECT_NAME
public static final String TOPOLOGY_PROJECT_NAME
Project name of the topology, to help us with tagging which topologies are part of which project. For example, if topology A and Topology B are part of the same project, we will like to aggregate them as part of the same project. This is required by Cap team.- See Also:
- Constant Field Values
-
TOPOLOGY_ADDITIONAL_CLASSPATH
public static final String TOPOLOGY_ADDITIONAL_CLASSPATH
Any user defined classpath that needs to be passed to instances should be set in to config through this key. The value will be of the format "cp1:cp2:cp3..."- See Also:
- Constant Field Values
-
TOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECS
public static final String TOPOLOGY_UPDATE_DEACTIVATE_WAIT_SECS
Amount of time to wait after deactivating a topology before updating it- See Also:
- Constant Field Values
-
TOPOLOGY_UPDATE_REACTIVATE_WAIT_SECS
public static final String TOPOLOGY_UPDATE_REACTIVATE_WAIT_SECS
After updating a topology, amount of time to wait for it to come back up before reactivating it- See Also:
- Constant Field Values
-
TOPOLOGY_ENVIRONMENT
public static final String TOPOLOGY_ENVIRONMENT
Topology-specific environment properties to be added to an Heron instance. This is added to the existing environment (that of the Heron instance). This variable contains Map<String, String>- See Also:
- Constant Field Values
-
TOPOLOGY_TIMER_EVENTS
public static final String TOPOLOGY_TIMER_EVENTS
Timer events registered for a topology. This is a Map<String, Pair<Duration, Runnable>>. Where the key is the name and the value contains the frequency of the event and the task to run.- See Also:
- Constant Field Values
-
TOPOLOGY_REMOTE_DEBUGGING_ENABLE
public static final String TOPOLOGY_REMOTE_DEBUGGING_ENABLE
Enable Remote debugging for java heron instances- See Also:
- Constant Field Values
-
TOPOLOGY_DROPTUPLES_UPON_BACKPRESSURE
public static final String TOPOLOGY_DROPTUPLES_UPON_BACKPRESSURE
Do we want to drop tuples instead of initiating Spout BackPressure- See Also:
- Constant Field Values
-
TOPOLOGY_COMPONENT_OUTPUT_BPS
public static final String TOPOLOGY_COMPONENT_OUTPUT_BPS
The per component output bytes per second (rate limit) in this topology. It works with the addConfiguration() function in ComponentConfigurationDeclarer class. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000); builder.setBolt(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);- See Also:
- Constant Field Values
-
-
Method Detail
-
setTopologyCapTicket
public static void setTopologyCapTicket(Map<String,Object> conf, String ticket)
-
setTopologyProjectName
public static void setTopologyProjectName(Map<String,Object> conf, String project)
-
setSerializationClassName
public static void setSerializationClassName(Map<String,Object> conf, String className)
-
setKryoFactory
public static void setKryoFactory(Map conf, Class<? extends IKryoFactory> klass)
-
setSkipMissingKryoRegistrations
public static void setSkipMissingKryoRegistrations(Map conf, boolean skip)
-
registerKryoSerialization
public static void registerKryoSerialization(Map<String,Object> conf, Class klass)
-
registerKryoSerialization
public static void registerKryoSerialization(Map<String,Object> conf, Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
-
registerDecorator
public static void registerDecorator(Map conf, Class<? extends IKryoDecorator> klass)
-
setFallBackOnJavaSerialization
public static void setFallBackOnJavaSerialization(Map conf, boolean fallback)
-
setEnableAcking
@Deprecated public static void setEnableAcking(Map<String,Object> conf, boolean acking)
Deprecated.Is topology running with acking enabled?
-
setComponentParallelism
public static void setComponentParallelism(Map<String,Object> conf, int parallelism)
-
setTickTupleFrequency
public static void setTickTupleFrequency(Map<String,Object> conf, int seconds)
-
setTickTupleFrequencyMs
public static void setTickTupleFrequencyMs(Map<String,Object> conf, long millis)
-
setTopologyReliabilityMode
public static void setTopologyReliabilityMode(Map<String,Object> conf, Config.TopologyReliabilityMode mode)
-
setContainerCpuRequested
public static void setContainerCpuRequested(Map<String,Object> conf, double ncpus)
-
setContainerDiskRequested
@Deprecated public static void setContainerDiskRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerDiskRequested(Map<String, Object> conf, ByteAmount nbytes)Users should use the version of this method at uses ByteAmount
-
setContainerDiskRequested
public static void setContainerDiskRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerRamRequested
@Deprecated public static void setContainerRamRequested(Map<String,Object> conf, long nbytes)
Deprecated.use setContainerRamRequested(Map<String, Object> conf, ByteAmount nbytes)Users should use the version of this method at uses ByteAmount
-
setContainerRamRequested
public static void setContainerRamRequested(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxCpuHint
public static void setContainerMaxCpuHint(Map<String,Object> conf, double ncpus)
-
setContainerMaxDiskHint
public static void setContainerMaxDiskHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxRamHint
public static void setContainerMaxRamHint(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerPaddingPercentage
public static void setContainerPaddingPercentage(Map<String,Object> conf, int percentage)
-
setContainerRamPadding
public static void setContainerRamPadding(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerCpuPadding
public static void setContainerCpuPadding(Map<String,Object> conf, double ncpus)
-
setMaxNumInstancesPerContainer
public static void setMaxNumInstancesPerContainer(Map<String,Object> conf, int n)
-
setComponentDiskMap
public static void setComponentDiskMap(Map<String,Object> conf, String diskMap)
-
setFFDSortingStrategy
public static void setFFDSortingStrategy(Map<String,Object> conf, String sortingStrategy)
-
setTopologyComponentOutputBPS
@Deprecated public static void setTopologyComponentOutputBPS(Map<String,Object> conf, long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);This function should not be used to set rate limiter in topology config.
-
setComponentRam
@Deprecated public static void setComponentRam(Map<String,Object> conf, String component, long ramInBytes)
Deprecated.use setComponentRam(Map<String, Object> conf, String component, ByteAmount ramInBytes)Users should use the version of this method at uses ByteAmount
-
setComponentCpu
public static void setComponentCpu(Map<String,Object> conf, String component, double cpu)
-
setComponentRam
public static void setComponentRam(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setComponentDisk
public static void setComponentDisk(Map<String,Object> conf, String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
-
setComponentJvmOptions
public static void setComponentJvmOptions(Map<String,Object> conf, String component, String jvmOptions)
-
setTopologyStatefulCheckpointIntervalSecs
public static void setTopologyStatefulCheckpointIntervalSecs(Map<String,Object> conf, int secs)
-
setTopologyStatefulStartClean
public static void setTopologyStatefulStartClean(Map<String,Object> conf, boolean clean)
-
setCheckpointManagerRam
public static void setCheckpointManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setTopologyStatefulSpillState
public static void setTopologyStatefulSpillState(Map<String,Object> conf, String spillState)
-
setTopologyStatefulSpillStateLocation
public static void setTopologyStatefulSpillStateLocation(Map<String,Object> conf, String location)
-
setStreamManagerRam
public static void setStreamManagerRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setMetricsmgrRam
public static void setMetricsmgrRam(Map<String,Object> conf, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
registerTopologyTimerEvents
public static void registerTopologyTimerEvents(Map<String,Object> conf, String name, Duration interval, Runnable task)
Registers a timer event that executes periodically- Parameters:
conf
- the map with the existing topology configsname
- the name of the timerinterval
- the frequency in which to run the tasktask
- the task to run
-
setDebug
public void setDebug(boolean isOn)
-
setTeamName
public void setTeamName(String teamName)
-
setTeamEmail
public void setTeamEmail(String teamEmail)
-
setTopologyCapTicket
public void setTopologyCapTicket(String ticket)
-
setTopologyProjectName
public void setTopologyProjectName(String project)
-
setNumStmgrs
public void setNumStmgrs(int stmgrs)
-
setSerializationClassName
public void setSerializationClassName(String className)
-
setKryoFactory
public void setKryoFactory(Class<? extends IKryoFactory> klass)
-
setSkipMissingKryoRegistrations
public void setSkipMissingKryoRegistrations(boolean skip)
-
registerKryoSerialization
public void registerKryoSerialization(Class klass)
-
registerKryoSerialization
public void registerKryoSerialization(Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
-
registerDecorator
public void registerDecorator(Class<? extends IKryoDecorator> klass)
-
setFallBackOnJavaSerialization
public void setFallBackOnJavaSerialization(boolean fallback)
-
setEnableAcking
@Deprecated public void setEnableAcking(boolean acking)
Deprecated.Is topology running with acking enabled? The SupressWarning will be removed once TOPOLOGY_ENABLE_ACKING is removed
-
setMessageTimeoutSecs
public void setMessageTimeoutSecs(int secs)
-
setTopologyReliabilityMode
public void setTopologyReliabilityMode(Config.TopologyReliabilityMode mode)
-
setComponentParallelism
public void setComponentParallelism(int parallelism)
-
setMaxSpoutPending
public void setMaxSpoutPending(int max)
-
setTickTupleFrequency
public void setTickTupleFrequency(int seconds)
-
setContainerCpuRequested
public void setContainerCpuRequested(double ncpus)
-
setContainerDiskRequested
public void setContainerDiskRequested(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerRamRequested
public void setContainerRamRequested(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxCpuHint
public void setContainerMaxCpuHint(double ncpus)
-
setContainerMaxDiskHint
public void setContainerMaxDiskHint(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerMaxRamHint
public void setContainerMaxRamHint(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerPaddingPercentage
public void setContainerPaddingPercentage(int percentage)
-
setContainerRamPadding
public void setContainerRamPadding(org.apache.heron.common.basics.ByteAmount nbytes)
-
setContainerCpuPadding
public void setContainerCpuPadding(double ncpus)
-
setTopologyContainerMaxNumInstances
public void setTopologyContainerMaxNumInstances(int n)
-
setComponentCpuMap
public void setComponentCpuMap(String cpuMap)
-
setComponentRamMap
public void setComponentRamMap(String ramMap)
-
setComponentDiskMap
public void setComponentDiskMap(String diskMap)
-
setComponentCpu
public void setComponentCpu(String component, double cpu)
-
setComponentRam
public void setComponentRam(String component, org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setComponentDisk
public void setComponentDisk(String component, org.apache.heron.common.basics.ByteAmount diskInBytes)
-
setFFDSortingStrategy
public void setFFDSortingStrategy(String sortingStrategy)
-
setUpdateDeactivateWaitDuration
public void setUpdateDeactivateWaitDuration(int seconds)
-
setUpdateReactivateWaitDuration
public void setUpdateReactivateWaitDuration(int seconds)
-
setTopologyStatefulCheckpointIntervalSecs
public void setTopologyStatefulCheckpointIntervalSecs(int secs)
-
setTopologyStatefulStartClean
public void setTopologyStatefulStartClean(boolean clean)
-
setCheckpointManagerRam
public void setCheckpointManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setTopologyStatefulSpillState
public void setTopologyStatefulSpillState(String spillState)
-
setTopologyStatefulSpillStateLocation
public void setTopologyStatefulSpillStateLocation(String location)
-
setStreamManagerRam
public void setStreamManagerRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
setMetricsmgrRam
public void setMetricsmgrRam(org.apache.heron.common.basics.ByteAmount ramInBytes)
-
registerTopologyTimerEvents
public void registerTopologyTimerEvents(String name, Duration interval, Runnable task)
-
setTopologyRemoteDebugging
public void setTopologyRemoteDebugging(boolean isOn)
-
setTopologyDropTuplesUponBackpressure
public void setTopologyDropTuplesUponBackpressure(boolean dropTuples)
-
setTopologyComponentOutputBPS
@Deprecated public void setTopologyComponentOutputBPS(long bps)
Deprecated.use the TOPOLOGY_COMPONENT_OUTPUT_BPS config with ComponentConfigurationDeclarer's addConfiguration() instead. Example: builder.setSpout(...).addConfiguration(Config.TOPOLOGY_COMPONENT_OUTPUT_BPS, 1000);This function should not be used to set rate limiter in topology config.
-
-