Class ComplexSource<R>

    • Constructor Detail

      • ComplexSource

        public ComplexSource​(Source<R> generator)
    • Method Detail

      • initState

        public void initState​(State<Serializable,​Serializable> startupState)
        Description copied from interface: IStatefulComponent
        Initializes the state of the function or operator to that of a previous checkpoint. This method is invoked when a component is executed as part of a recovery run. In case there was prior state associated with the component, the state will be empty. Stateful Spouts/Bolts are expected to hold on to the state variable to save their internal state

        Note that initialState() is called before open() or prepare().

        Specified by:
        initState in interface IStatefulComponent<Serializable,​Serializable>
        Overrides:
        initState in class StreamletSource
        Parameters:
        startupState - the previously saved state of the component.
      • open

        public void open​(Map<String,​Object> map,
                         TopologyContext topologyContext,
                         SpoutOutputCollector outputCollector)
        Description copied from interface: ISpout
        Called when a task for this component is initialized within a worker on the cluster. It provides the spout with the environment in which the spout executes.

        This includes the:

        Specified by:
        open in interface ISpout
        Overrides:
        open in class StreamletSource
        Parameters:
        map - The Heron configuration for this spout. This is the configuration provided to the topology merged in with cluster configuration on this machine.
        topologyContext - This object can be used to get information about this task's place within the topology, including the task id and component id of this task, input and output information, etc.
        outputCollector - The collector is used to emit tuples from this spout. Tuples can be emitted at any time, including the open and close methods. The collector is thread-safe and should be saved as an instance variable of this spout object.
      • nextTuple

        public void nextTuple()
        Description copied from interface: ISpout
        When this method is called, Heron is requesting that the Spout emit tuples to the output collector. This method should be non-blocking, so if the Spout has no tuples to emit, this method should return. nextTuple, ack, and fail are all called in a tight loop in a single thread in the spout task. When there are no tuples to emit, it is courteous to have nextTuple sleep for a short amount of time (like a single millisecond) so as not to waste too much CPU.