Interface IStateManager

  • All Superinterfaces:
    AutoCloseable

    @LimitedPrivate
    @Unstable
    public interface IStateManager
    extends AutoCloseable
    This file defines the IStateManager interface.

    Services across Heron use HeronStateMgr to get/set state information. Currently the primary things kept by state are: 1. Where is the the topology manager running. The topology manager is responsible for writing this information out upon startup. The stream managers query this upon startup to find out who is their topology manager. In case they loose connection with the topology manager, the stream managers query this again to see if the topology manager has changed.

    2. Topology and the current running state of the topology This information is seeded by the topology submitter. The topology manager updates this when the state of the topology changes.

    3. Current assignment. This information is solely used by topology manager. When it creates a new assignment or when the assignment changes, it writes out this information. This is required for topology manager failover.

    Clients call the methods of the state passing a callback. The callback is called with result code upon the completion of the operation.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Interface Description
      static class  IStateManager.LockName  
    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void close()
      This is to for disposing or cleaning up any internal state accumulated by the StateManager
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteExecutionState​(String topologyName)
      Delete the execution state for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteLocks​(String topologyName)
      Delete all locks for a given topology.
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteMetricsCacheLocation​(String topologyName)
      Delete the metricscache location for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deletePackingPlan​(String topologyName)
      Delete the packing plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deletePhysicalPlan​(String topologyName)
      Delete the physical plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteSchedulerLocation​(String topologyName)
      Delete the scheduler location for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteStatefulCheckpoints​(String topologyName)
      Delete the stateful checkpoints for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteTManagerLocation​(String topologyName)
      Delete the tmanager location for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> deleteTopology​(String topologyName)
      Delete the topology definition for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.ExecutionEnvironment.ExecutionState> getExecutionState​(WatchCallback watcher, String topologyName)
      Get the execution state for the given topology
      Lock getLock​(String topologyName, IStateManager.LockName lockName)
      Return a lock object backed by the state manager store.
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.tmanager.TopologyManager.MetricsCacheLocation> getMetricsCacheLocation​(WatchCallback watcher, String topologyName)
      Get the MetricsCache location for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.PackingPlans.PackingPlan> getPackingPlan​(WatchCallback watcher, String topologyName)
      Get the packing plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.PhysicalPlans.PhysicalPlan> getPhysicalPlan​(WatchCallback watcher, String topologyName)
      Get the physical plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.scheduler.Scheduler.SchedulerLocation> getSchedulerLocation​(WatchCallback watcher, String topologyName)
      Get the scheduler location for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.ckptmgr.CheckpointManager.StatefulConsistentCheckpoints> getStatefulCheckpoints​(WatchCallback watcher, String topologyName)
      Get the Stateful Checkpoints
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.tmanager.TopologyManager.TManagerLocation> getTManagerLocation​(WatchCallback watcher, String topologyName)
      Get the tmanager location for the given topology
      com.google.common.util.concurrent.ListenableFuture<org.apache.heron.api.generated.TopologyAPI.Topology> getTopology​(WatchCallback watcher, String topologyName)
      Get the topology definition for the given topology
      void initialize​(Config config)
      Initialize StateManager with the incoming context.
      com.google.common.util.concurrent.ListenableFuture<Boolean> isTopologyRunning​(String topologyName)
      Is the given topology in RUNNING state?
      com.google.common.util.concurrent.ListenableFuture<Boolean> setExecutionState​(org.apache.heron.proto.system.ExecutionEnvironment.ExecutionState executionState, String topologyName)
      Set the execution state for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> setMetricsCacheLocation​(org.apache.heron.proto.tmanager.TopologyManager.MetricsCacheLocation location, String topologyName)
      Set the location of MetricsCache.
      com.google.common.util.concurrent.ListenableFuture<Boolean> setPackingPlan​(org.apache.heron.proto.system.PackingPlans.PackingPlan packingPlan, String topologyName)
      Set the packing plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> setPhysicalPlan​(org.apache.heron.proto.system.PhysicalPlans.PhysicalPlan physicalPlan, String topologyName)
      Set the physical plan for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> setSchedulerLocation​(org.apache.heron.proto.scheduler.Scheduler.SchedulerLocation location, String topologyName)
      Set the scheduler location for the given topology
      com.google.common.util.concurrent.ListenableFuture<Boolean> setStatefulCheckpoints​(org.apache.heron.proto.ckptmgr.CheckpointManager.StatefulConsistentCheckpoints checkpoint, String topologyName)
      Set the Stateful Checkpoints
      com.google.common.util.concurrent.ListenableFuture<Boolean> setTManagerLocation​(org.apache.heron.proto.tmanager.TopologyManager.TManagerLocation location, String topologyName)
      Set the location of Tmanager.
      com.google.common.util.concurrent.ListenableFuture<Boolean> setTopology​(org.apache.heron.api.generated.TopologyAPI.Topology topology, String topologyName)
      Set the topology definition for the given topology
    • Method Detail

      • initialize

        void initialize​(Config config)
        Initialize StateManager with the incoming context.
      • close

        void close()
        This is to for disposing or cleaning up any internal state accumulated by the StateManager

        Closes this stream and releases any system resources associated with it. If the stream is already closed then invoking this method has no effect.

        Specified by:
        close in interface AutoCloseable
      • isTopologyRunning

        com.google.common.util.concurrent.ListenableFuture<Boolean> isTopologyRunning​(String topologyName)
        Is the given topology in RUNNING state?
        Returns:
        Boolean
      • getLock

        Lock getLock​(String topologyName,
                     IStateManager.LockName lockName)
        Return a lock object backed by the state manager store.
        Parameters:
        topologyName - the name of the topology
        lockName - any thread may get the Lock object bound to a given name, but only one thread may obtain the actual lock from that @{code Lock} object.
        Returns:
        an object representing an implementation of a lock.
      • deleteLocks

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteLocks​(String topologyName)
        Delete all locks for a given topology. Ideally locks should be deleted when released but it's possible that some state systems (e.g., ZooKeeper) will not delete all resources when a lock is released. This method should be invoked to clean all such lock resources.
        Returns:
        Boolean - Success or Failure
      • setTManagerLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> setTManagerLocation​(org.apache.heron.proto.tmanager.TopologyManager.TManagerLocation location,
                                                                                        String topologyName)
        Set the location of Tmanager.
        Returns:
        Boolean - Success or Failure
      • getTManagerLocation

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.tmanager.TopologyManager.TManagerLocation> getTManagerLocation​(WatchCallback watcher,
                                                                                                                                                 String topologyName)
        Get the tmanager location for the given topology
        Parameters:
        watcher -
        Returns:
        TManagerLocation
        See Also:
        WatchCallback
      • deleteTManagerLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteTManagerLocation​(String topologyName)
        Delete the tmanager location for the given topology
        Returns:
        Boolean - Success or Failure
      • setMetricsCacheLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> setMetricsCacheLocation​(org.apache.heron.proto.tmanager.TopologyManager.MetricsCacheLocation location,
                                                                                            String topologyName)
        Set the location of MetricsCache.
        Returns:
        Boolean - Success or Failure
      • getMetricsCacheLocation

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.tmanager.TopologyManager.MetricsCacheLocation> getMetricsCacheLocation​(WatchCallback watcher,
                                                                                                                                                         String topologyName)
        Get the MetricsCache location for the given topology
        Parameters:
        watcher -
        Returns:
        TManagerLocation
        See Also:
        WatchCallback
      • deleteMetricsCacheLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteMetricsCacheLocation​(String topologyName)
        Delete the metricscache location for the given topology
        Returns:
        Boolean - Success or Failure
      • setTopology

        com.google.common.util.concurrent.ListenableFuture<Boolean> setTopology​(org.apache.heron.api.generated.TopologyAPI.Topology topology,
                                                                                String topologyName)
        Set the topology definition for the given topology
        Returns:
        Boolean - Success or Failure
      • getTopology

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.api.generated.TopologyAPI.Topology> getTopology​(WatchCallback watcher,
                                                                                                                            String topologyName)
        Get the topology definition for the given topology
        Parameters:
        watcher -
        Returns:
        Topology
        See Also:
        WatchCallback
      • deleteTopology

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteTopology​(String topologyName)
        Delete the topology definition for the given topology
        Returns:
        Boolean - Success or Failure
      • setPackingPlan

        com.google.common.util.concurrent.ListenableFuture<Boolean> setPackingPlan​(org.apache.heron.proto.system.PackingPlans.PackingPlan packingPlan,
                                                                                   String topologyName)
        Set the packing plan for the given topology
        Returns:
        Boolean - Success or Failure
      • getPackingPlan

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.PackingPlans.PackingPlan> getPackingPlan​(WatchCallback watcher,
                                                                                                                                  String topologyName)
        Get the packing plan for the given topology
        Parameters:
        watcher -
        Returns:
        PackingPlan
        See Also:
        WatchCallback
      • deletePackingPlan

        com.google.common.util.concurrent.ListenableFuture<Boolean> deletePackingPlan​(String topologyName)
        Delete the packing plan for the given topology
        Returns:
        Boolean - Success or Failure
      • setPhysicalPlan

        com.google.common.util.concurrent.ListenableFuture<Boolean> setPhysicalPlan​(org.apache.heron.proto.system.PhysicalPlans.PhysicalPlan physicalPlan,
                                                                                    String topologyName)
        Set the physical plan for the given topology
        Returns:
        Boolean - Success or Failure
      • getPhysicalPlan

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.PhysicalPlans.PhysicalPlan> getPhysicalPlan​(WatchCallback watcher,
                                                                                                                                     String topologyName)
        Get the physical plan for the given topology
        Parameters:
        watcher -
        Returns:
        PhysicalPlan
        See Also:
        WatchCallback
      • deletePhysicalPlan

        com.google.common.util.concurrent.ListenableFuture<Boolean> deletePhysicalPlan​(String topologyName)
        Delete the physical plan for the given topology
        Returns:
        Boolean - Success or Failure
      • setExecutionState

        com.google.common.util.concurrent.ListenableFuture<Boolean> setExecutionState​(org.apache.heron.proto.system.ExecutionEnvironment.ExecutionState executionState,
                                                                                      String topologyName)
        Set the execution state for the given topology
        Returns:
        Boolean - Success or Failure
      • getExecutionState

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.system.ExecutionEnvironment.ExecutionState> getExecutionState​(WatchCallback watcher,
                                                                                                                                                String topologyName)
        Get the execution state for the given topology
        Parameters:
        watcher -
        Returns:
        ExecutionState
        See Also:
        WatchCallback
      • deleteExecutionState

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteExecutionState​(String topologyName)
        Delete the execution state for the given topology
        Returns:
        Boolean - Success or Failure
      • setSchedulerLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> setSchedulerLocation​(org.apache.heron.proto.scheduler.Scheduler.SchedulerLocation location,
                                                                                         String topologyName)
        Set the scheduler location for the given topology
        Returns:
        Boolean - Success or Failure
      • getSchedulerLocation

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.scheduler.Scheduler.SchedulerLocation> getSchedulerLocation​(WatchCallback watcher,
                                                                                                                                              String topologyName)
        Get the scheduler location for the given topology
        Parameters:
        watcher -
        Returns:
        SchedulerLocation
        See Also:
        WatchCallback
      • deleteSchedulerLocation

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteSchedulerLocation​(String topologyName)
        Delete the scheduler location for the given topology
        Returns:
        Boolean - Success or Failure
      • setStatefulCheckpoints

        com.google.common.util.concurrent.ListenableFuture<Boolean> setStatefulCheckpoints​(org.apache.heron.proto.ckptmgr.CheckpointManager.StatefulConsistentCheckpoints checkpoint,
                                                                                           String topologyName)
        Set the Stateful Checkpoints
        Returns:
        Boolean - Success or Failure
      • getStatefulCheckpoints

        com.google.common.util.concurrent.ListenableFuture<org.apache.heron.proto.ckptmgr.CheckpointManager.StatefulConsistentCheckpoints> getStatefulCheckpoints​(WatchCallback watcher,
                                                                                                                                                                  String topologyName)
        Get the Stateful Checkpoints
        Returns:
        Boolean - Success or Failure
      • deleteStatefulCheckpoints

        com.google.common.util.concurrent.ListenableFuture<Boolean> deleteStatefulCheckpoints​(String topologyName)
        Delete the stateful checkpoints for the given topology
        Returns:
        Boolean - Success or Failure