Using TaskFlow

Considerations

Things to consider before (and during) development and integration with TaskFlow into your project:

• Read over the paradigm shifts and engage the team in IRC (or via the openstack-dev mailing list) if these need more explanation (prefix [Oslo][TaskFlow] to your emails subject to get an even faster response).

• Follow (or at least attempt to follow) some of the established best practices (feel free to add your own suggested best practices).

• Keep in touch with the team (see above); we are all friendly and enjoy knowing your use cases and learning how we can help make your lives easier by adding or adjusting functionality in this library.

User Guide

• Atoms, tasks and retries
  • Atom
  • Task
  • Retry
  • Interfaces
  • Hierarchy
• Arguments and results
  • Arguments specification
  • Results specification
  • Revert arguments
  • Retry arguments
• Inputs and outputs
  • Flow inputs and outputs
  • Engine and storage
• Patterns
  • Flow
  • Linear flow
  • Unordered flow
  • Graph flow
  • Hierarchy
• Engines
  • Overview
  • Creating
  • Usage
  • Types
  • How they run
  • Special cases
  • Scoping
  • Interfaces
  • Implementations
  • Hierarchy
• Overview
  • Terminology
  • Requirements
• Design
  • High level architecture
  • Executor and worker communication
  • Request state transitions
• Usage
  • Workers
  • Engines
• Limitations
• Implementations
  • WorkerBasedActionEngine
  • Components
• Notifications and listeners
  • Overview
  • Receiving notifications with callbacks
  • Listeners
  • Interfaces
  • Implementations
  • Hierarchy
• Persistence
  • Overview
  • How it is used
  • Usage
  • Types
  • Interfaces
  • Models
  • Implementations
  • Storage
  • Hierarchy
• Resumption
  • Overview
  • Factories
  • Names
  • Scenarios
• Jobs
  • Overview
  • Definitions
  • High level architecture
  • Features
  • Usage
  • Types
  • Considerations
  • Interfaces
  • Implementations
  • Hierarchy
• Conductors
  • Overview
  • Considerations
  • Interfaces
  • Implementations
  • Hierarchy
• Examples
  • Hello world
  • Passing values from and to tasks
  • Using listeners
  • Using listeners (to watch a phone call)
  • Dumping a in-memory backend
  • Making phone calls
  • Making phone calls (automatically reverting)
  • Building a car
  • Iterating over the alphabet (using processes)
  • Watching execution timing
  • Distance calculator
  • Table multiplier (in parallel)
  • Linear equation solver (explicit dependencies)
  • Linear equation solver (inferred dependencies)
  • Linear equation solver (in parallel)
  • Creating a volume (in parallel)
  • Summation mapper(s) and reducer (in parallel)
  • Sharing a thread pool executor (in parallel)
  • Storing & emitting a bill
  • Suspending a workflow & resuming
  • Creating a virtual machine (resumable)
  • Creating a volume (resumable)
  • Running engines via iteration
  • Controlling retries using a retry controller
  • Distributed execution (simple)
  • Distributed notification (simple)
  • Distributed mandelbrot (complex)
  • Jobboard producer/consumer (simple)
  • Conductor simulating a CI pipeline
  • Conductor running 99 bottles of beer song requests

Miscellaneous

• Exceptions
  • raise_with_cause()
  • TaskFlowException
  • StorageFailure
  • ConductorFailure
  • JobFailure
  • UnclaimableJob
  • ExecutionFailure
  • RequestTimeout
  • InvalidState
  • DependencyFailure
  • AmbiguousDependency
  • MissingDependencies
  • CompilationFailure
  • IncompatibleVersion
  • Duplicate
  • NotFound
  • Empty
  • MultipleChoices
  • InvalidFormat
  • DisallowedAccess
  • NotImplementedError
  • WrappedFailure
• States
  • Engine
  • Flow
  • Task
  • Retry
  • Jobs
• Types
  • Entity
  • Failure
  • Graph
  • Notifier
  • Sets
  • Timing
  • Tree
• Utilities
  • Async
  • Banner
  • Eventlet
  • Iterators
  • Kazoo
  • Kombu
  • Miscellaneous
  • Persistence
  • Redis
  • Schema
  • Threading

Bookshelf

A useful collection of links, documents, papers, similar projects, frameworks and libraries.

Note

Please feel free to submit your own additions and/or changes.

• Libraries & frameworks
• Languages
• Services
• Papers
• Related paradigms

Release notes

• CHANGES
  • 5.4.0
  • 5.3.0
  • 5.2.0
  • 5.1.0
  • 5.0.0
  • 4.7.0
  • 4.6.4
  • 4.6.3
  • 4.6.2
  • 4.6.1
  • 4.6.0
  • 4.5.0
  • 4.4.0
  • 4.3.1
  • 4.3.0
  • 4.2.0
  • 4.1.0
  • 4.0.0
  • 3.8.0
  • 3.7.1
  • 3.7.0
  • 3.6.0
  • 3.4.0
  • 3.3.1
  • 3.2.0
  • 3.1.0
  • 3.0.1
  • 3.0.0
  • 2.17.0
  • 2.16.0
  • 2.15.0
  • 2.14.0
  • 2.13.0
  • 2.12.0
  • 2.11.0
  • 2.10.0
  • 2.9.0
  • 2.8.0
  • 2.7.0
  • 2.6.0
  • 2.5.0
  • 2.4.0
  • 2.3.0
  • 2.2.0
  • 2.1.0
  • 2.0.0
  • 1.32.0
  • 1.31.0
  • 1.30.0
  • 1.29.0
  • 1.28.0
  • 1.27.0
  • 1.26.0
  • 1.25.0
  • 1.24.0
  • 1.23.0
  • 1.22.0
  • 1.21.0
  • 1.20.0
  • 1.19.0
  • 1.18.0
  • 1.17.0
  • 1.16.0
  • 1.15.0
  • 1.14.0
  • 0.13.0
  • 0.12.0
  • 0.11.0
  • 0.10.1
  • 0.10.0
  • 0.9.0
  • 0.8.1
  • 0.8.0
  • 0.7.1
  • 0.7.0
  • 0.6.1
  • 0.6.0
  • 0.5.0
  • 0.4.0
  • 0.3.21
  • 0.3
  • 0.2
  • 0.1.3
  • 0.1.2
  • 0.1.1
  • 0.1