Data Flow Architecture

End-to-end data flows for common workflows and interactions.

1. Workflow Creation & Storage

graph LR U["User
Studio"] U -->|Design Workflow| S["Studio
State"] S -->|Save| A["API
POST /create"] A -->|Transform| P["ProcessStudio
Service"] P -->|1. Create Project| DB["Database"] P -->|2. Create App| DB P -->|3. Create Process| DB P -->|4. Create Threads| DB P -->|5. Create Elements| DB P -->|6. Create Connections| DB DB -->|Return IDs| P P -->|Response| A A -->|Update State| S S -->|Notify User| U style U fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style S fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style A fill:#238636,stroke:#3fb950,color:#c9d1d9 style P fill:#238636,stroke:#3fb950,color:#c9d1d9 style DB fill:#1158c7,stroke:#58a6ff,color:#c9d1d9
1
User designs workflow in Studio
Drag-drop nodes, configure parameters, define connections
2
Click "Save" button
Studio sends ProcessDefinition to backend API
3
ProcessStudio Service processes
Creates Project, App, Process, Threads, Elements, Connections
4
Data persisted to database
All entities stored with relationships intact

2. Workflow Execution Flow

graph TD U["User
Click Execute"] -->|1. POST /execute| A["API"] A -->|2. Validate| P["ProcessEngine
Service"] P -->|3. Load ProcessDef| DB["Database"] DB -->|4. ProcessDefinition| P P -->|5. Init OrchestrationProcessor| O["Orchestration
Processor"] O -->|6. Push Trigger| S["Execution
Stack"] S -->|7. Pop & Execute| E["ProcessElement
Executor"] E -->|8. Execute Logic| N["Node Type
Handler"] N -->|9. Output Result| R["Execution
Router"] R -->|10. Query Downstream| DB DB -->|11. Connected Nodes| R R -->|12. Push to Stack| S S -->|Loop until empty| E E -->|13. Emit Event| SG["SignalR
Hub"] SG -->|14. Broadcast| U P -->|15. Save Results| DB style U fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style A fill:#238636,stroke:#3fb950,color:#c9d1d9 style P fill:#238636,stroke:#3fb950,color:#c9d1d9 style O fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style E fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style N fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style SG fill:#238636,stroke:#3fb950,color:#c9d1d9 style DB fill:#1158c7,stroke:#58a6ff,color:#c9d1d9

3. AI Agent Invocation Flow

graph LR N["AiAgentExecutor
Node"] N -->|1. Parse Config
agentId, modelChoice| P["Configuration
Parser"] P -->|2. Extract Context
from InputData| C["Context
Builder"] C -->|3. Call Invoker| I["IAgentInvoker
Interface"] I -->|4. Invoke Agent| O["OctopusAgent
Invoker"] O -->|5. Reflection
GetService| D["Octopus
Service"] D -->|6. Query DB| DB["Octopus
DB"] DB -->|Agent Config| D D -->|7. Build LLM
Request| L["LLM API
OpenAI/Claude"] L -->|8. Generate
Response| D D -->|9. Return
Response| O O -->|10. Map to Result| R["Invocation
Result"] R -->|11. Output Data| E["Next Executor"] style N fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style P fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style C fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style I fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style O fill:#ae3f6e,stroke:#f97583,color:#c9d1d9 style D fill:#ae3f6e,stroke:#f97583,color:#c9d1d9 style DB fill:#1158c7,stroke:#58a6ff,color:#c9d1d9 style L fill:#ae3f6e,stroke:#f97583,color:#c9d1d9 style R fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style E fill:#9e6a03,stroke:#fb8500,color:#c9d1d9

4. Real-time Monitoring (SignalR)

graph TB E["Node Executes
NodeExecuting Event"] E -->|1. Emit Event| PH["ProcessEngine
Publishes Event"] PH -->|2. IExecutionEventHandler| H["ExecutionHub
Receives Event"] H -->|3. FormatMessage| M["ProcessHub
MessageResponse"] M -->|4. BroadcastAsync
to execution_[ID]| SG["SignalR Group"] SG -->|5. WebSocket
Send| F["Frontend
Studio"] F -->|6. Update State
executionStates Map| S["Store"] S -->|7. Re-render| V["Badge on Node
⏳ Running"] P["Polling Fallback
if disconnected"] -->|Poll /status| A["API"] A -->|Return Status| P P -->|Update UI| V style E fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style PH fill:#9e6a03,stroke:#fb8500,color:#c9d1d9 style H fill:#238636,stroke:#3fb950,color:#c9d1d9 style SG fill:#238636,stroke:#3fb950,color:#c9d1d9 style F fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style V fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style P fill:#1158c7,stroke:#58a6ff,color:#c9d1d9

5. Database Operations Pattern

Write Operations (EF Core)

1. Create Entity

Instantiate domain model object

2. Add to DbContext

dbContext.Entities.Add(entity)

3. SaveChanges

Executes INSERT/UPDATE/DELETE

4. Audit Trail

CreatedAt, ModifiedAt, CreatedBy set automatically

Read Operations (Query)

1. Query DbSet

dbContext.Entities.Where(...)

2. Filter by Tenant

Automatically scoped to tenant_id

3. Lazy Load Relations

Include() for related entities

4. Execute Query

ToList/FirstOrDefault to execute

6. Multi-Tenant Data Isolation

graph TD Request["Request with
JWT tenant_id claim"] Request -->|Extract tenantId| CTX["Execution
Context"] CTX -->|Set TenantID| Filter["Global Query
Filter"] Filter -->|WHERE tenant_id = @tenantId| DB["Database Query
Only tenant's data"] DB -->|Results| Response["Return filtered
data only"] DB2["Database
All Tenants
Data"] Tenant1["🏢 Tenant 1
Projects/Processes/Executions"] Tenant2["🏢 Tenant 2
Projects/Processes/Executions"] Tenant3["🏢 Tenant 3
Projects/Processes/Executions"] DB2 --> Tenant1 DB2 --> Tenant2 DB2 --> Tenant3 Request -.->|Only Tenant 1
data queried| Tenant1 style Request fill:#1f6feb,stroke:#58a6ff,color:#c9d1d9 style CTX fill:#238636,stroke:#3fb950,color:#c9d1d9 style Filter fill:#238636,stroke:#3fb950,color:#c9d1d9 style DB fill:#1158c7,stroke:#58a6ff,color:#c9d1d9 style Response fill:#238636,stroke:#3fb950,color:#c9d1d9 style Tenant1 fill:#ae3f6e,stroke:#f97583,color:#c9d1d9 style Tenant2 fill:#ae3f6e,stroke:#f97583,color:#c9d1d9 style Tenant3 fill:#ae3f6e,stroke:#f97583,color:#c9d1d9