Context Orchestration

I am working on a novel software architecture where orchestration of agentic workflow patterns adaptively programs in-context memory. With it, I am trying to solve the broader problem I see with the user interaction style of model inference.

In a nutshell, the magic trick we are all trying to master with is how to get the most amount of useful and profitable AI output from the least amount of human input and oversight.

Real world applications of agentic AI need to see both productivity gains and a return on investment. This requires trust, reliability, security and effectiveness from agentic systems. However, it also requires human designers to map domain specific knowledge and operational expertise into productive and profitable agentic resources that can be deployed. For the vast majority of inference use-cases, this will involve some form of in-context learning.

In my opinion, the user interaction style of model inference is the bottleneck. Effective human centered design needs cognitive insight about the nature of inference that designers can comprehend and reason about. Designers need these insights to develop design heuristics and best practices. Cognitive insight builds the roadmap to designer productivity and effectiveness. Without it, projects inevitably return to trial and error.

I now liken conversational prompting to writing assembly code. Yes its direct, but its also time consuming and inefficient. Instead, we need a higher level construct than prompting. Much like high level languages have replaced assembly language in software development, I have a sense that agentic inference needs a similar way to program context. One that optimizes context window utilization so models understand the problem domain via adaptive in-context learning. This requires a different kind of interaction style between agentic workflow designers and the context window.

The method I am researching involves orchestration of inference states, where reasoning patterns are composed from reusable sub-workflows that comprise a finite state machine. The orchestrator moves between these states based on the inference results the sub-workflows encapsulate. This architecture makes its logic transparent and easier to reason about from a high-level perspective.

While I am a relative newcomer to applied AI research, I know a thing about system software engineering and user centered experience design. I spent 8 years in Silicon Valley during the 90’s dot come era¹ where we worked on hardware based neural networks amongst other things, I also currently teach students UX Design at Central Queensland University.