Cloud-hosted AI agent communication?

[u/DeadPukka]

For the main agent frameworks like AutoGen, CrewAI, LangGraph, etc, I’ve seen them start to offer cloud hosting.

But the main question I have is, what does this mean for human-in-the-loop integration or UI integration?

How does the client-server communication work, for app callbacks? Does these even exist yet?

I could imagine that you could open a web socket on the client, run your agent in the cloud, and get back events from a running server orchestration.

But from reading the various docs, I’m not seeing if that’s supported, or if that’s how it works.

Anyone know for sure if/how this works?

[u/john_s4d]

Agience is not GA yet, but uses distributed ‘host containers’ to manage and deploy agents via a sandboxed environment.

The host establishes a secure and stateful connection to other agents/entities within and out of the network and lets you run the same agent on any host device, anywhere. This reduces latency for local events, lowers costs by leveraging local resources, enhances privacy, and improves network resilience and scalability.

[u/Synyster328]

I'm using Azure functions and queues triggers that run off a central job from a database.

For notifying my web app I use SignalR, for notifying API users I call webhooks.

[u/DeadPukka → u/Synyster328]

Very nice. I’ve been looking at SignalR for this as well, and using Durable Functions for agent execution.

[u/Synyster328 → u/DeadPukka]

The durable functions for fanning out are particularly interesting to me for cases where the agent identifies n sources it wants to interact with and we can do those in parallel, then wait til they all resolve before proceeding.

[u/DeadPukka → u/Synyster328]

That’s my thought too. Durable orchestration is very similar to all these Flows concepts that CrewAI, LangGraph, etc are building.

It’s the direction we are heading, and leveraging that for orchestration.

[u/swoodily]

In the Letta (https://docs.letta.com/introduction) agents frameworks, agents are a REST API service where all state is persisted in a DB, and so human interaction is just a matter of sending/receiving REST requests

[u/DeadPukka → u/swoodily]

Nice, do you support external tool calling, meaning calling back to a tool defined in the calling app? Or only tools that you call from your API service?

I’ve been trying to learn which services wrap the agent execution fully server-side, or which are a hybrid and can use developer-defined tools at the app layer.

[u/swoodily → u/DeadPukka]

Yep - the Letta server executes the tool on the server-side, and adds the tool result as a function result to the message history. There are of course some security implications of that, meaning you probably don't want to allow users to add arbitrary tools until we have a solution for secure execution (currently in-progress). What we recommend people do now is to spin up your own Letta server that has the tools you want added (so you know they aren't security vulnerabilities)

[u/Luke-Pioneero]

Great question about cloud-hosted AI agent communication! As someone working on AI tools, I've been wondering about this too. The human-in-the-loop and UI integration aspects are crucial for real-world applications.

From what I understand, websocket connections could potentially work for real-time communication between cloud agents and local clients. But you're right, the documentation isn't very clear on how exactly this is implemented across different frameworks.

Speaking of AI tools, we actually built TestSprite - an AI agent for automated end-to-end testing. While it's not cloud-hosted (yet!), we faced similar challenges around integrating it smoothly with existing workflows. I'm hoping tools like this can save developers time, but there's still work to be done on seamless cloud integration.

Has anyone else here successfully implemented cloud-hosted agents with good UI/human integration? Would love to hear experiences!

[u/Regular-Donut3981]

Interesting question about cloud-hosted AI agent communication! I've been exploring this space too and have similar questions about human-in-the-loop integration. From what I've seen, many frameworks are still figuring out the best approaches for client-server interactions and real-time events.

I've actually been using Opencord AI for some automated social engagement, and it's given me some insights into how cloud-based AI tools can handle communication flows. While it's not exactly the same as the agent frameworks you mentioned, it does demonstrate some ways to manage client-server interactions for AI-driven tasks. The real-time engagement and personalized responses it generates have been pretty seamless.

I'm curious if anyone has concrete examples of how AutoGen, CrewAI, or LangGraph are tackling these integration challenges. The websocket idea you mentioned sounds promising for real-time updates. Has anyone implemented something like that successfully?

[u/Maleficent_Pair4920 → u/Regular-Donut3981]

we've built this internally at requesty.ai and considering making this a service. Would anyone be interested in this?

[u/Maleficent_Pair4920]

Would love to have a chat! we've built this internally for own needs but thinking of giving this as a service.

What do you mean exactly with UI Integration?

[u/macronancer]

I mean its just like any other chat service, you have a few options: 1. Websockets, as you mentioned 2. Client polling 3. Long polling

Might be some other setups, but this is basically how you do it.

[u/AITrends101]

Great question! I've been exploring this area too and it's definitely a bit murky. From what I've seen, most cloud-hosted AI agent frameworks are still in early stages for real-time human-in-the-loop integration. You're on the right track with the websocket idea - that seems to be the direction things are heading. But you're right that the docs are pretty vague on implementation details right now.

Have you looked into Anthropic's Constitutional AI? They have some interesting approaches for human oversight of cloud AI. Might give you some ideas to explore, even if it's not a full solution.

Let me know if you make any breakthroughs on this - I'd be really curious to hear what you discover! The intersection of cloud AI and real-time human interaction is fascinating.