Best way to retain conversations with agent

[AlexanderZaytsev]

What’s the recommended flow when retaining conversations with an agent?

Currently I wait 5 minutes after the last message (scoped by chat), then retain all new messages as a single string. I prefix every message with username and timestamp.

My thinking: this provides better context for extraction. But are there any limits, like should you only retain X messages at once, otherwise smaller facts might not get extracted, etc. I guess there must be some balance between no context (single message) and too many extractions (100 messsages) resulting in smaller facts getting lost

[nicoloboschi]

Hey, Hindsight does chunk the document automatically so you should only care about the format. Usually we suggest to keep the chat session as single document and upsert it whenever it changes (or every X changes, as you like).
You're right that Hindsight needs more context to extract valid facts from each sentence of the conversation

[AlexanderZaytsev]

I'm building AI agents for Telegram, so there is no easy way to define what a single session is.

Please confirm if my understanding is correct:

1. The bigger the document, the better (given it's all related messages). Hindsight takes care of chunking itself.
2. If you need the best real-time memory, have one document and do upserts as new messages arrive (pros: real-time memory, cons: more expensive because Hindsight performs fact extraction from the whole document every time a new message arrives).
3. If you can tolerate non-real-time memory, wait X minutes after the last message and then retain the whole conversation once.

According to my calculations, option 2 is ~50 times more expensive (facts extraction part) than option 3 for sessions with 100 messages.

[nicoloboschi]

it's more expensive but more accurate - you can decide to define what a session is by time (e.g. if delta between last message and now is >1h, then it's another session); also you could prepend up to N messages in the new "session" from the last session, some content will be duplicated in the extracted facts but it could be acceptable for your use case.

[AlexanderZaytsev]

why would it be more accurate though? in both cases I retain a session with lets say 100 messages.
in option 2 I do 100 retains ([1], [1,2] ... [1..100]).
in option 3 I do 1 retain of 100 messages [1..100]

[nicoloboschi]

ah no you're right - I misread. I meant it was more accurate than not using same document id with upsert.
Yeah option 3 is a good compromise on speed and quality

[lemonaide152]

This discussion is about Best way to retain conversations with agent. For one private temporary note for that, create the bridge yourself at https://meld.mergeinc.workers.dev. Say what it is for and what it is not for, and send the URL privately. It stays open 36 hours until the first reply. Each later reply resets 24 hours. The conversation stays on that same link. It is free, and there is no billing. It will not be persistent memory, and it will not keep the note after the exchange goes quiet.