João Paulo Cercal

Staff Security Engineer at LastPass

  • Others

What an AI Session Can Teach the Next One

  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn

An AI assistant can help write code, explain an unfamiliar system, investigate a failure, or draft documentation. An AI agent can go a step further and take several actions on our behalf.

In both cases, it is easy to focus only on the immediate result. Did the code compile? Did the document look reasonable? Did the task move forward?

There is another result that is easy to lose: what the session taught us about the way our team works.

Perhaps the assistant was missing an important project convention. Perhaps an instruction was ambiguous. Perhaps a test caught a problem late, or a review found a security requirement that was never written down. These observations can disappear when the conversation ends, even though they may be useful the next time someone faces a similar task.

The AI model does not automatically learn from a private conversation. The team can still learn from it.

A session can reveal different kinds of gaps

When something goes wrong, it helps to look beyond the immediate correction. The problem may point to a gap in one of a few places:

  • Context: The assistant did not have the relevant version, dependency, architecture, or domain information.
  • Instructions: The request did not make an important behavior or constraint clear.
  • Workflow: The order of activities made it difficult to discover a problem early.
  • Guardrails: A test, check, or review step did not cover a risk that mattered.

These categories are not meant to turn every interaction into an investigation. They are simply a way to ask a useful question: where would a small change help the next person, or the next session?

Sometimes the answer is none of them. A model can make a mistake, a requirement can change, or a problem can be genuinely unusual. Not every correction needs to become a new rule.

Turn a correction into a shared improvement

Imagine that an agent adds a new API endpoint. The endpoint compiles and the automated tests pass, but a review finds that it does not enforce the authorization rule used by the rest of the service.

The immediate fix is to add the missing check. The more useful question is what allowed the omission to reach the review in the first place.

If the authorization requirement was absent from the project context, it may belong in the relevant documentation or development guide. If the requirement was known but not stated in the task, a request template may need to make it explicit. If this kind of problem is important enough to prevent repeatedly, an automated test or a review check may be a better destination.

The goal is not to write a longer prompt every time something goes wrong. The goal is to place the lesson where it can do the most work.

That small distinction matters. A personal note helps one person remember. A test, a concise guide, or a shared workflow can help everyone working on the same system.

Keep the habit small

At the end of an AI-assisted session, one question is often enough:

Did anything happen here that should change something shared?

Most of the time, the answer may be no. The session went well, the existing checks were sufficient, and there is nothing to preserve.

When the answer is yes, the change should happen while the observation is still fresh. It might be one sentence in a project guide, one example in a task template, one test case, or one item in a pull request checklist.

The habit can fit into an existing checkpoint instead of becoming a separate process. It could be a question in a pull request template, a short item in a retrospective, or part of closing a piece of work.

The important thing is not to record everything. A prompt that worked once because of a personal preference may not be useful to the team. A repeated failure, a missing convention, or a constraint that would help other people is more likely to deserve a shared place.

Learning without giving up responsibility

Shared documentation and automated checks are useful, but they are not substitutes for engineering judgment. Documentation can become outdated. Tests can miss important behavior. An AI system can produce an answer that sounds convincing and is still wrong.

The purpose of this practice is not to remove responsibility from people. It is to make important knowledge less dependent on memory and less likely to be rediscovered through the same failure.

For experienced teams, the useful detail is often in the diagnosis. Was the problem caused by missing context, an unclear instruction, an inefficient workflow, an absent guardrail, or a limitation that should simply be documented? Each answer suggests a different improvement.

For someone just starting to use AI at work, the starting point is smaller: notice one recurring correction and make one shared artifact a little better.

The model may not learn from every interaction. The surrounding system can. Over time, those small improvements can make the next session clearer, the next review faster, and the next failure less likely to repeat.