Where it goes wrong
The usual approach is: load everything into one index and make that index searchable. It works, it is quick to build and it promptly produces an impressive demonstration. It also invalidates every permission model in your organization in one move, because the index no longer knows folders, groups or exceptions — only text.
It is also noticed late. For the first weeks nobody asks a question they were not supposed to know the answer to. Then someone does, by accident, and it is an incident.
Rule 1 — Filter on the person asking, not on the outcome
The permission check should determine which documents are eligible at all, before anything is read or summarized. That is different from blocking an answer afterwards: in that second case the model has already processed the content, and it often leaks anyway in the form of a summary.
In practice this means the search step happens on behalf of the user and the source system determines the selection — not a separate permission list you maintain yourself and that drifts after three months.
Rule 2 — Copy as little as possible
Every copy is a second place where permissions can lapse, that has to be cleaned up and that counts in a data breach. Keep the documents where they are and retrieve only what is needed for the answer.
If you cannot avoid storing something, record three things: exactly what is kept, for how long, and what happens when the source file is deleted or the permissions change. The last one is the one most often skipped: a deleted document that stays in an index is a document you have not deleted.
Rule 3 — Choose the sources, do not include everything
Not every folder belongs in a knowledge base. Drafts give answers that formally do not apply, personnel files belong separately, and client documents are covered by agreements with that client. Fewer sources with a clear status produce better answers than indexing everything — and make the conversation with the administrator a lot shorter.
Make that explicit per source, even if the answer is "yes, everyone may read this". A recorded decision is something different from an assumption nobody has tested.
Rule 4 — Arrange exit at the start
At the outset, agree where the configuration, the credentials and the logs live, who manages them and what happens when the collaboration ends. At the beginning that is a paragraph and at the end it is a negotiation. This also applies to the least exciting part: the access tokens the connection uses should belong to your organization.
What this costs in speed
These four rules make the build slower than the "everything in one index" route, especially on the permissions side. That is the honest trade-off: you give up a few weeks for a solution that survives the security review and that you do not have to turn into a second project later.
What then makes a knowledge base usable for the people who use it — source, permission and date — is covered in internal knowledge base with AI.