Document Q&A Chatbot with RAG on SAP Business Technology Platform
Document chatbot demo on SAP BTP: an SAP CAP backend in Node.js querying SAP AI Core and GPT-4o mini in a Retrieval-Augmented Generation flow, citing the source documents in every answer. Freestyle SAPUI5 front end over OData V4.
Description
A chat application that answers natural-language questions using a set of company documents indexed in SAP AI Core as its only source, and closes every answer by citing the documents it drew on. It was built at my company as a demonstrator for a client, to show on concrete documents what Retrieval-Augmented Generation can do while staying inside SAP Business Technology Platform. The mock was built in five working days by a team of three developers overseen by the delivery manager: one colleague handled the integration that moves documents from an SFTP server into AI Core grounding, another developer and I worked on the AI side — indexing, classification, passage retrieval and answer construction — and I built the SAP Cloud Application Programming Model (CAP) backend in Node.js and the SAPUI5 chat application that queries it.
The problem
A company's operational knowledge lives inside documents that nobody reads end to end. When someone needs a specific figure, they open the files one by one and scroll through the pages: full-text search finds the keyword but does not answer the question, and anyone who does not already know which document to open has no starting point. The result is wasted time and answers that vary depending on who looks for them. The project started as a pre-sales demonstrator: we needed to show a client, on actual documents rather than a slide-deck example, that you can ask a question in plain language and get an answer built from their own documents, with the source file and page.
Constraints
Five working days from kickoff to the demo in front of the client: every decision had to be the one that works immediately, not the one that defends itself best in a code review. Everything had to stay inside SAP Business Technology Platform: no external provider API keys, model access only through SAP AI Core, reached via a destination configured on the subaccount and scoped to a dedicated resource group and deployment ID. The fifty documents loaded were mixed in type and format, with no specific preparation: retrieval had to hold up across all of them without per-document tuning. Fiori Elements templates do not cover a conversational interface, so the UI is freestyle SAPUI5. Authentication goes through XSUAA and the app is served by the HTML5 Application Repository: backend, database and front end have to be released together in a single MTA archive.
Architecture
documents on SFTP ──► ingestion integration ──► SAP AI Core grounding
(vector repositories)
freestyle SAPUI5 ──► Approuter (XSUAA) ──► CAP service
│
SAP AI Core (destination)
├─ 1. retrieval on grounding
└─ 2. GPT-4o mini, chat completions
- Ingestion: the company documents live on an SFTP server; an integration flow picks them up and loads them into SAP AI Core document grounding, where they are split, indexed and classified so they can be retrieved.
- Front end: freestyle SAPUI5 application with chat bubbles built at runtime; it sends the message through an unbound OData V4 action
sendMessage(text). - Backend: an SAP Cloud Application Programming Model (CAP) service in Node.js. The flow is two calls: first retrieval against the SAP AI Core vector repositories (up to 30 chunks), then inference on the GPT-4o mini deployment.
- Grounding: chunks are normalised into
[document - p.N]: textlines and injected into the system prompt, which restricts the model to that material. Every answer closes with the list of documents the content came from: an explicit client requirement, and what makes the chatbot's output verifiable. - When there is no answer: if retrieval returns nothing relevant, the user gets an explicit message — no information found in the available sources — rather than an answer the model made up.
- Data: SAP HANA Cloud through an HDI container; the CDS model holds a
DocumentChunksentity with aVector(1536)column, prepared for embeddings. - Deployment: an MTA archive with four modules — Node.js service, HDI db-deployer, approuter, HTML5 application.
Outcome
Five days, from blank page to demo#
The mock was built in five working days, deployed on the Cloud Foundry runtime and tried against fifty company documents of different types and formats, loaded with no preparation. In that time the constraint was not to build the best possible thing, but the thing that works in front of the client: the full chain from a question typed in the chat to an answer built from the documents, entirely inside SAP BTP and without a single API key pointing outside.
The part that matters: the answer is verifiable#
A chatbot that answers well but never says where the information came from is of no use inside a company: nobody can sign off a decision based on a sentence whose source is unknown. That is why every answer closes with the list of documents the content came from — an explicit client requirement — and why, when the sources do not hold the information, the system says so instead of filling the gap. It is the difference between a demonstrator you can take into a boardroom and one that holds up until somebody asks the wrong question.
My part#
I built the CAP backend and the chat application that queries it: the OData action receiving the question, the retrieval call, the construction of the model's context, error handling and the no-answer path, and the interface showing the result. On the AI side I worked with another developer on document indexing and answer construction; a third colleague handled the integration moving files from SFTP into AI Core grounding.
Where it stands#
It remained a demonstrator, with the scope that implies: no systematic accuracy measurement against a reference question set, no automated release pipeline. Neither was the point of five days. The client is evaluating whether to take the project further.