Do you need a vector database to build RAG?

Not usually. For most startups, Postgres with the pgvector extension handles embeddings alongside the rest of your data, which also makes permission filtering simpler. A dedicated vector database becomes worth considering at very large scale or when you need specialized search features. Start with the database you already run and move only when you measure a problem.

Should you fine-tune a model or use RAG?

Start with RAG. It keeps answers grounded in current data, lets you show sources and updates as soon as your content changes. Fine-tuning changes how a model writes or behaves, not what it knows today, and it's slower and costlier to iterate. Many products never need fine-tuning; good retrieval and clear prompts cover most use cases.

How do you stop an AI feature from making things up?

You reduce it rather than eliminate it. Ground answers in retrieved sources, instruct the model to say when the sources don't cover the question, show citations users can check, and prefer structured outputs for anything that feeds other systems. For high-stakes actions, require a human to approve before anything is sent, saved or charged.

Can you build AI features without sending customer data to a model provider?

Partly. You can minimize what you send, strip personal fields, use providers with data-retention controls or run open models on your own infrastructure. Check each provider's current data-use terms and your customer contracts before launch. For enterprise buyers, be ready to explain where data goes, how long it's kept and whether it's used for training.

All posts

Product design

Building AI Features in a Web App: Streaming, RAG and Cost Control

How to build AI features into a web app: streaming responses, retrieval (RAG) with citations, guardrails and cost control, from a team that designs and ships both.

By Mehdi HassanOct 29, 20266 min read

The short answer

To build AI features into a web app, call the model from your server, stream the response to the UI, ground answers in your own data with retrieval (RAG), and wrap every call in limits, logging and fallbacks. The model is the easy part. Data access, permissions, failure states and per-user cost control decide whether the feature works.

Key takeaways

Never call a model from the browser. Keep keys, prompts and permission checks on the server.
Stream responses so users see progress almost immediately, and design a stop button from the start.
RAG answers from your data: chunk, embed, retrieve with permission filters, then cite the sources.
Control cost with per-user limits, smaller models for simple steps, caching and capped context sizes.
Log every prompt, response and user action, then build a small evaluation set before you change prompts.

Part of Full-Stack Product Engineering for Startups: From Figma to Production

How do you build AI features into a web app?

You build AI features into a web app in five layers: a server route that calls the model, a streaming UI, a retrieval step that pulls in your own data, guardrails around inputs and outputs, and cost and quality monitoring. Each layer has a design side and an engineering side, and they need to be planned together.

This is the implementation companion to our guide on AI product UX, which covers how the feature should look and behave. If you're adding AI to a product people already use, run through the AI readiness checklist first. Both sit within our hub on full-stack product engineering for startups.

Server route
A route handler or server action holds the API key, builds the prompt, checks who the user is and what they may see, then calls the model.
Streaming UI
Tokens are sent to the browser as they're generated, with visible pending, streaming, done, stopped and error states.
Retrieval
Relevant chunks of your own data are fetched, filtered by the user's permissions and added to the prompt.
Guardrails
Inputs are validated, outputs are checked or structured, and risky actions need human approval.
Monitoring
Every call logs tokens, latency, cost and the user's response: accepted, edited, rejected or retried.

How do you stream LLM responses in a React or Next.js app?

Stream LLM responses by having your server forward the model's output as a stream, then rendering each chunk as it arrives in React. Most model providers support streaming, and libraries such as the Vercel AI SDK wrap the server and client sides so you get a streaming text response and a chat hook with little code.

Under the hood, the transport is usually server-sent events or a streamed HTTP response, described in MDN's guide to server-sent events. The design work matters as much as the plumbing: show that the request was received immediately, render partial text without layout jumps, give users a stop button, and keep the input usable so they can correct course.

What is RAG, and how do you add it to a web app?

RAG, or retrieval-augmented generation, means fetching relevant passages from your own data and putting them in the prompt, so the model answers from your facts instead of its training. In a web app, you split documents into chunks, store an embedding of each chunk, retrieve the closest matches for each question and show them as citations.

Chunk your content into passages small enough to be specific and large enough to make sense alone, keeping titles and section headings attached.
Embed each chunk with an embedding model and store the vector with the chunk, its source and its tenant ID. Postgres with the pgvector extension is enough for most products.
Retrieve the top matches for each query, always filtered by the user's workspace and permissions. Combining vector search with keyword search often improves results for names and codes.
Generate the answer with the retrieved chunks in the prompt and an instruction to say when the sources don't contain the answer.
Cite the chunks used, linking back to the original document, so users can check the answer.

If you're on Postgres already, as we recommend in Supabase vs Firebase vs custom Postgres, you can keep vectors in the same database as the rest of your data and reuse its permission rules.

Why permissions are the hardest part of RAG

A RAG system that ignores permissions is a data leak with a chat box. If user A can't open a document in your app, the AI acting for user A must not retrieve it either.

The safe pattern is to store the tenant and access information on every chunk and apply the same filter in the retrieval query that you apply to normal reads. Re-index when permissions change, not only when content changes. Test it by asking questions that only another workspace's data could answer, and confirm the answer is a polite refusal.

How do you control the cost of AI features?

Control the cost of AI features by limiting how much each user can spend, sending simple tasks to smaller models, capping context size, caching repeated work and measuring cost per feature. AI costs scale with usage, so a feature that's cheap in testing can become your biggest bill after a successful launch.

Lever	What it does	Trade-off
Per-user and per-plan limits	Caps spend; ties AI use to pricing	Needs clear UI when users hit a limit
Model routing	Small model for classification, large for hard reasoning	More prompts to test and maintain
Context caps	Retrieve fewer, better chunks; trim history	Too little context lowers quality
Caching	Reuse results for identical requests; use provider prompt caching	Stale answers if data changes
Background jobs	Run long tasks async and notify when done	Less instant, needs notification design

Tie limits to your plans. A higher tier with more AI usage is easier to explain than a surprise overage. That's both a billing and a UX decision, and it needs to be designed with the upgrade flow.

How do you test and monitor AI features in production?

Test AI features with a small evaluation set of real questions and expected answers, run it every time you change a prompt or model, and monitor production with logs of inputs, outputs, cost, latency and user reactions. Unit tests alone can't tell you whether an answer is good.

Evaluation set: start with a few dozen real examples, including tricky and adversarial ones, and grow it from user reports.
User signals: track accept, edit, reject, retry and copy actions. They're the most honest quality measure you have.
Traces: store the prompt, retrieved chunks and response together, so you can see why an answer went wrong.
Alerts: watch error rates, latency and daily cost per feature, not only total spend.

What stack should you use to build AI features in a web app?

For most startups, a practical stack is Next.js for the app and server routes, an AI SDK that abstracts providers and streaming, Postgres with pgvector for retrieval, a background job runner for long tasks and an observability tool for traces. That keeps everything in TypeScript and one database.

Keep the provider swappable. Models and prices change quickly, and an abstraction layer lets you route tasks to different models without rewriting your UI. The surrounding architecture, from tenancy to data access, is covered in building a SaaS app with Next.js, and the wider choices in our SaaS tech stack guide.

Frequently asked questions

Do you need a vector database to build RAG?: Not usually. For most startups, Postgres with the pgvector extension handles embeddings alongside the rest of your data, which also makes permission filtering simpler. A dedicated vector database becomes worth considering at very large scale or when you need specialized search features. Start with the database you already run and move only when you measure a problem.
Should you fine-tune a model or use RAG?: Start with RAG. It keeps answers grounded in current data, lets you show sources and updates as soon as your content changes. Fine-tuning changes how a model writes or behaves, not what it knows today, and it's slower and costlier to iterate. Many products never need fine-tuning; good retrieval and clear prompts cover most use cases.
How do you stop an AI feature from making things up?: You reduce it rather than eliminate it. Ground answers in retrieved sources, instruct the model to say when the sources don't cover the question, show citations users can check, and prefer structured outputs for anything that feeds other systems. For high-stakes actions, require a human to approve before anything is sent, saved or charged.
Can you build AI features without sending customer data to a model provider?: Partly. You can minimize what you send, strip personal fields, use providers with data-retention controls or run open models on your own infrastructure. Check each provider's current data-use terms and your customer contracts before launch. For enterprise buyers, be ready to explain where data goes, how long it's kept and whether it's used for training.

Mehdi Hassan · CEO & Design Director

Copywriter, designer and developer, and founder of Designify Studio. Over five years, 60+ projects across 14+ industries, for startups that have raised $300M+.

Want us to do this for you?

We take craft seriously over at Designify Studio. We do deep research and do what we do best - pay attention to the smallest details and make sure we speak to the right audience.

Schedule Call