> For the complete documentation index, see [llms.txt](https://documentation.proto.cx/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.proto.cx/docs/developers/on-premise-and-hybrid-hosting.md). # On-premise & hybrid hosting An alternative to default cloud hosting, bespoke on-premise or hybrid hosting gives you location control over data residency. {% hint style="info" %} On-premise or hybrid hosting are available with the Enterprise Max add-on. {% endhint %} ## **Hosting Options** Get in touch with your Proto account representative to discuss which hosting solution is right for you, and its manual deployment procedure. * **On-premise** means all database content (e.g. AI configurations, conversations, and profiles of registered people), bucket content (files & attachments), and the Proto AICX platform (web app) are all hosted independently by you. * **Hybrid Hosting** means all database and bucket content is hosting independently by you, but hosting and provision of the Proto AICX platform (web app) is maintained by us. *** ## **System requirements** #### Software * **OS:** Ubuntu 22.04.3 LTS * **Web Server:** Nginx 1.14.0 or later * **Docker:** 24.0.0 or later * **Docker Compose:** 2.23.0 or later * **AWS CLI:** 2.13.30 or later * **Python:** 3.10 or later #### Hardware * **Memory:** 32GB RAM minimum * **Disk Space:** 20GB required for safe installation.\ An additional 500 GB for database storage is recommended. #### **Network** * Allowed traffic through port 80/443 #### External Services * SendGrid API key * OpenAI or Azure OpenAI API key *** ## Chat-GPT data privacy If you have AI agent with a Chat-GPT LLM enabled, it's recommended to use Azure's OpenAI service instead of OpenAI directly. The benefits of Azure’s OpenAI service include: * Faster response times * 99.9% uptime SLA * Stronger data privacy From Microsoft's legal resource for [data, privacy, and security for Azure OpenAI Service](https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy): > Your prompts (inputs) and completions (outputs), your embeddings, and your training data: > > * are NOT available to other customers. > * are NOT available to OpenAI. > * are NOT used to improve OpenAI models. > * are NOT used to improve any Microsoft or 3rd party products or services. > * are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data). > * Your fine-tuned Azure OpenAI models are available exclusively for your use. > > The Azure OpenAI Service is fully controlled by Microsoft; Microsoft hosts the OpenAI models in Microsoft’s Azure environment and the Service does NOT interact with any services operated by OpenAI (e.g. ChatGPT, or the OpenAI API). *** ## Installation {% hint style="info" %} These installation steps are specific to on-premise hosting and are for reference only.\ Proto personnel will provide direct guidance on your host migration process. {% endhint %} There are three components that must be served by your local server: 1. The backend platform 2. The Proto AICX platform (web app) 3. The webchat library These three components can be served directly from Nginx. As components (2) and (3) are pre-built, they only need to be served as static files from Nginx without any configuration. You will be provided with the following files to scaffold the on-premise deployment: * A Python file: *proto\_scaffold.py* * A JSON file: *proto\_credentials.json* To scaffold: 1. Place both files in your desired installation directory. 2. Run python `proto_scaffold.py init` The command will *not* start Proto. It will create a folder with the relevant service configuration TOML files, authenticate your instance with our private docker container repository, and add a docker-compose.yml. *** ## Configuration In the newly created directory, you'll find a subdirectory named *backend/config/*. This contains all the configurable parameters for the backend platform. While many parameters might not require adjustments, the following are mandatory: ``` [app] base_url = "https://{YOUR_URL}" base_api_url = "https://{YOUR_URL}/api" [cors] allow_origins = ['https://{YOUR_URL}'] [sendgrid] api_key = "{YOUR_SENDGRID_API_KEY}" from_domain = "{YOUR_DESIRED_EMAIL_FROM}" inbound_parse_key = "{YOUR_SENDGRID_INBOUND_URL_KEY_PARAM}" [openai] api_key = "{YOUR_OPENAI_API_KEY}" [api] session_middleware_secret_key = "{YOUR_RANDOM_SECRET_KEY_1}" secret_key = "{YOUR_RANDOM_SECRET_KEY_2}" algorithm = "HS256" access_token_expire_minutes = 3600 frontend_url = "https://{YOUR_URL}" ... ``` ### Nginx example The backend platform will run its services on port 8080 of the containers. Each service should be prefixed with *api/*. The web app and the webchat library will be served from */* and */webchat*, respectively. ``` ... location / { # Serve web app root /home/ubuntu/proto/frontend; index index.html; try_files $uri $uri/ /index.html; } # Entry point; must never be served stale so version detection works. location = /index.html { add_header Cache-Control "no-cache"; } # Build assets are named with a content hash, so a changed file always gets a # new filename — safe to cache for a long time with no revalidation. location /assets/ { add_header Cache-Control "public, max-age=31536000, immutable"; } # Version manifest emitted at build time; must never be served stale. location = /version.json { add_header Cache-Control "no-cache, no-store, must-revalidate"; } location /webchat { # Serve webchat root /home/ubuntu/proto/webchat; index index.html; try_files $uri $uri/ /index.html; } location /api/platform/ { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Protocol $scheme; proxy_pass http://platform:8080/; } location /api/chatbot/ { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Protocol $scheme; proxy_pass http://chatbot:8080/; } location /msgrouter_ws/ { proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; proxy_pass http://msgrouter_ws:9000; } ... ``` ### Managing platform state * **Start the platform:** in *backend/* run `docker compose up -d` * **Stop the platform:** in *backend/* run `docker compose down` * **Update the platform from remote:** `python proto_scaffold.py update` * **Directly connect to database:** `psql -U postgres -d platform` ### Database connection pooling If you monitor your own database (for example, via RDS alerting) on an on-premise or hybrid deployment, it helps to know how the backend's connection pool behaves by default. This applies separately to the primary (`[db]`) and read-replica (`[db_read_replica]`) pools in your backend configuration: | Behaviour | Details | | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Pool size** | Up to 5 persistent connections, plus up to 10 additional overflow connections under load. | | **Connection recycling** | Each pooled connection is closed and re-established after being open for 30 minutes, rather than held indefinitely. This keeps the connection-count baseline from creeping upwards over long-running deployments. | | **Checkout timeout** | A request waits up to 30 seconds for a free connection before failing. | *** ## Frontend version updates Every build of the Proto AICX web app embeds a version string and emits a small `version.json` manifest alongside the built assets. The client uses this to detect when a newer deploy has replaced the build it's currently running. ### Detecting a new deploy While a user is signed in, the web app periodically compares its own version against `version.json`: | Behaviour | Details | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | **Check frequency** | Roughly once a minute, and again whenever the tab regains focus or the user navigates to a new page. | | **Condition** | Only checks while the tab is visible; skipped entirely in local development builds. | | **On mismatch** | Shows the update notification described in [Staying up to date](/docs/getting-started/staying-up-to-date.md), then stops checking further. | ### Handling a stale page If a signed-in user leaves a tab open across a deploy without refreshing, they may eventually navigate to a page whose JavaScript is no longer available at its previous location, because the newer deploy replaced it with differently named files. The web app does not retry automatically when this happens: | Behaviour | Details | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Detection** | A failed module load matching the signature of a stale-version failure (e.g. "Failed to fetch dynamically imported module") is recognised and flagged separately from other errors, for internal diagnostics. | | **What the user sees** | The same "Something went wrong" error page shown for any unhandled error, displaying the running app's version and an event reference the user can pass to support. | | **Recovery** | Refreshing the page loads the current build. There's no automatic retry or reload. | In practice, the update notification above is what keeps most users from hitting this: it's designed to appear well before a tab is likely to reach a stale chunk. A user who ignores it and later navigates to an affected page sees the error page and needs to refresh manually to continue. ### Caching requirements for self-hosted frontends If you serve the web app yourself, your web server's caching rules need to respect the following — the [Nginx example](#example-nginx-configuration) above sets these: | Path | Required behaviour | Why | | --------------------------------------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `version.json` | Never cached (`no-cache, no-store, must-revalidate`) | The client must always read the freshly deployed manifest — a cached response would make it believe it's already up to date. | | `index.html` / SPA routes | Never cached, or cached with mandatory revalidation | Old HTML would reference filenames from a previous build that may no longer exist. Since a stale chunk surfaces as an error page rather than an automatic retry, getting this right is what keeps users from landing on it in the first place. | | Hashed JS/CSS build output (e.g. `/assets/*`) | No specific rule required | Filenames change whenever content changes, so these files never need cache invalidation. How long they're cached is purely a performance choice. | *** ## Related {% content-ref url="/pages/Qr9pei9HMblePEnopM7Q" %} [Staying up to date](/docs/getting-started/staying-up-to-date.md) {% endcontent-ref %}