Privately hosted models
Privately hosted models can be used with Autologyx where a Sequence needs to call an AI or machine learning service that is hosted, managed, or controlled outside the standard OpenAI, Azure OpenAI, or AWS Bedrock actors.
This is a flexible pattern. Instead of using a dedicated AI actor, the Sequence uses the generic API integration capability in Autologyx to send structured requests to a model endpoint and use the response later in the workflow.
Privately hosted models can include large language models, smaller specialist models, conventional machine learning models, image or audio models, redaction services, named entity recognition services, or customer-specific AI gateways.
When to use privately hosted models
Use privately hosted models where the customer needs more control over the model, hosting environment, data handling, or inference behaviour than a standard public model integration provides.
This can be useful where:
- the model performs a specialist task that is not well served by a general-purpose LLM
- the customer wants stronger control over data location, data retention, logging, or access
- the customer has fine-tuned or otherwise adapted a model for a specific domain
- the model needs to process images, voice, video, documents, or other non-text inputs
- the customer already has a model-serving platform or AI gateway
- the workflow needs to call a conventional machine learning model, not an LLM
- the organisation wants to run AI workloads inside a private cloud, private network, or customer-controlled environment
- the workflow needs to combine structured Autologyx data with specialist model inference
This approach can be powerful because Autologyx is very good at managing structured records, workflow state, user tasks, permissions, documents, and integrations. A privately hosted model can add specialist prediction or analysis capabilities while the Sequencer continues to control the business process.
How privately hosted model integration works
A privately hosted model normally exposes an API endpoint.
The Sequence can use an API Call actor or another generic API integration pattern to:
- collect data from an Object Record,
- prepare a request body,
- authenticate to the private endpoint,
- send text, structured data, files, references, or metadata to the model service,
- receive a response,
- parse or store the result,
- update the record, create a task, generate a document, or route the workflow.
Important: Autologyx does not need to understand every detail of the model internally. The model endpoint needs to expose a reliable API contract, and the Sequence needs to know what to send and how to interpret the response.
Why customers choose private hosting
Privately hosted models are sometimes chosen for reasons that are technical, commercial, operational, or governance-related.
Common reasons include:
Specialist capability
Some models are designed for a specific type of processing.
For example, a customer may have a high-performance model for:
- PII detection and redaction
- named entity recognition
- legal clause detection
- document layout understanding
- image classification
- voice transcription or speaker analysis
- video frame analysis
- fraud detection
- risk scoring
- outcome prediction
- anomaly detection
These models may be smaller, faster, cheaper, or more accurate for a specific task than a general-purpose LLM.
Conventional machine learning
Not every AI use case requires a large language model.
Some workflow decisions are better supported by conventional machine learning, such as regression, classification, clustering, ranking, or scoring models. These models can be particularly useful when paired with Autologyx because Autologyx can supply structured data from records, tasks, forms, documents, and process history.
For example, a privately hosted model might return:
{
"risk_score": 0.82,
"predicted_category": "supplier_contract",
"requires_review": true,
"reason_code": "high_value_and_non_standard_terms"
}The Sequence can then apply deterministic logic to the returned values. For example, it can create a human review task where requires_review is true, or route the record differently where the risk score exceeds a configured threshold.
Fine-tuning and post-training optimisation
Some applications benefit from fine-tuning, retrieval tuning, adapter-based approaches, reinforcement learning from human feedback, domain-specific evaluation, or other post-training techniques.
Private hosting may be useful where a customer has a model that has been adapted for:
- their document types
- their terminology
- their risk taxonomy
- their clause library
- their intake categories
- their matter types
- their internal playbooks
- their historic decision data
This can help the model behave in a way that is more closely aligned to the customer’s domain and workflow than a general-purpose public model.
Privacy and control
Private hosting can provide the highest degree of control over where the model runs, who can access it, how data is logged, and how traffic is routed.
This may matter where:
- sensitive documents are being processed
- the customer has strict data residency requirements
- model inputs and outputs must remain inside a private network
- security teams require direct control over infrastructure
- the customer wants to apply their own monitoring, logging, encryption, retention, or access controls
- the organisation has an approved private AI gateway
Private hosting does not remove the need for careful AI governance, but it can give the customer more control over the infrastructure and data path.
Examples of privately hosted model use cases
Privately hosted models can support many different workflow patterns.
Examples include:
PII detection and redaction
A Sequence sends document text to a private named entity recognition or redaction model.
The model returns structured entities such as names, addresses, dates of birth, passport numbers, national identifiers, email addresses, or account numbers. The Sequence can then store the results, create a review task, or pass the output to a downstream redaction service.
This can be useful where the customer needs high-performance PII processing at scale and wants to keep the model and data inside a controlled environment.
Risk scoring
A Sequence sends structured matter, contract, or request data to a private scoring model.
The model returns a risk score, confidence value, and reason codes. The Sequence can then route the record to a low-risk path, create a review task, or escalate it for specialist handling.
This pattern is often more deterministic than asking an LLM to make a broad judgement because the model is designed to return defined numeric or categorical outputs.
Specialist document classification
A private model classifies documents using a customer-specific taxonomy.
The Sequence can use the returned category to select a workflow route, assign a task, apply a document template, or trigger a review process.
Image, audio, or video processing
A private model processes non-text content.
For example, a workflow might send an image for classification, audio for transcription, or video metadata for analysis. The returned structured result can then be stored on the record and used by later workflow steps.
Customer AI gateway
Some customers expose a single approved AI gateway rather than allowing different workflows to call different model providers directly.
The gateway may route requests to different models, enforce data policies, log usage, apply redaction, or standardise responses. Autologyx can call that gateway through a generic API actor, while the customer’s AI platform team manages the model selection and hosting behind it.
Cloud infrastructure and GPUs
Private hosting does not necessarily mean running models on physical hardware owned by the customer.
Many organisations use cloud providers to run private or controlled AI workloads. Cloud platforms can provide GPU-backed virtual machines, managed Kubernetes, container services, networking, identity controls, monitoring, and storage.
This can give customers flexibility. They can run a private endpoint using cloud infrastructure while still controlling the deployment, model weights, network route, security policy, and data handling approach.
Customers should work with their cloud, data science, and security teams to choose the right hosting pattern. Model size, throughput, latency, memory, GPU requirements, cost, security, and operational support all matter.
Model-serving platforms and tooling
Privately hosted model endpoints may be built using a range of platforms and tools.
Examples include:
- Hugging Face Inference Endpoints
- Hugging Face Text Generation Inference
- Hugging Face Text Embeddings Inference
- vLLM
- Ray Serve
- custom FastAPI, Flask, or containerised services
- Kubernetes-based model-serving platforms
- customer AI gateways
- cloud-native model-serving services
Autologyx does not require a specific model-serving platform. The key requirement is that the service exposes an API that the Sequencer can call and that the response can be handled predictably.
API design considerations
A privately hosted model endpoint should have a clear API contract.
Before connecting it to Autologyx, agree:
- the endpoint URL
- the authentication method
- the request method
- the request headers
- the expected request body
- whether files, text, JSON, or references are being sent
- the expected response format
- how errors are returned
- how timeouts and retries should be handled
- whether responses are synchronous or asynchronous
- how model versioning is controlled
- how requests and responses are logged
- who owns monitoring and support
The clearer the API contract, the easier it is to configure the Sequencer and make the workflow reliable.
Structured responses and deterministic processing
Private model responses are most useful when they are returned as structured data.
For example:
{
"document_type": "contract",
"pii_detected": true,
"entities": [
{
"type": "person_name",
"value": "Example Name",
"confidence": 0.94
}
],
"recommended_action": "human_review"
}A structured response lets the Sequence apply deterministic logic after the model has responded.
For example, the Sequence can check whether pii_detected is true, whether confidence is above a threshold, or whether recommended_action matches an allowed value.
This is usually safer and easier to maintain than relying only on free text output.
Human review and governance
Privately hosted models still need governance.
Even when the model is hosted in a customer-controlled environment, the output may still be uncertain, incomplete, or unsuitable for fully automated use. This is especially true where the model produces semantic output, narrative analysis, predictions, or recommendations.
Use human review where the output affects important decisions, external communications, sensitive data, record updates, or regulated processes.
Governance should also cover:
- what data may be sent to the model
- who can configure or call the endpoint
- how outputs are logged
- how model performance is monitored
- how model versions are changed
- how errors are handled
- when a workflow must stop and ask for review
Preparing a private model for Autologyx
In principle, the customer or implementation team should:
- choose the model or model-serving platform,
- decide where the model will be hosted,
- define the endpoint and API contract,
- configure authentication and network access,
- agree what data Autologyx will send,
- agree what structured response the model will return,
- configure the Authentication Object in Autologyx,
- configure the API Call actor in the Sequence,
- test using representative records,
- add validation and human review gates where needed,
- monitor model behaviour after release.
The best private model integrations are designed jointly by the Autologyx configuration team, the customer’s AI or data science team, and the customer’s security or cloud platform team.
Where to find more information
The following resources may be useful when planning privately hosted model patterns.
Cloud GPU and infrastructure options:
- AWS Deep Learning AMI GPU instance guidance
- Google Cloud GPUs
- Azure GPU-accelerated virtual machine sizes
Model and inference platforms:
- Hugging Face Inference Endpoints
- Hugging Face Text Generation Inference
- Hugging Face Text Embeddings Inference
- vLLM OpenAI-compatible server
- Ray Serve
These links are starting points only. Customers should use the current documentation for their chosen platform, hosting provider, model, and security architecture.
Important reminder about model outputs
Important reminder
Privately hosted models are not automatically deterministic or risk-free.
Some private models may return highly structured and predictable outputs. Others, especially LLMs and generative models, may produce inconsistent, incomplete, or inaccurate responses.
Prompt design, model choice, fine-tuning, structured JSON, validation rules, confidence thresholds, and deterministic Sequence logic can reduce risk, but they cannot absolutely guarantee that a model will always return a perfect, complete, or accurate response.
Where the model returns specific structured values, the Sequencer can often apply deterministic checks to decide what should happen next. Where the model produces semantic output, such as a narrative summary, interpretation, prediction, or recommendation, customers should strongly consider human review before the output is used for important decisions, external communications, or final record updates.
Things to remember
- Privately hosted models are integrated through generic API patterns rather than a single dedicated provider actor.
- Private hosting can support LLMs, specialist models, conventional machine learning, redaction, entity extraction, image processing, audio processing, video processing, and customer AI gateways.
- Private models are useful where the customer needs specialist capability, fine-tuning, post-training optimisation, or stronger control over data and infrastructure.
- The model endpoint should expose a clear API contract.
- Structured responses are easier for the Sequencer to validate and process deterministically.
- The customer’s AI, cloud, security, and Autologyx configuration teams should agree the data flow before production use.
- Use human review where outputs are semantic, high impact, externally visible, or difficult to validate automatically.