Build or buy: assembling the AI stack without locking yourself in

The build-or-buy conversation is almost always held too early, and about the wrong thing. Too early, because the question cannot be answered until you know which layer you mean. About the wrong thing, because "AI" is not a purchase — it is a stack of seven independent choices.

That last point is where the money is. The layers below can be chosen separately. Nothing obliges you to take your compute from the same supplier as your models, or your observability from the same supplier as your orchestration. The most expensive mistake we see in mid-sized organisations is not buying the wrong thing; it is buying one suite because it was one conversation.

The seven layers

Each layer has its own market, its own switching cost and its own rate of change. That last one decides whether building is sensible more often than anything else: in a layer that shifts every quarter, building means rebuilding.

Compute — where the GPUs are and who runs them
Model serving — how a model becomes reachable as an API
Training and fine-tuning — adapting an existing model to your work
Data and vectors — where your knowledge lives and how it is searched
Orchestration and agents — what happens in what order
Observability and evaluation — whether it works, and whether it keeps working
Governance and guardrails — what the system may not do, and how you show it

The landscape

Below are twenty solutions per layer that a mid-sized European organisation will realistically encounter, plus the self-build option in every layer. It is deliberately not a ranking: no market-share figure is published for most of these companies, and ordering them as though one existed would be inventing the most authoritative-looking number on the page.

140 of 140 solutions

Solution	Type	Where it fits
AWSCompute	Hyperscaler	EC2 GPU instances and SageMaker. The default when the rest of the estate is already there.
Microsoft AzureCompute	Hyperscaler	ND-series GPUs and Azure ML. Usually chosen because the enterprise agreement already exists.
Google CloudCompute	Hyperscaler	GPUs and TPUs behind Vertex AI. The only hyperscaler offering its own training silicon.
CoreWeaveCompute	Managed	Kubernetes-native GPU cloud built for AI rather than adapted to it.
LambdaCompute	Managed	Long-running training clusters with persistent storage.
NebiusCompute	Managed	European GPU capacity; relevant where data residency is a constraint.
RunPodCompute	Managed	Per-second GPU rental, community and secure tiers.
Vast.aiCompute	Managed	Marketplace pricing. Cheapest per hour, most variable in reliability.
ModalCompute	Managed	Serverless GPU with a Python-native developer experience.
PaperspaceCompute	Managed	DigitalOcean's GPU platform; simple pricing, smaller catalogue.
ScalewayCompute	Managed	French provider with EU-only regions.
OVHcloudCompute	Managed	European hosting at scale; strong on data-residency guarantees.
HetznerCompute	Managed	German provider. Very low cost, limited GPU catalogue.
Oracle Cloud InfrastructureCompute	Hyperscaler	Aggressive GPU pricing and large reserved capacity.
IBM CloudCompute	Hyperscaler	Relevant mainly where watsonx is already in the estate.
CrusoeCompute	Managed	Energy-first GPU cloud; competitive on cost per hour at scale.
Together AICompute	Managed	Compute and inference in one contract.
SpheronCompute	Managed	Aggregates bare-metal capacity across several providers.
NorthflankCompute	Managed	Deploys containers to GPUs without running Kubernetes yourself.
Own GPUs, on-premise or colocatedCompute	Self-build	Lowest marginal cost at sustained high utilisation. Carries procurement lead times, power, cooling and a hardware refresh cycle. Rarely justified below continuous multi-GPU load.
OpenAI APIModel serving	Managed	The broadest frontier catalogue and the most mature tooling around it.
Anthropic APIModel serving	Managed	Frontier models with a strong position on coding and long-document work.
Google Gemini APIModel serving	Managed	Long context and native multimodality; often the cheapest capable tier.
Azure OpenAI ServiceModel serving	Hyperscaler	OpenAI models inside an Azure tenancy, with EU region options.
Amazon BedrockModel serving	Hyperscaler	One API across several model families, inside an AWS account.
Google Vertex AIModel serving	Hyperscaler	Gemini plus third-party models with enterprise controls.
Mistral AIModel serving	Managed	European provider; relevant where EU processing is contractual rather than preferred.
CohereModel serving	Managed	Enterprise focus, strong retrieval and reranking models.
Together AIModel serving	Managed	Large open-weights catalogue at low cost per token.
Fireworks AIModel serving	Managed	Optimised serving of open models; competitive on latency.
BasetenModel serving	Managed	Bring your own model, deployed and scaled for you.
ReplicateModel serving	Managed	Container-packaged models with per-second billing.
GroqModel serving	Managed	Very high output speed on a fixed model set.
CerebrasModel serving	Managed	Wafer-scale inference; speed at the top of the market.
OpenRouterModel serving	Managed	One key across many providers. Useful for comparison and failover.
Hugging Face Inference EndpointsModel serving	Managed	Any model on the Hub, deployed to a managed endpoint.
AnyscaleModel serving	Managed	Ray-based serving for teams already using Ray.
vLLMModel serving	Open source	The de facto open serving engine. High throughput, you operate it.
OllamaModel serving	Open source	Local and small-scale serving; the fastest way to try an open model.
vLLM or TGI on your own GPUsModel serving	Self-build	Full control of model, version and data path, and the lowest cost per token at sustained volume. You own the uptime, the upgrades and the GPU capacity planning.
OpenAI fine-tuningTraining and fine-tuning	Managed	Supervised fine-tuning on hosted models. No infrastructure to run.
Azure AI FoundryTraining and fine-tuning	Hyperscaler	Fine-tuning and evaluation inside an Azure tenancy.
Vertex AI TrainingTraining and fine-tuning	Hyperscaler	Managed training jobs with TPU or GPU backing.
Amazon SageMakerTraining and fine-tuning	Hyperscaler	The broadest managed training toolchain; steep learning curve.
Together AI fine-tuningTraining and fine-tuning	Managed	LoRA and full fine-tunes on open models, deployed on the same platform.
Fireworks fine-tuningTraining and fine-tuning	Managed	Tuning paired directly with optimised serving.
PredibaseTraining and fine-tuning	Managed	Fine-tuning aimed at small task-specific models rather than frontier ones.
Hugging Face AutoTrainTraining and fine-tuning	Managed	Low-code tuning on the Hub.
AxolotlTraining and fine-tuning	Open source	The most widely used open fine-tuning harness.
UnslothTraining and fine-tuning	Open source	Memory-efficient tuning; makes single-GPU work feasible.
LLaMA-FactoryTraining and fine-tuning	Open source	Broad model coverage with a usable interface.
Weights & BiasesTraining and fine-tuning	Managed	Experiment tracking. Not training itself, but what makes it repeatable.
Determined AITraining and fine-tuning	Open source	Cluster scheduling and hyperparameter search.
Ray TrainTraining and fine-tuning	Open source	Distributed training for teams already running Ray.
Databricks Mosaic AITraining and fine-tuning	Managed	Training tied to a lakehouse; strong where the data already lives there.
ModalTraining and fine-tuning	Managed	Serverless training runs without a standing cluster.
Lightning AITraining and fine-tuning	Managed	Studio environments around PyTorch Lightning.
Nebius AI StudioTraining and fine-tuning	Managed	Tuning on European infrastructure.
Google Colab EnterpriseTraining and fine-tuning	Hyperscaler	Notebook-based experimentation with managed backends.
DeepSpeed or Ray on your own clusterTraining and fine-tuning	Self-build	Necessary for genuinely novel training work. For adapting an existing model to your data it is almost always more machinery than the problem needs.
pgvectorData and vectors	Open source	Vector search inside the Postgres you already run. The right first answer for most mid-sized firms.
PineconeData and vectors	Managed	Zero-operations vector search with built-in embedding and reranking.
QdrantData and vectors	Open source	Rust engine with fine-grained filtering and quantisation. Cloud or self-hosted.
WeaviateData and vectors	Open source	Hybrid search and multi-tenancy; self-hosting expects Kubernetes.
MilvusData and vectors	Open source	Built for billion-scale collections.
ChromaData and vectors	Open source	The quickest path from prototype to working retrieval.
LanceDBData and vectors	Open source	Multimodal and file-native; no server to run.
ElasticsearchData and vectors	Managed	Mature keyword search with vectors added. Strong hybrid retrieval.
OpenSearchData and vectors	Open source	The open fork; the same hybrid strengths without the licence.
RedisData and vectors	Managed	Vector search alongside a cache you may already operate.
MongoDB Atlas Vector SearchData and vectors	Managed	Vectors beside documents in one database.
Azure AI SearchData and vectors	Hyperscaler	Retrieval with enterprise access controls built in.
Vertex AI Vector SearchData and vectors	Hyperscaler	Google's managed index; scales without tuning.
Amazon OpenSearch ServerlessData and vectors	Hyperscaler	Managed hybrid retrieval inside AWS.
Databricks Vector SearchData and vectors	Managed	Retrieval governed by the same catalogue as the rest of the data.
Snowflake Cortex SearchData and vectors	Managed	Retrieval where the warehouse already is.
VespaData and vectors	Open source	Serving and ranking at very large scale; substantial operational weight.
TurbopufferData and vectors	Managed	Object-storage-backed vectors; unusually low cost at rest.
SingleStoreData and vectors	Managed	Transactional and vector workloads in one engine.
pgvector on your existing PostgresData and vectors	Self-build	No new vendor, no new backup regime, no new access model. The ceiling is real but most mid-sized corpora never reach it.
LangChainOrchestration and agents	Open source	The largest ecosystem of connectors and the most examples to copy.
LangGraphOrchestration and agents	Open source	Explicit state machines for agents; far easier to reason about than free-form chains.
LlamaIndexOrchestration and agents	Open source	Retrieval-first framework with strong document handling.
Semantic KernelOrchestration and agents	Open source	Microsoft's orchestration layer; natural fit alongside .NET.
HaystackOrchestration and agents	Open source	Production-oriented pipelines from deepset.
CrewAIOrchestration and agents	Open source	Role-based multi-agent orchestration.
AutoGenOrchestration and agents	Open source	Microsoft Research's multi-agent conversation framework.
Pydantic AIOrchestration and agents	Open source	Typed, validated agent definitions. Small surface, few surprises.
TemporalOrchestration and agents	Open source	Durable execution. The right answer when a workflow must survive a restart.
PrefectOrchestration and agents	Open source	Python-native workflow orchestration with a managed option.
Apache AirflowOrchestration and agents	Open source	Scheduled pipelines; the incumbent in most data teams.
n8nOrchestration and agents	Open source	Visual automation, self-hostable. Popular in European mid-market.
MakeOrchestration and agents	Managed	Visual automation with a large connector library.
ZapierOrchestration and agents	Managed	The lowest-effort route to connecting existing SaaS.
Microsoft Copilot StudioOrchestration and agents	Managed	Agents inside Microsoft 365, governed by existing tenancy controls.
Google Agent BuilderOrchestration and agents	Hyperscaler	Agent construction and deployment on Vertex.
Amazon Bedrock AgentsOrchestration and agents	Hyperscaler	Managed agents with action groups inside AWS.
DifyOrchestration and agents	Open source	Low-code application builder over your own models.
FlowiseOrchestration and agents	Open source	Visual chain building on top of LangChain.
Direct API calls with your own state machineOrchestration and agents	Self-build	For a single well-understood workflow this is often less code than the framework, and it never breaks on a framework upgrade. It stops scaling once you have several agents sharing state.
LangfuseObservability and evaluation	Open source	Open, self-hostable tracing and prompt management. The common default.
LangSmithObservability and evaluation	Managed	The smoothest path if you already build with LangChain or LangGraph.
BraintrustObservability and evaluation	Managed	Evaluation-first: experiments, scoring and review as the primary object.
Arize AXObservability and evaluation	Managed	Production monitoring with drift detection.
Arize PhoenixObservability and evaluation	Open source	Notebook-first tracing for experimentation.
LangWatchObservability and evaluation	Open source	Self-hosted tracing with online evaluation and guardrails together.
Weights & Biases WeaveObservability and evaluation	Managed	Tracing and evals beside experiment tracking.
GalileoObservability and evaluation	Managed	Fast inline scoring aimed at intercepting output in production.
Confident AI (DeepEval)Observability and evaluation	Open source	A test-suite model for LLM behaviour; runs in CI.
TruLensObservability and evaluation	Open source	Feedback functions for RAG quality.
HeliconeObservability and evaluation	Open source	A proxy that gives logging and caching with a one-line change.
PromptLayerObservability and evaluation	Managed	Prompt versioning and comparison for non-engineers.
Datadog LLM ObservabilityObservability and evaluation	Managed	Sensible where Datadog is already the monitoring system.
New Relic AI MonitoringObservability and evaluation	Managed	Same argument, different incumbent.
OpenTelemetry GenAIObservability and evaluation	Open source	The emerging standard. Vendor-neutral, and what keeps you portable.
WhyLabsObservability and evaluation	Managed	Data and model drift monitoring.
TrueFoundryObservability and evaluation	Managed	Serving and observability in one platform.
OpenlayerObservability and evaluation	Managed	Testing and monitoring with a strong evaluation story.
HumanloopObservability and evaluation	Managed	Prompt management with domain-expert review built in.
OpenTelemetry into your existing stackObservability and evaluation	Self-build	If you already run Grafana or the ELK stack, emitting spans into it costs little and adds no vendor. You give up the LLM-specific evaluation features, which is the trade.
NVIDIA NeMo GuardrailsGovernance and guardrails	Open source	Programmable conversational rails; topic and flow boundaries.
Guardrails AIGovernance and guardrails	Open source	Input and output validators with a shared library of checks.
Azure AI Content SafetyGovernance and guardrails	Hyperscaler	Managed content filtering with severity levels.
Amazon Bedrock GuardrailsGovernance and guardrails	Hyperscaler	Policy enforcement applied across models in Bedrock.
Vertex AI safety filtersGovernance and guardrails	Hyperscaler	Configurable safety thresholds on Google-served models.
LakeraGovernance and guardrails	Managed	Prompt-injection and jailbreak defence.
Protect AIGovernance and guardrails	Managed	Model and supply-chain security scanning. Now part of Palo Alto Networks.
Robust IntelligenceGovernance and guardrails	Managed	Automated red-teaming and runtime protection. Part of Cisco.
Credo AIGovernance and guardrails	Managed	Governance registry mapped to regulatory frameworks, including the EU AI Act.
Holistic AIGovernance and guardrails	Managed	Risk assessment and compliance reporting.
IBM watsonx.governanceGovernance and guardrails	Managed	Model inventory, risk and lifecycle documentation.
Fiddler AIGovernance and guardrails	Managed	Explainability and monitoring for regulated use.
ArthurGovernance and guardrails	Managed	Performance and bias monitoring in production.
BifrostGovernance and guardrails	Open source	An AI gateway that applies the same policy to every model call.
LLM GuardGovernance and guardrails	Open source	A practical scanner set for inputs and outputs.
Microsoft PresidioGovernance and guardrails	Open source	Personal-data detection and redaction. Directly useful under the GDPR.
RebuffGovernance and guardrails	Open source	Prompt-injection detection.
VijilGovernance and guardrails	Managed	Agent trust evaluation and certification.
TrustibleGovernance and guardrails	Managed	AI Act obligation tracking and evidence collection.
Policy checks in your own gatewayGovernance and guardrails	Self-build	You already need a gateway for keys and rate limits; putting redaction and logging there keeps one enforcement point. What you do not get is a maintained attack library, which is the part that ages fastest.

What most organisations miss

Three things come up in nearly every conversation.

The first is that building rarely means building everything. In almost every layer the sensible self-build option is small and specific: pgvector in the Postgres you already run, or policy checks in the gateway where you already manage keys and rate limits. That is a different proposition from running your own inference platform.

The second is that switching cost only counts if you work it out in advance. Ask of every layer: what would it cost to leave this in eighteen months? If the answer is unknown, that is the answer.

The third is that the AI Act does not transfer with the purchase. For most applications you are the provider, whoever's model sits underneath. A supplier can give you documentation; it cannot take on your obligation.

The framework

Ten dimensions, applied per use case and per layer. They rarely all point the same way — which is what makes it useful, because where they diverge is where the tension is.

Where this usually lands

For mid-sized organisations this rarely ends at "build" and rarely at "buy". It ends at compose: buy the layer, own the part that is yours. That part is almost always the same — your data, your prompts, your evaluation set and your interface. It is precisely the part suppliers give away most easily and replace least well.