Updated October 15, 2026. This guide is based on public product documentation, pricing pages, and policies. We did not create an account, run API requests, train a model, or test performance.
Image note: The featured image is retained from this listing’s March 2025 publication. Its acceleration and other marketing claims are historical and are not benchmark evidence.
Together AI is a developer cloud for running AI models, customizing them with training data, and renting GPU infrastructure. Its web console, APIs, and developer tools support different workflows, from a single inference request to a dedicated deployment. The important first decision is which service your application actually needs. Together AI overview.
For a small application, start by evaluating an existing model on a clearly defined task. Consider customization or reserved compute only when you can explain the quality, capacity, or operational requirement that justifies it. A bigger model catalog alone does not establish suitability.
What can you do with Together AI?
Serverless inference
Serverless lets an application call a supported model without provisioning GPUs. Billing depends on the model and modality: tokens for text, with other units for images, audio, or video. It suits experimentation and uneven traffic. Response times can vary with demand, and requests can be limited. Serverless does not let you select the serving region. Serverless documentation.
Dedicated model inference
Dedicated endpoints serve a model on reserved GPUs and support compatible fine-tuned models. You choose a deployment configuration and capacity rather than sharing the serverless pool. This is a candidate for workloads needing hardware control or their own model deployment; better economics depend on utilization, and performance still needs evaluation on your application. Dedicated inference overview.
Provisioned throughput
Provisioned throughput reserves capacity for a supported model or model family, with defined throughput and reliability targets. It is sales-assisted, has a minimum one-month term, and uses provisioned throughput units, or PTUs. This is another route for predictable production traffic; it should not be confused with renting GPUs to host a fine-tuned model. Provisioned throughput details.
GPU clusters
GPU clusters provide compute for distributed training, research jobs, and serving workloads. Together offers on-demand and reserved capacity with orchestration options including Kubernetes and Slurm. Treat this as infrastructure: your team still needs an appropriate workload, software configuration, and operating plan. GPU cluster product page.
Fine-tuning and post-training
The managed fine-tuning service adapts a supported base model to your dataset. LoRA trains adapter weights while leaving base weights fixed; full fine-tuning updates the base weights. Availability depends on the model. Supervised fine-tuning and preference training are distinct choices from LoRA versus full training. Fine-tuning overview.
The separate post-training workflow lets developers control a training loop using remote GPUs, including supervised training, DPO, and GRPO reinforcement learning. It is intended for teams that need to control the training process in code. Post-training overview.
Pricing and access
Do not assume a general free trial. Current billing documentation says an organization’s first credit purchase has a $15 minimum, with $5 minimum purchases afterward. An official support article still states a $5 access minimum. Both say there is no current free trial. Because these public pages disagree, check the amount shown in your account before purchasing. The self-serve platform is prepaid; active enterprise contracts can have different terms.
There are narrower exceptions: the public price list currently lists Ternary Bonsai 27B at $0 for input and output. A zero-priced model entry does not establish unrestricted free platform access or remove account requirements.
The Startup Accelerator is selection-based. It advertises credit awards of up to $15,000, $30,000, or $50,000 depending on funding tier; reserved GPU clusters are excluded. These are conditional program benefits, not automatic signup credits.
Dollar amounts below are presented as US dollars (USD), consistent with Together’s published USD billing examples. Prices were checked on October 15, 2026. Confirm current rates and the final amount before committing.
| Service | Example | Billing unit |
|---|---|---|
| Text inference | gpt-oss-120B: $0.15 input / $0.60 output | Per million tokens |
| Dedicated inference | H100: $5.49 | Per GPU-hour, billed by minute |
| GPU clusters | H100 on-demand: $3.99 | Per GPU-hour |
Source: Together AI pricing. These rows describe different services, not interchangeable ways to buy the same capacity.
Dedicated inference bills each ready replica for its running time, even without requests. GPU count and replica count multiply cost. A stopped deployment, or one scaled to zero replicas, stops compute billing; restarting requires a cold start. Set capacity bounds and an idle-shutdown policy appropriate for your application. Dedicated billing details.
Managed fine-tuning is billed for processed training and validation tokens, with rates varying by model, training method, and training type. Repeated epochs and evaluations add tokens. Model-specific minimum charges apply, and hosting the result is a separate expense. Use the job estimate before launching. Fine-tuning billing.
Post-training instead charges for GPU time while a model resource is ready, including idle time. Ending a training script does not necessarily stop that resource. Estimate the intended configuration and explicitly stop resources when finished. Post-training billing.
Cluster storage has its own lifecycle and can persist after compute is removed. Include retained volumes in your budget and cleanup plan. Cluster billing and storage rules.
A fictional API cost example
Imagine a shop wants drafts of short product descriptions from approved, non-sensitive specifications. Suppose it makes 10,000 requests, each consuming 500 input tokens and 150 billed output tokens. That is 5 million input tokens and 1.5 million output tokens.
Using the listed gpt-oss-120B rates of $0.15 per million input tokens and $0.60 per million output tokens, the arithmetic is (5 × $0.15) + (1.5 × $0.60) = $1.65 in inference usage.
This is an illustration, not a measured run or a quote. It assumes the stated billed token counts, one successful call per description, and no extra generation. Retries, longer prompts, extra reasoning output, and other services can change the total. Credit-purchase minimums are separate from the amount consumed by these calls. The shop should review factual accuracy and prohibited claims before publishing any generated description.
Taxes are additional under Together’s standard terms. Review replenishment settings and refund conditions before funding an account; the terms generally describe paid fees as non-refundable unless an exception applies. Terms of service, section 6.
Model choice, exports, and ownership
Choose an exact model ID and check its context window, input types, function-calling support, structured outputs, and serving availability. These are model-specific. The live model catalog is more useful than a fixed headline count of “open-source models.”
Together documents checkpoint downloads for completed fine-tuning jobs. LoRA jobs can export merged weights or an adapter; full fine-tuning has a different checkpoint-output option. Export capability makes migration possible, but the destination still needs compatible hardware and software. Fine-tuned model deployment and downloads.
Ownership needs careful wording. Together’s terms assign your content and output to you as between you and Together, while third-party models remain their providers’ intellectual property. Model-specific terms still apply. Paying for training or downloading weights does not give unrestricted ownership of the underlying base model. Terms, sections 2 and 7.
For example, OpenAI’s gpt-oss-120b license is Apache 2.0, while Meta’s Llama 3.3 license is a separate Community License with its own distribution, attribution, and acceptable-use conditions. Check the exact version’s license before commercial deployment or redistribution.
Privacy and retention
Together’s current privacy documentation says prompt and response storage is on by default, while permission to use organization data to train other models is off by default. These settings answer different questions. For requests made with an organization’s API key, organization settings govern; changing a personal-profile toggle does not change the team’s traffic. Privacy and security settings.
Zero data retention must be enabled. Turning off prompt storage enables ZDR for inference content and also disables passthrough models. ZDR applies prospectively. It does not remove billing metadata or account records, and explicitly uploaded fine-tuning or batch files remain stored until deleted. Temporary processing in memory can still occur. ZDR scope and exceptions.
Some models forward requests to an upstream provider under that provider’s data policy. Review the passthrough setting and selected model before sending data. Private networking and VPC options are available for appropriate dedicated deployments; serverless region selection is unavailable. Confirm the actual deployment and contract rather than assuming every request has the same privacy or residency arrangement. Passthrough and enterprise deployment details.
The Privacy Policy separately covers account data, service providers, international transfers, and retention for legal and operational purposes. A content-retention setting is therefore only one part of a data-governance review.
The standard terms also restrict submitting financial, medical, or sensitive personal information. Do not infer permission for such workloads from general security marketing; resolve the applicable terms and any separate agreement first. Terms, section 4.
How to evaluate the platform
- Define one task. Write down what a correct answer looks like, the expected traffic, and the longest realistic input.
- Select the exact model and service. Verify the features and license you need before planning customization.
- Review data handling. Identify what will be sent, which privacy settings apply, and whether the model uses passthrough.
- Set a budget. Include output tokens, repeat calls, training, deployment uptime, and retained storage as relevant.
- Evaluate quality and operations. Use a small, authorized, non-sensitive sample; review errors, latency, and cost before expanding. Follow the service’s applicable terms.
Existing OpenAI-style integrations can reuse supported client methods with a Together API key, Together’s base URL, and the correct model ID. Compatibility is not universal: the docs list unsupported endpoints and parameter differences. Check the methods your application actually calls before treating migration as a configuration-only change. OpenAI compatibility reference.
Who should consider it?
Together AI is worth evaluating when an engineering team wants hosted models and a path toward customization or dedicated compute. It requires application development and operational decisions. Someone seeking only a ready-to-use writing assistant may prefer a simpler consumer interface.
The strongest reason to choose a service should be a measured fit for your task, budget, and data requirements. This documentation review does not establish that Together is the fastest, cheapest, or most accurate choice.
If your main goal is comparing and routing requests across model providers, our OpenRouter guide is a relevant alternative to explore. OpenRouter’s documentation describes unified model access and fallback handling. Compare the actual model, provider policy, and total cost for your workload; an API-routing service and a training-and-compute platform address overlapping but different needs.
Visit the official Together AI website to check current services and account requirements.
Bottom line: choose the model first, match the deployment to your traffic, and check data settings before sending real information. Together’s range of services can support a growing application, but each step adds its own billing and operational responsibilities.
