Heroku Managed Inference and Agents
Reliable and Powerful Inference as a Service
Managed Inference
Managed Inference and Agents simplifies AI integration by providing access to powerful foundation models, including text, embedding, and diffusion models. Easily attach model resources to your Heroku app, and the add-on will automatically configure environment variables, enabling seamless API calls. Invoke models using the CLI plug-in or with API endpoints.
Agents
Extend Agents with tools that allow Large Language Models (LLMs) to execute actions within Heroku’s trusted environment. Deploy autonomous agents that can call APIs, run code, or interact with your app through tools like code_exec, http, or custom ones. Move from prototyping to production with optimized inference latency and minimal infrastructure management.
Model Context protocol
The Model Context Protocol (MCP) is an open standard that helps you extend Agents by connecting large language models to tools, services, and data sources. You can bring your own custom tools by deploying them as a heroku app and registering them by attaching the addon. Access all you mcp servers through a single toolkit.
Use Cases
Text Generation & Chat Use models like Claude-Sonnet to generate text, write code, or chat intelligently.
Retrieval-Augmented Generation (RAG) Bring your own data to power LLMs with up-to-date, domain-specific knowledge.
Personalize User Experiences: Leverage agents to deliver tailored content, recommendations, or support.
Data Analysis and Business Intelligence: Deploy agents that can analyze large datasets, identify trends, generate reports, and provide actionable insights.
Metered Billing
For those customers paying by credit card, Heroku Managed Inference and Agents uses metered billing, as set forth in the Plans & Pricing tables below
For enterprise customers, your usage of Heroku Managed Inference and Agents will consume your General Add-on Credits and/or Data Add-on Credits as set forth in the Plans & Pricing tables below.
Data protection
Heroku Managed Inference and Agents doesn’t store or log your prompts and completions. Heroku Managed Inference and Agents doesn’t use your prompts and completions to train any models and doesn’t distribute them to third parties for training.
Plans & Pricing
-
Claude-3-5-haiku (us)
Provisioned in: usA faster, more affordable large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $0.8 per million tokens
- Output Token $4 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:claude-3-5-haiku -a $APP_NAME -- --region=us -
Claude-3-5-sonnet-latest (us)
Provisioned in: usA state-of-the-art large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $3 per million tokens
- Output Token $15 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:claude-3-5-sonnet-latest -a $APP_NAME -- --region=us -
Claude-3-7-sonnet (eu)
Provisioned in: euA state-of-the-art large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $3 per million tokens
- Output Token $15 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model is hosted in both the
usandeuregions. By default, apps with ausregion provision theusplan, and apps with theeuregion provision theeuplan. To create your model resource, run:This model is hosted in both the
usandeuregions.oregon,virginia, andmontrealprivate space apps provision theusplan by default. All other private space apps provision theeuplan by default. To create your model resource, run:heroku addons:create heroku-inference:claude-3-7-sonnet -a $APP_NAME -
Claude-3-7-sonnet (us)
Provisioned in: usA state-of-the-art large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $3 per million tokens
- Output Token $15 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model is hosted in both the
usandeuregions. By default, apps with ausregion provision theusplan, and apps with theeuregion provision theeuplan. To create your model resource, run:This model is hosted in both the
usandeuregions.oregon,virginia, andmontrealprivate space apps provision theusplan by default. All other private space apps provision theeuplan by default. To create your model resource, run:heroku addons:create heroku-inference:claude-3-7-sonnet -a $APP_NAME -
Claude-3-haiku (eu)
Provisioned in: euA faster, more affordable large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $0.25 per million tokens
- Output Token $1.25 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
euregion. Apps with ausregion by default cannot provision this model. To override this, apply the--region=euflag:This model only runs in the
euregion. Private space apps in theoregon,virginia, ormontrealregions by default cannot provision this model. To override this, apply the--region=euflag:heroku addons:create heroku-inference:claude-3-haiku -a $APP_NAME -- --region=eu -
Claude-4-sonnet (eu)
Provisioned in: euA state-of-the-art large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $3 per million tokens
- Output Token $15 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
euregion. Apps with ausregion by default cannot provision this model. To override this, apply the--region=euflag:This model only runs in the
euregion. Private space apps in theoregon,virginia, ormontrealregions by default cannot provision this model. To override this, apply the--region=euflag:heroku addons:create heroku-inference:claude-4-sonnet -a $APP_NAME -- --region=eu -
Claude-4-sonnet (us)
Provisioned in: usA state-of-the-art large language model that supports chat and tool-calling.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Anthropic
Metered usage amounts
- Input Token $3 per million tokens
- Output Token $15 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:claude-4-sonnet -a $APP_NAME -- --region=us -
Cohere-embed-multilingual (eu)
Provisioned in: euA state-of-the-art embedding model that supports multiple languages. This model is helpful for developing Retrieval Augmented Generation (RAG) search.
- Type Text → Embedding
- API endpoint v1/embeddings
- Model Source Cohere
Metered usage amounts
- Input Token $0.10 per million tokens
- Output Token N/A
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model is hosted in both the
usandeuregions. By default, apps with ausregion provision theusplan, and apps with theeuregion provision theeuplan. To create your model resource, run:This model is hosted in both the
usandeuregions.oregon,virginia, andmontrealprivate space apps provision theusplan by default. All other private space apps provision theeuplan by default. To create your model resource, run:heroku addons:create heroku-inference:cohere-embed-multilingual -a $APP_NAME -
Cohere-embed-multilingual (us)
Provisioned in: usA state-of-the-art embedding model that supports multiple languages. This model is helpful for developing Retrieval Augmented Generation (RAG) search.
- Type Text → Embedding
- API endpoint v1/embeddings
- Model Source Cohere
Metered usage amounts
- Input Token $0.10 per million tokens
- Output Token N/A
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model is hosted in both the
usandeuregions. By default, apps with ausregion provision theusplan, and apps with theeuregion provision theeuplan. To create your model resource, run:This model is hosted in both the
usandeuregions.oregon,virginia, andmontrealprivate space apps provision theusplan by default. All other private space apps provision theeuplan by default. To create your model resource, run:heroku addons:create heroku-inference:cohere-embed-multilingual -a $APP_NAME -
Nova Lite
Provisioned in: euA fast and highly cost-effective model, perfect for applications requiring rapid text generation, summarization, and copywriting. It's optimized for high-throughput tasks and general-purpose use cases.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Amazon
Metered usage amounts
- Input Token $0.30 per million tokens
- Output Token $0.40 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
euregion. Apps with ausregion by default cannot provision this model. To override this, apply the--region=euflag:This model only runs in the
euregion. Private space apps in theoregon,virginia, ormontrealregions by default cannot provision this model. To override this, apply the--region=euflag:heroku addons:create heroku-inference:nova-lite -a $APP_NAME -- --region=eu -
Nova Lite
Provisioned in: usA fast and highly cost-effective model, perfect for applications requiring rapid text generation, summarization, and copywriting. It's optimized for high-throughput tasks and general-purpose use cases.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Amazon
Metered usage amounts
- Input Token $0.30 per million tokens
- Output Token $0.40 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:nova-lite -a $APP_NAME -- --region=us -
Nova Pro
Provisioned in: euA high-performance model designed for more complex tasks, including advanced question-answering, detailed content creation, and nuanced data extraction. It provides a significant step up in capability for applications that demand higher quality and deeper understanding.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Amazon
Metered usage amounts
- Input Token $0.80 per million tokens
- Output Token $1.60 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
euregion. Apps with ausregion by default cannot provision this model. To override this, apply the--region=euflag:This model only runs in the
euregion. Private space apps in theoregon,virginia, ormontrealregions by default cannot provision this model. To override this, apply the--region=euflag:heroku addons:create heroku-inference:nova-pro -a $APP_NAME -- --region=eu -
Nova Pro
Provisioned in: usA high-performance model designed for more complex tasks, including advanced question-answering, detailed content creation, and nuanced data extraction. It provides a significant step up in capability for applications that demand higher quality and deeper understanding.
- Type Text → Text
- API endpoint v1/chat/completions
- v1/agents/heroku
- Model Source Amazon
Metered usage amounts
- Input Token $0.80 per million tokens
- Output Token $1.60 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:nova-pro -a $APP_NAME -- --region=us -
gpt-oss-120b (us)
Provisioned in: usThe gpt-oss-120b model is a powerful, open-source large language model developed by OpenAI, designed for a wide range of generative AI applications. It offers advanced capabilities in natural language understanding, generation, and complex problem-solving, making it a versatile tool for developers and enterprises.
- Type Text → Text
- API endpoint v1/chat/completions
- Model Source OpenAI
Metered usage amounts
- Input Token $0.15 per million tokens
- Output Token $0.60 per million tokens
- Per image N/A
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:openai-gpt-oss-120b -a $APP_NAME -- --region=us -
Stable-image-ultra (us)
Provisioned in: usA state-of-the-art diffusion (image generation) model.
- Type Text → Image
- API endpoint v1/images/generations
- Model Source Stability AI
Metered usage amounts
- Input Token N/A
- Output Token N/A
- Per image $0.14
Availability
Region Available Dublin Available Frankfurt Available London Available Montreal Available Mumbai Available Oregon Available Singapore Available Sydney Available Tokyo Available Virginia Available Install Heroku Managed Inference and AgentsThis model only runs in the
usregion. Apps with aeuregion by default cannot provision this model. To override this, apply the--region=usflag:This model only runs in the
usregion. Only private space apps in theoregon,virginia, ormontrealregions can provision this model by default. To override this for apps in other private space regions, apply the--region=usflag:heroku addons:create heroku-inference:stable-image-ultra -a $APP_NAME -- --region=us
Heroku Managed Inference and Agents Documentation
Legal Notices
The Heroku Managed Inference and Agent add-on may employ third-party generative AI models to provide the Service. Due to the nature of generative AI, the output that it generates may be unpredictable, and may include inaccurate or harmful responses. Customer assumes all responsibility for such output, including ensuring its accuracy, safety, and compliance with applicable laws and third-party acceptable use policies. For more information, please see the Heroku Notices and License Information Documentation.