Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@
# 1. TIER ROUTING (the main routing knob — REQUIRED)
# ==============================================================================
# Format: TIER_<LEVEL>=provider:model[:variant]
# Supported providers: ollama, openai, azure-openai, azure-anthropic, openrouter,
# databricks, bedrock, vertex, zai, moonshot, llamacpp, lmstudio
# Supported providers: ollama, openai, atlas, azure-openai, azure-anthropic,
# openrouter, edenai, databricks, bedrock, vertex, zai,
# moonshot, llamacpp, lmstudio
#
# When all 4 TIER_* are set, Lynkr enters "tier routing mode":
# - MODEL_PROVIDER auto-detected from TIER_SIMPLE
Expand All @@ -45,7 +46,7 @@ TIER_REASONING=moonshot:kimi-k2-thinking
# Values: true | false
FALLBACK_ENABLED=false
# DESCRIPTION: Fallback provider when tier provider fails (cannot be local).
# One of: databricks, azure-anthropic, azure-openai, openrouter, openai, bedrock
# One of: databricks, azure-anthropic, azure-openai, openrouter, edenai, openai, atlas, bedrock
FALLBACK_PROVIDER=databricks
# DESCRIPTION: [DEPRECATED legacy knob] primary provider for credential validation.
# Auto-detected when TIER_* is set. Same allowed values as FALLBACK_PROVIDER plus ollama/llamacpp/lmstudio/vertex/zai/moonshot.
Expand Down Expand Up @@ -101,6 +102,16 @@ OPENAI_ENDPOINT=https://api.openai.com/v1/chat/completions
# DESCRIPTION: Optional OpenAI org id.
# OPENAI_ORGANIZATION=org-your-org-id

# ------------------------------------------------------------------------------
# Atlas Cloud (OpenAI-compatible)
# ------------------------------------------------------------------------------
# DESCRIPTION: Atlas Cloud API key from https://www.atlascloud.ai/console/api-keys.
# ATLASCLOUD_API_KEY=your-atlas-cloud-api-key
# DESCRIPTION: Default Atlas Cloud text model.
ATLASCLOUD_MODEL=qwen/qwen3.8-max
# DESCRIPTION: Atlas Cloud Chat Completions endpoint.
ATLASCLOUD_ENDPOINT=https://api.atlascloud.ai/v1/chat/completions

# ------------------------------------------------------------------------------
# Ollama (local models)
# ------------------------------------------------------------------------------
Expand Down Expand Up @@ -441,7 +452,7 @@ LYNKR_STREAM_TRANSFORM=true
# Values: true | false (default true — set false to stream)
LYNKR_OLLAMA_BUFFER_RESPONSES=true
# DESCRIPTION: Providers eligible for the cross-format stream transform.
# Values: comma-separated (default openai,azure-openai,openrouter,databricks,lmstudio,llamacpp)
# Values: comma-separated (default openai,atlas,azure-openai,openrouter,databricks,lmstudio,llamacpp,moonshot)
#LYNKR_STREAM_TRANSFORM_PROVIDERS=openai,azure-openai
# DESCRIPTION: Abort a passthrough stream when the upstream goes silent this long.
LYNKR_STREAM_IDLE_TIMEOUT_MS=60000
Expand Down
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<td align="center"><strong>84%</strong><br/>JSON Compression</td>
<td align="center"><strong>53%</strong><br/>Tool Token Reduction</td>
<td align="center"><strong>&lt;300ms</strong><br/>Semantic Cache Hits</td>
<td align="center"><strong>13+</strong><br/>LLM Providers</td>
<td align="center"><strong>14+</strong><br/>LLM Providers</td>
<td align="center"><strong>0</strong><br/>Code Changes Required</td>
</tr>
</table>
Expand Down Expand Up @@ -68,7 +68,7 @@ The fastest path is the interactive wizard:
lynkr init
```

It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 12 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated `.env` with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, rate limiting).
It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 14 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated `.env` with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, rate limiting).

Useful flags:

Expand Down Expand Up @@ -216,9 +216,21 @@ Claude Code / Cursor / Codex / Cline / Continue
| **Azure OpenAI** | Cloud | GPT-4o, o1, o3 | $$$ |
| **Azure Anthropic** | Cloud | Claude Sonnet, Opus | $$$ |
| **OpenAI** | Cloud | GPT-4o, o3-mini | $$$ |
| **Atlas Cloud** | Cloud | Qwen, DeepSeek, and other OpenAI-compatible models | $-$$$ |
| **DeepSeek** | Cloud | DeepSeek R1, Reasoner | $ |

**4 local providers** for 100% offline, free usage. **10+ cloud providers** for scale.
**4 local providers** for 100% offline, free usage. **11+ cloud providers** for scale.

<div align="center">
<a href="https://www.atlascloud.ai/?utm_source=github&amp;utm_medium=readme&amp;utm_campaign=lynkr">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://www.atlascloud.ai/logo-white.svg">
<img src="https://www.atlascloud.ai/logo.svg" alt="Atlas Cloud" width="196">
</picture>
</a>
<br>
<sub>Use Lynkr with Atlas Cloud's OpenAI-compatible API and route across its live model catalog with one optional provider configuration. <a href="https://www.atlascloud.ai/?utm_source=github&amp;utm_medium=readme&amp;utm_campaign=lynkr">Explore Atlas Cloud</a>.</sub>
</div>

---

Expand Down
13 changes: 12 additions & 1 deletion bin/lynkr-init.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,17 @@ const PROVIDERS = {
extras: [],
defaultModel: 'gpt-4o',
},
atlas: {
label: 'Atlas Cloud (OpenAI-compatible)',
local: false,
creds: [
{ key: 'ATLASCLOUD_API_KEY', label: 'Atlas Cloud API key', secret: true },
],
extras: [
{ key: 'ATLASCLOUD_ENDPOINT', label: 'Chat Completions endpoint', default: 'https://api.atlascloud.ai/v1/chat/completions' },
],
defaultModel: 'qwen/qwen3.8-max',
},
openrouter: {
label: 'OpenRouter (100+ models, one key)',
local: false,
Expand Down Expand Up @@ -163,7 +174,7 @@ const PROVIDERS = {

const PROVIDER_ORDER = [
'ollama', 'llamacpp', 'lmstudio',
'azure-anthropic', 'azure-openai', 'openai', 'openrouter', 'edenai',
'azure-anthropic', 'azure-openai', 'openai', 'atlas', 'openrouter', 'edenai',
'databricks', 'bedrock', 'vertex', 'zai', 'moonshot',
];
const TIERS = ['SIMPLE', 'MEDIUM', 'COMPLEX', 'REASONING'];
Expand Down
2 changes: 1 addition & 1 deletion documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Welcome to the comprehensive documentation for Lynkr, the self-hosted Claude Cod
New to Lynkr? Start here:

- **[Installation Guide](installation.md)** - Complete installation instructions for all methods (npm, git clone, homebrew, Docker)
- **[Provider Configuration](providers.md)** - Detailed setup for all 12+ supported providers (Databricks, Bedrock, OpenRouter, Ollama, llama.cpp, Azure OpenAI, Azure Anthropic, OpenAI, LM Studio, Moonshot AI, Z.AI, Vertex AI)
- **[Provider Configuration](providers.md)** - Detailed setup for all 14+ supported providers (Databricks, Bedrock, OpenRouter, Eden AI, Ollama, llama.cpp, Azure OpenAI, Azure Anthropic, OpenAI, Atlas Cloud, LM Studio, Moonshot AI, Z.AI, Vertex AI)
- **[Quick Start Examples](installation.md#quick-start-examples)** - Copy-paste configurations to get running fast

---
Expand Down
2 changes: 1 addition & 1 deletion documentation/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ To add support for a new LLM provider:
// Add to SUPPORTED_MODEL_PROVIDERS
const SUPPORTED_MODEL_PROVIDERS = new Set([
"databricks", "azure-anthropic", "ollama",
"openrouter", "azure-openai", "openai",
"openrouter", "azure-openai", "openai", "atlas",
"llamacpp", "lmstudio", "bedrock", "newprovider" // Add here
]);

Expand Down
2 changes: 1 addition & 1 deletion documentation/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ data: {}

**End-to-End SSE Through the Tier Router:**
- Native passthrough for Anthropic-format upstreams (Anthropic endpoints, Z.AI, Ollama v0.14+ with `LYNKR_OLLAMA_BUFFER_RESPONSES=false`) — upstream bytes piped straight through with backpressure
- OpenAI-format upstreams (openai, azure-openai, openrouter, databricks, llamacpp, lmstudio) transformed to Anthropic SSE in flight, including reassembly of split tool-call argument fragments into complete `tool_use` blocks
- OpenAI-format upstreams (openai, atlas, azure-openai, openrouter, databricks, llamacpp, lmstudio) transformed to Anthropic SSE in flight, including reassembly of split tool-call argument fragments into complete `tool_use` blocks
- `LYNKR_VISIBLE_ROUTING` badge injected into live streams as the first content block after `message_start`
- If the upstream fails before the first byte, the request falls back to the buffered path; after the first byte, errors surface as SSE error events
- Telemetry (latency, tokens, tool calls) recorded by a stream-close finalizer
Expand Down
38 changes: 37 additions & 1 deletion documentation/providers.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Provider Configuration Guide

Complete configuration reference for all 12+ supported LLM providers. Each provider section includes setup instructions, model options, pricing, and example configurations.
Complete configuration reference for all 14+ supported LLM providers. Each provider section includes setup instructions, model options, pricing, and example configurations.

---

Expand All @@ -19,6 +19,7 @@ Lynkr supports multiple AI model providers, giving you flexibility in choosing t
| **Azure OpenAI** | Cloud | GPT-4o, GPT-5, o1, o3 | $$$ | Cloud | Medium |
| **Azure Anthropic** | Cloud | Claude models | $$$ | Cloud | Medium |
| **OpenAI** | Cloud | GPT-4o, o1, o3 | $$$ | Cloud | Easy |
| **Atlas Cloud** | Cloud | Qwen, DeepSeek, and other OpenAI-compatible models | $-$$$ | Cloud | Easy |
| **Moonshot AI (Kimi)** | Cloud | Kimi K2 (thinking + turbo) | $ | Cloud | Easy |
| **LM Studio** | Local | Local models with GUI | **FREE** | 🔒 100% Local | Easy |
| **MLX OpenAI Server** | Local | Apple Silicon optimized | **FREE** | 🔒 100% Local | Easy |
Expand Down Expand Up @@ -725,6 +726,41 @@ OPENAI_MODEL=o1-mini # Smaller reasoning model

---

### 8a. Atlas Cloud (OpenAI-Compatible)

**Best for:** Accessing Atlas Cloud text models through Lynkr's static or tier-based routing

#### Configuration

```env
MODEL_PROVIDER=atlas
ATLASCLOUD_API_KEY=your-atlas-cloud-api-key
ATLASCLOUD_MODEL=qwen/qwen3.8-max
ATLASCLOUD_ENDPOINT=https://api.atlascloud.ai/v1/chat/completions
```

Get an API key from the [Atlas Cloud console](https://www.atlascloud.ai/console/api-keys). The endpoint is OpenAI-compatible, so Lynkr converts Anthropic messages and tools to Chat Completions requests and transforms streamed OpenAI SSE events back to Anthropic events.

Atlas can also be selected per tier:

```env
TIER_SIMPLE=ollama:qwen2.5-coder:latest
TIER_MEDIUM=atlas:qwen/qwen3.8-max
TIER_COMPLEX=atlas:qwen/qwen3.8-max
TIER_REASONING=atlas:qwen/qwen3.8-max
```

`ATLASCLOUD_MODEL` is the static-routing default. A model named in `TIER_*` takes precedence for that tier. Check the [live Atlas Cloud model catalog](https://www.atlascloud.ai/models) before selecting a different model; availability and pricing depend on that model.

#### Behavior

- Supports non-streaming and streaming Chat Completions
- Supports Anthropic-format tools through Lynkr's existing OpenAI conversion path
- Uses Bearer authentication and accepts a custom `ATLASCLOUD_ENDPOINT`
- Does not automatically replay billable Chat Completions POST requests

---

### 9. LM Studio (Local with GUI)

**Best for:** Local models with graphical interface
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
"dev": "nodemon index.js",
"lint": "eslint src index.js",
"test": "npm run test:unit && npm run test:performance",
"test:unit": "DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js",
"test:unit": "DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com LOG_FILE_ENABLED=false node --test test/routing.test.js test/hybrid-routing-integration.test.js test/retry-logic.test.js test/sse-transformer.test.js test/passthrough-stream.test.js test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js test/azure-openai-config.test.js test/azure-openai-format-conversion.test.js test/azure-openai-routing.test.js test/azure-openai-streaming.test.js test/azure-openai-error-resilience.test.js test/azure-openai-integration.test.js test/openai-integration.test.js test/atlas-integration.test.js test/toon-compression.test.js test/gcf-compression.test.js test/llamacpp-integration.test.js test/resilience.test.js test/telemetry-routing.test.js test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js test/distill.test.js test/large-payload.test.js test/prompt-cache-injection.test.js test/risk-analyzer.test.js test/interaction-block.test.js test/preflight.test.js test/token-reduction.test.js test/session-affinity.test.js test/cache-state.test.js test/cache-switch-cost.test.js test/lens-recommendations.test.js test/model-registry-cost.test.js test/output-format-guard.test.js test/tier-fallback.test.js test/wrap.test.js test/init.test.js test/tool-call-response-metadata.test.js test/degradation.test.js test/routing-telemetry-columns.test.js test/sticky-routing.test.js test/knn-ambiguous-escalate.test.js test/deescalator.test.js test/client-profiles.test.js test/strip-internal-fields.test.js test/complexity-tool-subtraction.test.js test/bandit.test.js test/routing-propensity.test.js test/reward-pipeline.test.js test/knn-cold-start.test.js test/calibration.test.js test/feedback-loop.test.js test/session-fingerprint.test.js test/side-request-guards.test.js test/verifier.test.js test/intent-score.test.js test/difficulty-classifier.test.js test/classifier-setup.test.js test/usage-stats.test.js test/loop-guard.test.js test/moonshot-model-mapping.test.js",
"test:memory": "DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/memory/store.test.js test/memory/surprise.test.js test/memory/extractor.test.js test/memory/search.test.js test/memory/retriever.test.js test/memory/distiller.test.js test/memory/distiller-freeze.test.js test/memory/wiki.test.js test/memory/skills-cache.test.js test/memory/tencentdb-launcher.test.js",
"test:new-features": "DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node --test test/passthrough-mode.test.js test/openrouter-error-resilience.test.js test/format-conversion.test.js",
"test:performance": "DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/hybrid-routing-performance.test.js && DATABRICKS_API_KEY=test-key DATABRICKS_API_BASE=http://test.com node test/performance-tests.js",
Expand Down
13 changes: 13 additions & 0 deletions src/api/providers-handler.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,19 @@ function getConfiguredProviders() {
});
}

// Check Atlas Cloud (OpenAI-compatible Chat Completions)
if (config.atlas?.apiKey) {
providers.push({
name: "atlas",
type: "atlas-cloud",
baseUrl: config.atlas.endpoint || "https://api.atlascloud.ai/v1/chat/completions",
enabled: true,
models: [
{ id: config.atlas.model || "qwen/qwen3.8-max", name: "Configured Model" },
]
});
}

// Check OpenRouter
if (config.openrouter?.apiKey) {
providers.push({
Expand Down
Loading
Loading