ModelFoundry documentation

Everything needed to install the client, authenticate a workspace, pull a verified checkpoint and serve it locally. The docs mirror the 1.8.0 CLI and 1.8.0 runtime.

Installation #

The installer detects your platform and drops a single static binary into ~/.foundry/bin. It does not touch a Python environment, a CUDA toolkit or a container runtime.

Every example below uses MF_HOST — the base URL of the hub you are talking to, scheme included. Set it to the host that served this page; nothing else in the client or the SDK is tied to a particular domain.

# point the tools at this host
export MF_HOST="https://your-mirror-host"

# macOS / Linux
curl -fsSL "$MF_HOST"/install.sh | sh

# Windows (PowerShell)
$env:MF_HOST = "https://your-mirror-host"
irm "$env:MF_HOST"/install.ps1 | iex

# or download a release asset directly
foundry --version

Quickstart #

Four commands: install, authenticate, pull, serve.

foundry login --key "$MF_KEY"
foundry pull mf-ember-7b-instruct --quant q4_k_m
foundry serve ./mf-ember-7b-instruct-q4_k_m.gguf --ctx 32768 --parallel 4
foundry chat --alias mf-ember-7b-instruct

The client verifies the manifest digest on every pull, prints the digest it received and refuses to write a file that does not match. Point it at a mirror with --registry if you operate a private copy of the catalog.

Authentication #

Workspace keys look like MF-XXXX-XXXX-XXXX-XXXX and are the only credential in the system. There is no OAuth flow, no session cookie and no password reset — a key is created, scoped, rotated and revoked by the node administrator.

# interactive
foundry login

# non-interactive, from a secret store
foundry login --key "$MF_KEY"

# inspect the key that is currently active
foundry whoami

Keys carry a scope: pull for weights, eval for the harness and hosted for third-party inference on our fleet. A key with the wrong scope fails with 403 and a message naming the scope it needs.

CLI reference #

Common flags

--quant Q4_K_M|Q5_K_M|Q8_0|F16|BF16   quantization to fetch
--include tokenizer,config,license      auxiliary files to fetch
--resume                            continue an interrupted transfer
--out ./dir                           destination directory
--registry https://host               alternate catalog mirror
--insecure-skip-verify-tls          only for local test registries
--dry-run                           resolve and verify without downloading

REST API #

The public surface is intentionally small. Everything that mutates state requires a bearer key; everything that reads is cacheable at the edge.

curl -sS "$MF_HOST"/v1/models/mf-ember-7b-instruct \
  | jq '{id, license, builds: [.builds[] | {quant, sha256, bytes}]}'

# anonymous pull attempt — flat 401, no model enumeration
curl -sS -o /dev/null -w '%{http_code}\n' \
  -X POST "$MF_HOST"/v1/pulls \
  -d '{"model":"mf-ember-7b-instruct","quant":"Q4_K_M"}'

Python SDK #

The client is a thin wrapper over the same endpoints and works with any OpenAI-compatible server.

import os
from modelfoundry import Client

# MF_HOST is the base URL of the hub, scheme included
client = Client(base_url=os.environ["MF_HOST"])
client.login(api_key="MF-KEY-REDACTED")

# catalog reads need no key at all
for m in client.models.list(task="code"):
    print(m.id, m.license, m.context)

# streaming chat against your own local server
for chunk in client.chat.create(
        model="mf-cobalt-13b-code",
        messages=[{"role": "user", "content": "add a regression test"}],
        stream=True, temperature=0.2):
    print(chunk.delta, end="")

Manifest format #

A manifest is signed, versioned and the single source of truth for a checkpoint.

{
  "id": "mf-ember-7b-instruct",
  "revision": "1.8.0",
  "license": "MF-RL-1.0",
  "tokenizer_sha256": "1b9f…7ac0",
  "eval_suite": "mf-eval/3.2",
  "builds": [
    { "quant": "Q4_K_M", "bytes": 4724461824, "sha256": "9f4c…ae21" },
    { "quant": "Q5_K_M", "bytes": 5476072448, "sha256": "2ab8…04cf" },
    { "quant": "Q8_0",  "bytes": 8160432128, "sha256": "77de…31b9" }
  ],
  "signature": "ed25519:…"
}

Serving & performance #

Three knobs decide almost everything: context length, concurrency and the scheduler.

foundry serve ./mf-atlas-32b-instruct-q4_k_m.gguf \
  --ctx 32768 \
  --parallel 8 \
  --sched continuous \
  --prefix-cache \
  --draft ./mf-sable-1.5b-instruct-q4_k_m.gguf \
  --draft-tokens 5 \
  --port 8080

• Continuous batching is the default and should stay on for any mixed workload.
• Prefix cache pays for itself in chat and agent loops; turn it off for single-shot completions.
• Speculative decoding helps interactive traffic and hurts pure batch throughput.
• Admission control is a feature: interactive sessions should get a fast rejection instead of an unbounded queue.

Evaluation harness #

Every release pins a suite version, a judge model and decoding parameters. The harness is part of the public surface: if a number in a model card cannot be reproduced, it is a bug in the card, not in your setup.

[suite]
version = "mf-eval/3.2"
tasks = ["mmlu-pro", "ifeval", "humaneval+", "longbench", "mt-bench"]

[decode]
temperature = 0.0
seed = 1337
max_tokens = 2048

[judge]
model = "mf-granite-judge-8b"
temperature = 0.0

Mirroring this site #

The catalog itself is a set of static files. Anyone can mirror it: copy the tree, point a web server at it, and optionally redirect the CLI at your copy with --registry. Weight transfers stay on the origin unless your key is provisioned for your storage.

rsync -a --delete ./site/ mirror.example:/srv/modelfoundry/
foundry search --registry https://mirror.example
foundry pull mf-ember-7b-instruct \
  --registry https://mirror.example \
  --registry-weights https://weights.example

Troubleshooting #

Contributing #

Conversion tools, evaluation tasks, runtime fixes and documentation are all welcome. Publishing new weights is a smaller door: a pull request must include a model card, a license choice, reference results from the pinned suite and someone willing to maintain it.

• Bug fixes and docs land directly after review by one maintainer.
• New evaluation tasks must be reproducible and must not leak answers into the prompt set.
• New weights need a signed manifest and a passing foundry verify --strict run.
• Derivative models must be renamed and carry their own license; the mf- prefix is reserved.

Licenses #

Quantizing or fine-tuning produces a derivative work under every license above. Say so in your distribution, keep the license file attached, and do not present a derivative as an official build.

Terms of use #

The catalog, documentation and evaluation harness are provided as-is for research and internal use. Weight downloads are subject to the license attached to each checkpoint and to a workspace key. We do not offer warranties on benchmark parity across quantizations, and we may withdraw a build whose provenance cannot be established.

Privacy #

The mirror is static and sets no cookies. Server logs keep request metadata for abuse control — address, path, user agent, status — and nothing else. Pull transfers are recorded against a key identifier so an administrator can answer "was this build downloaded by us?" without storing request bodies or credentials.

FAQ #

Why is there no public download button?

Because a public bucket cannot be revoked. Keys are cheap to rotate; a re-upload of a 68 GB checkpoint is not.

Can I use the models commercially?

MIT and Apache-2.0 builds, yes. MF-RL-1.0 builds require a key for hosted third-party access, and the terms are in the license file.

Do you train the models?

We train most of the chat, code and embedding families. Mixture-of-experts releases include work from partner labs, credited on the model page.

Why does my score differ from the model card?

Usually quantization. Run the published harness against both builds and compare; the difference should be small, and if it is not, that is a reportable finding.

Can I mirror the catalog?

Yes — it is static HTML. See mirroring this site. Keep the license files and manifest links intact.

Documentation revision docs-1.8.0 · built alongside CLI 1.8.0 · Changelog · Status · Security