Synqtune

Task templates a pipeline can start from, with the inputs each needs and the data format it expects.

Takes no parameters.

Recent pipeline runs, newest first. status: all, running, completed, failed or cancelled.

Status of one run: current step, progress, error, and a summary of its evaluation and comparison.

The comparison between the fine-tuned model and the LLM it should replace: verdict (ready / close / not_ready), reasons, scores, the weakest labels or fields, and the examples where the fine-tuned model was worse.

Saved models with their latest and production versions.

Takes no parameters.

Score a saved model version again on a held-out file in S3, optionally against an LLM. Nothing is trained. version_id: a model version (latest_version_id from list_models). eval_uri: .jsonl, .json or .parquet in the format the model was trained on, with rows it never trained on. Leave compare_* blank to compare with the LLM the training run compared with, if any. compare_system_prompt: the prompt the LLM gets in production; blank keeps the training run's, and with none the LLM is sent the task instruction. inference_engine: what serves the fine-tuned model in the comparison, huggingface or vllm (blank keeps the training run's); vllm also times the examples sent 8 and 32 at a time and needs a g5 instance or newer. Without confirm=true this only returns the plan and cost estimate: show it to the user and call again with confirm=true once they agree; it starts a GPU instance billed to the user's AWS account.

GPU cost of a training run (per hour, per day, rough total for the run) and, if you give current LLM usage and prices, the monthly API spend vs serving the model on the same GPU.

Train a model from a task template on data in S3, in the user's AWS account. Without confirm=true this only returns the plan and cost estimate: show it to the user and call again with confirm=true once they agree. Launching starts GPU instances billed to the user's AWS account. inputs: the template's inputs (see list_templates), e.g. {"labels": "billing, refund, shipping"}. compare_*: optionally compare with the LLM the model should replace (provider anthropic, openai or gemini). The provider key must already be saved in SynqTune under Settings > LLM provider keys; keys are never passed here. holdout_uri: a held-out file (same format as the source) to test on instead of a random split of the source. compare_max_samples: how many held-out examples to send to the LLM (default 300). compare_price_input_per_mtok / compare_price_output_per_mtok: the LLM's USD price per 1M tokens, so the report shows its cost next to the fine-tuned model's. max_steps, batch_size, base_model: override the template's training. inference_engine: what serves the fine-tuned model in the comparison, huggingface (default) or vllm. vllm gives production-like speed and cost, also times requests sent 8 and 32 at a time, and needs a g5 instance or newer.