AI, LLM & Machine Learning Engineering

Training & Fine-Tuning Compute Calculators

The arithmetic behind a training run: parameter counts from architecture, the FLOPs a run consumes, how long it takes on given hardware, and whether fine-tuning ever pays for itself against a longer prompt.

4 calculators in this category

Transformer Parameter Count CalculatorGive this calculator the six numbers from a model config — hidden size, layer count, feed-forward width, attention heads…
LLM Training Compute (FLOPs) CalculatorC = 6ND scaling estimate (Kaplan et al. 2020; Hoffmann et al. 2022)This calculator turns a parameter count and a token budget into the three numbers that decide whether a pre-training run is…
Fine-Tuning vs Long-Prompt Break-Even CalculatorFixed-versus-variable cost break-even analysisFine-tuning trades a one-off training bill, and usually a higher per-token serving rate, against never sending your few-shot…
Model Training Time Estimate CalculatorData-parallel scaling modelThis calculator converts a dataset size, a per-device batch size and an epoch count into the number the scheduler cares about…

More ai, llm & machine learning engineering categories

LLM Inference Cost & Serving Model Memory & GPU Sizing RAG, Embeddings & Vector Search Model Evaluation & Metrics

Related categories in other subjects

Where this work overlaps other trades and disciplines.

Bandwidth, Throughput & LatencyComputing Computer Science, Data & Application MetricsComputing