ForgeLM Toolkit

Advanced Model Merging

Create specialized AI models by mathematically combining task vectors. Add skills, remove toxicity, and optimize performance without retraining.

task_vector_lab.py

Surgically add or remove capabilities by manipulating model weights directly.

Algorithm: TIES-Merging

Base Model

Task Vectors

Code-Instruct

Storyteller

Guardrails

Merge Equation

Llama-3-70B

Resulting Capabilities

reasoning80%

coding60%

creativity70%

safety70%

Generated Config

{
  "base_model": "llama3",
  "merge_method": "ties",
  "parameters": {
    "density": 0.5,
    "weight": 0.5
  },
  "slices": [

  ]
}

Treat model capabilities as vectors. Add 'Coding' or subtract 'Bias' directly from the weights.

Resolve interference between different skills using our proprietary sparsity-based merging algorithm.

Merge models from different families (e.g., Llama + Mistral) using advanced layer mapping.