Petals

What is Petals?

Petals is an Inference tool. Run LLMs at home with BitTorrent-style distribution for faster inference and fine-tuning. It is available on Web App. It is aimed at developers, researchers and data scientists.

Petals allows users to run large language models (LLMs) locally using a BitTorrent-style network, enabling faster inference and fine-tuning compared to traditional offloading methods. It supports models like Llama 3.1, Mixtral, Falcon, and BLOOM, and provides tutorials, benchmarks, and contributing guidelines.

Built for developers, researchers, data scientists and ai enthusiasts.

Key capabilities

• Text Generation
• Chat

Features

• BitTorrent-style Distribution

  Uses a decentralized network to distribute AI model processing.

• Model Support

  Supports large language models like Llama 3.1, Mixtral, Falcon, and BLOOM.

• Tutorials and Examples

• Benchmarks

• Contributing Guidelines

• Citiation Resources

Use cases

• Run large language models locally using a decentralized network for faster processing
• Fine-tune models with improved speed compared to offloading methods
• Access tutorials and benchmarks to optimize model performance
• Contribute to the project by following provided guidelines

Pros & cons

AI models

• Llama 3.1

Developer & API

FAQ